Python code reflection and modification

Question

Perhaps I'm not using the proper term, but I'm looking to take a block of Python code (in Python), get the token tree for the code, make some kind of modification, and re-assemble it to Python.

For instance, consider this block of code:

def x(y):
    b = 2
    if y == b:
        foo(y)

I would like to be able to take this and programmatically do this:

def x(y):
    b = 2
    if y == b:
        bar(y)

I can't imagine that there's not a library that does something like this. Thanks in advance.

EDIT

Perhaps I wasn't entirely clear. I'm looking for a tool to read and manipulate arbitrary code, not code that I'm writing. I'd like to be able to modify code on-the-fly. The project I'm working on is a test app: it uses the Netflix philosophy to try to randomly break the functionality of an app in as many ways as it can, running the test suite each time. When the tests don't fail, there's an indication that there's either a gap in code coverage and/or the code is dead.

What about passing the function the function you want to execute as an argument? — Santiago Alessandri
– Santiago Alessandri, Commented Jan 21, 2011 at 22:40
Have you taken a look at the library support for what you want? docs.python.org/library/language.html — Apalala
– Apalala, Commented Jan 21, 2011 at 22:49

Andrew Clark · Accepted Answer · 2011-01-21 23:57:27Z

2

I was curious about this so I looked into the link Apalala posted, here is what I came up with:

from token import NAME
from tokenize import generate_tokens, untokenize
from StringIO import StringIO

source = """def x(y):
    b = 2
    if y == b:
        foo(y)"""
result = []
tokens = generate_tokens(StringIO(source).readline)
for toknum, tokval, _, _, _ in tokens:
    if toknum == NAME and tokval == "foo":
        tokval = "bar"
    result.append((toknum, tokval))

print untokenize(result)

And the result is:

def x (y ):
    b =2 
    if y ==b :
        bar (y )

Yeah I know, the spacing is ugly. I could not figure out how to maintain the format from the original code, but as far as functionality goes, this does what you want.

answered Jan 21, 2011 at 23:57

Andrew Clark

210k36 gold badges286 silver badges310 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

mattbasta Over a year ago

Awesome! This looks exactly like what I'm searching for! Thanks!

Paul Sasik · Accepted Answer · 2011-01-21 22:54:17Z

1

In Python functions can be passed just like any other object so you should be able to do something like:

def x(fx, y):
    b = 2
    if y == b:
        fx(y)

and then call:

x(foo, y)

or

x(bar, y)

Here's a stripped down, tested snippet (v 3.1) that demonstrates the concept w/o any bizlogic, just sample execution and arbitrary numeric manipulation:

def x(fx, y): return fx(y)
def foo(x): return x
def bar(x): return x+1

print(x(foo,1))
print(x(bar,1))

output:

>>> 
1
2

edited Jan 21, 2011 at 22:54

answered Jan 21, 2011 at 22:46

Paul Sasik

81.9k23 gold badges152 silver badges194 bronze badges

2 Comments

Exelian Over a year ago

Hmm, your answer seems to be more elegant then mine. Should've thought of this myself :P

mattbasta Over a year ago

The point is to be able to do this from the text from a file: I want to be able to accept arbitrary code and modify it on the fly. Nothing complex, but for instance, I'd like to be able to delete, say, an if statement and replace it with the contents of its else block.

Paolo · Accepted Answer · 2013-07-28 07:43:43Z

You can change your program classes by reflection like this


    class A(object):
        def __init__(self, x):
            self.x = x
        def show(self):
            print self.x

    def chglobals():
        import newmod
        reload(newmod)
        o = newmod.A(0)
        globals()['A'] = o.__class__

    def main():
        a = A(10)
        a.show()

        #change and wirte new code
        src = inspect.getsource(a.__class__)
        src = src.replace("print self.x", 'print "foo" ')
        text_file = open("newmod.py", "w")
        text_file.write(src)
        text_file.close()
        chglobals()

        #new istances
        b = A(20)
        b.show()

        #convert old istances
        a.__class__ = b.__class__
        a.show()


    main()

Exelian · Accepted Answer · 2011-01-21 22:43:50Z

-1

I'm going to have to guess about the usecase here, but I'll assume that at a certain point in time you want to switch for 1 function to the other. Doing this can be done relatively easily be using a dictionary filled with functions. For example:

def foo(x): return x
def bar(b): return x+1
fn = { 'foo': foo, 'bar': bar }

def x(y):
    b = 2
    func = 'foo'
    if y == b:
        fn[func](y)

This will let you choose which function you want based simply on the dictionary key. You can do so simply by changing the value of func (for example to 'bar').

answered Jan 21, 2011 at 22:43

Exelian

5,9381 gold badge33 silver badges51 bronze badges

1 Comment

mattbasta Over a year ago

Check out the comment I left on Paul's answer. I'm looking for something more lex-yaccy.

Collectives™ on Stack Overflow

Python code reflection and modification

4 Answers 4

1 Comment

2 Comments

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

1 Comment

2 Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related