Hot-swapping code in Python

superjoe on 2011/08/26 19:52

If you watched Notch coding during Ludum Dare, you might have, like me, been jealous that he could do hot swapping of code to tweak his game engine very quickly.

Well there's no reason we can't do that in Python!

Over the next week, I'm going to investigate and/or develop a framework that will perform automatic change detection and apply those changes, using the "reload" keyword of Python, to the running instance of the game.

A couple articles I'm going to look at:
http://stackoverflow.com/questions/6751263/hot-swapping-of-python-running-program
http://pypi.python.org/pypi/hotswap/0.1

If anyone has any knowledge about this already, please share!

My goal is to, by theme voting commencement (September 4), release code and/or document a strategy that anyone can use during PyWeek.

(log in to comment)

Comments

Mat on 2011/08/27 17:46:

cool :D

interested to see how it works...

bitcraft on 2011/08/30 15:40:

since the python interpreter treats everything as an object even to the byte code level, it's possible to create objects that are normally only created when the file is parsed and translated into byte code.  reloading a module at runtime isn't a bad idea, but you may also keep in mind that if you only have small chunks of code changing, you can restrict it to that and you won't be forcing the interpreter to check an entire module.  on some systems, that may be slow.

http://pastebin.com/XzTnfgAd

this pastebin has some code from a mud i coded a while back.  it uses source code in the strings in get_source and set_source, does string substitution for a variables name, creates python functions and then wraps them into a property that can be added to a class...at runtime.  last i knew it worked, but that was probably python2.4 and i haven't looked at the mud for years.  it might be helpful, though.

mauve on 2011/08/30 18:06:

A quick look at hotswap.py makes it look like it's along the right lines in most circumstances, though it looks like it won't work with metaclasses among other things. And that's the general way of things - Python is very dynamic and any change in code might not result in datastructures not being set up the right way for the code to operate on them.

The most general approach might be to save your gamestate (just to memory would suffice), drop all your datastructures, reload the modules, and load the gamestate back in a error-tolerant way. Effectively I'd focus on getting your code reloaded correctly, and risk breaking your current gamestate, which is less important in many regards.