Distribute 101

As Cosmologicon mentioned a way of packaging - a bunch of files in a directory - that certainly works, I thought I'd discuss the way the Python community has built for packaging and installing software, including Pyweek games.

You probably know about pip. pip is the current system of choice for installing packages, and is more powerful than easy_install.Well, distribute is the current system of choice for packaging. It's largely backwards-compatible with previous systems setuptools and distutils. In fact, the Python package is called setuptools! Distribute gives you tons of features, but here are a few:

You're probably thinking "Man, I have to go through all the hassle of packaging this properly? Too much effort!" But really, it's just a copy-and-paste (and fill in the blanks) job.

First, structure your app right, something like

├── gamename
│   ├── data
│   ├── __init__.py
│   └── __main__.py
├── run_game.py
└── setup.py

My preference is to not use holder directories like "gamelib" or "lib" or "src". But your game must be a package, not a bunch of modules in a directory. That's to say, all your Python source must be in a directory with a sensible name, with an __init__.py. And the data files must be within that directory too.

There's something else - run_game.py can go into your source distribution, but it won't be installed. Don't put functionality into it, or that functionality won't be available to your game once installed. run_game.py should contain just this:

from gamename.__main__ import main

And __main__.py should be the place where your game logic starts:

def main():
    print "Your game starts here"

if __name__ == '__main__':

(You'll notice you can now run your game without run_game.py, by just running "python gamename"!)

The last thing you need is to be able to access files in data. This is rather easy now, just use pkg_resources (no os.path.join() - it's forward slashes all the way here):

from pkg_resources import resource_stream, resource_string

levels = resource_string(__name__, 'data/levels.txt')
sprite = pygame.image.load(resource_stream(__name__, 'data/sprite.png'))

So far, this is all standard Python trickery. Now comes distribute. All we do is create a setup.py in the root that describes some metadata about the game:

from setuptools import setup, find_packages

        'gamename': ['data/*'],
        'console_scripts': [
            'gamename = gamename.__main__:main',

You also need to write a MANIFEST.in that states what files go into the sdist:

include run_game.py
include README
recursive-include gamename/data *

You can write this however you like - all you need to ensure is that your development crap doesn't go in, and all the files you want to distribute do go in.

And that's a wrap!

Building the source package:

python setup.py sdist

Installing the package (either directly or having downloaded it):

python setup.py install

Running the game, having installed the package :-D:


Installing the package directly from the intertubes:

pip install http://example.com/downloads/gamename-1.0.tar.gz

And this also works with virtualenv, so that you don't need to install the package system-wide:

virtualenv env
python setup.py install

As I understand it, all of this is covered by Richard's Skellington framework! But I've saved all of the layout above into a simple tarball for you to adapt.

(log in to comment)


Thank you very much! As someone who doesn't usually produce python applications (only one-off scripts for the most part), I'm definitely not very familiar with the process. I'll give it a shot this PyWeek.

(Bumping this post from 4 years ago.)

I never got around to trying this setup, but I always wanted to. Maybe I'll give it a shot this time. Would you say that the information here is still the best way to do it?

Yes, this is all still about right, except that distribute got rolled back into setuptools.

Awesome, thanks!

One thing I've noticed so far: the link for pkg_resources doesn't work anymore, because of the whole distribute/setuptools thing. The new link is probably:

(You'll notice you can now run your game without run_game.py, by just running "python gamename"!)

I'm having a little trouble setting up my imports with this option. I have my directory set up like you said, and in __main__.py I try to import another module in the same directory, in this case settings.py. As I understand it, I should be using relative imports:

from . import settings

That works fine if I run the game as "python run_game.py" in both python 2 and 3, but if I turn to run it as "python gamename" it only works in python 3. In python 2 I get:

ValueError: Attempted relative import in non-package

Is there a way to write the import so this works in python 2 as well? Thanks!

That's interesting. I guess the __main__.py isn't actually executed as a submodule of the package. From the docs, it's implied that it uses sys.path to load the package, but that the <pkg>.__main__ module is executed as the main module. In any case the upshot seems to be that you can't use relative imports in this module.

But if I don't use relative imports and instead just say "import settings", then it fails in python3 when I try to run it via run_game.py:

ImportError: No module named 'settings'
You'll need to do "from gamename import settings" if it isn't executing as a sub module of the package.
But if I do that then it fails with "ImportError: No module named gamename" in both Python 2 and 3 when I invoke it as "python gamename".
Oh! I think it should have read "python -m gamename"!