skellington-2.3 make sdist does not copy data folder

phoe6 on 2011/04/10 15:02

When developing the game, I had a particular folder structure and and then before uploading, I made it suitable in the format obtained by doing create_skeleton.py.
I had to copy my resources from resources folder to data folder and then change a path in program. I tried using python run_program.py and it works properly. 
 
If I do make sdist, a proper build is created, but data folder is not copied. Should I do anything special? With data folder, my program created using python setup.py sdist won't work.

The one the final version, I manually tar gzipped it so that data folder is present, but I would still like to resolve this issue.

(log in to comment)

Comments

gizmo_thunder on 2011/04/10 16:19:

source distribution is just for the game source i guess.. I found it a little annoying too but i think they did it because it was just "sdist" :). Also is it just me or is it really too many files in there... too many files i never used.. i could have deleted them but for a first timer it would be a little too much to handle may be?

tartley on 2011/04/10 18:23:

My impression is that data files ought to be included in an sdist.

You need to add either a 'data_dir' or a 'package_data' entry to the config passed to setup in your setup.py. They both work differently.

'data_dir' sounds initially most appealing, because it handles data directories that live next to your source code (as opposed to in the source code directory.) However, this option has a complication, in that it puts the data in one of two different places depending on whether the user installs your program from an unzipped sdist, using 'python setup.py install) or from PyPI (using easy_install or pip install.) This makes it harder for your source code to find stuff (you'll have to search both locations) Plus, In the PyPI case, it puts your data right at the root of the user's installed Python directory. Based on this, the nice people on stackoverflow have helped me to understand that data_dir is best used for documentation and for files which are shared between this package and other packages (which is pretty rare.)

The other option, then, is 'package_data'. This handles data directories that live in amongst your source code (ie. within a python package.) This sounds a little unusual to me, but because of the unusual behaviour of data_dir, package_dir is what I'm currently using for things like game images, sounds, etc.

I've written a small function called 'get_package_data', to help enumerate files for package_data. You can see it defined and used here:
https://bitbucket.org/tartley/gloopy/src/ba7e774dfadb/setup.py

Tee on 2011/04/10 18:50:

I also had the same problem and I ended up packing it up manually since I had no time to look deeper. This version seems to be more customizable, but, based on a shallow first impression, I prefer the previous version which is simpler to use. Furthermore, I like the fact that the previous version only asked the name when packing, where as in this one I had to rebuild the skeleton and copy the files only to change the name. The option to change it might be there, but, again, I didn't have time to look deeper. In my opinion, the simpler, the better, especially when you're at 30 minutes to deadline. The default should be able to take care of everything.

richard on 2011/04/11 04:52:

Yeah, unfortunately something strange happened and 2.3 was a bizarre hybrid of two approaches. Moving the "data" directory into the game's package should fix things. Well, except you'll probably have to modify paths to the data files in your code.

The new setup is nice because of the added distribution avenues it offers, but the increased complexity is not a good thing.

illume on 2011/04/11 13:58:

It did copy the $ROOT/yourgamename/data directory.  The idea is that you can just take your $ROOT/gamename/ directory, and that is all of everything you need.  To get your data directory is easy, since it should be at the root where your files are.

It looks like a $ROOT/data directory was added instead.  Was it tested after being added?

We should have added a function into the skellington to get the data dir.  Something like the old lib/data.py file did.

This works ok for me in a $ROOT/yourgamename/util.py file, if your data is in $ROOT/yourgamename/data.
_DATA_DIR = None
def data_dir(*args):
    ' data_dir() returns the path to where the data is stored.  Use it like os.path.join() where the first part of the path is the data dir.'
    global _DATA_DIR
    if _DATA_DIR is None:
        _DATA_DIR = os.path.join('yourgamename', 'data')
        if not os.path.exists(_DATA_DIR):
            _DATA_DIR = os.path.join(os.path.split(__file__)[0], 'data')
    return os.path.join(*([_DATA_DIR] + list(args)))


I'm not sure about the setup.py approach to putting in data with package_data.  Since it only works for distutils/pip/setuptool based installations.

However, there should definitely be a data_dir() type function added to the skellington.


cya!