Next Pyweek Guidelines

This pyweek, I've had attempted to run several entries that don't work without a lot of legwork on my end to hunt down the correct 3rd party library, compile it, etc.  I was guilty of the same thing in the past when I created a game that used Box2d and some very flaky python bindings.  I got a lot of comments from judges that they were dissapointed because it was a pain to run.

I know pyweek is very relaxed in respect to restrictions on external libraries, python versions, etc, but maybe it is time to set some guidelines on what is acceptable to expect the judges to do to to play the entries.

For some people, it is difficult to get support for some libraries to work on their system.  It may take a person 15-30-60 minutes to get an external library downloaded, compiled and finally installed, only to play a pyweek submission that lasts 10 minutes.

My suggestion is that we break down general parts of games (graphics, networking, physics, etc), and make a list of "acceptable 3rd party libraries", and encourage people to use those.  It would be a starting point for the newcomers for python/pyweek and make judging less of a hassle if all the games had a common set of libraries.

I'll start a list, all of these are just off the top of my head and just here to serve as a catalyst to get a discussion going.

    2.6 or 3.2 (both ok, i guess)



    PyBox2d or PyMunk (just choose one.  discuss)

P.S. could me make it a requirement to use os.path?  Nothing is more annoying that correcting a bunch of "C:\Documents and Settings\coolman23\python\pyweek\game\art\logo.jpg" statements.

P.P.S why are people using python 3.2 for pyweek?  The standard library has less to offer and the syntax in general isn't much different from python 2.6.

(log in to comment)


Some of this (eg using os.path) is covered in a post where I was trying to compile all the tips like this.

I'm opposed to having "acceptable" libraries or python versions. I would prefer a list of "common" libraries and versions, so people understand that the judges might have a problem. One thing I'd like is for someone to get together a big list of libraries that get used and give me the command lines to install them on a fresh Ubuntu install. Something like this:

sudo apt-get install python-pip libavbin0
sudo easy_install pygame pyglet cocos2d twisted

I guess that won't work for Windows users, but they're probably used to things being a pain to install.
Don't use os.path, use pkg_resources, as I covered in a post where I tried to introduce people to distribute :)

I believe buildout solves the sort of problem you're alluding to. It can build "configure, make, make install"-style C dependencies as well as building C Python modules and installing pure Python packages.

> P.P.S why are people using python 3.2 for pyweek?

I'll bite :-) As an occasional Python developer, I just grabbed the latest and greatest of Python and PyGame for Win7 32bit. They both worked first time so I stuck with them.

Installing the stuff and dependencies on Windows is not too bad (if you can find the right web page). Linux is much easier as libs are packaged more consistently yet Users not keen to upgrade? Odd that one :-)
I've noticed that of all the Python versions, our game ran significantly faster on Python 3.x. I don't even use for loops/range so I don't know what made the huge difference. Furthermore, the joystick is busted on the 2.x versions of PyGame.

I usually do testing on everything from 2.5 to 3.2 (I'm pretty meticulous about writing code that works on both) and try to avoid any and all libraries more exotic than PyGame like the plague. Not just for the sake of judges, but also to make it easier to distribute my game to non tech-savvy friends and family. I walked my mom through installing Python and PyGame over the phone but I think that's the limit before one of us would have given up. If NumPy or anything else was involved, it would have tipped into the "I'll just have to wait until the next time you visit for Christmas".

As far as os.path/os.sep goes, I use /'s throughout my code for the directory separator and wrap all methods that touch files with my own functions that replace them with the proper directory separator using os.sep and convert the paths to lowercase. Since I do my development on Windows, it pretty much guarantees I don't miss any. I also make sure no resource files are checked in by any team members that use uppercase letters.

I don't really mind installing obscure libraries to play the games as long as it's available for all platforms and how to acquire/install them is clearly described in the README.

However, if someone does leave a hardcoded path (e.g. C:\documents and settings\coolman23\etc) that's a DNW in my book.

If you support both 2.5 and 3.x, you can't handle exceptions properly - 2.5 doesn't support as, 3.x doesn't support comma. Sounds like a fool's errand to me. 2.6 and particularly 2.7 is where 3.x forward-compatibility starts to arrive.

Using forward-slashes and converting to os.sep sounds sensible. Afaict this is roughly what pkg_resources does (as well as making paths relative to package directories, loading resources from compressed eggs and so on).

I think your premise is flawed. The reason many of us choose Python is the availability of libraries.

Or put another way, if I created a list of things you're allowed to use, it would probably be completely unsuitable for you. I reckon the rules should limit you to these choices:

Python: CPython 2.7 or PyPy (Both Ok, I guess)
Graphics/Sound: Kivy, PyOpenGL, gst-python
Networking: gevent, tornado
Physics: retrogamelib or PyODE (Just choose one. Discuss.)
The entries page already lists "Python (typically 2.5+). PyGame, pyglet or PyOpenGL. I just think we need to update this to include some more common libraries. We should be bundling anything that isn't in that list, or providing reliable install instructions.

I don't mind if people only support 3.X and not 2.X, but something like what Cosmologicon suggested would really help.

I only had pygame installed for 2, and it took me ages to get it working on 3, because it wasn't in my repositories. I had to install a bunch of dev packages to be able to install it through pip.

What I don't like is when the game fails to work because I have a different *minor* python version. I think we should set a minimum version people need to support.
I think the most important thing is to write in the readme what dependencies (library and version!) an entry needs. Because I encountered some games that only run with 2.7 exclusively, which is annoying if the readme does not mention that! Also just saying it used pyglet for example isn't enough, because one games use, others 1.1.4 and others yet another unknown version. Instead of trying all that versions I just could say DNW after trying one pyglet version if it is not mentioned in the readme.
In the spirit of compatibility, CPython 2.7 should be the preferred version of python, with python 3.x acceptable in certain special circumstances.  In the Python universe, there is a myriad of choices to choose from in terms of libraries and interpreters, but that doesn't mean we should use them all in PyWeek.

Lets just consider that we had about 50 entries/judges, but on average, only 12 judges per entry.  Some entries had as little as 4 judges.  To make the event more enjoyable and fair, we should make it easier for judges to actually play our submissions.  IMO, the best way to accomplish this is [slightly] more strict guidelines (and one new rule...see below).

I think a few people are not understanding my intentions, so I will attempt to clarify.  I don't want to make it against the rules to use exotic dependancies.  I want to encourage established libraries so it is easier for everyone to play and judge.  There have been very interesting and fun games in PyWeek that use exotic libraries, and I think that is great, but there have been instances where the entry could have used a common library to accomplish the same thing that would not have required installing an obscure dependancy.

Since python and its its related software is cross platform, and the participants develop on a variety of platforms, I think we should set guidelines that help PyWeek entries work smoothly across different platforms.

As far as rules go, it will certainly meet some resistance, but requiring all submissions to use either skellington or distribute would cut down on the DWN's and headaches for everyone.  What are your opinions about that?

The README should contain the libraries used along with their version.  This will be a big step forward.  As you mentioned, there are examples of entries that list their libraries in the readme, but not the version and it makes a difference.  (Thanks for being tactful, that was my mistake this pyweek!)  This guideline should be mentioned somewhere on the website.

Do you mean joystick support is broken in Python 2.x?, because you wrote PyGame 2.x.  If that is what you mean, I'm interested in seeing where it is broken, as I've used it before and never had any problems.

Not all linux distributions are easy to upgrade.  There are people that don't use deb/rpm package management systems and upgrades are not always straightforward.  Also, "latest and greatest" hardly ever means the most appropriate.  Just turn on the radio and listen to the most popular music and you'll understand.

@Cosmologicon, mauve
Since we all have brought this up, maybe we can draft some guidelines based on these posts and get richard on board with us to post it somewhere prominent on the webpage.  If the guidelines we have been discussing are buried in the discussion page, then I don't see it making a big impact next time around.  These guidelines need to be in a more visible part of the PyWeek website, like on the front page, right under "Rules".

Your post about distribute is very informative and makes using it seem easy.  My [personal] concern would simply be that I don't necessarily want to 'install' every game that I judge, or to manage virtual environments with them.  If I am mistaken here, correct me please.

I think your premise is flawed. The reason many of us choose Python is the availability of libraries.
I know what you are getting at, and I agree with you: Python has a huge amount of ready to use libraries.  However, in the context of PyWeek and in consideration the judges' time, I think it would be beneficial to implicitly discourage (not forbid) the use of the more exotic libraries for PyWeek.

The list of things that I put together was just to serve as a starting point that seem to me to be common and understood in the python/pygame community.  You chose some pretty obscure additions for your own list, but if you can justify them as being suitable for a large audience to install on their system, then it would be compatible with the purpose of my original post: to make it less of a chore to judge the games.

"Python (typically 2.5+). PyGame, pyglet or PyOpenGL" should probably read "Python (2.7), Pygame, or pyglet".  It needs to be updated.
I'd be happy to help write/edit some guidelines. There are surveys somewhere that cover the dependencies people are using, which might be useful.

Btw the existing guidelines are part of this page:

I'd really like for this to be a wiki, and incorporate stuff from and other threads (such as mauve's distribution guide), so these things are easier to find. But obviously that's up to Richard as he'd be hosting it :)
bitcraft: Lets just consider that we had about 50 entries/judges, but on average, only 12 judges per entry. Some entries had as little as 4 judges.  To make the event more enjoyable and fair, we should make it easier for judges to actually play our submissions.  IMO, the best way to accomplish this is [slightly] more strict guidelines

Having scrolled through a few peoples ratings I've noticed somebody repeatedly commented 'it ran' and pulled down the ratings of some games. This is obviously unfair and I think this may or may not be tactical voting by one of the teams but I must say it's not what voting is about. My team only received 2 low ratings, 1 didn't leave a comment and 1 said it ran. Clearly they were useless votes and potentially cost us the week win. Not to discredit any other teams, it's just blatently wrong.

Furthermore, I don't think that the best game should or even could be decided by just:
Fun, Production and Innovation.

I played several games which i thought were great, I gave thorough feedback, but I didn't feel were particularly fun or innovative, but were still great games.

My reason in saying that is that teams who enter this competition may feel they want to create a challenging game that isn't necessarily 'fun'. Like space invaders, tetris, pong... most of the great arcades are purely innovative and not really fun but just challenging. A game doesn't have to be 'fun' to be an amazing game.

As for the production category the prompt is (graphics, sound, polish). Pong has 2 shapes and 1 sound effect. Who anybody argue against the greatness of that game?

Innovation also isn't the best way to rate a game. The most popular game on the app store at the moment I believe is draw something. That's not innovative, that's just pictionary for the social network era.

Continuing the discussion above, I do feel that judging is completely flawed.

Why are so few people voting? We need to find out the reason FIRST and then we can work out a solution.
@bitcraft - concerning PyGame 2.x Joystick
Sorry, I should have been more specific. The joystick module technically works, however, each time you poll a status of a button/axis/hat it prints out some debug text on the command line. These printf statements are in the pygame source code itself. With 5 or 6 buttons * 60 frames per second, it makes your own debug print statements impossible to see and can actually slow game execution down. The printf is surrounded by #ifdef DEBUG, but it seems the windows version of pygame available for download must have been built with this flag set even though pygame.version.ver says 1.9.1release. *shrug* So whenever I plug in a joystick, I switch over to 3.x.
(I'm a bit confused by what you meant by "because you wrote PyGame 2.x") 

@mauve - You can omit setting the exception value and the bare bones "try: ... except: ..." syntax works on both. If you're using a try/except, you generally know what sort of exception you're about to get anyway. However this can usually be danced around altogether. I don't think I used any try blocks in my entry. 

@Luke - I'll admit that I generally don't vote much. I'm fairly competitive with PyWeek and so I worry that I'd either rate too low or worry too much that I'd be rating too low and overcompensate and rate too high.

One solution is to open voting to the general public, but it's pretty easy to forsee a problem of people reaching into their social network to get friends to vote on their game without seeing other entries to calibrate fairly. Or another approach would be to designate a small panel of judges to go through all games. This would solve a few problems including being able to adapt the criteria when fun/innovation/production isn't a fair representation and (to a small degree) the original topic of not being able to play the games due to unusual dependencies. No entry got 100% DNW so chances are someone on the panel would be able to figure it out. 
Or maybe (although this might be a bit more complicated to arrange, and less helpful when people are looking for game improvements), whenever you test a game then instead of ranking each section from one to five you say where that game ranks compared to others for each of gameplay, production and innovation.

As an example, at the moment you say:

Game 1 - G: 5 - P: 2 - I: 5
Game 2 - G: 3 - P: 4 - I: 2
Game 3 - G: 4 - P: 5 - I: 4

Instead you would do:


Game 1
Game 2
Game 3


Game 3
Game 2
Game 1


Game 1
Game 3
Game 2


This would stop people from being able to vote all 1's (although they could still pull down other entries that they think are strong by just putting them at the bottom of their list). This may also be more complicated.

I agree that making a simple mean of Fun/Innovation/Production isn't a good way to rate a game.
I think that an "Overall" vote should be given as well.

I like the ranking idea. When you rate a game, instead of giving a vote you would just give it a ranking compared to the ones you have already rated.

This is my first Pyweek and I didn't vote because I didn't finish, but I've been thinking about how I would rate all the games and my idea was that I would have liked to give an overall vote too, which would equate roughly to the following question "If you had a certain sum to spend in a game store and all these games would cost the same, which one would you pick first?"
(i.e. which one has the highest overall value for you - which one you like the most).

Also, making a ranking by saying "this game is better than this one on Production" is easier than to say "this game is worth 4 while this one is 3" (and what about those who would deserve 3.5 or 3.7?)

After everyone has given the complete rankings of all the games he has tried, there are probably various algorithms to combine these single-judge rankings in a global ranking (e.g. like the instant runoff we use for theme selection).
There are an infinite number of possible voting systems. As far as I see the current system has proven itself to lead to good rankings. I don't think there has been significant disagreement over the winners in any year. It generally leads to some useful feedback too. I'm pretty satisfied with it.

I try to rate all entries. I usually manage to do it or come close. This time I got 100%, although I handed out a few DNWs too. I felt that the time for judging is a bit too short. How about giving 14 days instead of 13? This would add another Sunday, so if you can only spend time on it during the weekend it would give you 50% more time.
What I'd like is a new rating page that first asks "Did the game work for you?" and if not you enter a mini support forum area where you can post a stack trace or problem details, which can be seen and resolved by the entrant immediately. This would be more convenient than looking at diary entries, and would separate it from the process of rating working entries, make it easier for judges to get a game working, and ultimately decrease the number of DNWs.
@mauve - I would gladly accept code contributions to that effect!
Also, thanks to everyone else contributing to this discussion. I've been trying to keep up with it but sadly at the moment I have too much else on my plate. I will revisit it soon.
I will if I can find the time, richard. May I scrape a small test dataset (eg. latest Pyweek) from the site?
The pyweek code currently has absolutely no testing setup whatsoever (either in test data or unit tests.) That really needs to be addressed before contributions can happen. A snapshot of the site wouldn't really cut it (nor be entirely appropriate in my mind.) Please consider contacting me via email if you'd like to contribute. Note that I'm keynoting at PyCon APAC next week so I'm not likely to be responsive until after that :-)
This idea deserves some more attention.  Does this mean that the 24-hour upload period will officially become a bug-fix period as well?
@bitcraft it already is! I have always been fully behind anyone uploading a new release during the upload period if they find a game-killing bug they can fix in time. I'm just not behind anyone who doesn't manage any upload before the time runs out :-)
ok, never knew that.
We could make it all a bit simpler and make the next one like PyGame.Draw 

I realise this is merely whining on my part, but I can't help but think that instead of limiting libraries or Python versions, it would be lots better (for pyweek and the whole Python world) if it was easier to bundle PyWeek games up into executables for Windows, Linux and Mac.

That way, Pyweek entrants could choose whatever versions of Python they liked, and use any libraries at all, and all games would "just run" for all judges, without requiring any dependency installation.

I realise that in the big picture, this is eminently do-able, but the (percieved?) difficulty of it puts people off in practice, so it rarely ever happens. I think this hampers Python adoption in many ways. It should be easy to bundle up an executable and send it to non-developers (friends, clients, customers.)
Also: I wonder if a 'quick fix' for creating the 'mini-support forum' idea would be to simply create a forum thread for each entry, under which bugs, support questions and answers could be posted.
@tartley interesting idea - so while entrants can currently post a "diary" entry against their entry and it appears in the "forum" I think what you're proposing is that other entrants are also able to post a message against an entry...
Yep, that sound right. There seems great value in the 'development diary' aspect though, I wouldn't want to lose or dilute that..