Reducing my own voting bias

scav on 2012/09/18 20:17
— edited on 2012/09/19 09:52

This year, before I even look at any of the entries, I want to establish a consistent voting scheme for myself, to reduce the effects of the order in which I look at entries, and how I am feeling at the time.

I am not suggesting anyone else should use this scheme or even have one, but I know I can be a little inconsistent, and I want to be fair.

Approximately:

Fun
==
1 : No fun at all, or actually irritating
2 : It was a bit of a chore and I couldn't be bothered putting in the effort to finish it.
3 : I liked playing it once but probably wouldn't again
4 : I played it a few times for fun
5 : means I fully intend to play the game again occasionally even after the competition.

A game I couldn't get working because of pathological dependencies or wrong-case filenames will get a 2. Them's the breaks.
edited: not a 1, because mauve correctly pointed out that was unreasonable.

Innovation
======
1 : A blatant copy of a game I am already aware of
2 : A near copy of a game I am already aware of, but with distinguishing features.
3 : A game in a well-worn genre but mostly unlike any specific game I can think of
4 : A game with an unusual genre or with novel features I haven't played with before
5 : An entirely new gaming experience for me

Production
======
1 : DNW. If it doesn't even run, testing FAIL
2 : Rough edges or bugs that reduce my enjoyment of the game
3 : Competent but not well polished. Homely but not ugly.
4 : Aesthetically pleasing in at least one way (music, artwork, good writing, clean code)
5 : A delight to the senses and the sensibilities.

In all of these, 5 is hard but not unattainable. I wouldn't give my entry a 5 for any of them :(

P.S. You can if you like. Who am I to tell you how to vote? ;)

(log in to comment)

Comments

mauve on 2012/09/18 21:30:

That's interesting, thanks for sharing. I think that is close to how I vote, but I don't have a problem with judging subjectively.

Innovation is a bit difficult to rate: I don't feel I can always separate it from the fun score. Innovative ideas that aren't fun aren't valuable, while I want to reward an entry that brings together old ideas - selected appropriately and judiciously combined, in a novel combination - to make a fun entry.

I give only three or four 5-point ratings in an average Pyweek, as I want to be able to distinguish the games that I think are truly exceptional.

I wouldn't penalise people for using complicated dependencies though. Actually I won't penalise people at all - I don't think it's appropriate to shit on people who spent a week writing me a game just because they didn't tick some arbitrary box.

scav on 2012/09/19 09:51:

Yeah, it would be mean to deliberately downvote someone's hard work for the wrong reasons. I suppose using unusual dependencies that are difficult to get working on more than one platform is more a production and testing issue, so it doesn't belong in the "fun" score. I won't be giving anyone a 1 for fun or any other category unless I can't honestly give them more.

But you can't really say a game was fun if you can't run it, so.. I dunno. A 2 is maybe not too harsh if applied consistently. It'll be a DNW anyway in that case.

allefant on 2012/09/19 10:01:

I think I generally tend to rate one higher than that. Let's see...

For fun, I give 5 to games I like. 4 to games I think are above average. 3 to games that I think did OK for a pyweek. 2 if I found playing it annoying. 1 only if it has no discernible gameplay at all.

Innovation I give 5 if it's a new idea to me (I don't play many games so that's easy). 4 if it's a well executed take on an idea. 3 if the gameplay itself isn't all that new but there's some interesting idea. 2 if it's just a clone of some game but with some slight changes. 1 for a 1:1 clone.

For production I think I rate identical to scav.

Cosmologicon on 2012/09/19 14:35:

Your ratings don't count if you mark it DNW. I usually mark DNWs 1/1/1 but it shouldn't matter.