Thanks for feedback; and a counter comment
gummbum on 2018/05/06 01:13
— edited on 2018/05/06 01:13
Thanks to those who took some time to leave helpful feedback. It is much appreciated.
Except for this curmudgeon:
"I was completely confused by this entry, the obvious premade assets, the confusing dialog and
backgroud, the fact that this pretty much relates nothing to the challenge, ill pass on this."
You didn't visit us in IRC. We could have cleared up all this stuff. Or you could have read the docs I took the time to write. Or you could have read the rules and understood that publicly available pre-made assets are permitted. If you passed on it, why did you even rate it? So that you could leave a nasty comment?
To the Pyweek managers: I've noticed nasty comments with accompanying low scores on more than a few games in the past and have not said anything. I wish I'd kept track. I'm starting to think someone may be engaging in sabotage. Maybe you could look into this comment and the person's past activity. Thanks.
(log in to comment)
Comments
IvanTHoffmann on
2018/05/06 06:34:
mauve on
2018/05/06 07:46:
I think we should change the feedback system somehow to discourage the negativity. Maybe the time for anonymous ratings is over. If you can't stand by your comments when your name is alongside them, then you shouldn't be posting them. Or a rating panel, as you suggest.
I will investigate your (1, 1, 1) rating.
DR0ID on
2018/05/06 08:08:
It seems that many had problems running kivy (and maybe not using the right python/kivy version because "from kivy.lang.builder import Builder" => "ImportError: No module named builder" might be related to the import mechanism change from py 2.x to py 3.x ). Some info about the systems you have tested and which python and libs version you used would have been valuable info. Also for the judges that don't get an entry running they should also report their system and the used versions of python and libs.
I remember when I first installed kivy for another entry I had my troubles too, but got it working.
This time I got all games working, although some really needed some minor and easy bugfixes (yours worked out of the box).
As the rules states Rule 9 Target Platform:
All entries must run in Python on the latest available libraries
Maybe it should be emphasized more in future challenges. Maybe the forms should be updated so one has to provide the python version and libs on upload or when marking an entry as DNW (like a bug tracker form to get some info as why does not work to prevent this in future pyweeks)
IIRC there has been some compilations of "do" and "don't" for pyweek entries. Maybe there should be page where those tips are maintained
mauve on
2018/05/06 08:23:
I think we probably have to give some leeway in library versions, both to creators and raters of games. For something like Kivy, which doesn't ship wheels on Mac or Linux, and instead recommends installing a distro package, it's pretty hard to select exact versions. And even if you get that working, you might encounter shader issues because the compilation of GLSL is performed by drivers that differ between vendor and OS.
The thing I really think we need is a "support thread" that can be opened for each game during rating week, where instead of rating that you couldn't get the game to work, you get support about how to make it work.
xmzhang1 on
2018/05/06 08:33:
prake on
2018/05/06 08:39:
mauve on
2018/05/06 09:11:
IvanTHoffmann on
2018/05/06 09:30:
I only tried it on computers running Windows 10, so that's almost definitely part of it. I also assumed that people would run it with the latest version of kivy and figured it wouldn't take long to debug if they wanted to use python 3.
I also went into this knowing that kivy would be a bit difficult to distribute and attempting to compile it was my way to account for that. It was a little funky for me (an error message pops up every time I start the game) but I just click ok and it runs fine. I mentioned it on my pyweek page and figured there's nothing more I can do. Maybe somebody else knows what the problem is or could help to release a better copy. I think the idea of a troubleshooting period is awesome. Everybody contributes and more people can play. I would also be more than happy to stand by my reviews.
Again, i apologize for losing my cool before. This was my first pyweek and I have been really looking forward to joining the community, so this whole fiasco was kind of a bummer.
You guys rock though! thanks a lot
mauve on
2018/05/06 10:40:
mit-mit on
2018/05/07 05:51:
The danger with "named ratings" is that people will potentially be afraid to say what they really think about games (particularly if theres something they honestly didn't like) because they'd fear someone would take it personally: lets face it, people put alot of time and effort into pyweek entries and it can be hard to not take criticism personally. I just think we want to avoid a situation in which people are afraid to say what they really think (or even worse, afraid to say anything at all) because they fear its not "nice enough" ... I'm not sure about others, but one thing I like about pyweek is getting honest feedback on what you have done, learning from mistakes etc. and making better games in the future.
That said, I agree we need to put something in place to avoid unhelpful comments: perhaps scores which are very low (or even very high) would not count towards the total unless there is a minimum word count in the comments? (I dunno, I guess this isn't perfect either). I think we need to remind everyone that pyweek is supposed to be about having fun and being supportive of others in the python/game building community.
Mauve: totally agree about the support thread idea: that would be awesome. It's a bummer when you see people couldn't play your game because they couldn't get the dependancies to install, or misunderstood the requirements, and you find out after two weeks.
Hey IvanTHoffmann: I did notice your game crashed quite often when I ran the windows binary (although it didn't effect my ratings of your game). It seemed to have been happening randomly at the end of a level, and was an unhelpful standard "this program has stopped working" in Windows 7. I'd be happy to chase it up further if you wanted to get to the bottom of the issue.
mauve on
2018/05/07 10:02:
The danger with "named ratings" is that people will potentially be afraid to say what they really think about games (particularly if theres something they honestly didn't like) because they'd fear someone would take it personally: lets face it, people put alot of time and effort into pyweek entries and it can be hard to not take criticism personally.
Exactly, people take criticism personally, and I think receiving very negative comments is off-putting to beginners. Pyweek's niche is in being beginner-friendly. So I want people to think twice before offering criticism that is too blunt, not be granted anonymity to do so.
People will just have to say what they really think without being actively nasty and negative.
We can either do open feedback, or remove peer ratings in favour of a judging panel, but I think the competition has to evolve.
DaFluffyPotato on
2018/05/07 11:31:
One time I found out what happens if you just give all criticism. Basically, the creator of a game specifically asked me to give him as much criticism as possible, so I did. I left a comment with a lot of criticism (I think somewhere around 700 words) which resulted in someone calling me out for it. The person who asked me to do it still defended my comment though. :P
mit-mit on
2018/05/07 12:11:
xmzhang1 on
2018/05/07 22:30:
Hi, DaFluffyPotato, how to do the pixel art,just use mouse to draw them in the painter or photoshop? I am really a noob in drawing, I am not sure one day I have a chance to draw something, is there any classical tutorial for beginners?
Hi, IvanTHoffmann, how to make a 3D game in kivy? I wan to add a 2D control pannel in your game and package it into android, how to do it, I am a beginner, please, thank you very much:)
DaFluffyPotato on
2018/05/08 01:45:
I really don't have much of a recommendation for a tutorial jsyk. I mostly just figured it out from looking at pixel art(and watching Bob Ross helped with a couple things I've drawn).
IvanTHoffmann on
2018/05/08 08:09:
My source code is all there if you want to take a look. It got a bit messy at the end, but i think most of it is alright. There really isn't much information on 3D with kivy. I learned by looking at kivy's 3D example code (python_dir/share/kivy-examples/3Drendering) and by learning about 3D in general. There are plenty of tutorials on GLSL, matrices, modeling (I used blender), and whatever else you would like to accomplish is probably available elsewhere if you're feeling creative and willing to learn. But again, just let me know if you would like to get in touch. I'm still very excited to get involved in the community.
-----
mit-mit, I would love your help. I don't think i'll continue development on this game in particular, but I would like to know the issue in the future. Is there any way that we can get in touch privately? I feel like personal communication would be an incredibly inefficient misuse of these discussions.
-----
Getting back on topic though, I think users should have a text field available at each category to write a brief explanation of why they chose to rate it how they did.
1. This would put emphasis on each category as being separate from one another. When writing my comments, I was very careful in making sure that I rated it based on the description of each category. I didn't want to let my emotions about the game affect how I reviewed it. It gave me a little bit of time to reflect and to make sure that I was rating it fairly. I think this is how it was intended, but isn't always how it's used. It seems as if some people are rating the game overall and then selecting the numbers to reflect that score. In my case, the (1, 1, 1) score that I received was an obvious example of this.
2. If there are any complaints, this would give the admins the ability to follow up on the reviews, deciding whether or not it was genuinely how the person felt. What you write in these fields will be used in your defense. ((1, "didn't like it"), (1, "didn't like it"), (1, "didn't like it")) is much harder to defend than (1, 1, 1). In contrast, ((1, "After 35 minutes of trying to beat the first level, I just wanted to stop playing"), (1, "It had tons of bugs, no music, and the art was just a bunch of stock photos"), (1, "It was literally an exact rip-off of another game")) is a very negative, but overall a pretty solid review and you can't argue with it. Unless it's all made up. It seems unlikely, but in that case it should be pretty obvious.
3. The developer would have much more information on how they could improve. If the person leaving the comment only covers certain aspects in their review, you can only assume as to what each individual rating means. There's valuable feedback that is lost in the numbers. I think people would be more than willing to give some extra information if it was presented to them in an organized format. Not everyone is a writer. I think the assistance would be appreciated.
cauch on
2018/05/08 12:46:
I don't have the perfect answer, but maybe the following can help:
Maybe a lot of those issues can be mitigated with a more visible instruction text for the judge (for example, the first time they access the form, they first have a message "rules for voting").
This message can say, for example:
1) Please follow the instructions given with the package. Read the instruction files and at least the entry description carefully. Ideally, have a quick look at the diaries entries before complaining that something is not working as intended.
2) The admins are able to identify the authors of the vote. Please be fair-play: if requested, the admins will check for abuse
3) Please find arguments for each score you give, and mention those reasons in the comments in a constructive way.
...
I think the support thread is a great idea. I also think that fixing those kind of bugs should still be allowed during voting time, or at least during some period of time. After all, pyweek is targeting beginners who don't have the opportunity/knowledge to predict that the software working on their machine will not work on a machine they cannot test. Someone on the pyweek IRC channel (I've forgotten and I've deleted my logs) pointed me to some tips to avoid typical problems (example: case sensitivity of the media files). This kind of help could be more visible.
About the idea of using the latest version, I agree that it's not always possible (some distributions are providing stable packages which are not always the latest version).
xmzhang1 on
2018/05/08 13:36:
Pygasm on
2018/05/08 19:26:
I wrote that review you mentioned.
I was tired and irritated from poor sleep and the rush of getting my game as perfect as possible, and eventually I went though this thought process
Why was the game vertical? -> Why do I see the same boy from the twins game -> Wait! -> This game was a copy-paste from sources, theres no way that two people could make the same assets?!?! -> Im writing this! Too cheap!
I am all in favor of the removal of anonymity and a system redesign, its too reliant on 2000's internet thinking of "Every man for themselves" and a restricted catagorical rating system.
Tee on
2018/05/08 23:47:
First, I don't think removing anonymity would stop us from getting honest feedback on our games. At the very least, we know that there are a bunch of us here that would continue to provide constructive criticism without anonymity, so at least some honest feedback is sort of guaranteed. Also, the more the reviewers see high-quality feedback in their games, the more likely they will follow the same trend in their next Pyweek. There will always be low-quality comments, but I think if we have some sort of critical mass of high-quality comments, we'll be fine.
I actually like the idea of removing anonymity for another reason as well. I think it might improve the sense of community in Pyweek. Sometimes I see a thoughtful comment in my game and I'd like to engage in a discussion with the person who wrote that particular comment. At this stage, the best I can do is write a post replying to all the anonymous comments. In the same spirit, perhaps even a particularly negative comment could lead to a friendly discussion clarifying the differing viewpoints. Providing constructive criticism is not always easy; I have previously found myself spending time rewriting feedback that wasn't particularly constructive.
Digressing a bit, this actually goes beyond removing anonymity, as the current platform is not great for this sort of post-game discussion. Currently I have to keep checking my diary entry to see if anyone commented. Pyweek is awesome, but sometimes I feel a bit disconnected with the community. I admit that it's my own fault to not participate in IRC/Twitter/Discord -- part of the reason (besides time) is that I like to keep my game under wraps until it's done. However, it's very rewarding to see people engaged in discussion about a game you spend a lot of effort building after it's done, and I wish there were a more solid platform to encourage this kind of thing. In this sense, perhaps removing anonymity along with providing a better platform for discussions would help with this. Of course, this is a more ambitious goal, but I think the reward may be worth it. There are incredibly talented people around here and it's a shame that I only get to interact with you all so briefly. I feel like there's a lot to learn from each other but the community is not connected in that way.
On other topics:
On the idea of a panel of judges: I think we would be stepping in a mine field of issues around that, such as who to pick and whether they are being fair. It generally seems like a non-inclusive direction, which I feel doesn't resonate too well with the atmosphere of Pyweek.
@IvanTHoffman: I agree that ideally the judge should explain why they gave their scores, and a lot of people actually do that. However, my impression is that splitting up the review wouldn't stop a bad reviewer to continue giving a bad review. In the current set-up, reviewers never need to defend themselves, so a bad reviewer won't really care if they write something arbitrary in all fields.
I'm also in favor of some sort of technical support thread. Right now we kind of use the diary entries, but it's not great because someone might not post a diary entry, or they might not check. The implementation could be discussed, but in general it seems like a good idea to me.
paulpaterson on
2018/05/09 03:01:
I looked at some numbers from all the Pyweek's so far ...
The ratings seem remarkably stable over time, including the min and max. There is a gradual drop-off in the number of entries over time but this may be flattening off after Pyweek 20.
I'm not sure if too much extra can be gleaned from this so onto the softer stuff ...
I agree with the idea that there is an opportunity to grow the community (and maybe reverse the drop-off in numbers). Like Tee, I really enjoy the post competition discussion of the games and the current approach of putting this in the diary entries hides that.
Also, when I first took part what I loved was the diary discussion during the competition. It was great to see the progress of others. I can definitely understand when people want to keep aspects under wraps and also that it takes time and energy to post. However, for beginners it is great to see that others are struggling and in the same boat as you. It really helps to keep people moving towards a conclusion.
So I'm +1 for,
1. The game support thread
2. A post competition discussion area with a UI that fosters discussion (with RSS feed / notification or some other way to bring people to it)
3. Maybe try to foster collaboration by including a rating on contribution. Maybe this doesn't count to the overall score but could be a badge / honour system.
4. I'm for non-anonymous rating / comments in the context of a wider community push. I don't think there is an overall problem with the current approach but if we could use it to kick-start deeper post competition discussion then I think its a win.
5. Add easy to use capability to post video walkthrough / trailers to the site. It's not that hard to do it by hand but anything that makes it easier would be a win for me.
paulpaterson on
2018/05/09 03:04:
mit-mit on
2018/05/09 12:09:
+1 from me for the point raised by Tee and paulpaterson regarding having post-competition discussion area with a UI that fosters discussion (i.e. I can get an alert somehow when others have posted/responded to things).
Hey IvanTHoffman: my email address is randomprojectlab@gmail.com: if you send me your address, I can send back some screen shots of the issue etc.
IvanTHoffmann on
2018/05/06 04:19:
I completely agree. I got a comment that just said "yuk" and gave me 1/5. I spent a ton of time on my game and my friend and i found no bugs after hours of play on several different machines. Yet people said that after their "fixes" it crashed after every level. overall 2.9/5. It was complete in every sense of the word. Incomplete games got a higher ranking than I did. There should be a panel of unbiased judges that rank each game based on the stated criteria. When unbiased people ran it as intended, they ranked it very well. I think I could have won.