pouët.net

One thing I didn't see yet:

Give people points to hand out over a compo (e.g. 2 stars per entry in compo -> you can give max N stars to a single entry to avoid excessive self voting).

This way maybe there'll be less noise, as people are forced to choose instead of just give 5 stars to their most liked entries and 0 to the others?

Indeed. This seems like a good system to me. I think it's just natural that if you're a lazy voter you only vote for your friends entries and for stuff that really stands out for you.

The only downside I see is if you stick to the 5 star system star #3 becomes obsolete this way. An even number of stars would probably "feel" better for those who take their time to vote for all entries, because then every vote has a true impact.

Another idea:
At most parties the releases are presented in an order decided by the orgas (from worst to best). What if visitors would only say if the current entry was better than the previous or not (something like selection sort or bubble sort)?

I'm perfectly fine with the current method.

I would prefer avg or that sum based with neutral. Though i'm not weeping if current mostly used one stays and nothing changes :)

Well, thank you KeyJ for confirming my fears that Spiral by Resistance should have beaten me and Gasman.. :P

It's really good to see the changes on the actual data, and seeing it for a party with smaller attendances would be great and help this discussion. If I get a vote on how we all vote in future, I'd choose the +2..-2 idea at the moment, but with spiderj's modification to an even split away from no vote so that you can have slightly more resolution for how you feel about the entry. Call it +2.5..-2.5 ?

Overall we are always going to be relying on the social side to get people to vote, and that means those who are good at the social side, and can get lots of people they know to vote in general, will do better. I don't have a problem with this, in order for it too happen you need to have been contributing to the scene for a while and bringing more people to parties. I want more demos and more people at parties, so I'm happy for them to get a small advantage.

What teo said.

Not a fan of the neutral for "not voted" - every click is an investment of time, if I don't like something, I don't want to invest any time to vote for it. The current system in partymeister would probably be a little more transparent if there was a 0 to 5 score instead of "no stars" to "5 stars". But at the end of the day, that's pretty much the same.

I think, most issues we see with demoscene votes are mainly caused by not enough visitors voting at all or bias introduced by human error, not by the used voting system.

And while it's rather clear that the evoke is biased, the result is actually what the people voted. Of the people who used live voting, probably roughly 20 didn't place their last vote. That's bad IMHO.

KeyJ: Can you please add an option for giving unvoted entries 2 stars (or, equivalently, shift the range to -1..3)? This would come pretty close to the old Breakpoint voting system that (IIRC) had a range from -1 (SUCKS) to 5 (RULES).

I like that we're getting into the discussion of voting - maybe we will have a better + easier to use system in the end.
If we stay with the current system I like the idea of trying out to take the average, but I see the point that this probably could only work on bigger parties and has other flaws.

Looking at the discussion I think we can safely summarize that we have the following goals for a voting system:

• Easy to sign-up
  
• Easy to use at demoparty, especially during live-voting
  
• Doesn't punish non-voted prods (newcomers, unknown groups, compofillers)
  
• Minimises possibilities to harm/skew results through "Fake-Voting"
  

Proposed System:

• Possibility to Sign-up via Scene-ID to make it as easy as possible
  
• Voting Scheme : +1; 0; -1 (similar to pouets ruletz, ok, sucks) (This has the following live-voting advantage: It doesn't matter what comes next, either you like a prod or not, or you don't care / it's ok. You don't have to save up your "5 stars" for a demo which is yet to come, or to adjust your votes constantly)
  
• The default vote is 0
  
• All Prods start with no of registered users = Points -> Everybody starts at the same base -> possibility to go down to 0 or to go up to 200%
  
• Super easy live-voting interface (see mock below)
  
• (Optional) Being able to add a comment at the party, ability to allow it to be shown during prize giving or to be transferred automatically to pouet/demozoo/both (Opt-in!)
  
• (Optional) On-Party comments will be marked on Pouet as being such
  
• (Optional) All votes will made public (anonymiyed) to enable statistics and for transparency, eg. to make patterns visible like 20 people upvoted only one prod and downvoted all others
  

v3nom: Giving negative points is really a terrible idea.

For the sake of curiosity I'd like to see the outcome of concordet-style ranking as well (a > b if both have been voted for and a has more stars than b).

It could have similar issues as averaging for entries with few votes but it would also reduce the impact of people giving their own entry 5 stars and everyone else's 1 star or no vote at all.

Oh, while looking at KeyJs "probably nicer" results, keep in mind that the experiment is a little flawed. It's a bit trying to compare apples and pears. Voters might vote totally differently depending on the voting system used. Throwing data gathered with one voting system through a filter to generate the data from another voting system - unfortunately - just does not work.

If we wanted to pick the best voting system based only on KeyJs experiment we would have to pretend that we changed nothing to the voting system and filter the data in post without telling anyone.

hfr: fair point. It works the same with 0 +1 +2 though, with having +1 the default if you don't like negative votes.

I prefer +0 as default. I don't want to invest any energy into voting something which I don't care enough about to vote (i.e.: your entry is bad enough that I probably don't want this anywhere near the winning entries and I really don't care about it).

It would still be simple and not discriminating non-voted prods (eg because a screenshot is missing, or because nobody can remember the first demo of a 28-entry democompo.

One more data point to consider: With Breakpoint's -1...5 voting system, there was probably only one entry that ever achieved a negative score, and that was because it somehow ended up in the wrong compo.

las: In my opinion the need to press one button to express your opinion is not too much to ask for when getting a fairer overall result.

We had -1..5 at Solskogen for a while. Don't remember negative end results, plus there's something to be said about having an entry so bad that it gets negative results.

v3nom: On a +/-1 voting scale you have no possibility to differ between two good entries.

v3nom: I agree that it's not too much to ask, but the question is whether people (overall) are going to A) understand that this is what they're meant to do, and B) actually do it

it would be better imho to design the vote system around the assumption that people aren't going to press a button for every single entry, because that (in my experience) is the reality.

hfr/farfar: sure, every system has it's pros and cons. For this proposal I had easy in use and 'actively selecting the worst option' != 'Not voting' in mind, with the added Charme of being pouet compatible.
I have to say I like a -1...5 or -1...3 with 0 being the default as well

Thanks KeyJ for compiling the results!

Even if I agree with:


And that's also apply to the current system: are you sure that all voters understand how the current voting system works?
Especially new-comers? People who are used to non-partymeisters demoparty? Drunk people?
With this incertitude, you can't say the results reflected what the visitors wanted, including the care / didn't care factor.
(To have a better idea on how much people understand the current system, we should make... a poll :p)

I like v3nom UI & the -1..3 range proposal (equivalent to current system with 2/5 for non-voted prod).
It's more fair to non-voted prods.
And it's better cognitively (sucks / neutral / why not / good / great) while still giving a fair expressiveness. Especially in a live-voting context.

(The more I think about it, the more I believe there is a bad mix between the current partymeister system & the livevoting. Two important biases in livevoting have been shown in this discussion: the voting-window bias, and the "I give 4 stars (and not 5) to a great prod shown early"-bias.).

Absolutely. That's why I noted that the -2..2 results are just a simulation based on the 1..5-based input data, neglecting the psychological effects of the different scale.

However, regarding average-based ranking with the current 1..5 voting scheme, I think the results would be more or less the same, regardless of whether people know what happens with uncast votes or not. Those who think that an uncast vote is a neutral vote will be represented better, and those wo want to downvote entries with the least possible score can continue to do so; it's just 1 instead of 0 points then.

In general, I'm surprised that so few people see the appeal of a truly neutral vote option. If I missed an entry because I came too late to the compo, had to pee, had a phone call, had a too interesting conversation, or worked on my own prod, I don't want to assign any vote to that entry. I certainly don't want to downvote it, but I don't necessarily want to give it a seemingly neutral "was OK, nothing special" default vote either, because I don't fucking know how good or bad it was. I might have just missed "the product 2" -- or another boring Färjanmaker compofiller.

Sure, you may say now that I can review all the entries later on -- but seriously, who does that? It involves downloading all the entries (except those which screenshots in the graphics compos), relistening to music compos with headphones in a loud hall, maintaining a zoo of emulators for all possible and impossible platforms to run entries on ... that's a lot of work I can't imagine many people doing just because they want their votes to be as fair as possible.

FWIW, there are preferential voting systems where you can rank candidates equally; you don't have to distinguish between entries you don't care about. So if you have a compo with 20 entries, you can rank the (say) five on top that you care about, and then put all the others on split 6th (or split 6th–20th, depending on how you look at it). Or possibly with that troll entry you really hate at the 20th, alone :-P

With AVG scoring system, "Bold" would be the "best" demo on pouet (highest upvotes on all perfect 1.00 avg entries), disregarding all the nonvoters, and of course only if potential trolls resist to give it a pig right now ;)

And finally i know why i don't win compos more often ... the voting system! i knew it all along ^^