Page 1 of 2

Site change - 12/12/07 - Rating system update

Posted: Wed Dec 12, 2007 6:04 pm
by anonymous_object
The rating system has gone through a restructuring. Images are now rated on a 1-10 scale instead of the previous 1-5. This will allow for more accurate ratings across the entire database. Also, each image now has a Bayesian average rating calculated in addition to the original mathematical mean. As of now, both appear in the rating section of an image.

When you rate an image, you still rate it the same way. The Bayesian average is calculated automatically when a new rating is submitted for an image. A script will also run every night or every other night to resync all ratings and recalculate the Bayesian ratings using some global variables.

The main reason for doing was to make the Top Images page more accurate. There were some images that were rated with just 4 ratings of all 5s and caused them to be rated higher than another image with 15 5s and 1 rating of 2. The Bayesian average in a sense weights the image ratings based on how many ratings it has and what the average number of ratings any image in the database has.

I would like to focus more on weighting images using methods like this. Due to how the Bayesian algorithm works, the more ratings in the database, the more accurate the rating for an image can be calculated.
So, everyone please remember to rate images! This will make the whole rating system work better. I really don't want to see images like this one rated so high =P The more people we have rating images, the better the system will become.

I plan on working with the favorites system to come up some ways on weighting images as well. More in the future ^^

Posted: Wed Dec 12, 2007 8:03 pm
by A.Dantes
I would suggest writing clear definitions for the rating scale.

I once had a debate with one of my friends about what rating on aniDB to assign a series we had both just finished. Our had essentially the exact same opinion of the series, namely that it was enjoyable once but was not particularly original and we would never choose to watch it a second time on our own. On the other hand, it wasn't so awful that we would mind watching it again if a friend wanted to see it. Series either one of us feel like this about typically have an average score of 7, so that was the vote I cast. He, however, decided that on a scale of 1-10 a show that is merely average should only be a 5 and so that was what he voted.

Exact same opinion, not a small degree of variation.

Alternatively, I think more people would actually vote with a far simpler system. On a scale of 1-10, I have to sit and think about exactly where I believe this picture should fall. If the options were instead changed to:
  • Thumbs Down
  • Neutral
  • Thumbs Up
  • Favorite
I would be able decide pretty much instantly which category the image belonged in and would vote more often.

Posted: Wed Dec 12, 2007 9:29 pm
by Myu-chan
Good Job anonymous_object-san. ^^
I really was confuse about the Top Images.

Posted: Wed Dec 12, 2007 9:29 pm
by anonymous_object
A.Dantes wrote:Alternatively, I think more people would actually vote with a far simpler system. On a scale of 1-10, I have to sit and think about exactly where I believe this picture should fall. If the options were instead changed to:
  • Thumbs Down
  • Neutral
  • Thumbs Up
  • Favorite
I would be able decide pretty much instantly which category the image belonged in and would vote more often.
No thanks... there's not enough variance in a rating system with only 4 possible ratings. That's why I moved from a 1-5 system to 1-10. Anyway, rating images is not hard to do with the current system. The thing is, up until now, there hasn't been much incentive for users to rate images as it didn't really amount to anything. That is why I have created the Top Images page. Users' votes have an actual effect on this page. Also, I have plans to extend the ratings further with a compatibility match system and possibly make image suggestions based on a comparison of one user's ratings to another's.

Posted: Thu Dec 13, 2007 7:42 am
by Fhant
It's great to see that every person who rated 5 on a pic suddenly gets a 10 rating. It's even better with the voting, now you can give a 5 and not a 3 or 2 to say if you find the pic not THAT bad and not THAT good.

Well done Anony, cookie and some resting time for you!!

Posted: Thu Dec 13, 2007 9:16 am
by Ran
Still working on the code? The top Shana image is like a 27.4/10 right now. :lol:

I like the new rating system though and it should definitely help with giving good images the credit they deserve. Kudos! :D

Posted: Thu Dec 13, 2007 12:58 pm
by anonymous_object
Ran wrote:Still working on the code? The top Shana image is like a 27.4/10 right now. :lol:
Heh... seems some one found a little part of code I forgot to update when I changed from 1-5 to 1-10. Well... I've fixed the problem and taken care of the person that exploited it ;)

Posted: Thu Dec 13, 2007 1:38 pm
by Ran

Posted: Thu Dec 13, 2007 1:44 pm
by Ran
Image

Minor script adjustment needed here. Just needs that break between the Rating and Favorite lines. Strangely enough, once the image is rated by me, the Favorite line goes to where it's supposed to be.

I'm sure you're still working on the script, so if I start to be annoying, let me know.

Posted: Thu Dec 13, 2007 2:10 pm
by anonymous_object
Should be fixed for good now. I'm resyncing the ratings... it'll take a few minutes.

Posted: Thu Dec 13, 2007 2:15 pm
by anonymous_object
Also fixed the line break.

Posted: Thu Dec 13, 2007 3:29 pm
by Ran
anonymous_object wrote:Also fixed the line break.
Now we can't add pics to favorites. It gives an error.

Posted: Thu Dec 13, 2007 3:31 pm
by anonymous_object
Ran wrote:
anonymous_object wrote:Also fixed the line break.
Now we can't add pics to favorites. It gives an error.
>_<
Fixed.

Posted: Thu Dec 13, 2007 3:50 pm
by Ran
anonymous_object wrote:>_<
Fixed.
The fun never ends. :lol:

Posted: Fri Dec 14, 2007 3:08 pm
by A.Dantes
I didn't think my suggestion would fly, given previous responses to similar questions on this forum. I still reserve the right to believe that a simpler system ( like used on Digg or DeviantArt ) would work better, though. It could be argued ( which doesn't necessarily mean that I believe it is ) that sorting by image views is currently a better measurement than sorting by rating or Bayesian rating.

A request for some form of written guidelines for what each rating should mean still stands. Past experience indicates that ten point scales tend towards 7 as "Average, nothing special" in the absence of guidelines, for some reason.


And for a random bug report,

Code: Select all

Warning: array_merge() [function.array-merge]: Argument #1 is not an array in /var/www/e-shuushuu.net/includes/functions_main.php on line 374
appears when viewing http://e-shuushuu.net/image.php?mode=view&image_id=52