Site change - 12/12/07 - Rating system update

Important stuff goes here?
User avatar
anonymous_object
Site Admin
Posts: 1584
Joined: Tue Jan 24, 2006 2:04 pm
Location: Good old US of A
Contact:

Site change - 12/12/07 - Rating system update

Post by anonymous_object »

The rating system has gone through a restructuring. Images are now rated on a 1-10 scale instead of the previous 1-5. This will allow for more accurate ratings across the entire database. Also, each image now has a Bayesian average rating calculated in addition to the original mathematical mean. As of now, both appear in the rating section of an image.

When you rate an image, you still rate it the same way. The Bayesian average is calculated automatically when a new rating is submitted for an image. A script will also run every night or every other night to resync all ratings and recalculate the Bayesian ratings using some global variables.

The main reason for doing was to make the Top Images page more accurate. There were some images that were rated with just 4 ratings of all 5s and caused them to be rated higher than another image with 15 5s and 1 rating of 2. The Bayesian average in a sense weights the image ratings based on how many ratings it has and what the average number of ratings any image in the database has.

I would like to focus more on weighting images using methods like this. Due to how the Bayesian algorithm works, the more ratings in the database, the more accurate the rating for an image can be calculated.
So, everyone please remember to rate images! This will make the whole rating system work better. I really don't want to see images like this one rated so high =P The more people we have rating images, the better the system will become.

I plan on working with the favorites system to come up some ways on weighting images as well. More in the future ^^
e-shuushuu!
Image
User avatar
A.Dantes
Posts: 172
Joined: Wed Nov 14, 2007 11:13 pm

Post by A.Dantes »

I would suggest writing clear definitions for the rating scale.

I once had a debate with one of my friends about what rating on aniDB to assign a series we had both just finished. Our had essentially the exact same opinion of the series, namely that it was enjoyable once but was not particularly original and we would never choose to watch it a second time on our own. On the other hand, it wasn't so awful that we would mind watching it again if a friend wanted to see it. Series either one of us feel like this about typically have an average score of 7, so that was the vote I cast. He, however, decided that on a scale of 1-10 a show that is merely average should only be a 5 and so that was what he voted.

Exact same opinion, not a small degree of variation.

Alternatively, I think more people would actually vote with a far simpler system. On a scale of 1-10, I have to sit and think about exactly where I believe this picture should fall. If the options were instead changed to:
  • Thumbs Down
  • Neutral
  • Thumbs Up
  • Favorite
I would be able decide pretty much instantly which category the image belonged in and would vote more often.
User avatar
Myu-chan
Posts: 1925
Joined: Sat Dec 30, 2006 2:33 am
Location: Try to find me

Post by Myu-chan »

Good Job anonymous_object-san. ^^
I really was confuse about the Top Images.
User avatar
anonymous_object
Site Admin
Posts: 1584
Joined: Tue Jan 24, 2006 2:04 pm
Location: Good old US of A
Contact:

Post by anonymous_object »

A.Dantes wrote:Alternatively, I think more people would actually vote with a far simpler system. On a scale of 1-10, I have to sit and think about exactly where I believe this picture should fall. If the options were instead changed to:
  • Thumbs Down
  • Neutral
  • Thumbs Up
  • Favorite
I would be able decide pretty much instantly which category the image belonged in and would vote more often.
No thanks... there's not enough variance in a rating system with only 4 possible ratings. That's why I moved from a 1-5 system to 1-10. Anyway, rating images is not hard to do with the current system. The thing is, up until now, there hasn't been much incentive for users to rate images as it didn't really amount to anything. That is why I have created the Top Images page. Users' votes have an actual effect on this page. Also, I have plans to extend the ratings further with a compatibility match system and possibly make image suggestions based on a comparison of one user's ratings to another's.
e-shuushuu!
Image
User avatar
Fhant
Posts: 2108
Joined: Wed Sep 05, 2007 2:31 am
Location: Netherlands

Post by Fhant »

It's great to see that every person who rated 5 on a pic suddenly gets a 10 rating. It's even better with the voting, now you can give a 5 and not a 3 or 2 to say if you find the pic not THAT bad and not THAT good.

Well done Anony, cookie and some resting time for you!!
[Signature space. $10]
User avatar
Ran
Posts: 2663
Joined: Thu May 04, 2006 11:57 am
Location: Canton, Ga
Contact:

Post by Ran »

Still working on the code? The top Shana image is like a 27.4/10 right now. :lol:

I like the new rating system though and it should definitely help with giving good images the credit they deserve. Kudos! :D
Image
User avatar
anonymous_object
Site Admin
Posts: 1584
Joined: Tue Jan 24, 2006 2:04 pm
Location: Good old US of A
Contact:

Post by anonymous_object »

Ran wrote:Still working on the code? The top Shana image is like a 27.4/10 right now. :lol:
Heh... seems some one found a little part of code I forgot to update when I changed from 1-5 to 1-10. Well... I've fixed the problem and taken care of the person that exploited it ;)
e-shuushuu!
Image
User avatar
Ran
Posts: 2663
Joined: Thu May 04, 2006 11:57 am
Location: Canton, Ga
Contact:

Post by Ran »

Image
User avatar
Ran
Posts: 2663
Joined: Thu May 04, 2006 11:57 am
Location: Canton, Ga
Contact:

Post by Ran »

Image

Minor script adjustment needed here. Just needs that break between the Rating and Favorite lines. Strangely enough, once the image is rated by me, the Favorite line goes to where it's supposed to be.

I'm sure you're still working on the script, so if I start to be annoying, let me know.
Image
User avatar
anonymous_object
Site Admin
Posts: 1584
Joined: Tue Jan 24, 2006 2:04 pm
Location: Good old US of A
Contact:

Post by anonymous_object »

Should be fixed for good now. I'm resyncing the ratings... it'll take a few minutes.
e-shuushuu!
Image
User avatar
anonymous_object
Site Admin
Posts: 1584
Joined: Tue Jan 24, 2006 2:04 pm
Location: Good old US of A
Contact:

Post by anonymous_object »

Also fixed the line break.
e-shuushuu!
Image
User avatar
Ran
Posts: 2663
Joined: Thu May 04, 2006 11:57 am
Location: Canton, Ga
Contact:

Post by Ran »

anonymous_object wrote:Also fixed the line break.
Now we can't add pics to favorites. It gives an error.
Image
User avatar
anonymous_object
Site Admin
Posts: 1584
Joined: Tue Jan 24, 2006 2:04 pm
Location: Good old US of A
Contact:

Post by anonymous_object »

Ran wrote:
anonymous_object wrote:Also fixed the line break.
Now we can't add pics to favorites. It gives an error.
>_<
Fixed.
e-shuushuu!
Image
User avatar
Ran
Posts: 2663
Joined: Thu May 04, 2006 11:57 am
Location: Canton, Ga
Contact:

Post by Ran »

anonymous_object wrote:>_<
Fixed.
The fun never ends. :lol:
Image
User avatar
A.Dantes
Posts: 172
Joined: Wed Nov 14, 2007 11:13 pm

Post by A.Dantes »

I didn't think my suggestion would fly, given previous responses to similar questions on this forum. I still reserve the right to believe that a simpler system ( like used on Digg or DeviantArt ) would work better, though. It could be argued ( which doesn't necessarily mean that I believe it is ) that sorting by image views is currently a better measurement than sorting by rating or Bayesian rating.

A request for some form of written guidelines for what each rating should mean still stands. Past experience indicates that ten point scales tend towards 7 as "Average, nothing special" in the absence of guidelines, for some reason.


And for a random bug report,

Code: Select all

Warning: array_merge() [function.array-merge]: Argument #1 is not an array in /var/www/e-shuushuu.net/includes/functions_main.php on line 374
appears when viewing http://e-shuushuu.net/image.php?mode=view&image_id=52
Post Reply