Additional variables for a potential weighted rating system | Waifudex® | Page 1

exotic plank May 7, 2023, 5:36 PM

#

Could there possibly be a chance for a weighted system to be introduced for rating anime, where the age of the anime and the amount of people voting for the anime are also taken into account?

An issue I've seen with MAL and AniList is recency bias regarding anime made in the last 5 years dominating the top 10-50.

31 of the top 50 on AniList were made in the past 5 years, and a majority of the remaining anime were made in the past 10-15.
With only five anime being made a considerable amount of time before that:
One Piece, Monster, Ashita no Joe, and Legend of the Galactic Heroes

MAL notably had Oshi No Ko at the very top for at least a couple weeks before being bumped down to number 6, which is still way too high IMO.

Same with AOT Final Chapters and Bleach TYBW, which were both released within the past 6 months and are both in the top 5. Plus Final Chapters hasn't even finished airing yet.

I'd like to see a way to curb this, whether by taking into account age and number of people into the weighting process, or some other factor entirely, and I know that it would probably take a very long time for it to take any meaningful effect but its still worth considering IMO.

fossil chasm May 7, 2023, 5:46 PM

#

@exotic plank ive thought about the need for this too after seeing oshi no ko at #1. i will keep it in mind when doing leaderboards for different categories, i havent thought too deeply about what would be the best method to display reviews.

the app will have two things: ratings and reviews. ratings are private and are for training the recommendations, reviews are public posts and those count towards the overall star count of what you review.

the star count/total reviews could be a simple average, but their actual leaderboard position could shift depending on other factors 🤔 , though it may confuse people why a 4 star is higher than a 4.5
i think it's probably best to factor in everything when displaying the star count so it's easy to understand, then the question is what to factor in
i don't want to have private ratings directly affect the star count because that gives a small incentive to not be fully truthful to your taste in the chance that you could boost something for fun, which i dont want as signals for the AI

i'll think about good solutions later because i dont plan on having leaderboards until after beta, or at least not initially during the beta

#

i think steam reviews has something they do that's special, maybe you could look into it and let me know what they're doing. i saw one time that they say they do something with weighting reviews

exotic plank May 7, 2023, 5:48 PM

#

holy shit thats genius

having the reviews be the ratings that count towards overall ratings/averages

#

and using just normal animelist ratings as training data rather than a value that contributes to the average rating of an anime

#

allowing the site to have a more constructive overview of how good an anime is

fossil chasm May 7, 2023, 5:50 PM

#

yes also to be clear this is data we would build from the ground up, where people submit it while using the app. i actually dont plan on using data from other sites anymore and instead just build a good app people use and train on our own data. small side note tho

exotic plank May 7, 2023, 5:51 PM

#

yeah ik that

#

i was using "animelist" to refer to a possible anime list that waifudex could have

fossil chasm May 7, 2023, 5:51 PM

#

m isee

#

but yea we will figure out good solutions. https://steamdb.info/blog/steamdb-rating/ this seems interesting to read too

SteamDB

Introducing Steam Database's new rating algorithm

A short write-up on how the rating algorithm was made, and why it's probably a better estimate of the score.

exotic plank Oct 6, 2023, 11:01 PM

#

This is 5 months late but i also wanted to add that we would have to enforce strict guidelines on review submissions to maintain a standard of accuracy and reliability for the average scores of anime on waifudex, on top of reviews being the only weighted item that counts towards the average score of an anime

#

because many other non-anime sites are infamous for review bombs and low effort copy paste review templates

#

steam and metacritic especially

fossil chasm Oct 7, 2023, 1:38 AM

#

exotic plank This is 5 months late but i also wanted to add that we would have to enforce str...

my view is there would be the 0-10 review score and a 0-100% match score for each entertainment. the match score is different for every user and i think people would value that the most. reviews would be like social posts that also attribute to the total review score.

looking at this again i have a few ideas, i forgot all my old ones but i can read them again once reviews are implemented. i think there should be a delay from when a review is posted to when it's incorporated into the total score to prevent review bombs. another idea is to use the ai embeddings we use for recommendations to get a smarter average for the review score. i think the smart average could start happening once the reviews reach a large number like over 100-1000

exotic plank Oct 8, 2023, 6:50 PM

#

Yeah but I meant more basic guidelines like "The Review has to be this length and tick these checkboxes in terms of the analysis of the anime itself, and the review actually has to have relevance to the anime you're talking about and not be a joke or meme."

#

those types of guidelines are nonexistent on other websites

#

you're just allowed to write whatever you want

#Additional variables for a potential weighted rating system