The future of ratings

Update: See feedback summary here

A topic that has come up at the summit is what we want to do with ratings going forwards (MB & CB).

We are looking for user feedback, for instance:

  • How do you use ratings (or why do you not)?
  • What if they went away?
  • What if they changed (were only on LB, only on CB, etc)?
  • What if we displayed them less prominently?
  • What if we displayed them more prominently?
  • What if we left them the same?
  • etc

Broad (a few words) as well as detailed feedback welcome. Your feedback will help dictate the future of ratings.

9 Likes

I don’t currently use ratings in MusicBrainz because the way I use MusicBrainz (for editing music that I very often do not own or have listened to) doesn’t lend itself to adding ratings. I think I’d be more likely to add them if they were in ListenBrainz, where I’d be asked to rate only stuff I’ve actually listened to. As such, if there was a preference to hide ratings from MusicBrainz, I’d use it (even if I was using ratings in ListenBrainz at the same time).

I use ratings for books (in Goodreads) and I would like to eventually use ratings for books in BookBrainz as well, probably.

I feel having the option to rate something both in MusicBrainz and in CritiqueBrainz is confusing, and I would expect both to be merged. I’d expect them to live in CritiqueBrainz, but be able to access (display, add, edit) them from the other sites if needed.

I would be against dropping ratings entirely.

17 Likes

Personally I don’t use ratings, because I don’t find the concept of rating my music on a scale particularly useful. If I don’t like something, I would just not put it into my library/collection.

I would also strongly support moving the ratings to CB! That just makes tons of sense. A simple “click the stars” rating could just be a CB review without text.

4 Likes

I mainly use RateYourMusic for rating music rather than MusicBrainz. The ratings alongside the genre and descriptor systems there help with their recommendation algorithms and I’ve found music I’ve enjoyed a lot as a result.

I do sparingly use ratings when I remember. The main reason for not using it constantly is that the rating systems from the two sites mismatch (RYM uses half-star ratings from 0.5 to 5, where MB uses 1-5), so I only tend to rate things I’ve given whole-number ratings to. If I was able to provide those half-star ratings, I would probably sync my ratings between the two sites. Same thing goes for CritiqueBrainz.

I’d be happy merging ratings solely to CB, too. Just makes sense that the database for storing reviews should also include the rating data, rather than it be part of the database housing the entities themselves. I do think if MB is to get a redesign, the ratings should be retrieved from CB and displayed prominently.

4 Likes

for what it’s worth, CritiqueBrainz saw this coming and actually stores ratings from 1-100, but divides and multiplies by 5 when showing it in stars. If there is a consensus that a 1-10 rating is more useful (or 1-5 with 0.5s), then this is something that could be done

2 Likes

I use MB ratings and they help me A LOT.
I use it on artists and release groups mostly.
But also on recordings and only rarely on works.

Artist ratings

Very important.

I use it for my music discovery and collecting.
I use them to know which artist no need to try listening any more records.

Which artists I should try to find new stuff (helped by collection highlighter to see what I don’t have yet).

Release group ratings

Very important.

See my favourite records in my collection.
See what releases I would like to buy.

Recording ratings

See what versions I prefer.
Making compilations easier.
See what releases I would like to buy.

5 star system ★★★★★

… Is perfect for me.

  1. I really don’t like it
  2. I don’t like it very much
  3. Not bad
  4. I like it
  5. I love it

I would lose too much time thinking, if it was more detailed.

3 Likes

I don’t use them on MusicBrainz (I’d appreciate if we could hide them like before NGS and without a script). I might use them on ListenBrainz if it helped recommendations but like like or not like (not a five star system). For me CritiqueBrainz is kinda pointless… With all the streaming platforms that have a free tier I can easily get an idea of an album by just listening to it.

3 Likes

This is not true:

1 Like

I don’t use ratings on MusicBrainz much, mostly because I forget they’re there. most of my ratings are in my MusicBee library, embedded in the music files. if there were a way to sync ratings with media players, I’d likely use them more.

edit: for example, I rate a track on BlackPlayer (or some other Android music player), the rating gets synced to whichever MusicBrainz database, and then next time I boot up MusicBee, my rating shows up there.

(this is mostly because MusicBee doesn’t support syncing ratings from synced devices, only to synced devices)


on moving ratings to CritiqueBrainz, I wouldn’t personally mind that. however, I could see ratings being used less often if you’ve got to make a separate account to use them. I do also think they should still be visible on MusicBrainz, even if they’re stored in CB or LB. cross-promotion and all that jazz


I really only use 1-5 stars, no .5 ratings, but if we make it possible for a 10-point rating scale, I wouldn’t mind. (frankly, I wouldn’t mind a 0-100 point system, as long as it’s easy to just use 5 stars)

I also prefer a 5-star system to a thumbs up/thumbs down system. though I can imagine there might be different uses for both systems, and we could have both.

4 Likes

I don’t use ratings. Personal opinions about artists/albums/songs seem irrelevant to MB’s purpose, in my view. If I were to rate my music (I don’t, generally), I’d likely just do it in Plex, where I actually play the music.

The current implementation of ratings in MB is noticeable, but not intrusive. Making them more prominent would not be helpful for my usage. I would opt to leave them as they are. I agree with others who have said that ratings make more sense in CB than MB.

8 Likes

I just got to this part in the stream, apologies for the delay - life gets in the way…

My responses are in italic

  • How do you use ratings (or why do you not)?
    Ratings are a core point of my ability to organize my playlist. As a lazy sysadmin I like to automate, so one way I can do this with my ever growing digital collection is to use auto-playlists in my library software (currently MusicBee, previously MediaMonkey and iTunes). Auto-playlists for the unititated are effectively queries that run against the local library database and create playlists set to a selection of conditions. For me for example I can then say to my digital library software, “hey, sync all music with a 4 star rating or higher to the device please”. This means that I don’t have to worry about pulling individual tracks to and from playlists to ensure they are synced to my portable device…

Now I use MusicBrainz to effectively feed and organize my entire library; this means that I have the rating set in MBz on a recording, I then use Picard to apply those ratings in MBz to the ID3 tag of the digital audio file, and my library picks up the file and decides where it goes from there. (this is a similar method which I also use for genres)

Now the question comes do I need all 5 stars for a rating… probably not. Could the simple LB like and dislike functin work, yes it could. Although the issue with that is that if I wanted to at a later date make a playlist of my 5 star tracks then I would be a bit stuck if limited with a binary yes/no system.

  • What if they went away?
    It would be a pain and interrupt my workflow.

  • What if they changed (were only on LB, only on CB, etc)?
    So long as Picard can read it somehow, I’m not too fussed. Accessibility to set the rating however must be maintained across the projects… so if we’re going to include ratings on MBz, LB and CB then if I set it in MBz it should also be set in LB and CB and the same vice versa.

  • What if we displayed them less prominently?
    Again this would be a pain and if they get hidden behind another silly tab, i.e. causing additional clicks, I might do one of my little rants again (like I did when Reviews link got moved). I’m not against change but I don’t like to move things unnecessarily. At the moment I’d say leave it as it, and if it does need to be moved (for whatever decided reason) include it with the big UX overhaul that is hopefully due someday soon; this is because I’m going to be re-learning my muscle memory so would then get used to it.

  • What if we displayed them more prominently?
    Within MBz I’m not sure how more prominently you can make it, aside from maybe making the stars “bigger” (I run MBz at a zoom level of of 200% on a 2560x1440 resolution monitor due to a mixture of both poor eye-sight and a general feeling that MBz doesn’t cope overly well in high DPI situations - so the stars are quite big already :grin:) . I think that in CB its pretty obvious too. LB I don’t think such a thing really exists?

  • What if we left them the same?
    No skin off my nose but I think it’s going to possibly change, at least in apperance, via the UX change… will that change interaction is something that is ultimately unknown.

(next post with some more thoughts on its way…)

5 Likes

So Ratings are a point of contention across every platform. I’m unsure which speaker at the summit mentioned it, but they’re right, ratings are just one of those things that we have and feels like we’ve always had “online”. Although I’d argue that 5-star ratings exist way before this, certainly recalling things such as newspapers and magazines prior to the internet giving anything and everything (certainly media) a star rating.

But, the question is what value are they? And I’m sad to say it, probably they’re worth diddly squat.

When used to rank lists, ratings can get a bit pointless. The most popular, most liked, and sometimes the most “critically acclaimed” content will always rate high. The most popular to hate and “critically disliked” content will always rate low; and sometimes the individuals placing those ratings may have never even listened/watched/played/read the content in question… they’re just wanting to “stick it to the man” and give something a 1/5 to show their disdain for whatever trivial reason they behold at the time. Then everything else in the middle often lumbers along with a middling score of around 3.something because lets be honest for most art content you’ve got those that love it (giving it 4’s and 5’s) and those that are simply so unphased about it they’ll just give it a fair 3/5.

Now without going into some kind of philosophical diatribe, there is question that is a 3/5 actually more damning than a 1/5 (think of how many “terrible” movies are still actively talked about today, but those average releases are nearly permenantly forgotten to the sands of time).

I guess that ratings have always existed because they’re easy to work with. It’s a scale, 4 is higher than 2 and 1 is lower than 3. So you can sort it easily, but the problem is like I just mentioned you’ll have a very small percentage at each end of the scale and the rest will sort of fall somewhere in the middle.

The other arguement is that giving any entity a star rating actually means nothing to anyone; sometimes even to the original submitter later on (there are ratings on Discogs or RYM that I probably wouldn’t be able to rationalise now). Where as even a short sentence written will give more insight to how an entity makes you feel and probably becomes worth at least something.

Again without getting too sidetracked, and I know there’s more CB talk tomorrow that I’ve been duly invited to, I think encouraging people to make short statements is probably a higher priority than getting people to either give a star rating or pressing a heart or a smiley face etc. If that individual then wants to write more than a short statement (thus becomming an actual “review”) then that’s even better.


Let’s think of this in a scenario…

You go to a social place like a bar, a complete stranger appears and starts a conversation with you. Turns out you both have opinions on music and you begin to talk about the subject. You say to this person “So buddy, what do you think of the latest song by Johnny Goodhair?” and they just sit there, smack their lips and blurt “4/5”. You look confused, and say what do you mean “4/5 - for me its a 5/5, I think its their best work yet” and the stranger simply sits there and responds “4/5”. The night goes on, and this seems to happen all the time - you never can find out why the stranger gives one song a 4, the next a 1 and the next a 3. You become fustrated because the conversation has no substance, pay your tab and leave.

Now what you really wanted from that interaction was the stranger to sit there and say “Well I’m not sure, I think although Johnny Goodhair is showing that he can certainly come back and hold his place in the charts, I think his teeny bopper days are over… ever since losing his producer Mr. Musicman, who let’s be honest had the midas touch, the music won’t be the same; thats why I think the latest track is good but not as good as it was 5 years ago”

Now with that simple sentence you can start a proper discussion instead of blurting numbers at each other.

OK so I get the Zoomers (or whatever you call them) will often say “ehh mid” to a lot of things; I guess its their generations “meh” but at least that even says more than a 3/5.

3 Likes

At least on the website, it seems to be the case that you can only pick 1-5. Was aware of it being saved as 0-100 internally.

I have moved players (to Quod Libet specifically) so I might look at syncing ratings from there.

5 Likes

Never use ratings. Turn them off when I can.

I don’t rate my own music. And even less interested in a random other person’s opinion of my music. Glory in our differences.

Not everyone is going to agree on “good” and “bad” music. Too many different opinions.

If they went away - great. Double plus good. When I get a customisable GUI - first thing I turn off. (And I have scripts that kill them today)

I also don’t rate my music on my media player as I don’t want to be steam rollered into listening to the same stuff all the time. Different moods, different music. One thing I love about MB is it has made me listen to music I had forgotten I owned. “Opinion” blocks discovery.

I also agree with other editors above. MB is a database. Raw data. Just the facts. Ratings seem to be in the wrong place here among the data. Does Wikipedia rate their articles? “Five stars, must read”.

4 Likes

My 2 cents:

  • Increments of 0.5 from 0 to 5 make the most sense to me. I think there is some nuance between 4, 4.5 and 5. But I don’t think there anyone can really justify why they think one thing is an 86 and another an 87, or, god forbid, an 86 vs an 86.1
  • But since some people are already invested in 0-100 ratings or 1-5 ratings, these could be alternatives settings you could toggle to if you desired more or less detailed ratings
  • One problem with the MB website is that you can’t easily see the average rating after rating it yourself. I think the site average should be displayed as a number next to the stars after you fill them
  • Removing them entirely would be a mistake. A toggle in the settings to hide them might work better
  • Nobody uses the ratings because nobody uses them. You can like or dislike a recording on LB, maybe allowing you to sync a rating between MB and LB on a recording would make more people rate things? Then seeing more names and votes across the website would make rating feel more useful, so more people rate
  • Possibly controversial idea: I think it would be a good idea to allow individually rating releases and allowing that to be seperate from the release group rating. I have bought some CDs that were terrible brickwalled needledrops. The album itself is good, but those releases were terrible. You might get some audiophile users to add release and rate the different masterings with this, Discogs has by-release ratings that some people try to do this with, but everyone hits 5/5 because they want you to buy it from them
  • The artist pages could by default give a rating based on your average rating of their release groups, but you could override it if you think they are more or less than the sum of their parts. For example, your favorite band has a few bad albums after having a reunion, or has hundreds of live releases that you collect and not all of them are great
  • CB ratings and MB ratings should be the same, if they aren’t already. I don’t see why they shouldn’t be?
2 Likes

Part of our discussions at the summit this week has been about how to better integrate the accounts of all of our apps. In this case, we would expect that a user on MB (or LB) would be able to seamlessly view and make ratings without needing to explicitly or manually go to another site to create an account. The details of this are still being worked out, but in the case that we were to move ratings to CB, the workflow from a user in MB wouldn’t change

5 Likes

Me too, and it’s usually thanks to my ratings that I can enjoy this. :wink:

I am a heavy MB ratings user but rarely look at the others’ ratings.
I think the ratings are first made for yourself.

1 Like

I don’t use ratings on MB (sometimes I do accompany a CB review with a rating), but I do use them elsewhere.

I don’t use them here partly because of how I use MB, partly because how they’re implemented.

No matter what I don’t think me not using them is any justification for removing them, if others are using them.

How and where I use ratings:

  • CritiqueBrainz: I like reading CB reviews (particularly by sound.and.vision) and I think the rating gives an interesting context at a glance. Of course, the review is more interesting than the rating. I like reading reviews with really low ratings :stuck_out_tongue:

  • Rate Your Music (RYM): I use RYM all the time. Not to find new artists, but if I have an artist that I want to listen to I will go to the RYM page, sort their releases by rating, and then grab the top one or a few of the top ones. That I can sort releases by the rating column makes the whole thing very easy and saves trawling through reviews or whatever. I don’t want to faff about, I just don’t want to accidentally listen to a bands worst album and then dismiss them by unlucky accident. I don’t go entirely off rating - maybe I’ll try a highly rated album from early on in an artists career, and a highly rated one from later. Or a low rated album that has a huge amount of ratings, which means it was their break out hit, or radio album or whatever. This is pretty much my only interaction with RYM.

  • Rotten Tomatoes: Since I don’t watch many movies I will often have a quick check on RT if my partner suggests something to watch (I know I know, poor Lucy!) Depending on the genre and other vague stuff the rating won’t necessarily dictate if I watch it or not. Horror or dumb comedy movies I don’t really care if it rates low (they all do more or less). But if it’s something that requires more investment I am interested in the critical opinion. I am more aligned with ‘critic’ reviews than ‘public’ reviews in these cases, and I find the fact that it splits them out and displays them next to each other super useful. And very interesting - critic and public often have wildly different ratings and that always tells a story. I will sometimes look at RT after watching a movie, particularly if one gave me something to grapple with or process, to see if people thought the same as me (both the score and the reviews). I do give things stars on RT (if I can be bothered) because at some point I thought I may as well track what I’ve watched and I don’t think you can ‘save’ stuff on RT without rating it. Which is an interesting UX choice… My rating system is different to most (5 = an average movie, not 7) but I don’t really have concerns that I’m breaking anything. I rarely look back at my ratings but when I do it is kinda cool to be reminded of different movies, and the rating helps trigger memories.

image

  • GoodReads: Note that I’ve dipped in and out of using GoodReads over years (maybe because I don’t read digital books? So it’s not that natural a transition after reading something from the library), but this is what I do when I’m using it. When I’ve read a good book and am looking for more from the author the rating is helpful (or simply number of ratings, if I want to find their ‘biggest’ books). I avoid reading longer discussions/reviews until after I’ve read it, after which it’s often interesting to dig in and read the commentary. I don’t rate books much, but I do always add books to my ‘read’ collection, when it will pop up the option to rate and/or review. I usually don’t bother rating because I find it really hard to put how I feel about books into a rating, but I will leave a rating if I think my feelings are clear. I think books are already known for how much value there is in discussing them with others (we write school reports, have book clubs…), so it’s natural and socially encouraged (seen as ‘a respectable way to pass your time’) for communities to form around authors and discuss/critique them. I think ratings are secondary to discussions in GoodReads, but certainly not useless.

  • Steam: Steam only has a ‘thumbs up/thumbs down’ way of rating games. Honestly, this is pretty annoying as a user, because SO often you want to leave a neutral review. But I suspect forcing users to make this hard call (I disliked X, is that enough to dislike the game…) actually works for their system, potentially keeping things out of the ‘everything is a 7’ trap. They don’t use a number scale, but text, something like ’negatively received’, ‘overwhelmingly positive’, and so on. I find these ratings actually very accurate in terms of deciding if a game is good or not (unlike pretty much every site I use, where I take them with a big ol grain of salt). Context still matters, e.g. if a niche game genre is included in a big sale or bundle its rating will drop a little (because it’s reaching a different audience). They combat this (and because of the nature of games, where they are often updated or changed) by having a ‘all time’ and a ‘recent’ rating, which again tells a story in tandem. They have a huge issue with ‘review bombing’/abuse of the system, which they try to combat with automatic systems. When they changed this they also added a great graph of ratings over time, which can be fun to peek at. Beause you have to write a review, you end up with a lot of throwaway reviews. Also a lot of ‘joke’ reviews make it to the product page, because people vote them up. That doesn’t break the system imo, it’s often pretty funny (some of you technical folks may be more concerned by the typical Steam review joke…)

image

Because Steam (and other game sites, Metacritic etc, and recently film sites too, as entertainment attracts more political discussion) has a lot of ‘review bombing’ problems, they have some cool additional views:

(we can see that most people loved the March update, some didn’t)

  • Rideshare apps etc: I usually give everyone 5*, but it’s still nice to know in advance/have time to inform your family that you may be murdered if your ride is turning up and has 1*

image

  • Google maps: Depends on the place. I don’t trust reviews that much for restaraunts, but for something like a mechanic or a plumber it means a lot if they have a massive amount of ratings and they’re all positive. I think it is useful that they have a display of the rating distribution, because if 90% are 5*, and 10% are 1* I will suddenly be very interested in reading what the 1* people had to say.

  • Random music torrent sites: I find ratings interesting, but dismiss them as entirely useless (e.g. people hate-voting etc). Entertainment value is high watching a new Kanye West album ratings go up and down : D
    With more niche artists the ratings suddenly become a bit useful.

  • A bunch more random stuff

A lot of these are just interesting - for my music interests/needs RYM is the best comparison :+1:

6 Likes

@ApeKattQuest_MonkeyPython doesn’t have access to the forums at the moment, I am posting this comment on their behalf:

rate something both in MusicBrainz and in CritiqueBrainz is confusing, and I would expect both to be merged. I’d expect them to live in CritiqueBrainz, but be able to access (display, add, edit) them from the other sites
put on CB, show/edit on LB/MB/BB/whatever

7 Likes