Idea: Editor experience points

MusicBrainz
21

To give some actual numbers:
2k edits is only few days of edits for some editors: https://musicbrainz.org/statistics/editors
If you look at Top editors overall Top 25 (all are autoeditors afaik), editors have between 300k and 2.3M edits) as today.

The XP system would just help to list possible candidates for wide autoeditor powers.

But what if someone is very proficient in a narrow field, shouldn’t the person granted more powers in this narrow field? Specialization is something useful too.

As an example, some editors are almost never creating new releases, but rather fix existing ones.
Some others are just adding/fixing cover art, while others never uploaded any cover art.
Some are adding tags to entities, while others never do.
Some are recording merging specialists, some are creating and associating works to recordings.
Someone can be a specialist in one band, being a total fan, IMHO this person could have auto-editor permissions for this artist and related entities (something we don’t have know).
Plus languages, countries, genres, time periods, in fact we have less wide-spectrum editors than narrow-spectrum editors IMHO.

Yes, it makes sense, i’m especially worried about editors obviously doing bad job, but still have a lot of ways to edit without any vote. Such system could at least restrict their actions a bit. Then they can either continue this way till a ban, or improve and get back to the right track.
That said, negative score is a bad idea, the worse score cannot be less than a new editor starting score, else people will just create new accounts.
But someone with 0 XP (because it was reduced or because it’s a new account) should have 0 auto-edit possibility.

I can’t think of any system that would not involved human or advanced AI.
What if we could just like/dislike edit notes (like we do with posts here) ?

Yes, that’s what i call “activity”, so voting should give XP as any other editor actions.

I think yes, but not sure if it’s technically feasible atm.

Good point, but the XP system on MB will not replace the vote system nor peer reviews.

But the current system isn’t that great as it let people do a lot of edits without any votes.

For example, an editor who joined only 3 months ago, currently very active, adding a lot of cover art (which are auto edits for all), with almost no one checking if correct, i’m the only subscriber (since today), no one ever votes on his edits, most of his edits have no edit notes (cover art is not “sourced”).
Fortunately in this case the editor isn’t doing too bad job AFAIK.

Do you think we can ensure quality if no one is reviewing / voting ?
Is there any reward for this very time consuming task ?
How do we encourage people about this ?

IMHO, having fine-grained privileges could help, only giving auto-edits to those that got accepted/voted Yes edits in this field.

7 Likes
22

Top 25 seems like a very small number, especially for all time. I’d be curious about the top 100 or even more. Where are the break points in terms of edit numbers/percent of people?

I’m a pretty experienced editor (I feel like!) and I’ve made about 25K edits over 9 years (with more and less active periods) - everything from scanning my own artwork to detailed relationships to button-press Guess Case and feat artists fixes. Would love to do more, but life gets in the way. (And lately I’ve been less active because I went back to school for programming, partly inspired by the desire to get involved on the coding side - but time will still be the biggest obstacle!)

6 Likes
23

You’re right, that’s probably better.

Yeah, that’s what I thought of too. It’s not ideal, but I can’t think of anything better. It should be made very clear though that the like/dislike function should be used to rate the quality of the note as such and not convey whether you agree or disagree with the content of the note. Maybe every user should have a very small limited number of daily likes and dislikes so that more thought goes into when it is used.

Only as soon as you’re in the top 25 voters in the last week or of all time. I admit the opportunity to rise to the #4 spot did motivate me.

I’d like that too. Unfortunately it will probably only be possible to distinguish between edit types. It would be really great though to also distinguish between experience with edits concerning different parts of the style guide, like different languages, classical, audiobooks, …

2 Likes
24

I’m actually surprised that edit note isn’t a required field.

4 Likes
25

Perhaps a weighting based on vote differential – number of “No” votes minus the number of “Yes” votes.

1 Like
26

It isn’t required, because else people just enter “edit note” as edit note…
Also edit notes don’t always make sense, many edits are self-explaining:

  • most edits adding a link (one can check it is correct by following it)
  • trivial changes (like quote to apostrophe or capitalization fixes, according to guidelines)

But edit notes are very needed for:

  • new cover art images (scans, web, etc…)
  • removed data (why?)
  • adding dates or places (sources?)
  • most (but not all) relationships (sources, reason)
  • etc…

Imho that’s not a good approach, likes/dislikes can easily be confused with votes, i wonder if a system merging both could work.
One could “rate” each edit on a scale +2 to -2:

  • +2 = i love this edit, and i think it is very needed, and the edit note is very useful and well written
  • +1 = i like this edit, but it could have been better
  • 0 = i don’t care
  • -1 = i don’t like this edit (=i don’t care, but i feel it isn’t needed, or it can be better)
  • -2 = I’m against this edit (=No vote)

Visually, it could be something like (i’m not a graphic designer, and that’s just an example): test

7 Likes
27

I like the idea of a scoring system and I also believe that it will be hard to find the right balance between justice and effort.

Hopefully we don’t see an increasing number of newly created recording and work duplicates. Some kind of penalty for such edits would be needed. This could be achieved by applying a malus to the duplicate creator’s account during the approved merging process.

4 Likes
28

I’m extremely uncomfortable with this idea.
I’ve worried since I joined MB that having had edits voted down would affect how much weight my votes would hold going forward. Often times (primarily new) editors don’t know that they can cancel their edits, so I worry that such a system will discourage new editors from joining and contributing.
Additionally, having a visible score will encourage editors to compete against each other and may lead to a form of tribalism wherein more seasoned editors discriminate against and belittle newer/less active editors. We’ve occasionally had issues with this under the current system; I don’t want to see it rewarded or becoming commonplace.

7 Likes
29

Good points, it’s really not my goal to encourage such things.
I don’t think such behaviors (discriminate/belittle) are related to a specific system, there are against our Code of Conduct anyway.

In fact, i was thinking about this to make all editors equal, currently they’re not regarding auto-editor status, because it’s based on suggestions by other editors and because the system is kinda broken (Call for nominating new autoeditors (AE))

So XP system would just be a way to put some light on those experienced editors who deserved to have more privileges (because they do good work) but aren’t “popular” enough to be nominated.

I also think current beginner/normal/auto-editor privileges levels aren’t enough. A more fine-grained system would help limiting damages to database. Think about a more progressive way to reach the current auto-editor status.

Something we could do is to calculate XP for all editors, only editor, auto-editors and admins could see the actual score, and it wouldn’t be visible by all.

It could still be used to trigger stuff, like an auto-editor election, without having the “competition” drawback.

6 Likes
30

I talked about a similar idea in "Advanced editor" status discussion however I didn’t say how one could acquire such status. I mainly talked about a different UI for the different editor groups but having “more power” for the more experienced editors wouldn’t also be too bad.

For example I can see myself as an experienced editor but every time I see someone nominated for the auto-editor status I notice that they usually have 10 times more edits than me :slight_smile:

4 Likes
31

That sounds confusing. The votes are a plain clear system for quality control. Yes if you agree and know the data\edit is correct, No if you know the data is wrong or against policy, Abstain if you want to discuss it.

Confusing this with facebook type likes\dislikes will confuse what a vote means. Someone may “like” that images are being updated, but other users are hitting “No” because they know the artwork is wrong. That will then lead to edits passing which shouldn’t.

4 Likes
32

This sounds like a decent idea, as long as there is still transparency on how the score is generated and what it means.

2 Likes
33

I don’t mind it. But it would need to reflect actual effort spent. Adding a release is one thing - spending 4 hours adding a release plus all ARs, includingcreating new artists/works/… is another.

As for negative points for downvotes, on StackExchange sites you a) only get the ability to downvote after you reach some basic level of reputation, and b) you yourself lose a point for each downvote. That should probably kerb misuse somewhat. (https://stackoverflow.com/help/privileges/vote-down)

It’s a different kind of site of course, but I feel there are ideas to be adapted from their rep system.

5 Likes
34

It’s impossible to evaluate the amount of time spent in research for one edit, the editor could tell, but i don’t think we would trust this info, other editors could estimate the effort, but it seems overcomplicate.

Interesting, thanks for sharing. It prevents downvote abuse.

Currently, on MB, anyone can downvote, once they can vote (that is 2 weeks / 10 accepted edits, according to Introduction to Voting - MusicBrainz).

Also interesting: Privileges - Stack Overflow

Of course, MB differs but not that much, IMHO.
MusicBrainz is looking for quality of edits, StackOverflow quality of posts. Both needs data quantity and quality, volunteers, privileges, …

2 Likes
35

That is interesting. And it raises the point that we’ve talked quite a bit about how to calculate a score, but not much about what privileges might be gained with higher scores, or how they would be tiered. Some thoughts:

  • new users might have to have all their edits in the voting queue at first, with types of auto-edits unlocking as they gain experience (maybe measured by entity type: edit lots of events? get more event auto-edits). (This maybe should even be based on % passed edits vs edits voted down)
  • ability to approve other’s edits at a certain experience level (again, probably by entity type)
  • reserve ability to make destructive edits for more experienced editors
  • give more experienced users ability to “flag” edits for discussion/voting (sort of an automated version of the voting request thread)
  • weighted votes based on experience level? (iffy on this)

That’s off the top of my head, gotta go for now…

1 Like
36

I was not suggesting using the actual time as a metric. Just that each AR created/modified should also accrue points. So that a marathon editing session like that would grant a correspondingly large number of points.

6 Likes
37

Yes, it might work.

For each Edit Type (https://musicbrainz.org/doc/Edit_Types) we could have requirements in XP and votes.

Let’s imagine what it could be (just an example, might not be realistic or accurate or anything):

New editor: 0 XP

Day 1: editor Add disc ID , this action has no requirement, earn 5 “Add disc ID” XP + 5 XP per Yes vote if accepted, and defaults on “vote required”
Day 2: editor “Add disc ID” again, he has 2 edits in vote queue

After few days:
First edit is accepted, it got 3 Yes votes, “Add disc ID” XP = 5 + 3x5 = 20
Second edit was accepted too, it got only 1 Yes, “Add disc ID” XP = 20 + 5 + 1x5 = 30

On next week, editor wants to “Move discid ID”, 0 XP.
Requirements are: 40 “Add discid ID” XP or a total of 100 XP, and it defaults on “not possible”

Editor cannot do that action yet, but he can create an edit request.

This is like submitting an edit to vote, but instead it first goes to another queue: only editors having a certain amount of XP in this kind of edit will be able to “vote” for few days. If the edit gets 3 votes it’s applied immediatly, each voter earn 5 “Move discid ID” XP for this and editor earns 10 + 3x5 “Move discid ID” XP.
If after few days, the edit isn’t yet “approved” it goes back to normal vote queue.

Having an “edit request” queue will focus attention on new editors from more experienced editors.

So this “Move discid ID” is approved, our editor has now:
30 “Add disc ID” XP
25 “Move discid ID” XP

Let’s continue:

Editor does another “Add disc ID” but this action has a rule “total XP > 50” = “autoedit”, so this time, no vote queue, the edit is automatically accepted, earning 10 “Add discd ID” XP (no vote bonus).

Now editor wants to “Remove disc ID”, this action has requirements:
50 “Add discid ID” XP, 50 “Move discid ID” XP and a total of 500 XP
Since the editor is far from this, the edit will go in Edit Request queue first.

It’s a bad edit, got 2 No votes and one Yes from autoeditors (for this kind of edit), no one earns any XP (the edit wasn’t accepted).

He does another one, this time 2 Yes, 1 No, the edit is accepted:

  • editor earns 10 XP + 2x5 XP = 20 “Remove disc ID” XP
  • No voter loses 5 “Remove disc ID” XP (edit was accepted, but he voted No)
  • Yes voters earn 5 XP each

After a while, he becomes autoeditor on “Remove discid ID”, but he has a doubt, and ask for votes:
the edit goes in Edit Request queue, he will earn an XP bonus if the edit is accepted.

So main ideas:

  • an “Edit request” queue: only editors having a certain amount of XP in this type of edit can vote for first days, they get an alert when such edit is created, if the edit isn’t rejected but not accepted, it goes to normal vote queue
  • Voting earns XP on the type of edit, if accepted
  • Voting No has a penalty, if accepted
  • asking for votes even if one could autoedit may earn bonus XP
  • XP requirements on each type of edit
  • a default for each type among 3 choices: Edit request, Normal Vote Queue, AutoEdit
  • requirements could be total editor XP and/or specific type XP.
  • XP are accounted per edit type
3 Likes
38

At an admittedly brief read-through this sounds about right. My only immediate thought (back to the scoring part, sorry) is that no votes shouldn’t be penalized only if the votes passes. Mainly I don’t think someone should be punished for disagreeing with the majority, and also if an edit is voted down those voting should feel strongly enough about it to take the hit.

39

Yes, that and I’m now convinced losing XP is a bad thing, i’d rather not take No/Yes/Abstain in account, but only “has voted”.
If someone vote on one edit he gains a small amount of XP, but only if the edit is finally accepted.
It would be much simpler, less controversial, and still encourage people to vote.

5 Likes
40

So what’s the next step in making this happen?