Picard / genre scripts / Wikidata Genre plugin

I think it was a change to the topic tags.

I just moved it under MusicBrainz Picard category.

So I didn’t post it there?
I would have sworn I had. Sorry.

@outsidecontext I agree with this statement and I suggest that we set the following rules and implement them into Picard / plugins:

A. Picard and all genre plugins set hidden variables for the genres they provide;
B. Plugins always add to the genres rather than replacing them.

If we do this, then you don’t get plugins overwriting existing genre information, and people can script for any alternative approach that they want.

4 Likes

That brings me to another aspect I seem to have had a wrong and vague notion about; folksonomy tags.

I somehow always assumed folksonomy tags were also genre tags that were retrieved form some other source than MusicBrainz. Some third party database called ‘folksonomy’ that was much less curated but could provide useful genres.

But it looks like what is called ‘folksonomy tags’ in Picard, is the same as what is called ‘other tags’ on MusicBrainz?

Yes. The folksonomy tags are the original capability of how users could tag all entities on MB. This can be used for anything, including genres. See https://musicbrainz.org/doc/Folksonomy_Tagging.

MusicBrainz’ genre feature builds on the existing folksonomy tags. At it’s core it is a whitelist of tags that should be considered genres.

Before MusicBrainz had explicit genre support Picard already allowed you to use the folksonomy tags for genres.

2 Likes

Now that it has, and the genre database is getting better and better, I have some other food for thought:

Picard has a default setting of 90% match for tags to be retrieved.
I always left it like that since otherwise a lot of non-genre, or incorrect genre tags would get retrieved.

But doing some testing, setting it at 1% immensely improved the amount of valid and usable genres retrieved. (but of course that will also add many undesired ‘other’ tags)

So it seems that for the ‘genre’ tag the setting of 90% is much too high.
But for ‘folksonomy/other tags’, a high value is much more apropriate.
So it might be good (if possible) to separate them in Picard?
And maybe have the option to disable ‘other tags’ getting retrieved and mingled with ‘genre’ at all?

1 Like

Another question:
I would like to do more testing on how Picard retrieves genres with regards to how it retrieves them form recordings vs releases vs release groups etc. (what trumps what exactly?)

There is a MusicBrainz testing database that you can use to experiment with editing. So I am guessing you could test entering and editing genre tags there too.
Is there a way to get Picard retrieving the data from that test database instead of from the real database?

Or is there perhaps a testing release/release group in the real database that one can fool around with?

It depends on what you want to have. If your goal is to retrieve all genres anyone ever tagged this with, set it very low. If you want to filter out outliers a higher value (something between 60-90 %) is more appropriate. This is still subjective crowd sourced data, with the idea that given enough people tagging the most relevant genres will get visible, forming kind of a crowd consensus. The idea of the min genre usage value is to filter out those genres that got significantly less votes than the highest voted genre. If a hundred people tagged something as “Neoclassical metal” and one put in “Hopepunk” (just picking some stuff from recent discussions) I would give some trust that “Neoclassical metal” is a proper classification of this piece of music while “Hopepunk” isn’t (even if I have no idea what either of them actually is). But if 90 people had tagged it “Hopepunk” then probably both genres describe this music well.

I had written a bit how this works at Picard 2.x and genres. Good practices? Advice on plugins? - #12 by outsidecontext

But I agree that we could set this lower to maybe 70% now that we default to actual genres.

You can already do this, and actually it is the default on new install. Just disable “Use folksonomy tags as genres”.

The genres from recording, release and release group are combined (including their votes). If you have enabled artist genres the genres of the artist get also merged in.

Yes, set test.musicbrainz.org as the server address in Options > General.

3 Likes

Thanks for all the answers!

about:

I was thinking that it might be useful to be able to set ‘genre’ tags to e.g. 20%, and ‘other’ tags to 80%.
And have it possible to get them written to two different tags so that they don’t mingle.
Just thinking out loud for now.

The tags feature in Picard always was only about genres. The “Use folksonomy tags” mainly is for backward compatibility, so users could retain the exact behavior that was before explicit genres were added. But it’s all based on the same tags. If you ask for all folksonomy tags of a release you might get something like “fixme, rock, pop, french”. If you ask it for the genres only you get “rock, pop”.

We could of course make all folksonomy tags available as a separate variable, but then I wouldn’t mix this up with the genre settings. I’m also not sure there is much use for it, so far nobody asked for non-genre tags.

1 Like

The use would be that you will also get genres that were added by users that considered them to be a genre, but were/are not listed in MusicBrainz’ genre database.

If somebody adds ‘disco rap’ as a tag, it will not be retrieved by Picard if you have disabled folksonomy tags because MusicBrainz does not know the genre. (yet)
If somebody added prog-rock as a tag, it will not get retrieved by Picard as a genre, since MusicBrainz only accepts/knows ‘progressive rock’.
Same for ‘jazz-rock’ vs ‘jazz rock’, ‘dance-pop’ vs ‘dance pop’, etc. etc.

But if you want get genres considering all folksonomy tags you’d just enable that option.

Yes, but now this discussion is going a bit in circles.
As I tried to explain: for ‘genres’ it makes sense to get them all. Or at least anything with e.g. a 20% match.
For ‘folksonomy’ it is useful to be able to get the tags only if their occurrence is for example >80%.
Else you would miss out on a lot of genres that users have put in that are either spelled slightly different, or not (yet) validated by MusicBrainz as a genre.

Just having ‘on/off’, or the percentage slider working simultaneously both for ‘genre’ and ‘folksonomy’ seems sub-optimal to me.

1 Like

I have been noticing some odd results using the Wikidata plugin that I couldn’t explain.
So I tried to narrow it down by doing a lot of testing, and there is one issue that has come to light:

It seems related to the ‘use Artist genres only if no Release Group exists’ setting.

Here are two releases that clearly show the issue:

When running each release separately, these are the results:

But when running them both I get these results for the second one:

So it looks like it gets all artist genres added, while ‘Use artist genres only if…’ is checked.

So somehow the processes seem not completely isolated between the releases, and they can ‘contaminate’ each other?

settings:

edit:
Perhaps I should ping @ the creators of the Wikidata plugin in this post?
But I don’t know what avatar/alias to use for that?
(ah well, I’ll just rely on @outsidecontext :wink: )

Since:

A. A lot of genres have been added to the ‘validated’ genre list recently.
B. I have gained a better insight in the workings of how Picard handles genres and the related settings.
C. There seems to be an increased interest and activity in possible further improvements on the matter.

I am getting more and more confident that using Picard for retrieving genres and relying on the results is a good and promising option and worth investing more time and effort in.

So, I have created a script that will:

  • Make the names of genres as provided by the Wikidata plugin and MusicBrainz’ database uniform.
    So there won’t be duplicates such as ‘Pop music’ and ‘Pop’, or ‘Dark wave’ and ‘Darkwave’, etc.

  • It will ‘sentence case’ all genre names. Just because I think that looks best.

  • Adjust some genre names to my strictly personal preference.

  • Remove duplicates.
    (doesn’t seem to work perfect yet…) is fixed now

And I thought to share it to see if others find it useful or have interesting or useful suggestions.

.

Suggested settings for trying this out:

Picard’s Genres settings:

Wikidata plugin settings:

Happy testing!

$setmulti(genre,$lower(%genre%))
$setmulti(genre,$map(%genre%,$upper($substr(%_loop_value%,0,1))$substr(%_loop_value%,1,)))

$replacemulti(%genre%,Ebm,EBM)
$replacemulti(%genre%,Edm,EDM)
$replacemulti(%genre%,Mpb,MPB)
$replacemulti(%genre%,Idm,IDM)
$replacemulti(%genre%,Stage & screen,Stage & Screen)
$replacemulti(%genre%,Brass & military,Brass & Military)
$replacemulti(%genre%,Uk funky,UK funky)
$replacemulti(%genre%,Uk hardcore,UK hardcore)
$replacemulti(%genre%,Us power metal,US power metal)

$replacemulti(%genre%,2-step garage,2-step)
$replacemulti(%genre%,A cappella,A capella)
$replacemulti(%genre%,African popular music,African)
$replacemulti(%genre%,Afro pop music,African)
$replacemulti(%genre%,Afro-trap,Afro trap)
$replacemulti(%genre%,Alternative country,Alt-country)
$replacemulti(%genre%,Aor,Adult oriented rock)
$replacemulti(%genre%,Arabic pop music,Arab pop)
$replacemulti(%genre%,Argentine tango,Tango)
$replacemulti(%genre%,Art music,Classical)
$replacemulti(%genre%,Axé musica,Axé)
$replacemulti(%genre%,Bard song,Bard)
$replacemulti(%genre%,Baroque era,Baroque)
$replacemulti(%genre%,Berlin school of electronic music,Berlin school)
$replacemulti(%genre%,Blues-rock,Blues rock)
$replacemulti(%genre%,Bolero,Boléro)
$replacemulti(%genre%,Bop,Bebop)
$replacemulti(%genre%,Breaks,Breakbeat)
$replacemulti(%genre%,Brit pop,Britpop)
$replacemulti(%genre%,Celtic music,Celtic folk)
$replacemulti(%genre%,Club,Club-house)
$replacemulti(%genre%,Dance-pop,Dance pop)
$replacemulti(%genre%,Dark wave,Darkwave)
$replacemulti(%genre%,Deep soul,Southern soul)
$replacemulti(%genre%,Drill 'n' bass,Drill and bass)
$replacemulti(%genre%,Electro-funk,Electro)
$replacemulti(%genre%,Electronic dance music,EDM)
$replacemulti(%genre%,Electronica,Electronic)
$replacemulti(%genre%,Electronicore,Trancecore)
$replacemulti(%genre%,Euro disco,Euro-disco)
$replacemulti(%genre%,Folk-pop,Folk pop)
$replacemulti(%genre%,Fusion,Jazz fusion)
$replacemulti(%genre%,Future jazz,Nu jazz)
$replacemulti(%genre%,Glam,Glam rock)
$replacemulti(%genre%,Go-go,Gogo)
$replacemulti(%genre%,Hip hop soul,Hip-hop soul)
$replacemulti(%genre%,Humor,Comedy)
$replacemulti(%genre%,Intelligent dance music,IDM)
$replacemulti(%genre%,Irish traditional music,Irish folk)
$replacemulti(%genre%,Italo disco,Italo-disco)
$replacemulti(%genre%,Italo house,Italo dance)
$replacemulti(%genre%,Jazz-pop,Jazz pop)
$replacemulti(%genre%,Jazz rock,Jazz-rock)
$replacemulti(%genre%,Laïko,Laïkó)
$replacemulti(%genre%,Latin american music,Latin)
$replacemulti(%genre%,Low fidelity,Lo-fi)
$replacemulti(%genre%,Mangue bit,Mangue beat)
$replacemulti(%genre%,Martial music,Brass & Military)
$replacemulti(%genre%,Midwest hip hop,Midwest hip-hop)
$replacemulti(%genre%,Minimalism,Minimal)
$replacemulti(%genre%,Minimalist music,Minimal)
$replacemulti(%genre%,Mizrahi music,Mizrahi jewish)
$replacemulti(%genre%,Music of africa,African)
$replacemulti(%genre%,Music of israel,Israeli)
$replacemulti(%genre%,Music of latin america,Latin)
$replacemulti(%genre%,Music of puerto rico,Puerto rican)
$replacemulti(%genre%,Música popular brasileira,MPB)
$replacemulti(%genre%,Musical theater,Musical)
$replacemulti(%genre%,Neapolitan song,Canzone napoletana)
$replacemulti(%genre%,Neo-classical metal,Neoclassical metal)
$replacemulti(%genre%,Neoclassical new-age music,Neoclassical new age)
$replacemulti(%genre%,New flamenco,Flamenco nuevo)
$replacemulti(%genre%,New romanticism,New romantic)
$replacemulti(%genre%,New-age music,New age)
$replacemulti(%genre%,Nuevo tango,Tango nuevo)
$replacemulti(%genre%,Old-school hip hop,Disco rap)
$replacemulti(%genre%,Old-school rap,Disco rap)
$replacemulti(%genre%,Philadelphia soul,Philly soul)
$replacemulti(%genre%,Political hip hop,Political hip-hop)
$replacemulti(%genre%,Pop art,Art pop)
$replacemulti(%genre%,Pop-music,Pop)
$replacemulti(%genre%,Popular music,Pop)
$replacemulti(%genre%,Prog rock,Progressive rock)
$replacemulti(%genre%,Prog-rock,Progressive rock)
$replacemulti(%genre%,Protopunk,Proto-punk)
$replacemulti(%genre%,Psychedelic trance,Psytrance)
$replacemulti(%genre%,R&b,Rhythm and blues)
$replacemulti(%genre%,Rapping,Rap)
$replacemulti(%genre%,Rave,Breakbeat hardcore)
$replacemulti(%genre%,Rock & roll,Rock and roll)
$replacemulti(%genre%,Romantic era,Romantic)
$replacemulti(%genre%,Rune singing,Runolaulo)
$replacemulti(%genre%,Runo song,Runolaulo)
$replacemulti(%genre%,Samba reggae,Samba-reggae)
$replacemulti(%genre%,Shoegazing,Shoegaze)
$replacemulti(%genre%,Song-cycle,Song cycle)
$replacemulti(%genre%,Southern gothic,Gothic country)
$replacemulti(%genre%,Southern hip hop,Dirty south)
$replacemulti(%genre%,Stage and screen,Stage & Screen)
$replacemulti(%genre%,Symphonic music,Symphony)
$replacemulti(%genre%,Synthpop,Synth-pop)
$replacemulti(%genre%,Synthpunk,Digital hardcore)
$replacemulti(%genre%,Traditional heavy metal,Heavy metal)
$replacemulti(%genre%,Trip-hop,Trip hop)
$replacemulti(%genre%,Urban,Hip hop)
$replacemulti(%genre%,Urban contemporary,Hip hop)
$replacemulti(%genre%,Western classical music,Classical)

$set(_genremusic,$replacemulti(%genre%,Bass music,Bass xxxxx))
$set(_genremusic,$replacemulti(%genre%,Chamber music,Chamber xxxxx))
$set(_genremusic,$replacemulti(%genre%,Early music,Early xxxxx))
$set(_genremusic,$replacemulti(%genre%,Epic music,Epic xxxxx))
$set(_genremusic,$replacemulti(%genre%,Furniture music,Furniture xxxxx))
$set(_genremusic,$replacemulti(%genre%,Incidental music,Incidental xxxxx))
$set(_genremusic,$rreplace($rreplace(%_genremusic%,\( music;\),;),\( music\)\$,))
$set(genre,$replace(%_genremusic%,xxxxx,music))

$setmulti(_genretemp,%genre%)
$set(genre,$unique(%_genretemp%))
3 Likes

Im experimenting with the vanilla musicbrain genres and having mixed results.
Take this example:

Recording “My Last Request” by Grim - MusicBrainz

from the compilation album

Release “Human Traffic: Essential Selection Presents Music From the Motion Picture” by Various Artists - MusicBrainz

This track is blatantly rap / hip hop but MB wants to tag it as “Techno”. I looked through the artist, recording, release and cant find where this is coming from

All sub-options unticked under genre in MB settings (except use MB genres of course), 90% match selected

Any ideas?

2 Likes

It’s getting this from the release group: Release group “Human Traffic: Essential Selection Presents Music From the Motion Picture” by Various Artists - MusicBrainz

Picard is currently always using release / release group genres in addition to the recording genres. We have an open ticket https://tickets.metabrainz.org/browse/PICARD-1848 to allow disabling the release genres. Or probably only use them as fallback if no recording genre is available, this needs some consideration.

2 Likes

I looked at the ticket and the proposed would be really good. Has there been any progress implementing this yet? I see the ticket still reads unresolved. I’m holding out on a full library re-tag until this is sorted

Does this work if you also have lastfm plugin installed, so that it can get genres from all three.