Picard doesn't recognize existing MusicBrainz ID3 data?

notie5 · November 8, 2022, 1:49am

Hi everyone, I’ve been using Picard for about a month now and this may just be something I’m doing wrong or a setting that’s not checked but I’m not finding it in my searches. I’m big on physical media and still buy the vast majority of my music on CD and then rip them to FLAC using EAC. When I originally ripped my CD’s about 10 years ago, I manually created the folder structure while letting EAC create the file names and add ID3 data. I’ve been using Picard to go through my FLAC library and manually write a lot of the ID3 tag data that was missing or incomplete. I’ve got it setup to write to my preferred folder structure and life is good with my FLAC’s.

I do use dBpoweramp to do a mass conversion of the FLAC files to MP3 so I can fit them on a thumb drive for my car or use them on my phone. dBpoweramp just mirrors the FLAC file structure into a different directory. Meaning all my FLAC files are in D:/FLAC and all of my MP3’s are in D:/MP3. On top of this I do occasionally buy MP3’s from iTunes for certain music that does not have physical media. I would Picard to organize my converted MP3 folders/files without having to go through and manually select the specific releases of each album like I did with FLAC. When I have Picard look at MP3’s that were converted from my FLAC files it DOES seem to see the musicbrainz ID3 tags written by Picard. Like this.

You can see that screenshot has the album already in a cluster. It clusters all of the albums. However, if I choose the “Scan” button, it tries to add new ID3 tags, just in a different format. The ID’s themselves are the same but the Tag name is different. The previous tag will say “MUSICBRAINZ_ALBUMARTISTID” while the new tag will say “MusicBrainz Artist Id”.

Anyone have any suggestions on how I can make it recognize the existing tags and not add a bunch of extra tags. From tagging my FLAC’s I can tell you that it will select the wrong version of many of my albums. Thanks in advance.

outsidecontext · November 8, 2022, 7:32am

When dBpoweramp converts tags between different format, it handles those tags it knows how to handle (e.g. title and artist), but all the unknown tags it finds in the FLAC it just copies over to the ID3 tag as a free form text tag. In ID3 those tags are known as TXXX tags, and they can have any arbitrary name + a value.

Know in FLAC / Vorbis tags for example the MusicBrainz artist ID gets stored in a tag called MUSICBRAINZ_ARTISTID. EAC copies this over to ID3 as TXXX:MUSICBRAINZ_ARTISTID . But what actually is expected is TXXX:MusicBrainz Artist Id (1).

What I suggest you to do is using a script to fix the tag names in the MP3s first, then re-tag your files. Please try the below with a few files first, of which you have made backups, before you try it with all your files at once. If everything works as expected you can apply it to the rest of your files.

Open Options > Scripts, make sure “Enable tagger scripts” is activated, add a new Script, give it a name (e.g. “Fix MBIDs”) and paste the following code (2):

$if(%MUSICBRAINZ_ARTISTID%,$set(musicbrainz_artistid,%MUSICBRAINZ_ARTISTID%))
$delete(MUSICBRAINZ_ARTISTID)
$if(%MUSICBRAINZ_DISCID%,$set(musicbrainz_discid,%MUSICBRAINZ_DISCID%))
$delete(MUSICBRAINZ_DISCID)
$if(%MUSICBRAINZ_ORIGINALARTISTID%,$set(musicbrainz_originalartistid,%MUSICBRAINZ_ORIGINALARTISTID%))
$delete(MUSICBRAINZ_ORIGINALARTISTID)
$if(%MUSICBRAINZ_ORIGINALALBUMID%,$set(musicbrainz_originalalbumid,%MUSICBRAINZ_ORIGINALALBUMID%))
$delete(MUSICBRAINZ_ORIGINALALBUMID)
$if(%MUSICBRAINZ_TRACKID%,$set(musicbrainz_recordingid,%MUSICBRAINZ_TRACKID%))
$delete(MUSICBRAINZ_TRACKID)
$if(%MUSICBRAINZ_ALBUMARTISTID%,$set(musicbrainz_albumartistid,%MUSICBRAINZ_ALBUMARTISTID%))
$delete(MUSICBRAINZ_ALBUMARTISTID)
$if(%MUSICBRAINZ_RELEASEGROUPID%,$set(musicbrainz_releasegroupid,%MUSICBRAINZ_RELEASEGROUPID%))
$delete(MUSICBRAINZ_RELEASEGROUPID)
$if(%MUSICBRAINZ_ALBUMID%,$set(musicbrainz_albumid,%MUSICBRAINZ_ALBUMID%))
$delete(MUSICBRAINZ_ALBUMID)
$if(%MUSICBRAINZ_RELEASETRACKID%,$set(musicbrainz_trackid,%MUSICBRAINZ_RELEASETRACKID%))
$delete(MUSICBRAINZ_RELEASETRACKID)
$if(%MUSICBRAINZ_WORKID%,$set(musicbrainz_workid,%MUSICBRAINZ_WORKID%))
$delete(MUSICBRAINZ_WORKID)

Do not activate this script (keep the checkbox in front of the script name unchecked), we will run the script manually later on.

Load your files into Picard, so they show up in the unclustered files pane.
Select the files, right click and choose “Run scripts” > “Fix MBIDs” (or whatever you named the script).
Check the changes in the metadata view at the bottom. It should show the all uppercase tags like “MUSICBRAINZ_ARTISTID” getting removed (shown in red and with a strike-through), and the value instead being added to a new tag with a nice name “MusicBrainz Artist Id” (3). If this does not show as expected, please abort here and report back, then we need to investigate what went wrong.
For now disable “Rename Files” and “Move Files” in the Options menu, but make sure “Save Tags” is enabled. We only want to save tags for now, not change file names.
Select the files and “Save”. This will write the tag changes to the files.
Remove the files from Picard again.
Load the files into Picard. By default Picard should now detect the stored MBIDs properly and automatically load the files into the right pane of tagged files to the proper release and recording. If this does not happen, please check that Options > General > “Ignore MBIDs when loading files” is not checked.

If the files have been properly loaded you can now use Picard normally to organize them. Don’t forget to activate “Rename files” and / or “Move files” again if you need it.

It would be great if you could report back if the above worked for you. I haven’t tested it on real files, but it should work.

(1) For details of the different tags and how they get stored in the specific formats see Appendix B: Tag Mapping — MusicBrainz Picard v2.9alpha1 documentation
(2) The script probably handles more tags then you have in your files, but I wanted to be complete.
(3) For Tags Picard knows about it always shows a more human friendly name in the metadata list

Sophist · November 8, 2022, 7:57am

Perhaps the easiest route forward is to try to find a format conversion tool that understands a bit more about the FLAC and ID3 metadata (though I fear you may end up with the same results as with dBpoweramp.

I have used MediaMonkey to transcode from Flac to ID3, but since my library is in MP3 format, I do this before I tag it with Picard not after, so I cannot comment on how good this will be at converting the FLAC tags to ID3.

But if you can not find a tool that stores the tags in the correct place, at least dBpoweramp preserves them somewhere, and you should definitely give @outsidecontext 's suggestions a go.

InvisibleMan78 · November 8, 2022, 8:16am

What about to inform dBpoweramp/EAC about this issue and tell them to do that conversion in a “correct” way as @outsidecontext suggest it?
It should be not that hard to use the correct tag names right away as they are listed above and well documented.

Sophist · November 8, 2022, 8:45am

LoL.

There is no independent industry standard on how tags work or what tags even exist across all formats - at best there is a de-facto standard agreed where a few key players have kind-of agreed to have some consistent naming conventions. So as a single organisation out of many, we have a defined tag mapping for all the metadata names that Picard uses as standard, and these have a lot of n/a entries where tags don’t exist in some formats, but these are not necessarily agreed to or followed by other tagging or player software. And should we really expect other software to support the tags that are specially created to enable Picard functionality and are not applicable to anyone else?

There may also be minor tags that are defined for one or more formats (like ID3v2.3 or ID3v2.4) that are not supported by Picard - like lyrics.

And then of course we get plugin writers that create a raft of other metadata names and expect them to be mapped to tags in all the different formats.

So, I am really not sure that it will be that easy to persuade dBpoweramp/EAC to put the effort into creating this functionality, nor that easy for them to implement it.

Some options for Picard enhancements to consider:

Add support in Picard to map all known tags to metadata so that e.g. Picard can load, display and save all well defined tags in the various formats. From this we can then propose new Picard-specific tags for format’s that don’t current support them and see if other taggers will agree to support them i.e. create an extended de-facto standard.
MediaMonkey runs external processes to transcode - perhaps Picard should add similar functionality to call an external open source transcoder - because we can then control the writing of metadata into the transcoded files.

notie5 · November 9, 2022, 2:33am

Well this absolutely is moving in the right direction. Here’s what I’m getting now

You can see all of the MusicBrainz fields are matching and it’s not trying to update them. And Picard does load them in the right hand pane. However, there are some additional fields like MIXER vs Mixer, RELEASECOUNTRY vs Release Country, RELEASESTATUS vs Release Status, RELEASETYPE vs Release Type, ENGINEER vs Engineer, ORIGINAL YEAR vs Original Year that are still being duplicated. So I suppose the script would need to be updated to include those fields?

I don’t know if all those other fields above are standard Picard fields? I’m not familiar enough yet. I’m concerned that across 1000+ CD’s that have been converted that there will be other fields?

There are still other fields like Genre that are different - I’m not sure if that’s because the database got updated since I processed my FLAC files or what? And I’m not really sure what’s going on with ORIGINALDATE vs Original Release Date.

outsidecontext · November 9, 2022, 6:53am

That’s exactly the same thing for these these tags. Again you can use scripting to fix those. For each tag use a script like:

$if(%RELEASECOUNTRY%,$set(releasecountry,%RELEASECOUNTRY%))
$delete(RELEASECOUNTRY)

The first statements checks if the uppercase tag RELEASECOUNTRY is present, and if it is it uses the value of this tag to set the tag known to Picard as releasecountry . You can find the tags Picard knows about at Tags & Variables — MusicBrainz Picard v2.10 documentation, and information on how these tags get stored in the different formats at Appendix B: Tag Mapping — MusicBrainz Picard v2.10 documentation.

The second statement then deletes the uppercase RELEASECOUNTRY tag. Extend this for all the tags you encounter, e.g. “ORIGINALDATE” vs. “originaldate”, “RELEASESTATUS” vs " releasestatus" etc. I suspect in your case the difference will be uppercase in the files vs. lowercase for the Picard name, but there might be exceptions (e.g. for the MB Recording ID there are some historical differences, see above).

When you add this to the script you can also activate the script, so it runs automatically when releases get loaded on the right pane.

Worst case is that one of the uppercase fields stays in the files. Nothing bad happens. If you don’t need the data there is no problem anyway, and the data is not lost.

There will be of course other changes. The data on MB changes over time as users update the entries. There will be both new and old fields. Also your PIcard settings can be different. I don’t know when you originally tagged your files, but most likely you used a different source for e.g. the genre tags.

You have not really stated your goals. If you want to update your tags with the latest data from MB then the above would be expected. If you just want to use Picard to rename and organize your files but you don’t want to change the tags you can also disable saving the tags and only do move + rename.

notie5 · November 10, 2022, 2:15am

Okay, I added a few more fields to the script. Here’s what I ended up with in case anyone sees this in the future and wants to copy it.

   $if(%MUSICBRAINZ_ARTISTID%,$set(musicbrainz_artistid,%MUSICBRAINZ_ARTISTID%))
$delete(MUSICBRAINZ_ARTISTID)
$if(%MUSICBRAINZ_DISCID%,$set(musicbrainz_discid,%MUSICBRAINZ_DISCID%))
$delete(MUSICBRAINZ_DISCID)
$if(%MUSICBRAINZ_ORIGINALARTISTID%,$set(musicbrainz_originalartistid,%MUSICBRAINZ_ORIGINALARTISTID%))
$delete(MUSICBRAINZ_ORIGINALARTISTID)
$if(%MUSICBRAINZ_ORIGINALALBUMID%,$set(musicbrainz_originalalbumid,%MUSICBRAINZ_ORIGINALALBUMID%))
$delete(MUSICBRAINZ_ORIGINALALBUMID)
$if(%MUSICBRAINZ_TRACKID%,$set(musicbrainz_recordingid,%MUSICBRAINZ_TRACKID%))
$delete(MUSICBRAINZ_TRACKID)
$if(%MUSICBRAINZ_ALBUMARTISTID%,$set(musicbrainz_albumartistid,%MUSICBRAINZ_ALBUMARTISTID%))
$delete(MUSICBRAINZ_ALBUMARTISTID)
$if(%MUSICBRAINZ_RELEASEGROUPID%,$set(musicbrainz_releasegroupid,%MUSICBRAINZ_RELEASEGROUPID%))
$delete(MUSICBRAINZ_RELEASEGROUPID)
$if(%MUSICBRAINZ_ALBUMID%,$set(musicbrainz_albumid,%MUSICBRAINZ_ALBUMID%))
$delete(MUSICBRAINZ_ALBUMID)
$if(%MUSICBRAINZ_RELEASETRACKID%,$set(musicbrainz_trackid,%MUSICBRAINZ_RELEASETRACKID%))
$delete(MUSICBRAINZ_RELEASETRACKID)
$if(%MUSICBRAINZ_WORKID%,$set(musicbrainz_workid,%MUSICBRAINZ_WORKID%))
$delete(MUSICBRAINZ_WORKID)
$if(%RELEASESTATUS%,$set(releasestatus,%RELEASESTATUS%))
$delete(RELEASESTATUS)
$if(%RELEASECOUNTRY%,$set(releasecountry,%RELEASECOUNTRY%))
$delete(RELEASECOUNTRY)
$if(%RELEASETYPE%,$set(releasetype,%RELEASETYPE%))
$delete(RELEASETYPE)
$if(%ENGINEER%,$set(engineer,%ENGINEER%))
$delete(ENGINEER)
$if(%MIXER%,$set(mixer,%MIXER%))
$delete(MIXER)
$if(%MEDIA%,$set(media,%MEDIA%))
$delete(MEDIA)
$if(%ORIGINALYEAR%,$set(originalyear,%ORIGNALYEAR%))
$delete(ORIGINALYEAR)
$if(%ORIGINALDATE%,$set(originaldate,%ORIGNALDATE%))
$delete(ORIGINALDATE)
$if(%ARTISTSORT%,$set(artistsort,%ARTISTSORT%))
$delete(ARTISTSORT)

I do want Picard to rename, organize and tag the files, both FLAC and MP3’s. Worst case scenario, as you said, is that I end up with a few extra tags. The important part is that the musicbrainz tags are being corrected so that Picard can get the most current tag data without me having to go back through all the MP3’s and identify each album version. Thank you for your help.