Matching accuracy and Apple iTunes Match

Alex1 · July 14, 2020, 7:34pm

The release that MusicBrainz Picard selects after scanning a file is based on the file metadata and the AcoustID. I wish you could have MusicBrainz Picard look for files based on the audio fingerprinting alone. You can set the matching thresholds in Option, but unfortunately you cannot turn that metadata matching off completely.

Apple’s iTunes Match service has been available for a while, but I am only now looking through my library, replacing lower-quality songs for so-called “iTunes Plus” AAC at 256 kbps files.

The problem is the in-transparency. Apple does not reveal which release the recording in iCloud Music Library is from. It is like a black box. So, I am left to obtain the recording and then audio fingerprint it. The metadata from your local copy gets applied to the recording obtained from Apple. And because Picard uses this metadata, I cannot get a “pure” match-making, based on AcoustID alone.

I hope it became clear what I mean. I think it is sad that Apple does not let you see the release it uses to match your local copy with.

I am wondering: How to search for releases based on AcoustID alone?

To give an example: When I have a digital recording that was taken from vinyl, I will certainly NOT replace that with an iTunes Plus version, because the vinyl version is precious. I’d rather want to have both versions, just for the sake of collecting. So, I would create a copy of my version, then replace that for the iTunes Plus version, then check with MusicBrainz Picard to determine which release Apple might have used. But because Picard mixes the metadata of my local copy into its match-making algorithm, the recording might actually be from a different album… this means there is too much ambiguity.

What do you think?

sibilant · July 14, 2020, 8:57pm

Even if you ignore all other metadata, an AcoustID won’t necessarily reveal the precise release of the track. I haven’t used iTunes Match in a few years, but if Apple still writes a track ID to the files, it may be possible to use this to identify the source album.

See my post about this here: Number of tracks in database
You might find this interesting, too: iTunes UPC Database

However, MusicBrainz and AcoustID do give you a way to find some of the possible albums. After you use Picard to match a track by AcoustID:

Select the matched track in Picard, then click “Lookup in Browser”.
In the MusicBrainz web page that opens, click the “Fingerprints” tab. (Example)
Click the fingerprint of your track (found in the AcoustID frame of your file’s tags).
If more than one album is associated with this AcoustID (or another one that has been judged to be essentially the same), the albums are listed under “Additional user-submitted metadata”. (Example)

psychoadept · July 14, 2020, 9:15pm

Yeah, I really wish they did this. I’ll see if the iTunes UPC database is any help

outsidecontext · July 14, 2020, 10:47pm

If you use the scan function in Picard this really is primarily an AcoustID search. Existing metadata is only used to select the best match if multiple recordings are linked with the same AcoustId.

But as others have written above this won’t help you to find the exact release if there are multiple editions. If it is the same recording it should get the same AcoustId.

Vinyl and other analog media can be tricky, as the exact start time and length of a digitalized track could vary. Also people probably don’t submit too many fingerprints for Vinyl rips. But anyway, if your Vinyl rips match the digital tracks close enough in start time and length they will probably get the same AcoustId.

Alex1 · July 15, 2020, 3:36pm

Thank you for leading me on. Let me split this topic in two.

First, reagarding Picard and AcoustID:

My workflow is usually like this:

Drag a song from iTunes to Picard, fingerprint it, discern among the results.
Replace the song (or a copy of it) for an iTunes Plus version and repeat step 1.

Most of the time, Picard will assign the iTunes Plus version to the same recording. (I have processed around 1200 songs so far in this manner, album by album.) This led me to think that the matching in Picard is somehow biased, because I think it is likely that iTunes matches songs to other (newer) releases - whatever they have available that matches the fingerprint of my local copy.

This is what makes the upgrade to an iTunes Plus version so risky. It might be the same recording, but it might be from a different album! That is only acceptable, if I can verify the album/release myself – thanks to MusicBrainz.

To avoid bias in Picard, I thought it might be a good idea to turn off preferred release types in Options, matching thresholds in Options and the file metadata itself from Picard’s search.

I see now that this not such a good idea. I would still have to discern, as @outsidecontext clarified.

To resolve my suspicion of “bias”, I look up the MB recording page to see which other possibilities there are. I will now also use the Fingerprints tab, as @sibilant pointed out.

Second, regarding iTunes Match:

Using the Kid3 audio tagger on macOS, I see that the m4a iTunes Plus files have an “Arstid ID” and “Album ID” tag. Neither of these tags are revealed in Picard.

With “Artist ID” I can do a https://itunes.apple.com/lookup?id=18310489&entity=album query.

I put the result through https://jsonformatter.curiousconcept.com

I take collectionId from the result and use that in the “iTunes album ID” field at https://etc.marlonob.info/atisket/.

OK, that’s a way to get data, but does not serve to determine the album, which iTunes uses to supply an iTunes Plus file.

My hope lies with the “Album ID” tag. There is a lookup URL for AMG Album ID, but that is not what “Album ID” represents, as the results will show you.

Does anyone know what it represents? If it is an iTunes ID, why on earth not let us use it? Please Apple.

outsidecontext · July 15, 2020, 4:17pm

Yes, there usually should be a separate release for the e.g. Vinyl and iTunes version on MB. But AcoustID itself is not suited for finding the correct release. Let’s assume you have both the files from the Vinyl rip and the iTunes download. Both versions will have essentially the same existing metadata (same album, artist and title tags). If we also assume fingerprints for both versions will result in the same AcoustID it becomes clear that Picard will choose the same release in both cases.

Normally if you have found the correct release with your search already the other versions of the same release (Vinyl, CD, iTunes etc.) will be in the same release group. So one good way to check is to right click on the release in Picard (right pane) and choose the proper versions via the “Other versions” submenu.

I don’t know much about iTunes match, though.

sibilant · July 15, 2020, 5:58pm

Using the Kid3 audio tagger on macOS, I see that the m4a iTunes Plus files have an “Arstid ID” and “Album ID” tag. Neither of these tags are revealed in Picard.

Can you post one of the album IDs you copied from Kid3?

Alex1 · July 15, 2020, 6:42pm

For testing, I am using the album “The Rain” by Ghazal (MB release group), which I obtained some time before 2004 as a digital version. It has 3 tracks.

Today I obtained the iTunes Plus versions. For the 3 tracks, Kid3 reveals:

Album ID: 18310487
Artist ID: 18310489

Using the Artist ID number in the following lookup URL reveals that the iTunes Store has one “The Rain” album, next to 9 other albums where the artist is involved:

itunes.apple.com/lookup?id=18310489&entity=album

The same artist id number can be retrieved by dragging the artist name from iTunes Store into a plain text file, which will give you this link:

music.apple.com/de/artist/ghazal/18310489?l=en

The iTunes Store link to the album is this:

music.apple.com/de/album/the-rain/1442466683?l=en

The number 1442466683 however is the collectionId.

If the Album ID tag would have this collectionId, we could delight in looking up which album was used by Apple to supply the iTunes Plus version.

sibilant · July 15, 2020, 8:07pm

Well, I think I have good news for you. If you put the album ID into the following iTunes URL, it will redirect you to the album in Apple Music (in this case, to collectionID 1442466683):

So, you should now have a way to determine the precise release that each track belongs to, and you can use the collectionID with a-tisket to submit any missing albums to MusicBrainz.

Alex1 · July 15, 2020, 9:10pm

Woah! I am glad you tried this. You made my day!!

The redirect also works with these URL variants:
https://music.apple.com/album/18310487
https://music.apple.com/de/album/18310487

Over the next days, I will use this in my workflow.

What if the scope of titles in the iCloud Music Library is not the same as the scope of titles in the iTunes Store? If iCloud Music Library is larger, I might get a title without an album id? We’ll see if that case occurs…

Tomorrow I will return with The Gentle Giant and The Rain - both albums contain oddities. Good for learning I think.

Thank you for helping to make this iTunes Plus upgrade and subsequent matching process more precise!

sibilant · July 15, 2020, 10:04pm

What if the scope of titles in the iCloud Music Library is not the same as the scope of titles in the iTunes Store? If iCloud Music Library is larger, I might get a title without an album id?

I think you are safe: All of Apple’s services use the same music catalogue. Of course, certain albums and tracks are only available to customers in certain regions, but the catalogue itself is unified.

psychoadept · July 16, 2020, 5:18am

Is it possible this is a mac-only function? I tried putting one of my m4a files (just downloaded) in Kid3 and can’t see any ID tags. (I’m on Windows, just to be clear.)

Alex1 · July 16, 2020, 1:33pm

Can you send me the file and a screenshot? I would open it in Kid3 on macOS and send you a screenshot in turn.

psychoadept · July 16, 2020, 4:15pm

Sure, although maybe you could share a screenshot of where you’re seeing it, first, so I know we’re looking at the same thing?

Alex1 · July 16, 2020, 5:08pm

I’ve highlighted the relevant iTunes-specific tags in grey. This is the release of album “Songs of Innocence” that was free to download for all iTunes users for a while in 2014.

Alex1 · July 16, 2020, 8:23pm

Using the Album ID tag can serve the goal of avoiding incorrect data. That is a proposition I want to test here. If you know for certain that you obtained a music file through iTunes, then Album ID gets you the data to identify (or verify) the correct MB recording to link its fingerprint with.

Just because the fingerprint of your file already exists on AcoustID servers doesn’t always mean it has been linked to the correct release (by “release” I mean the specific release of an album). I might want to trust that others have put in enough scrutiny, but then I also like to verify against facts.

I am trying this workflow to get at “facts” (which is difficult because the source media is not physical but I would like to consider that Apple knows very well where it gets its data from):

Step 1 In iTunes, select a song obtained through iTunes Store and show its location in Finder.

Step 2 Open the file in Kid3.

Step 3 Copy the Album ID number and look up the album in a web browser:
https://music.apple.com/album/[Album ID]

(Don’t use the “Show in iTunes Store” link in iTunes - it is totally unreliable)

At this point you might be satisfied, look up the album in MB and pull it into Picard.

But how would you select among the releases of the album? What about the release country?

I am not sure how to infer it. If there is only one worldwide iTunes music catalogue, as @sibilant said, then the digital distribution through iTunes would classify as Worldwide I guess. It is a worldwide release.

The question is, can you find out from which release Apple obtained the digital files that it provides?

There is a country tag in the JSON. But there is no documentation what it stands for.

Step 4 Notice that the URL in Step 3 redirects to
https://music.apple.com/album/[collectionID]

Step 5 Take the collectionID and use it with https://etc.marlonob.info/atisket/ to get a UPC.

Step 6 Use the UPC to look up the album:

https://itunes.apple.com/lookup?upc=[UPC]

https://www.discogs.com/search/?type=all&barcode=[UPC]

https://musicbrainz.org/search?query=barcode%3A[UPC]&type=release&limit=25&method=advanced

The UPC narrows the possible releases. That’s good.

Let’s try:

Step 1 I chose “Heartbreak Hotel (Hex Hector RIP Mix)” by “Whitney Houston feat. Faith Evans & Kelly Price” (Discogs r7073939). At least that is what I think the correct metadata is at this point.

Step 2
Album ID = 251090652.

Step 3
https://music.apple.com/album/251090652

So, this tells us already that my song in iTunes has the wrong metadata. The recording supplied by iTunes is not from “Heartbreak Hotel (Dance Vault Mixes)” but from “My Love is Your Love”. There is even a third album by Whitney Houston in the iTunes Store with this recording.

The album page does not reveal any country information. So I looked up http://itunes.apple.com/lookup?id=13952&entity=album and see that country":"USA but I don’t know if that JSON key means “release country”.

Step 4
collectionID = 251090652

Step 5
UPC = 078221903721

@marlonob’s a-tisket tool also reveals that country = United Kingdom (gb).
Hm, where does this information come from? It is not part of the JSON result from the iTunes Store.

The problem is, the UPC is not tied to a United Kingdom release! Check it out:

Step 6

https://www.discogs.com/search/?type=all&barcode=078221903721
Discogs has 4 releases for that UPC, all of which are US releases.

https://musicbrainz.org/search?query=barcode%3A078221903721&type=release&limit=25&method=advanced

MusicBrainz has 2 releases, one US, and one DE.

What do you make of that?
Something is wrong.

When I pick the US release on MB, I am looking at track 2 which has a duration of 4:41.

My version has a duration of 3:39.

The fingerprint of my m4a version is presently linked to this AcoustID. As you can see there, nobody so far has linked this recording to album “My Love is Your Love”.

This feels like a dilemma!

If I trust the iTunes Album ID then people before me have probably just followed into the footsteps of people before them, propagating an incorrect link between AcoustID and MB recording.

If I trust MB Picard, then I should match the song against a completely different artist: C.C.Catch.

Wow. As you may guess, by now I am confused. I would really appreciate your comments on either of these problems:

The release country
The UPC problem
Picard suggesting a recording on a different album by a different artist
Nobody so far having linked this “official” iTunes Plus version and its fingerprint to a corresponding “My Love is Your Love” album on MB.

sibilant · July 17, 2020, 12:37am

I would say the best thing is to use the release country that a-tisket suggests. Its method of determining whether or not a release is “[Worldwide]” is the result of a lot of thought and discussion. My personal feeling is that it is a reasonable formula; it’s what I would have chosen, too. When you use this tool, you know that the releases have the same tracks, UPC, and cover art for every region.

This code has the same meaning as the country code in the URLs:

https://itunes.apple.com/**gb**/album/id1140681685
https://music.apple.com/**fr**/album/1442466683

The UPC does not change by region. If an album in a certain region has a different UPC, we consider that to be a different release. Fortunately, you don’t have to think about that. Just use the album ID and you’ll be fine.

Before you paste an iTunes or Apple Music URL into a-tisket, you may want to change the country code. If the URL has no country code and you don’t tell a-tisket which country to use, it will select from its own priority list, which is not tailored to the release.

My personal practice is to use the code of the country that seems to be the primary market for the release. Sometimes that’s the artist’s home country; other times it’s the publisher’s. If it’s a “various artists” collection, I use the label’s country. If it isn’t clear what the label’s country is, I use the country code from the ISRCs.

psychoadept · July 17, 2020, 2:30am

Ah. For the sake of apples-to-apples, I downloaded the same album (which is considered one of my “purchased” albums). It does indeed have all the iTunes specific tags, as in your screenshot. But my “matched” tracks don’t have those.

Alex1 · July 17, 2020, 4:00pm

My m4a file (the iTunes Plus version) has a duration of 3:39. The track 2 on My Love is Your Love has a duration of 4:41. So, the Album ID number provided by Apple is wrong.

sibilant · July 17, 2020, 4:51pm

Have you listened to the iTunes M4A file? What’s in it?