Matching accuracy and Apple iTunes Match

Can you send me the file and a screenshot? I would open it in Kid3 on macOS and send you a screenshot in turn.

Sure, although maybe you could share a screenshot of where you’re seeing it, first, so I know we’re looking at the same thing?

I’ve highlighted the relevant iTunes-specific tags in grey. This is the release of album “Songs of Innocence” that was free to download for all iTunes users for a while in 2014.

Using the Album ID tag can serve the goal of avoiding incorrect data. That is a proposition I want to test here. If you know for certain that you obtained a music file through iTunes, then Album ID gets you the data to identify (or verify) the correct MB recording to link its fingerprint with.

Just because the fingerprint of your file already exists on AcoustID servers doesn’t always mean it has been linked to the correct release (by “release” I mean the specific release of an album). I might want to trust that others have put in enough scrutiny, but then I also like to verify against facts.

I am trying this workflow to get at “facts” (which is difficult because the source media is not physical but I would like to consider that Apple knows very well where it gets its data from):

Step 1 In iTunes, select a song obtained through iTunes Store and show its location in Finder.

Step 2 Open the file in Kid3.

Step 3 Copy the Album ID number and look up the album in a web browser:
https://music.apple.com/album/[Album ID]

(Don’t use the “Show in iTunes Store” link in iTunes - it is totally unreliable)

At this point you might be satisfied, look up the album in MB and pull it into Picard.

But how would you select among the releases of the album? What about the release country?

I am not sure how to infer it. If there is only one worldwide iTunes music catalogue, as @sibilant said, then the digital distribution through iTunes would classify as Worldwide I guess. It is a worldwide release.

The question is, can you find out from which release Apple obtained the digital files that it provides?

There is a country tag in the JSON. But there is no documentation what it stands for.

Step 4 Notice that the URL in Step 3 redirects to
https://music.apple.com/album/[collectionID]

Step 5 Take the collectionID and use it with https://etc.marlonob.info/atisket/ to get a UPC.

Step 6 Use the UPC to look up the album:

https://itunes.apple.com/lookup?upc=[UPC]

https://www.discogs.com/search/?type=all&barcode=[UPC]

https://musicbrainz.org/search?query=barcode%3A[UPC]&type=release&limit=25&method=advanced

The UPC narrows the possible releases. That’s good.


Let’s try:

Step 1 I chose “Heartbreak Hotel (Hex Hector RIP Mix)” by “Whitney Houston feat. Faith Evans & Kelly Price” (Discogs r7073939). At least that is what I think the correct metadata is at this point.

Step 2
Album ID = 251090652.

Step 3
https://music.apple.com/album/251090652

So, this tells us already that my song in iTunes has the wrong metadata. The recording supplied by iTunes is not from “Heartbreak Hotel (Dance Vault Mixes)” but from “My Love is Your Love”. There is even a third album by Whitney Houston in the iTunes Store with this recording.

The album page does not reveal any country information. So I looked up http://itunes.apple.com/lookup?id=13952&entity=album and see that country":"USA but I don’t know if that JSON key means “release country”.

Step 4
collectionID = 251090652

Step 5
UPC = 078221903721

@marlonob’s a-tisket tool also reveals that country = United Kingdom (gb).
Hm, where does this information come from? It is not part of the JSON result from the iTunes Store.

The problem is, the UPC is not tied to a United Kingdom release! Check it out:

Step 6

https://www.discogs.com/search/?type=all&barcode=078221903721
Discogs has 4 releases for that UPC, all of which are US releases.

https://musicbrainz.org/search?query=barcode%3A078221903721&type=release&limit=25&method=advanced

MusicBrainz has 2 releases, one US, and one DE.

What do you make of that?
Something is wrong.

When I pick the US release on MB, I am looking at track 2 which has a duration of 4:41.

My version has a duration of 3:39.

The fingerprint of my m4a version is presently linked to this AcoustID. As you can see there, nobody so far has linked this recording to album “My Love is Your Love”.

This feels like a dilemma!

If I trust the iTunes Album ID then people before me have probably just followed into the footsteps of people before them, propagating an incorrect link between AcoustID and MB recording.

If I trust MB Picard, then I should match the song against a completely different artist: C.C.Catch.

Wow. As you may guess, by now I am confused. I would really appreciate your comments on either of these problems:

  • The release country
  • The UPC problem
  • Picard suggesting a recording on a different album by a different artist
  • Nobody so far having linked this “official” iTunes Plus version and its fingerprint to a corresponding “My Love is Your Love” album on MB.

I would say the best thing is to use the release country that a-tisket suggests. Its method of determining whether or not a release is “[Worldwide]” is the result of a lot of thought and discussion. My personal feeling is that it is a reasonable formula; it’s what I would have chosen, too. When you use this tool, you know that the releases have the same tracks, UPC, and cover art for every region.

This code has the same meaning as the country code in the URLs:

https://itunes.apple.com/**gb**/album/id1140681685
https://music.apple.com/**fr**/album/1442466683

The UPC does not change by region. If an album in a certain region has a different UPC, we consider that to be a different release. Fortunately, you don’t have to think about that. Just use the album ID and you’ll be fine. :slight_smile:

Before you paste an iTunes or Apple Music URL into a-tisket, you may want to change the country code. If the URL has no country code and you don’t tell a-tisket which country to use, it will select from its own priority list, which is not tailored to the release.

My personal practice is to use the code of the country that seems to be the primary market for the release. Sometimes that’s the artist’s home country; other times it’s the publisher’s. If it’s a “various artists” collection, I use the label’s country. If it isn’t clear what the label’s country is, I use the country code from the ISRCs.

1 Like

Ah. For the sake of apples-to-apples, I downloaded the same album (which is considered one of my “purchased” albums). It does indeed have all the iTunes specific tags, as in your screenshot. But my “matched” tracks don’t have those.

My m4a file (the iTunes Plus version) has a duration of 3:39. The track 2 on My Love is Your Love has a duration of 4:41. So, the Album ID number provided by Apple is wrong.

Have you listened to the iTunes M4A file? What’s in it?

I’ll keep documenting my experience using Album ID in the list below, with a rating.

Rating = 1: Album ID is wrong or misleading
Rating = 2: Album ID is helpful
Rating = 3: Straightforward process from Album ID to UPC to selecting a release with confidence

Documentation of using Album ID

Whitney Houston feat. Faith Evans & Kelly Price
Heartbreak Hotel
Heartbreak Hotel (Hex Hector RIP Mix)
251090652, 1
Wrong Album ID (duration mismatch)

Yusef Lateef
The Gentle Giant
The Poor Fisherman
734113, 1
URL doesn’t redirect, a-tisket returns strange release year 1987 (link), UPC only found at mora.jp.

Ghazal
The Rain
[all tracks]
18310487, 3
Matched, though barcode (UPC) 044006662725 is missing

Willie Bobo
Verve//Remixed
Spanish Grease (Dorfmeister con Madrid de los Austrias Muga Reserva mix)
2091079, 1
Wrong Album ID (duration mismatch)

Astrud Gilberto
Verve//Remixed
Who Needs Forever? (Thievery Corporation remix)
4183088, 1
URL does not redirect

Dinah Washington
Verve//Remixed
Is You Is or Is You Ain’t My Baby? (Rae & Christian remix)
417693, 3
Matched

Nina Simone
Verve//Remixed
Feelin’ Good (Joe Claussell remix)
714237726, 1
URL does not redirect

Ella Fitzgerald
Verve//Remixed
Wait 'Till You See Him (De-Phazz remix)
83348264, 1
URL does not redirect

Nina Simone
Verve//Remixed
See-Line Woman (Masters at Work remix)
79835, 1
URL does not redirect

Billie Holiday
Verve//Remixed
Strange Fruit (Tricky remix)
1288388, 1
URL does not redirect

Sarah Vaughan
Verve//Remixed2
Whatever Lola Wants (Gotan Project remix)
2237453, 3
Matched

Ella Fitzgerald
Verve//Remixed2
Angel Eyes (Layo & Bushwacka remix)
2237453, 3
Matched

Hugh Masekela
Verve//Remixed2
Mama (Metro Area Birthday dub)
2237453, 3
Matched

Astrud Gilberto
Verve//Remixed2
Here’s That Rainy Day (Koop remix)
4183088, 1
URL does not redirect

Nina Simone
Verve//Remixed2
Black Is the Color of My True Love’s Hair (Jaffa remix)
2237453, 3
Matched

Yes, I am very thorough :face_with_monocle: :slight_smile:

I used Shazam with this and this result.

I verified audibly against this and this version on music.youtube.com.

I checked whether the song might have been trimmed or is missing the end. It is not.

I am pretty sure it is the R.I.P Radio Edit. Hex Hector’s name is erroneously listed in the song name as others have noted.

Wow, so all these years later, that is still a problem. Supposedly, Apple Music used to overly rely on file tags for identification, whereas iTunes Match incorporated acoustic fingerprinting. Both methods sometimes returned a different version of a song. I don’t think either service uses metadata-only matching any longer, but your experience shows that it still can’t distinguish between different versions of a song.

I don’t know how much consideration iTunes Match gives to file tags, but that’s the easiest place to begin troubleshooting. If the match failed even though the original file had complete, accurate tags, here are two options:

  1. Complain to customer service. In the past, they have given vouchers to customers so that they can buy the track.
  2. Submit a different copy of the track, and hope that it matches properly.

I came across 2 albums (15 tracks) this afternoon, which are not given the Album ID and Artist ID tags, after having downloaded the iTunes Plus versions. So ya, maybe I was too fast to think this can be depended on.

I have not experienced failed matching so far - in the sense that an iTunes Plus version was clearly different from the one it replaced.

The upgrade is a free service, but still, it is a matter of accountability (of business ethics) to reveal which album was used. Image you’re the company offering this matching service. Would you hide this information from your customer? I don’t like this about iTunes Match.

It is like offering someone a bottle of wine without a label on it.

When I started with this whole review and upgrade process, I did sometimes trigger a re-match if I wanted to try my luck on songs with iCloud status “Uploaded”, following this tip by Jeff V. But it takes time, and most of the time a song got uploaded again, not matched.

Yeah, the service has some problems, and lack of transparency is one of them. I’m disappointed that this didn’t turn out to be a reliable process. It doesn’t make sense to me that they would choose the wrong track and also add the wrong metadata. If they chose the track from their library, they have the track ID. It’s just bad.

I used to do that, too, and sometimes it was successful. In this case, I was suggesting using a recording from a different source, if you have one—such as a re-rip from the same CD. Try feeding them the radio edit; maybe they’ll give you the album version. :laughing:

Anyway, the outcomes can be hard to predict. Sometimes it accurately matches corrupt MP3s, but is confused by high-quality recordings from official sources. Oftentimes it accepts every track but one from a perfect digital media release.

My plan is to run a filter over my entire iTunes library in Kid3 (found under Tools > Filter…) in order to see which tracks have the Album ID field set in the MP4 tag. With 891 tracks upgraded to 256 kbps m4a files so far, I could then see which percentage of those have this tag set.

I am in the process of figuring out (issue 527), how the expression for this filter should look like - something like this:

not %2{Album ID} equals ""

I don’t know what the field name for “Album ID” is to use in the curly braces. I tried different things:
albumid
ALBUMID
@albumid
albid

etc.

Does anyone know where to look this up? A resource like this mapping table?

Kid3 has its own mapping table at https://docs.kde.org/trunk5/en/extragear-multimedia/kid3/commands.html#table-frame-list

But it does not list Album ID, and I actually don’t know how Apple stores this in their files. If you’d install the mutagen Python library it comes with a command line tool called mutagen-inspect which you could run on the file and it would output all tags with their technical name. With that we could figure out how it is stored. It would even make sense to add support for this to Picard (as “iTunes Album ID” or such).

2 Likes

The internal name for Album ID is plID. Urs Fleisch explained how to find it in the Kid3 user interface.

With the expression not %2{plID} equals "" then applied as a filter, I have 732 tracks with an Album ID set. That indicates that this tag gets applied more often than not. @psychoadept

I haven’t been able to find any files that didn’t come from iTunes originally that have those tags. Since I’m coming from a non-iTunes library, only using iTunes for matching, that’s not terribly surprising. Do you know for sure that it’s not just retaining the tags they already had, when you do the match?

Yes, for sure. the Album ID and Artist ID tags were not already there before replacement with a matched file in iCloud Music Library.

To check, in Kid3 I exported the filtered track list as a playlist and imported that playlist into iTunes. Then I compared it to a playlist containing all the upgraded tracks (the 256 kbps m4a aka “iTunes Plus” files). There are only 19 of the 732 tracks that are not also in the “upgraded” playlist, and that’s because they are already in iTunes Plus format. With other words, “all” of them are in the “upgraded” playlist.

Of the 892 tracks in the upgraded playlist, 179 didn’t receive the Album ID and Artist ID tags.

This is enough proof that these tags get added by Apple when a download is requested.

That’s good – just need to see how reliably these can be used to infer the source album.

I might have discovered something – I am going to describe what I did first, and then say what I think it means, regarding if and when Apple adds the Album ID and Artist ID tags:

I have several tracks from “When a Man Loves a Woman”. I matched them via Picard with this release. Some of the tracks have iCloud status “Matched”. Rather than downloading m4a versions to replace them with, I first created copies of the audio files and imported them into iTunes.

In Tunes, I waited for the imported set to become matched again with iCloud Music Library.

Now I had two sets of matched tracks - the original set and the imported set.

Then I replaced the imported set by downloading the m4a versions. Then I checked with Kid3 whether Album ID and Artist ID tags are present. They are not.

Then I replaced the original set by downloading the m4a versions. This time, Album ID and Artist ID tags present.

So, this is what I think this means:

I signed up for the iTunes Match service 11 months ago. This means my library was scanned 11 month ago and the “iCloud Status” information on all of the tracks is from that time.

When I replace one of the tracks that were matched 11 months ago, Album ID and Artist ID are added. When I replace any track that was matched today (or more recently), that does not happen.

That means the scheme (for lack of a better word) that Apple uses to describe the link between a customer’s local copy and a track in iCloud Music Library has changed. They are no longer writing Album ID and Artist ID into the files.

However, the information that instructs what to write into the files, does still exist. So, when I request a download today for a track that was matched some time ago, the information from some time ago is used and that’s why I get Album ID and Artist ID for this track.

I will continue to test this theory with the tracks I am going to process next.