How is listenbrainz failing to identity the correct recording?

Levi_OP · May 30, 2023, 5:29pm

I use foobar2000 and the foo_listenbrainz2 plugin to track listens. Unfortunately, foobar2000 cannot read the ID3 UFID tag, which is where picard stores the recording url. foo_listenbrainz2 sends album id, artist id, track id, etc. (but can’t send the recording id) yet my listens are still having to be seemingly “guessed” by listenbrainz based only on the title and album name.

For example, here are the listen of Dead Presidents II, which should have been identified as this recording, but was identified as this, instead, a clean version from a different release.

Listen details

{
  "track_metadata": {
    "track_name": "Dead Presidents II (new lyrics)",
    "artist_name": "JAY‐Z",
    "release_name": "Reasonable Doubt",
    "additional_info": {
      "date": "1999-01-26",
      "duration": 266,
      "discnumber": "1",
      "totaldiscs": "1",
      "track_mbid": "f7c70bc5-418c-3a2c-b3f6-3aa23489a554",
      "albumartist": "JAY‐Z",
      "totaltracks": "15",
      "tracknumber": "4",
      "artist_mbids": [
        "f82bcf78-5b69-4622-a5ef-73800768d9ac"
      ],
      "media_player": "foobar2000",
      "release_mbid": "37c0a6b5-f723-4e70-92b8-805a569bc436",
      "submission_client": "foobar2000",
      "release_group_mbid": "a7f852ba-08bc-36f1-92f3-4dc127f4b70a",
      "media_player_version": "1.6.9",
      "submission_client_version": "1.3.2",
      "recording_msid": "5b9199d7-dc10-45fc-a38f-7ff40279bba0"
    },
    "mbid_mapping": {
      "recording_mbid": "2808921a-3018-4d53-b331-b7f97eeb5ffb",
      "recording_name": "Dead Presidents II (new lyrics)",
      "release_mbid": "3c4633aa-043e-4327-9900-43613691c135",
      "artists": [
        {
          "artist_mbid": "f82bcf78-5b69-4622-a5ef-73800768d9ac",
          "artist_credit_name": "JAY‐Z",
          "join_phrase": ""
        }
      ],
      "artist_mbids": [
        "f82bcf78-5b69-4622-a5ef-73800768d9ac"
      ],
      "caa_id": 1585893879,
      "caa_release_mbid": "e9a5f123-447f-4430-8efb-0921e1761ddf"
    }
  },
  "listened_at": 1685391339,
  "recording_msid": "5b9199d7-dc10-45fc-a38f-7ff40279bba0",
  "user_name": "Levi_OP",
  "inserted_at": 1685391340
}

It seems as though listenbrainz should be able to ascertain the recording id simply from the track id, but fails to. Is there something I am doing wrong, or does listenbrainz just need to be improved to look at things like the track ID and more accurately get the recording. I imagine I could make a little script that went through all of my listens and checked if the identified recording id didn’t match the recording id from track, but I’d like to not have to do this.

Thanks, Levi.

lucifer · May 31, 2023, 10:18am

Hi! We currently don’t utilise track_mbid or release_mbid or artist_mbids to guide the automatic identification. We might do that in future but for now its only track name and artist name.

But you can manually link the listen to a different recording using the ListenBrainz UI if needed. It should be possible to create a script to the same automatically I guess.

rob · June 1, 2023, 2:29pm

Hi!

I think everyone, including the team, is a bit unsure about exactly how we and handling MBIDs – it is clearly time that we document what we currently have, review that and decide if this is how we wish to continue it and in the end document the whole thing and link it from the web interface.

This appears to be growing in need, so I’ll see about getting these docs written sometime next week.

Thanks!

aerozol · June 1, 2023, 11:45pm

+1 for eventually using track_mbid, when it’s available! It’s strange on LB to see albums mainly attributed correctly, but with different covers (from singles, compilations etc) peppered throughout.

o3 · June 2, 2023, 3:41am

Hi, sorry to be annoying again.

I was originally going to post the following wall of text in its own post titled ‘Mapping issues related to the ‘Link with MusicBrainz’ function’, but then real-life got in the way and I forgot to post it. Then I saw this thread, and it appears to be relevant to my post, so I figured that I could just post here. Is that OK?

The post itself:

The following issues were observed over the course of a few months. Initially, I had planned on waiting for them to be fixed as they aren’t really critical bugs. But as weeks turned into months, I thought, “Maybe the ListenBrainz team isn’t aware of these issues because no one has reported them?” So I figured that I would.

I apologize for the length of this post, but the issues are numerous and I wanted to be as helpful as possible by providing as much detail as possible. I ask in advance for your patience.

Prefaces:

All of the example songs used in this post, unless otherwise noted, are by TWICE, the K-pop girl group.
The listens that demonstrate the following issues are available on my ListenBrainz profile (but the profile is otherwise empty because the issues were discovered on another profile).
I don’t think that I found any problems with re-mapping the Artist category when using the ‘Link with MusicBrainz’ function. The problems I found lie with the Album category.
I wrote the bulk of this post right before the ListenBrainz redesign went live, but all of these issues are still present in the redesign.
The (meta)data for my listens is sourced from Melon, the #1 Korean music streaming service. (It has the biggest market share.)

Issue #1: the mapped album is wrong

Example: ‘KNOCK KNOCK’ is a song that was released on the re-release/repackage album ‘TWICEcoaster : LANE 2’. Originally, my ‘KNOCK KNOCK’ listens weren’t mapped to anything, so I used the ‘Link with MusicBrainz’ option to do it manually. But for some reason, that caused my ‘KNOCK KNOCK’ listens to be mapped to the live/concert album ‘TWICELAND - THE OPENING’. On both the ‘Listens’ page and ‘Stats’ page, listens for ‘KNOCK KNOCK’ all wrongly show the album as ‘TWICELAND - THE OPENING’. <ListenBrainz profile link>
- Side note: Hovering over the cover art for ‘TWICELAND - THE OPENING’ on the ‘Listens’ page produces a tooltip with the correct album name, ‘TWICEcoaster : LANE 2’, which confused me at first. But after some experimentation by hovering on other listens, I discovered that that’s just ListenBrainz displaying the the album/release data that I submitted, as-is. Not sure how helpful this is, but I figure that it’s worth a mention, just in case it’s relevant.
Closing remarks: This is the biggest issue in this list, for sure. I have numerous more listens/tracks that are affected by this bug, and it’s caused my ‘Stats → Top Albums of This Week/This Month/etc.’ pages to become filled with albums that I’ve never listened to. If further examples are needed, then I can provide a more exhaustive list of affected listens/tracks of mine.

Issue #2: the track is mapped to the original album, when it is only available on the repackage

Examples: My listens for the songs ‘Heart Shaker’ and ‘Merry & Happy’ originally were not mapped to anything. When I used the ‘Link with MusicBrainz’ option to do so manually, ListenBrainz then proceeded to categorize under the album ‘twicetagram’ (October 2017). This is problematic because these two tracks were only released on the repackage of said album released two months later, titled ‘Merry & Happy’ (December 2017). The ‘Inspect listen’ function shows that the submitted album data is ‘Merry & Happy’ (December 2017), so I’m not sure why it’s getting mapped to ‘twicetagram’ (October 2017). <ListenBrainz profile link>
How it should be: the track ‘STAY BY MY SIDE’, which similarly was first released on a repackage album, is correctly filed under the album ‘BDZ -Repackage-’ (December 2018), instead of being wrongly mapped to the original album ‘BDZ’ (September 2018). <ListenBrainz profile link>
- Side note: Please don’t be thrown off by the fact that the wrong cover art is displayed next to the listen for ‘STAY BY MY SIDE’; if you click on the cover art, then you are taken to the Release page for ‘BDZ -Repackage-’, which shows that the listen is linked to the correct Release. The cover art being wrong is a separate issue, which I describe below as issue #3…

Issue #3: listen shows wrong cover art, despite being linked to the correct Release

Explanation: Sometimes, a listen will have the wrong cover art displayed to the left of it, despite the listen pointing to the correct Recording ID and the correct Release ID with the correct cover art.
Example: On my ‘Listens’ page, listens for ‘STAY BY MY SIDE’ show the cover art for the ‘BDZ’ album instead of the correct one: the cover art for ‘BDZ -Repackage-’. Strangely enough though, on the ‘Stats’ page, the correct cover art is shown, so the problem appears to only affect the ‘Listens’ page?

Issue #4: the track is mapped to the album that it was first released on, but it’s not the album that I listened to

Explanation: A certain track may have originally been released on the album that ListenBrainz mapped it to, but ‘Inspect listen’ shows that I listened to the track on a repackaged version of that album. It would be preferable if ListenBrainz were to file the listens under the repackage album, not the original album.
Example: ‘LIKEY’ and ‘LOVE LINE’ were originally released on the album ‘twicetagram’ (October 2017), but they are also on the repackage album ‘Merry & Happy’ (December 2017). But even though ‘Inspect listen’ shows that I listened to them on the album ‘Merry & Happy’ (December 2017) , ListenBrainz still files my listens under ‘twicetagram’ (October 2017). I would much appreciate it if ListenBrainz filed them under the correct album and displayed the ‘Merry & Happy’ cover art instead of the ‘twicetagram’ cover art. <ListenBrainz profile link>
Addendum: At first, I just thought to myself, “Huh, that’s mildly annoying, but I guess ListenBrainz just always ties the tracks to the albums that they were originally released in, even if you listen to that track on a different album.” and that would have been the end of it. But then I discovered that that’s not how it work for other tracks in similar situations, as seen in the following issue #5…

Issue #5: the mapped album is correct (correct Release Group), but it’s not the version of the album that I listened to (wrong Release)

Explanation: This sounds similar to issue #4, but it’s a somewhat different issue. What makes this different is that ListenBrainz correctly ignores the original album that the track was originally released on, and maps the track to a subsequent album that’s in a different Release Group. So far so good. The problem, however, is that ListenBrainz fails to go a step further and map it to the correct Release in that Release Group, which in my case was a repackage album.
Example: ‘Candy Pop’ was originally released on the single album ‘Candy Pop’ (February 2018), and subsequently included on the album ‘BDZ’ (September 2018) and its repackage album ‘BDZ -Repackage-’ (December 2018). According to the supposed logic that I laid out in issue #4, when I listen to the song ‘Candy Pop’ on the album ‘BDZ -Repackage-’ (December 2018), my ‘Candy Pop’ listens should be categorized under the album that it was originally released in, i.e. the single album ‘Candy Pop’ (February 2018), right? But that’s not what happens. ListenBrainz half-correctly categorizes it under the album ‘BDZ’ (September 2018). I say “half-correctly” because I listened to it on ‘BDZ -Repackage-’ (December 2018), not ‘BDZ’ (September 2018). Both are in the same Release Group, but they are still separate Releases with different track counts and cover arts’. <ListenBrainz profile link>
How it should be: already mentioned in issue #2 but also applicable here, the track ‘STAY BY MY SIDE’ is correctly filed under the album ‘BDZ -Repackage-’ (December 2018) and not the original ‘BDZ’ (September 2018). So it doesn’t seem like ListenBrainz is incapable of distinguishing between specific Releases inside a Release Group or anything like that.

That’s it for the mapping issues specific to albums, but I found a few more issues having to do with the ‘Link with MusicBrainz’ function that I feel aren’t completely irrelevant to the topic, so I will include them here.

Issue #6: unclear as to why some tracks are automatically mapped successfully, while similarly-named sister tracks aren’t

Example: This is a screenshot of my ‘Listens’ page, before I performed any manual linking with ‘Link with MusicBrainz’ (taken a day or so before the redesign went live):

As you can see in the screenshot above, out of eight songs by TWICE, five were mapped automatically, while three weren’t. And I don’t understand why, as all eight had the same Artist field “TWICE (트와이스)”, the same Album Artist field “TWICE (트와이스)”, and so on. What do the three not-automatically-mapped tracks have in common that is throwing off ListenBrainz’s automatic mapper?

Issue #7: no way to un-link wrongly mapped listen if it’s a song that’s not on MusicBrainz

Explanation: When a listen gets wrongly mapped, it’s simple enough to use the ‘Link with MusicBrainz’ function to point it to the correct Recording ID. However, that only works when the data for the song exists on MusicBrainz. The problem: if it’s a song for which data is not avaiable on MusicBrainz, then there is no way to un-link the listen from the wrong Recording ID.

Issue #8: no way to separate listens for tracks that have the same track title, artist, and album, but different duration/length

Example:
- Track 1.01 야야야 and track 1.08 야야야 (remix) are songs released by the artist Baby V.O.X together on the same album.
- However, the music service that I use, the ‘(remix)’ bit is absent from the title of track 1.08. So both track 1.01 and track 1.08 end up being titled ‘야야야’. This causes ListenBrainz to file listens for both tracks under one Recording ID, the one for track 1.01. I tried to manually link my ‘야야야 (remix)’ listens to the recording ID for track 1.08, but that ended up also changing my listens for the regular ‘야야야’ to ‘야야야 (remix)’. In short, it’s impossible to affect one without the other.
Possible solution: use the “duration_ms” tag in the submitted listening data (available for viewing with the ‘Inspect listen’ option) to recognize that these are not the same song, and allow them to be linked to separate Recording IDs.

Issue #9: matching doesn’t work when Artist field doesn’t start with Latin script

Explanation: When the Artist field is in a format such as ‘English name (Korean name)’ (like ‘TWICE (트와이스)’), ListenBrainz is for the most part successful at matching listens to Recording IDs. But when the Artist field is entirely in Korean, or is in a format such as ‘Korean name (English name)’, then ListenBrainz fails to map the listen, despite the data for the track existing on MusicBrainz.
Example: ‘After School’, or ‘애프터스쿨’, is a K-pop girl group that is listed on Musicbrainz. Now, were I to submit a listen with the Artist field set as ‘After School’ or ‘After School (애프터스쿨)’, then ListenBrainz would successfully map it. However, Melon, the #1 Korean music streaming service, has their artist name set as ‘애프터스쿨’ with no English (cf. ‘TWICE (트와이스)’). This causes ListenBrainz to fail to map the listen, with the listen (wrongly) remaining grayed-out as missing data.
More examples: Songs by the following artists also fail to map properly due to this issue: ‘베이비복스’, ‘오마이걸 (OH MY GIRL)’, ‘에이프릴 (APRIL)’. As you can see, they all share a commonality, which is that either there is no Latin script or the Latin script is in parentheses.
How it should be: As ‘애프터스쿨’ is already listed as an alias for ‘After School’ on their MusicBrainz artist page, ListenBrainz should recognize that it’s an alias. Then, it should substitute in ‘After School’ for ‘애프터스쿨’, and search for matching tracks that are listed under that artist name as well, instead of searching just the one and failing to find any matches.

Again, apologizes for the long post.

Edit: attach screenshot that I had forgotten to attach

RandomMushroom128 · June 2, 2023, 11:28pm

Hello, i just got linked this thread to post a related issue to.
Another issue that could potentially be solved if listenbrainz starts using track_mbid’s in the future is scrobbling different recordings with the same name in the same release.

Kero Kero Bonito - Intro Bonito is an album i listened to today and it has two songs titled “Cat vs. Dog”, an english and a japanese version.
Another release which came to mind is notnow000 - , having 8 songs titled “-”

VBZPPlNQyJ · October 27, 2023, 2:42pm

I have noticed a related odd matching between scrobbled data, recording and release.
I listened to the track https://musicbrainz.org/track/984bf78e-bd7f-3ae2-9151-4258d48e11f8 from release https://musicbrainz.org/release/41134b9a-049e-415d-b621-4af4a9b3ea71 and scrobbled it through Pano Scrobbler (android). Listenbrainz did not match it automatically. This was unsurprising.

Using the web interface, I manually linked the listen to the recording Recording “Piano Sonata in E minor, op. 7: I. Allegro moderato” by Leif Ove Andsnes - MusicBrainz. This is the only appearance of that recording in MusicBrainz.

However, ListenBrainz now displays the wrong album art.
Inspecting the listen, it appears that ListenBrainz has mapped the listen to the correct recording but the wrong release. This recording doesn’t even appear on the appear on the release that is linked (https://musicbrainz.org/release/6e4d622d-01e1-4b20-959e-91674ccf7343).

{
  "track_metadata": {
    "track_name": "Piano Sonata in E minor, op. 7: I. Allegro moderato",
    "artist_name": "Edvard Grieg",
    "release_name": "Piano Concerto / Sonata Op. 7 / Lyric Pieces Opp. 43, 54 & 65",
    "additional_info": {
      "duration_ms": 264000,
      "submission_client": "Pano Scrobbler",
      "submission_client_version": "3.6 - 2023, Aug 16",
      "recording_msid": "6002a93c-b7b1-4a8a-8065-684a4b56e2fa"
    },
    "mbid_mapping": {
      "recording_mbid": "cce4fdfe-390d-44d5-b4cb-6c93685b1b1d",
      "recording_name": "Piano Sonata in E minor, op. 7: I. Allegro moderato",
      "release_mbid": "6e4d622d-01e1-4b20-959e-91674ccf7343",
      "artists": [
        {
          "artist_mbid": "2c094b77-1268-4609-ba95-7545c2c5eb62",
          "artist_credit_name": "Leif Ove Andsnes",
          "join_phrase": ""
        }
      ],
      "artist_mbids": [
        "2c094b77-1268-4609-ba95-7545c2c5eb62"
      ],
      "caa_id": 27868074266,
      "caa_release_mbid": "6e4d622d-01e1-4b20-959e-91674ccf7343"
    }
  },
  "listened_at": 1698411991,
  "recording_msid": "6002a93c-b7b1-4a8a-8065-684a4b56e2fa",
  "user_name": "VBZPPlNQyJ",
  "inserted_at": 1698412123
}

gnurfling · November 1, 2023, 8:43am

This is the most significant issue I have with ListenBrainz.

I need a way to link a listen to the triple (album_mbid, disc number, track number), which shoud be equivalent to track_mbid. Everything else can be deduced from that.

Otherweise the data is going to be incorrect in subtle ways quite frequently.

VBZPPlNQyJ · November 1, 2023, 9:29pm

Or just use track_mbid if provided manually or by the scrobbler. Presumably, it would even reduce the computational load required to match tracks if it could just default to the corresponding track when track_mbid is provided.

VikingSchism · November 12, 2023, 11:41pm

I mean now I’m seeing a lot of songs not automatically matching even when they have exact matches for artist name and track title. Which is like… the whole point of the automapper.

For examples:

If we were able to use submitted track ID then it would solve issues like this. I only need to make query the musicbrainz db once to get the track ids and then can go on without needing to worry about micromanaging listenbrainz to get the data sort of correct (only “sort of” because of any issues with release that using recording id can produce)

outsidecontext · November 13, 2023, 7:36am

Submitted listens can include the recording ID, and if it is present it will be used for the matching.

VikingSchism · November 13, 2023, 9:29pm

Yes I know this, but this is still subject to a lot of error (regarding releases mainly - many recordings appear across multiple releases. Track ID is unique to specific releases so would not be subject to the same error)

I also cannot submit recording ID correctly currently as my music server of choice currently does not correctly implement it (waiting for the next release for it to be fixed - ironically it currently uses the track ID where recording ID should be)

VBZPPlNQyJ · February 8, 2024, 3:05pm

As @VikingSchism says, this is a good start but breaks down quickly if a recording appears on more than one release. I find that this happens frequently with the music I listen to - mostly classical for what it’s worth. It may be less of a problem in other genres.

Nicknickcdf · March 16, 2024, 5:42pm

It’s bitten me already (a month or so in of light use) with Frank Zappa, the king of regurgitating tunes on many albums. Also live albums of rock bands, e.g. AC/DC.
What I don’t understand is that all these submissions of mine have included the album title to, but this appears to simply be ignored. None of my music has MBIDs and I’m not precious about which pressing the track came from, so the data would have to be a little approximate. At present i guess a decision was taken to ignore parsing album metadata in absence of an MBID? Why was that?

boysmithers · April 3, 2024, 3:43pm

I wouldn’t mind so much if you could correct them manually… it’s important for me to know my top albums of the week, month, year etc. Listenbrainz is much better than Last.FM for handling artists, but can’t do albums so well, unfortunately…
I’d love to bin off Last completely, but we’re not quite there yet…

boysmithers · April 15, 2024, 2:15pm

[Replying to myself, but others may be interested]

It turns out that if you DELETE a listen and then manually add it, then you can link to the correct version!

This is going to be very useful to me as I’m listening to a lot of Grateful Dead albums - they have a huge discography mainly comprised of live albums, so hundreds of different versions of the same songs. A lot of deleting and manually adding, but I’m so pleased to be able to do it!