Harmony: Music Metadata Aggregator and MusicBrainz Importer

I’ve also found Deezer sometimes gives the true release date (or at least what seems to be a true release date). I haven’t checked against Mora or 7digital yet to confirm this tho

4 Likes

Yes, Deezer is also usually good about the month-day in when Apple & Spotify only have the year sometimes.

Hi, I plan to change the behavior of external links on the Harmony website.
Currently most of them automatically open in a new tab, (but some do not) and there was never a good reason (like interrupted playback, or loss of state) why this behavior should be enforced on users with the target='_blank' HTML attribute.
After all it is simple to force regular links to open in new tab (with middle click or using a modifier key, depending on your browser) but not easy to force links with target='_blank' to open in the same tab.

Since I am already revisiting this topic for links, I am considering to also change the behavior of the seeder form (aka the import button) which also opens a new tab currently, just like a-tisket did it before.

Here I would like to hear your opinion before I make a call:

After the seeded release editor has opened in a new tab, I …
  • close the Harmony tab
  • reuse the Harmony tab for a new lookup
  • compare the Harmony tab and the seeded release side by side
  • keep the Harmony tab open because I am too lazy to close it
  • have a different use case for it (please comment)
0 voters
The seeded release editor should be opened in…
  • a new tab
  • the same tab
0 voters
3 Likes

I can’t speak for others, but my work flow has had the same browser window open for over a year, with my usual streaming sites to find releases, and atisket. After I finish a release I just go back and do another. Harmony is now one of the tabs that would stay open indefinitely. I like everything in the same spot always exactly where I left :smiley:

4 Likes

Forceful new tabs can be very annoying. If I wanted a new tab I’d do it myself.
In the case of release imports, especially when I add many releases at once these “tabs” are deadweight, take up space, require more steps to finish an import session, methods to force current tab don’t work reliably, etc.

There are more bad reasons to force new tabs than good ones[1]

Perhaps not so well known (anymore), using target='_blank' can also pose a security [2][3] and performance issue.[4]

BTW, ever wondered about the history of target='_blank'?[5]

Fun fact: target='_blank' has been considered user hostile since the 90s[6] (via German blog post with more than 400 comments)

Also consider that a new tab/window has no history → back button doesn’t work or that every new tab/window requires extra resources.
(There was a time I exclusively used a Chromebook with 4GB RAM for a year. Many tabs? Not pleasurable)

If anything there should be a preference, or stay with the default and let the user decide whether navigation triggers a new tab/window.


  1. When to use target="_blank" | CSS-Tricks - CSS-Tricks) ↩︎

  2. The target="_blank" vulnerability by example - DEV Community ↩︎

  3. About rel=noopener ↩︎

  4. The performance benefits of rel=noopener - JakeArchibald.com ↩︎

  5. HTML postmortem: frameset and its legacy - DEV Community ↩︎

  6. The Top 10 Web Design Mistakes of 1999 ↩︎

3 Likes

Cross-posting, since it’s also relevant here. Harmony gives an error message about different GTIN when this happens.

3 Likes

That would be this issue, of which I became aware when I was quickly checking Harmony’s logs yesterday. But thanks for reporting, I can’t always check logs and find such issues myself as they are not always obvious from the logs.

If anyone has insights under which circumstances the Deezer API does this, please let me know. There are (at least) two possibilities:

  • Deezer does not have the release with the requested barcode at all (I hope that this is the case)
  • Deezer suddenly tries to be smart and return the “proper” release for the region its API was queried from (I hope not, that would be the second API after iTunes which becomes unusable in certain cases)
5 Likes

Usually expected behaviour:

  • click: same tab
  • shift+click: new tab
  • ctrl+click: background tab

This might explain some (but not all) of the misbehavior caused by the Deezer API. At least that’s what I thought after testing some of these Deezer releases (each one a valid release that’s different from whatever the API might return). I got different results depending on whether I was manually calling the Deezer API through URL ( api.deezer.com/album/[releaseID] ) or testing them on Harmony and Atisket (not only the mirror on pulsewidth but also marlonob’s original).

For example:

A deluxe release for which the Deezer API returns me the album’s standard edition instead (in terms of release IDs 613896 instead of 1345882).

Another one:

Another deluxe album that both Harmony and Atisket return the standard release instead. This time however, manually calling the Deezer API returns me the correct version: https://api.deezer.com/album/466836945

A more extreme example:

This time I got the correct Deezer release on all tests, except on marlonob’s original instance of Atisket: https://etc.marlonob.info/atisket/?deez_id=1609760&preferred_vendor=spf&no_spf=no&no_itu=no
It’s still the same 12-track album, but a different Deezer release with another UPC and a whole different set of ISRC codes.

Correct me if I’m wrong, but looks like the Deezer API doesn’t have the option of specifying the country/locale for searching a release like some other services do (e.g. market on the Spotify API). If that’s the case, I’m afraid that would render it unusable on such cases like @kellnerd already said.

3 Likes

So, I just went to run this release found on Deezer. https://www.deezer.com/album/2315441. On both a-tisket (pulsewidth) & harmony, it changes the release to https://www.deezer.com/album/1227235. However, on a-tisket (marlonob) it doesn’t do this. It gives me the proper release. I checked the API directly, https://api.deezer.com/album/2315441, and it’s a current release that I can access in the US. I have US set as country code in all 3. https://www.deezer.com/album/1227235 isn’t even available for me in the US. So, not sure why it’s forcing it on me. Even if I put the 2315441 code in the Deezer spot and have the correct barcode, it still changes it.

3 Likes

I was just wondering about if this might be the problem. Marlonob’s a-tisket, I believe, is likely a server in Mexico, where he lives. Pulsewidth has a UK extension. Is it only looking up the releases where the server is located? Because the release in my example above both try to force the GB release whereas Marlons is giving me the MX release. Since it’s open sourced would I be able to run a local version off my laptop for US releases and then from the website for UK releases? Or am I way off base on this?

2 Likes

Yes, that seems to be exactly the issue, see the ticket on GitHub. Deezer has changed the behavior of their API about a week ago (to make it even worse than iTunes since neither GTIN nor ID lookups are reliable now).
So far I didn’t have the time to check if there is any trick to force a specific region to be used for the API.

If there is no solution, I think I will have to change the Deezer provider to return no release (and a warning with the ID/GTIN of the wrong release it returned instead) rather than letting this cause a merge conflict.
Running your own instance for “local” releases would be the simplest workaround then, the pulsewidth server will probably only return Deezer releases which are available in France.

The deluxe vs standard release problem is the same issue, the deluxe version is not available in France:

In France they seem to have their exclusive standard release only:

4 Likes

To add to the fun, querying both the ID 2315441 and 1227235 return the ID 1242277 for me (querying the API from Germany).

3 Likes

Any chance a “copy to clipboard” button could be added to the unavailable country list? I still use atisket for these annotation lists.
I would also love a button to copy all external links, though not sure it would make sense for the average person, I have a userscript that automatically splits them when pasted in the external links field.

4 Likes

Good idea, I thought I had copy buttons on my list already, but apparently not until now:

I think they won’t happen sooner than the region lists being part of the annotation (optionally), but I think they would still be useful to have.

7 Likes

Some minor issues

MB resolving seems to fail if the scheme differs (http vs https) as is the case with Infacted Recordings which still uses http for Beatport

(Ideally these are normalized in the db)

https://harmony.pulsewidth.org.uk/release?url=https%3A%2F%2Fwww.beatport.com%2Frelease%2Fder-maschinist%2F4600510&category=all

https://harmony.pulsewidth.org.uk/release?beatport=4600510&spotify=0jXVDJ8g4PlOaCqDxNkVQ8&gtin=3617388995315&deezer=&itunes=&tidal=&region=GB&ts=1718537351


Post-submission page is missing artist links to other vendors when there is more than one artist it seems (and MB artist links in general):
https://harmony.pulsewidth.org.uk/release/actions?beatport=4570224&spotify=6zClg9wXDH5gRYxplHLZNC&deezer=585995582&itunes=1746031869&tidal=362835466&region=GB&release_mbid=cd5a27d1-73ee-4c1a-b919-a4ccf91874f0

image

https://harmony.pulsewidth.org.uk/release/actions?beatport=4591813&spotify=7aOmoLOodQQfzp4m1Tokg6&deezer=593378262&itunes=1748761552&tidal=365889339&region=GB&release_mbid=1c1f0fc9-0479-4bff-8958-f6a4fe454ef8

Screenshot 2024-07-04 142637


Artist links to Tidal and Deezer are missing for some reason here (although icons are displayed):
https://harmony.pulsewidth.org.uk/release?beatport=4591813&spotify=7aOmoLOodQQfzp4m1Tokg6&gtin=5056686146773&deezer=&itunes=&tidal=&region=GB&ts=1718636419

Also searched page source for 223290835 (Deezer) and 40961876 (Tidal) but no results. I haven’t gone blind, it’s really not there.

image


This release has many track images but none of them seems to be detected. I haven’t seen that feature in action often, is there a threshold?

https://harmony.pulsewidth.org.uk/release?url=https%3A%2F%2Fglis.bandcamp.com%2Falbum%2Fphoenix-2024-deluxe-edition&gtin=&region=US&deezer=&itunes=&spotify=&tidal=&beatport=

https://harmony.pulsewidth.org.uk/release?bandcamp=glis%2Fphoenix-2024-deluxe-edition&ts=1719072535


This problem I’ve mentioned by mail before.
(18. Handle release artists that are track artists more gracefully )

Here is a more extreme example where resolving of track artists failed at first because there are too many releases artists.
I’ve linked most artists to Spotify already but that didn’t help much. It looks like Harmony tries to match the releases artists first:

https://harmony.pulsewidth.org.uk/release?url=https%3A%2F%2Fwww.beatport.com%2Frelease%2Fno1-circuit-djs-compilation-vol-1%2F2160974&gtin=5054283319811&region=GB%2CUS%2CDE%2CJP&spotify=

I guess this falls under “Combine release and track artists which share identifiers or names to avoid inconsistent results in edge cases”?

Edit
After adding a few more Beatport links it finally started resolving the track artists.

1 Like

TL;DR: Many of these issues will be solved by the upcoming MB provider and by Improve artist matching and combining · Issue #54 · kellnerd/harmony · GitHub.

Exactly, I won’t bother to try multiple URLs per provider for a single artist (or label), there are already too many of these failed URL lookups for my taste, especially with the growing amount of providers.
The only maintainable solution is to have as many standardized URLs in MB as possible, so I hope that I will be able to release the artist linking feature on the release actions page soon :crossed_fingers:

URLs with country codes are even more problematic in that regard, as there is no ideal solution for these. Best would be to use the region of the artist in most cases and only use other regions for releases which are not available there.

Yes, the problem is that the artist matching algorithm is way too simple right now, it trips over the artists being in a different order for the different sources :person_facepalming:

So the merge algorithm starts with the Beatport artists here, thinks the artist credits from the other sources are not compatible and only uses the Beatport ID to resolve to an MBID.

That part is working as intended currently, there is little value in trying to resolve external artist IDs again on the release actions page when the artist MBIDs should already be available via the MB release.
In my development branch the new MB provider is used to lookup the MB release and merge it with the other data, so every (matching) artist should have an MBID on the release actions page once that is deployed.

You are right, the alternative artist values (in grey) do not show all external links. Identification of alternative values is a generic feature which does not know that the external IDs/URLs of the artists could be combined, it just groups values (here: artist credits) which are considered the same and the displayed value is in fact the first one of a group (here: Spotify artist credit).
That is why the second alternative artist credit value has a hyperlinked Spotify logo on the left of each artist. Deezer’s and Tidal’s artist credit are considered the same (in a text representation), so their icons are also displayed to the right of the whole artist credit.
Admittedly this is suboptimal here, I will see if it is possible to address this without too much overhead.

(In my original solution alternative artist credits did not even display the hyperlinked icons because it felt a bit weird to display the same icon left and right of an artist in many cases. But without them it is sometimes impossible to see why artist credits are different, for example Apple would also have the same artist credit text representation as Spotify/Deezer/Tidal, but it is only one artist entity instead of two separate entities.)

The feature is currently only used for pre-releases, for which the Bandcamp provider already has to do the extra request which contains the track image data anyway. For releases which have already been released, the extra request is skipped as it usually contains no additional data which is relevant.

Some metadata providers already have options on how much data they should return, which they use to decide how many extra requests are necessary. This feature is not exposed so far, but I plan to use it to further optimize lookups and to give the user options to lookup additional data. Bandcamp’s track images would be one such case.

Yeah, I should really address this Beatport oddity. I had hoped someone who actually uses Beatport would come up with a clever solution, but I guess I will simply copy the threshold of four release artists from the userscript for now…

Exactly, I already have code to detect duplicate artists for the “Link with MusicBrainz” feature, I guess it can also be used here.

3 Likes

This Bandcamp URL Tone Science Module No.9 Theories and Conjectures (DiN:TS09) | Various | DiN returns errors.

No provider returned a release

Bandcamp:
Failed to extract embedded ‘tralbum’ JSON: Tone Science Module No.9 Theories and Conjectures (DiN:TS09) | Various | DiN

Not sure if the issue is with Harmony or Bandcamp.

Regards,
Gary.

1 Like

Thanks for reporting. The Bandcamp description of your example contains an unexpected, invisible Unicode line separator (U+2028) which broke data extraction. Should be fixed now.

5 Likes

Tremendous. Thanks for the speedy fix. Added the release.