A multi‐source seeder for digital releases

Small bug: A Spotify release without cover art still has link which just goes to the same page:

Screenshot 2021-05-10 at 18-08-29 Tai Sondeho Kori by RH Ovi a‐tisket A multi‐source seeder for MusicBrainz

The Spotify release actually doesn’t have cover art on Spotify though. Tai Sondeho Kori - Single by RH Ovi | Spotify

FYI, when I open the complimentary link to the iTunes art, the https://qsniyg.github.io/maxurl/ userscript reloads a bigger picture/url :slight_smile:

Any idea why The Holy Bible 20 (Deluxe) - Album by Manic Street Preachers | Spotify is showing a blank list of countries? Spotify says it’s available everywhere.

3 Likes

I tried excluding Spotify to see if it was an issue with their API, but the Deezer and iTunes APIs aren’t returning any countries either.
I tried a different Spotify release to see if this was an API issue, and I got countries then. Based on my own past experiences, this almost certainly means that Manic Street Preachers album is no longer available to stream anywhere. However, a cursory look at the listing doesn’t show any greyed-out titles, so I’m not sure what’s going on. Maybe Spotify has a feature that allows certain artists and labels to block their music from being embedded or queried by third parties? :confused:

1 Like

There was a period after its initial release where the album wasn’t available on Spotify (or at least not for free users). I’ve checked some other releases either by the band or previously premium-only and the countries are showing up fine for those. Since Deezer and iTunes are also affected, I’m assuming the problem is on a-tisket’s end.

1 Like

This is a problem for releases with more than 50 tracks which is caused by Deezer’s API and the non-existing rate limiting of a-tisket. Basically it tries to query the country availability for each track (there is no release level availability) and Deezer does not let us do more than 50 requests per 5 seconds.
On marlonob’s server this is not a notable problem because it takes more than 5 seconds to process the release but since atj’s mirror is faster it runs into this kind of problems. You can confirm this by simply ignoring Deezer.

I have to admit that I had already noticed this issue and prepared a (dirty) fix in March but I did not have the time to write a cleaner solution. Since I did not have this problem again and no one has complained so far, it fell out of my focus.
The structure of the code makes it hard to spot why it is completely failing to return any countries in that case, but querying only the first 50 tracks on Deezer solves the problem. We still avoid doing heavy refactoring of the code (like introducing proper rate limiting) as long as it is still running, and meanwhile @atj has been working on a more easily maintainable replacement written from scratch.
I will have another look…

7 Likes

Apologies for the lack of updates on my instance for the last month or two - I’ve been busy with other things.

I’ve now deployed a fix for the Deezer API ratelimiting problem, so releases with 50+ tracks should show the correct country availability, however the page may take a while to load when the ratelimit is hit (e.g. 10-15 seconds). Once again many thanks to @kellnerd for his help on this.

Also, after some discussion on IRC, the country availability info is no longer included in the release annotation by default.

The next issue I plan on looking at is the change from iTunes to Apple Music.

8 Likes

If you do change to Apple Music on the links, please change to “streaming page” instead of “purchase for download”. The whole purchase at Apple Music annoys me, because you can’t purchase at Apple Music, only stream. Also, I know I was the one that first pointed out the Deezer availability list, but since those have been being removed it’s caused some issues. On any release dealing with Deezer, I’ve been having to run on the original. This is because the mirror removes China & Curacao from Deezer releases causing true Worldwide releases to no longer show as Worldwide, marlonob’s site shows these 2 countries almost every time it comes up that these 2 countries are excluded on the mirror.

You couldn’t make purchases at the old iTunes pages either; it was always only a preview.
I do agree with changing them to “streaming page” by default, though. So many releases I’ve imported recently have been streaming-only, either in all territories or all except one (in the case of Japanese idol music).

1 Like

There appears to be more streaming only new releases than there are purchase only new releases, at least. There will always be exceptions. I see your point in that it’s always been a link to open the iTunes software for purchases, so technically, it’s the same. I guess it’s just always been in my mind that iTunes is for purchases and Apple Music is for streaming.

1 Like

Oh, and just as a reminder

1 Like

Could you give me a specific example of this so I can investigate. I’ve seen Curacao listed as excluded once or twice but never China, and I use Deezer as the primary source 99.9% of the time.

1 Like

Here’s the last release I ran across it:

2016 - 2018 by LiL Lotus | a‐tisket: A multi‐source seeder for MusicBrainz (pulsewidth.org.uk)

vs.

2016 - 2018 by LiL Lotus | a‐tisket: A multi‐source seeder for MusicBrainz (marlonob.info)

1 Like

Release where Curacao, but so small not as big of deal
Release “Pray 4 Love (deluxe)” by Rod Wave - MusicBrainz.

1 Like

And the very next release I just added, amazingly, was an example of both China & Curacao, which is the most common I run across:
Release “Prove It on Me” by Rory Block - MusicBrainz

1 Like

@atj Would you consider adding Bandcamp support? They provide all album details as an LD+JSON, including barcode, image link and track list. So from a data point of view it should be easy to add. I don’t know how easy it is to extend this tool with a new provider, though :slight_smile: Is the source available somewhere?

5 Likes

Is there any copyright restriction to use in MB the images found by the tool?

This is something I’m going to attempt over this week. I can get the TIDAL credits for an album at the moment, maybe can combine/check against Spotify credits.

I have no clue how to handle inserting these relationships into MB though, as there only seems to be a release seeding support: Development / Release Editor Seeding - MusicBrainz

1 Like

Seems TIDAL (right) has much better credits data

2 Likes