A multi‐source seeder for digital releases

Confirmed by API, 2/25/2021: Antigua and Barbuda, Armenia, Azerbaijan, Bahamas, Barbados, Belize, Bhutan, Botswana, Burundi, Cambodia, Cameroon, Chad, Comoros, Curaçao, Dominica, Equatorial Guinea, Swaziland (Eswatini), Fiji, Gabon, Gambia, Georgia, Grenada, Guinea, Guinea-Bissau, Guyana, Haiti, Jamaica, Kiribati, Kyrgyzstan, Laos, Lesotho, Liberia, Macao, Malawi, Maldives, Mali, Marshall Islands, Mauritania, Federated States of Micronesia, Mongolia, Namibia, Nauru, Niger, Palau, Papua New Guinea, Rwanda, Samoa, San Marino, Sao Tome and Principe, Senegal, Seychelles, Sierra Leone, Solomon Islands, Saint Kitts and Nevis, Saint Lucia, Saint Vincent and Grenadines, Suriname, Timor-Leste, Togo, Tonga, Trinidad and Tobago, Tuvalu, Vanuatu & Zimbabwe

https://atisket.pulsewidth.org.uk/?preferred_countries=US%2CGB%2CCA%2CMX%2CDE%2CAU%2CJP%2CDK%2CBE&spf_id=4otkd9As6YaxxEkIjXPiZ6

3 Likes

Not sure if this is the right topic to mention this, but the “ISRC submit” option isn’t working for like 3 days now. I keep getting this error:

Capturar

I am regularly checking the Wikipedia article on Spotify and out of 85 announced countries only 8 countries are still missing, they will likely be available by tomorrow. Since I have already prepared the changes for all 85 at once, the most sensible way is to wait for the missing ones instead of pulling the update apart, it just would make it more likely that the list is already deprecated again by the time atj has deployed the update.

By the way, according to my local test server your example release has become “non-worldwide” in the mean time because in addition to the 8 missing countries Belarus (since July 2020), Brunei, Burkina Faso, Nepal and Uzbekistan (since February 2021) are not available (yet).

1 Like

It has been moved to d.ontun.es a few months ago and marlonob’s instance of a-tisket has not received updates for a long time. If you need the ISRC submission tool, I would recommend you to use the a-tisket mirror which is actively maintained and links to tatsumo’s new site.

5 Likes

I knew that’d be a problem. I think on those it depends on what countries they are, and many won’t change as most releases on Spotify are shared with Deezer and Deezer was already available in most of these new countries. But yeah, I’m not going to change a worldwide to a long list if it’s only Cabo Verde, which I never even heard of (and I have a minor in Geography and have studied world maps for 45 years) until this news came out.

4 Likes

@tatsumo announced 3 months ago in the Fetching ISRCs from Spotify thread that they were migrating the code to a new host, and that the old instance would stop working in the future.

Great tool!

I think the ISRC is very much needed, and I understand that there is a development block at the moment. It ties into my recommendation here… add a distinction for explicit and non explicit. For example:


There are two versions of this release:
  1. Explicit, iTunes 1521943207 (tracks explicit: 1,2,3,4,6,7,8,10,11)
  2. Not Explicit, iTunes: 1521946560

There is also a barcode issue that I see, but I will only state the issue vs debating it here. The UPC on the release in MB is 00810045114187, which does not correspond to the UPC codes used in the ISRC database. This is a concern as the UPC is still trying to be used as an identifier, but the usage is inconsistent through channels like it is for physical releases… since it is printed right on the release itself. I think there should be some way to reference the different barcodes used for the same release… otherwise I could very well duplicate the above release three times as there are three different barcodes available for it, all separate from what is already there. Adding both versions, explicit and not, there is a total of 6 barcodes, all differing from the retail stores.

@aerozol - FYI

Here is the data for ISRC on the above release:
1.Your Plug - explicit QMJMT2002929, not explicit QMJMT2002930
2.Why - explicit QMJMT1902576, not explicit QMJMT1902577
3.Careful - explicit QMJMT2002960, not explicit QMJMT2002961
4.Show Me - explicit QMJMT2002933, not explicit QMJMT2002934
5.All I Do Is 4 U - QMJMT2002935
6.More In the Morning - explicit QMJMT2002937, not explicit QMJMT2002938
7.Red Flag - explicit QMJMT2002962, not explicit QMJMT2002963
8.Next Episode - explicit QMJMT2002939, not explicit QMJMT2002940
9.By Your Side - QMJMT2002941
10.Regrets - explicit QMJMT2002964, not explicit QMJMT2002965
11.Love Too Hard - explicit QMJMT2002943, not explicit QMJMT2002944

1 Like

Hello,

Based on the different discussions about Digital Media different barcode makes a different release so you can create the 6.
We can review after if you unsure about some points

PS: Here is a good example of well edited release group with the different digital releases
(except the two versions showing all the different countries for release)

1 Like

That is sort of the point I am raising… I would not consider them different releases (and I am picky on digital releases). Also, I have no idea where the barcode in MB right now even came from. Most of these barcodes are coming from searches and data mining and not from the actual release.

The danger I see here is that the barcode is not really a good identifier for a digital release, yet here, we would have 6 releases with no real way to tell which is which… since those barcodes will never appear in the release. They will be replaced by the vendor barcode. If I add them and you ask me to distinguish between them, I could say that 3 are clean lyrics and the other three are explicit, otherwise, the differentiator is the barcode… a barcode that will never be on a release. In my opinion, that makes the topic sort of useless as the data holds no value to the user.

To clarify that, I would like it, as those barcodes would be a good fit for the methods I use to catalog my music. But I do not use barcodes to identify a specific release, but a general release. I hope that makes sense.

But to the point, I think this tool should have a way to match the barcodes to releases. That is the main intent of my post here, something I think would be nice to this tool in addition to the ISRC listings. The ISRC data is there, but is often hard to properly interpret. Example on my release above, if you simply go to the ISRC lookup tools, you will find those ISRCs, but you will not see a distinction between clean and explicit, and the barcodes will not match the stores, leaving you with no way to distinguish. Obviously there are other ways to get data, like a pull from Spotify API, etc. I personally place very high value on ISRC. If I had to choose only one piece of information to label a digital file with, I would choose ISRC.

3 Likes

I see your points but may be better to discuss it there Digital releases :slight_smile:

1 Like

I agree on the details, that is why I made the last paragraph… I think an addition would be to incorporate the barcodes into this, as the barcodes are all referenced via online sources. That in addition to an upvote on the need for ISRC, which the maker of this tool has already mentioned.

The barcodes with the extra zeroes are actually the correct ones. They are 14-digit GTINs. IMO, they shouldn’t be truncated off, but they are using this tool. These are the barcodes returned by the Spotify APIs & the iTunes barcode file that we now have access too. We must be aware however, that some times certain labels will use a 14-digit code outside of US and then have a 12-digit code for US release on Spotify. But they always have a different ID. iTunes, Spotify & Deezer always use the same code in different countries if they are indeed considered the same release by them. Different labels or variations of their name, are typically associated with different IDs or barcodes as well. I know on physical media that the barocde is printed right on the release. On Spotify & Deezer they are printed in the metadata (Deezer for some reason truncates every single barcode without leading zeroes, even if all other media outlets don’t). Just to add to the confusion :-). ISRC Search always truncates the barcodes without the leading zeroes as well. So, if you copy & paste a barcode from MB (which is taken directly from API or image filename or iTunes barcode file, etc.) just take off the zeroes and it should work. Of course, ISRC Search also is just missing a lot of releases.

2 Likes

I have no idea why you say barcodes aren’t on the release, when on every single iTunes, Spotify & Deezer release they are on the release and never change. If they change, it’s a new release. And yes, if iTunes, Spotify & Deezer all have their own unique barcode for both clean & explicit releases, that is 6 releases.

2 Likes

https://atisket.pulsewidth.org.uk/?preferred_countries=US&itu_id=930345999&upc=886444857697&preferred_vendor=itu&search_itu_countries=si
vs.
https://etc.marlonob.info/atisket/?preferred_countries=us&spf_id=5gUiAoK3N24NEA4DQivROt&deez_id=8924833&itu_id=930345999&upc=886444857697&preferred_vendor=itu&search_itu_countries=si

Why is the mirror removing legitimate countries from the list? Who are we to decide which country is worth listing and which one is not? I think the information is still valid. I know if the only country that a release is unavailable in is British Indian Ocean Territory, that that’s a Worldwide release, so I can see if has a trigger for that exception, but that’s a lot of countries that is just not being counted. What if a release was ONLY in those island nations, would the mirror even recognize that release as being released?

update: Ok. I didn’t mean for it to sound that harsh. I guess what I’m asking is are countries being deleted after being reported from an API. I don’t think they should be. Or is it because of the additional Spotify countries not being fully shown on the count, so it’s saying Worldwide even though it’s not.

update again: Ok. I see both are saying 193 countries. So, should this fix itself when the final Spotify countries up the counts?

@tigerman325: I appreciate your enthusiasm but please don’t jump to conclusions.

No countries are being deleted. @kellnerd made an update which added all the new Spotify territories, and I commented out the 8 which aren’t currently live. If I did not do this then worldwide releases on Spotify would be flagged as having excluded those 8 countries.

We will keep the country list up to date as the new Spotify countries go live.

4 Likes

I don’t see a problem with different barcode = new release. Musicbrainz has all the tools you need to disambiguate them - and if you don’t want to add 6 releases, then don’t, just make the one and add the links from all :slight_smile:

I don’t feel any urge to merge digital releases, I’m fine with there being just as many as we would do with physical.

6 Likes

Yeah. Sorry about that, but you do see why I’d think that. One is saying no exemptions and the other is saying a lot of exemptions. And none of those exemptions were added to the mirror as being available. It’s like they just don’t count anymore, but then when I noticed the 193 countries being the same I remembered the Belarus situation after South Korea became available. So, it’s the count that triggers if all available or not? And if the count says it’s worldwide, it just omits the not available in certain countries part? It just bothers me a little because there will be releases being marked as worldwide that actually aren’t until Spotify finishes with their expansion. But once that’s done, it’ll update the count and this wouldn’t happen? Thanks for the hard work.

The process is:

  1. Get a list of all the countries where Spotify is currently available ($spotify_countries).
  2. Retrieve the list of countries where the release is available from the Spotify API ($release_countries).
  3. Find the difference between the two, e.g. $missing_countries = $spotify_countries - $release_countries.
  4. If $missing_countries is empty then the release is available worldwide, if not then those countries are listed as being excluded.

So it’s not a count, it’s actually comparing the countries.

2 Likes

You are right, atj’s answer describes how we handle this since the latest update, but the old code did indeed only compare the counts first (as a shortcut) and the difference set was only calculated when count($release_countries) was smaller than count($spotify_countries). This lead to the scenario you describe above and which most likely still happens on marlonob’s server in certain cases.

This discrepancy you are seeing there is completely unrelated to the calculation of worldwide availability for Spotify, these are countries which Deezer reports as unavailable. The release events which are reported by the mirror however do not include Deezer results (for whatever reason) as indicated by the heading “Countries where available (iTunes + Spotify)”.
A quick look at the code shows that this occurs when the seeder has no release country data for a specific vendor (Deezer in this case), not sure why this happens. We will try to investigate this soon.

3 Likes