A multi‐source seeder for digital releases

@tatsumo announced 3 months ago in the Fetching ISRCs from Spotify thread that they were migrating the code to a new host, and that the old instance would stop working in the future.

Great tool!

I think the ISRC is very much needed, and I understand that there is a development block at the moment. It ties into my recommendation here… add a distinction for explicit and non explicit. For example:


There are two versions of this release:
  1. Explicit, iTunes 1521943207 (tracks explicit: 1,2,3,4,6,7,8,10,11)
  2. Not Explicit, iTunes: 1521946560

There is also a barcode issue that I see, but I will only state the issue vs debating it here. The UPC on the release in MB is 00810045114187, which does not correspond to the UPC codes used in the ISRC database. This is a concern as the UPC is still trying to be used as an identifier, but the usage is inconsistent through channels like it is for physical releases… since it is printed right on the release itself. I think there should be some way to reference the different barcodes used for the same release… otherwise I could very well duplicate the above release three times as there are three different barcodes available for it, all separate from what is already there. Adding both versions, explicit and not, there is a total of 6 barcodes, all differing from the retail stores.

@aerozol - FYI

Here is the data for ISRC on the above release:
1.Your Plug - explicit QMJMT2002929, not explicit QMJMT2002930
2.Why - explicit QMJMT1902576, not explicit QMJMT1902577
3.Careful - explicit QMJMT2002960, not explicit QMJMT2002961
4.Show Me - explicit QMJMT2002933, not explicit QMJMT2002934
5.All I Do Is 4 U - QMJMT2002935
6.More In the Morning - explicit QMJMT2002937, not explicit QMJMT2002938
7.Red Flag - explicit QMJMT2002962, not explicit QMJMT2002963
8.Next Episode - explicit QMJMT2002939, not explicit QMJMT2002940
9.By Your Side - QMJMT2002941
10.Regrets - explicit QMJMT2002964, not explicit QMJMT2002965
11.Love Too Hard - explicit QMJMT2002943, not explicit QMJMT2002944

1 Like

Hello,

Based on the different discussions about Digital Media different barcode makes a different release so you can create the 6.
We can review after if you unsure about some points

PS: Here is a good example of well edited release group with the different digital releases
(except the two versions showing all the different countries for release)

1 Like

That is sort of the point I am raising… I would not consider them different releases (and I am picky on digital releases). Also, I have no idea where the barcode in MB right now even came from. Most of these barcodes are coming from searches and data mining and not from the actual release.

The danger I see here is that the barcode is not really a good identifier for a digital release, yet here, we would have 6 releases with no real way to tell which is which… since those barcodes will never appear in the release. They will be replaced by the vendor barcode. If I add them and you ask me to distinguish between them, I could say that 3 are clean lyrics and the other three are explicit, otherwise, the differentiator is the barcode… a barcode that will never be on a release. In my opinion, that makes the topic sort of useless as the data holds no value to the user.

To clarify that, I would like it, as those barcodes would be a good fit for the methods I use to catalog my music. But I do not use barcodes to identify a specific release, but a general release. I hope that makes sense.

But to the point, I think this tool should have a way to match the barcodes to releases. That is the main intent of my post here, something I think would be nice to this tool in addition to the ISRC listings. The ISRC data is there, but is often hard to properly interpret. Example on my release above, if you simply go to the ISRC lookup tools, you will find those ISRCs, but you will not see a distinction between clean and explicit, and the barcodes will not match the stores, leaving you with no way to distinguish. Obviously there are other ways to get data, like a pull from Spotify API, etc. I personally place very high value on ISRC. If I had to choose only one piece of information to label a digital file with, I would choose ISRC.

3 Likes

I see your points but may be better to discuss it there Digital releases :slight_smile:

1 Like

I agree on the details, that is why I made the last paragraph… I think an addition would be to incorporate the barcodes into this, as the barcodes are all referenced via online sources. That in addition to an upvote on the need for ISRC, which the maker of this tool has already mentioned.

The barcodes with the extra zeroes are actually the correct ones. They are 14-digit GTINs. IMO, they shouldn’t be truncated off, but they are using this tool. These are the barcodes returned by the Spotify APIs & the iTunes barcode file that we now have access too. We must be aware however, that some times certain labels will use a 14-digit code outside of US and then have a 12-digit code for US release on Spotify. But they always have a different ID. iTunes, Spotify & Deezer always use the same code in different countries if they are indeed considered the same release by them. Different labels or variations of their name, are typically associated with different IDs or barcodes as well. I know on physical media that the barocde is printed right on the release. On Spotify & Deezer they are printed in the metadata (Deezer for some reason truncates every single barcode without leading zeroes, even if all other media outlets don’t). Just to add to the confusion :-). ISRC Search always truncates the barcodes without the leading zeroes as well. So, if you copy & paste a barcode from MB (which is taken directly from API or image filename or iTunes barcode file, etc.) just take off the zeroes and it should work. Of course, ISRC Search also is just missing a lot of releases.

2 Likes

I have no idea why you say barcodes aren’t on the release, when on every single iTunes, Spotify & Deezer release they are on the release and never change. If they change, it’s a new release. And yes, if iTunes, Spotify & Deezer all have their own unique barcode for both clean & explicit releases, that is 6 releases.

2 Likes

https://atisket.pulsewidth.org.uk/?preferred_countries=US&itu_id=930345999&upc=886444857697&preferred_vendor=itu&search_itu_countries=si
vs.
https://etc.marlonob.info/atisket/?preferred_countries=us&spf_id=5gUiAoK3N24NEA4DQivROt&deez_id=8924833&itu_id=930345999&upc=886444857697&preferred_vendor=itu&search_itu_countries=si

Why is the mirror removing legitimate countries from the list? Who are we to decide which country is worth listing and which one is not? I think the information is still valid. I know if the only country that a release is unavailable in is British Indian Ocean Territory, that that’s a Worldwide release, so I can see if has a trigger for that exception, but that’s a lot of countries that is just not being counted. What if a release was ONLY in those island nations, would the mirror even recognize that release as being released?

update: Ok. I didn’t mean for it to sound that harsh. I guess what I’m asking is are countries being deleted after being reported from an API. I don’t think they should be. Or is it because of the additional Spotify countries not being fully shown on the count, so it’s saying Worldwide even though it’s not.

update again: Ok. I see both are saying 193 countries. So, should this fix itself when the final Spotify countries up the counts?

@tigerman325: I appreciate your enthusiasm but please don’t jump to conclusions.

No countries are being deleted. @kellnerd made an update which added all the new Spotify territories, and I commented out the 8 which aren’t currently live. If I did not do this then worldwide releases on Spotify would be flagged as having excluded those 8 countries.

We will keep the country list up to date as the new Spotify countries go live.

4 Likes

I don’t see a problem with different barcode = new release. Musicbrainz has all the tools you need to disambiguate them - and if you don’t want to add 6 releases, then don’t, just make the one and add the links from all :slight_smile:

I don’t feel any urge to merge digital releases, I’m fine with there being just as many as we would do with physical.

6 Likes

Yeah. Sorry about that, but you do see why I’d think that. One is saying no exemptions and the other is saying a lot of exemptions. And none of those exemptions were added to the mirror as being available. It’s like they just don’t count anymore, but then when I noticed the 193 countries being the same I remembered the Belarus situation after South Korea became available. So, it’s the count that triggers if all available or not? And if the count says it’s worldwide, it just omits the not available in certain countries part? It just bothers me a little because there will be releases being marked as worldwide that actually aren’t until Spotify finishes with their expansion. But once that’s done, it’ll update the count and this wouldn’t happen? Thanks for the hard work.

The process is:

  1. Get a list of all the countries where Spotify is currently available ($spotify_countries).
  2. Retrieve the list of countries where the release is available from the Spotify API ($release_countries).
  3. Find the difference between the two, e.g. $missing_countries = $spotify_countries - $release_countries.
  4. If $missing_countries is empty then the release is available worldwide, if not then those countries are listed as being excluded.

So it’s not a count, it’s actually comparing the countries.

2 Likes

You are right, atj’s answer describes how we handle this since the latest update, but the old code did indeed only compare the counts first (as a shortcut) and the difference set was only calculated when count($release_countries) was smaller than count($spotify_countries). This lead to the scenario you describe above and which most likely still happens on marlonob’s server in certain cases.

This discrepancy you are seeing there is completely unrelated to the calculation of worldwide availability for Spotify, these are countries which Deezer reports as unavailable. The release events which are reported by the mirror however do not include Deezer results (for whatever reason) as indicated by the heading “Countries where available (iTunes + Spotify)”.
A quick look at the code shows that this occurs when the seeder has no release country data for a specific vendor (Deezer in this case), not sure why this happens. We will try to investigate this soon.

3 Likes

By the way, recently I also discovered why a-tisket does not set the release event to just [Worldwide] in some cases although there are no excluded countries listed at all. This happens e.g. if a release is not worldwide on Spotify but worldwide on Deezer, in a way that the missing Spotify countries are available on Deezer.
However, the code checks whether all queried vendors are worldwide, not if there are no excluded countries.

As a first step I changed the “Only a few countries have been excluded from the release” message (which is confusing when there are no excluded countries listed below) to “Only a few countries have been excluded from the release (by at least one vendor)”.
I think we should also change the behavior and set the release event to [Worldwide] in these cases nevertheless, what do you think?

6 Likes

I can address this straight away.
810026072734
810026072710
810026072697
810026072727
810026072703
810026072680

Maybe you can find a release that actually has these barcodes in the metadata?

EDIT: To clarify, those are 6 legit barcodes for the release I mentioned, both the clean and explicit versions, and broken down by the ISRC that I listed… Which releases are these tied to?

EDIT2: I should also clarify I have no interest in indexing streaming releases. Please do take my statements in relation to actual ownership based releases. For the streaming, you can have a barcode for every radio station or whatever, to me it is all the same.

Just because this is absolutely not clear for me from your previous posts: would you consider this just one release in MB, or 2 (explicit vs. cleaned) or 6 (one per barcode)?

2 Likes

Deception Szn 1 by Angelica Vila | a‐tisket: A multi‐source seeder for MusicBrainz (pulsewidth.org.uk)

First code isn’t on iTunes, Spotify or Deezer, so no result was found. ISRC reports barcodes for other services as well and even other medium types. Second code is seeded above, to give new release that I just added for you:
Release “Deception Szn 1” by Angelica Vila - MusicBrainz

Not sure what else to add to this. I know other than the barcode, the releases are basically the same, but the barcode is enough to make it a different release, just like it would be on a physical release. If editors say a pressing code is enough on a physical release to be considered a different release, I’m really not sure why a barcode wouldn’t be enough on a digital one.

3 Likes

Also, just because a release is streaming doesn’t mean that it can’t also be purchased. If the only difference is streaming vs. purchasing but everything else is the same, it’s still the same release, just on a different service.

3 Likes