A multi‐source seeder for digital releases

Tags: #<Tag:0x00007f739d6b6288> #<Tag:0x00007f739d6b6198> #<Tag:0x00007f739d6b60d0> #<Tag:0x00007f739d6b6008>

I’ve just deployed these fixes to the live instance. Thanks @kellnerd :slight_smile:

4 Likes

Deezer still not always showing up in country list.
https://atisket.pulsewidth.org.uk/?preferred_countries=US%2CGB%2CCA%2CMX%2CDE%2CAU%2CJP%2CDK%2CBE&spf_id=3rWJsuu7ukoZZhp7YYkjNZ&upc=731454681520&preferred_vendor=itu&search_itu_countries=si

vs.

https://etc.marlonob.info/atisket/?preferred_countries=us&spf_id=3rWJsuu7ukoZZhp7YYkjNZ&deez_id=249678&itu_id=1440912389&upc=731454681520&preferred_vendor=itu&search_itu_countries=si

It appears to be the same, but what confused me is for some reason on the mirror, it says Countries where available (iTunes + Spotify) even though it adds the Deezer link at top., whereas on the original it’s correct as Countries where available (iTunes + Deezer + Spotify). Country list is the same.

Also, on above example the dates are still screwy on the mirror suggestions. Deezer date (correct date): 2007–11-22 vs. 2007-11-22. If you try to copy & paste the suggested date into the override it doesn’t work because it’s using a “m” dash, instead of hyphen.

I think this is a HTML5 validation issue, so it should be easy to fix.

Not sure about the Deezer country list issue. I’ll investigate - I just wish it wasn’t so painful! :expressionless:

2 Likes

OK, I’ve had a look into why Deezer is not showing up in the country availability lists. The track API endpoint is returning an empty list for the available_countries attribute:

$ curl https://api.deezer.com/track/2548816 |jq .available_countries
[]

This doesn’t make much sense as the track is playable for me in the web interface and values are returned when querying other tracks:

$ curl https://api.deezer.com/track/1121653642 |jq .available_countries
[
  "AE",
  "AF",
[..]
]

Unfortunately there’s nothing I can do about this - it’s seems to be a Deezer API issue with certain releases :man_shrugging:

I’ve just pushed an update that fixes this once and for all!

2 Likes

But if it’s on Deezer’s end, why is marlonob’s site showing it, but the mirror isn’t? I guess that’s all I was pointing out.

1 Like

Or is it actually not getting a result, but showing like it is because it found the link?

1 Like

New Markets

Angola, Antigua and Barbuda, Armenia, Azerbaijan, Bahamas, Bangladesh, Barbados, Belize, Benin, Bhutan, Botswana, Brunei Darussalam, Burkina Faso, Burundi, Cabo Verde, Cambodia, Cameroon, Chad, Comoros, Côte d’Ivoire, Curaçao, Djibouti, Dominica, Equatorial Guinea, Eswatini, Fiji, Gabon, Gambia, Georgia, Ghana, Grenada, Guinea, Guinea-Bissau, Guyana, Haiti, Jamaica, Kenya, Kiribati, Kyrgyzstan, Lao People’s Democratic Republic, Lesotho, Liberia, Macau, Madagascar, Malawi, Maldives, Mali, Marshall Islands, Mauritania, Mauritius, Micronesia, Mongolia, Mozambique, Namibia, Nauru, Nepal, Niger, Nigeria, Pakistan, Palau, Papua New Guinea, Rwanda, Samoa, San Marino, Sao Tome and Principe, Senegal, Seychelles, Sierra Leone, Solomon Islands, Sri Lanka, St. Kitts and Nevis, St. Lucia, St. Vincent and the Grenadines, Suriname, Tanzania, Timor-Leste, Togo, Tonga, Trinidad and Tobago, Tuvalu, Uganda, Uzbekistan, Vanuatu, Zambia, and Zimbabwe.

6 Likes

As I see it there are 2 possibilities:

  1. Deezer is returning an empty available_countries attribute to requests only from certain IPs or countries
  2. marlonob’s instance is not interpreting the empty country list correctly

I think option 2 is much more likely than 1. You can help to confirm this is if you want by visiting the Deezer track API and telling me if it returns an empty array for available_countries.

1 Like

@kellnerd has already submitted a PR to update the list of Spotify countries. We’re going to wait until Spotify is confirmed as live in those countries before deploying the update.

Your link iswas incomplete, but when I visit https://api.deezer.com/track/2548816 from your previous post, it indeed shows an empty array for available_countries. The issue seems to be that some tracks (including track 1 = the above example) are only available for Deezer premium users:

Not sure if there is anything we can do about this, even ignoring all tracks with an empty available_countries array during the calculation of the intersection for the release will not work as even the two tracks (5 and 11) which I can play with Deezer free are reported with no available_countries through the API…

1 Like

Marlonob’s original server does not really handle this example correctly, it always says Countries where available (Deezer + Spotify), even if the release is not available anywhere on Deezer (according to the API). It returns the same list of available (Spotify) countries regardless of whether you are using Deezer+Spotify or only Spotify as source (the Countries excluded list then contains the Deezer-only territories).

At the time of testing this we already have 100 available Spotify territories for this example by the way, which means that (at least) seven out of the new 85 countries are already available.

2 Likes

Thanks, fixed!

AFAIU available_countries is unrelated to whether a track is available in Deezer free or premium:

https://developers.deezer.com/api/track

List of countries where the track is available

However, an empty value here clearly doesn’t indicate that the release is unavailable in all countries. It feels to me like a Deezer data issue given the number of tracks affected seems tiny so far.

2 Likes

Some further research: What the tracks of this album also have in common is that the API reports the attribute readable as false for all of them, even for Things That U Do which I can play just fine with a free account. Maybe because this track is also part of a different album without restricted access?
When I lookup this album with a-tisket it reports the Deezer countries as expected because the track has a different ID for this album which does not return an empty list.
So it looks like the availability of a track does depend on the subscription tier… and also on the related album, so Deezer’s whole system of storing the availability at track level seems pointless!?
Or are there any albums where some tracks are not availble for all territories if we ignore the cases where tracks are only playable because they are part of a different album which is available there?

Yes e.g.

Tracks 6, 7, 9 and 14 aren’t available to me and the popup says “this track is not available in your country”.

Your right. It does indeed return an empty country list, so it’s just not available on Deezer any longer at that link.

Confirmed by API, 2/23/2021: Bangladesh, Ghana, Kenya, Nigeria, Pakistan, Sri Lanka, Tanzania & Uganda

https://atisket.pulsewidth.org.uk/?preferred_countries=US%2CGB%2CCA%2CMX%2CDE%2CAU%2CJP%2CDK%2CBE&spf_id=7JhGOo4ojHVgNbFdxHGUgD&upc=00602435658919&preferred_vendor=itu&search_itu_countries=si

1 Like

Confirmed by API, 2/25/2021: Antigua and Barbuda, Armenia, Azerbaijan, Bahamas, Barbados, Belize, Bhutan, Botswana, Burundi, Cambodia, Cameroon, Chad, Comoros, Curaçao, Dominica, Equatorial Guinea, Swaziland (Eswatini), Fiji, Gabon, Gambia, Georgia, Grenada, Guinea, Guinea-Bissau, Guyana, Haiti, Jamaica, Kiribati, Kyrgyzstan, Laos, Lesotho, Liberia, Macao, Malawi, Maldives, Mali, Marshall Islands, Mauritania, Federated States of Micronesia, Mongolia, Namibia, Nauru, Niger, Palau, Papua New Guinea, Rwanda, Samoa, San Marino, Sao Tome and Principe, Senegal, Seychelles, Sierra Leone, Solomon Islands, Saint Kitts and Nevis, Saint Lucia, Saint Vincent and Grenadines, Suriname, Timor-Leste, Togo, Tonga, Trinidad and Tobago, Tuvalu, Vanuatu & Zimbabwe

https://atisket.pulsewidth.org.uk/?preferred_countries=US%2CGB%2CCA%2CMX%2CDE%2CAU%2CJP%2CDK%2CBE&spf_id=4otkd9As6YaxxEkIjXPiZ6

3 Likes

Not sure if this is the right topic to mention this, but the “ISRC submit” option isn’t working for like 3 days now. I keep getting this error:

Capturar

I am regularly checking the Wikipedia article on Spotify and out of 85 announced countries only 8 countries are still missing, they will likely be available by tomorrow. Since I have already prepared the changes for all 85 at once, the most sensible way is to wait for the missing ones instead of pulling the update apart, it just would make it more likely that the list is already deprecated again by the time atj has deployed the update.

By the way, according to my local test server your example release has become “non-worldwide” in the mean time because in addition to the 8 missing countries Belarus (since July 2020), Brunei, Burkina Faso, Nepal and Uzbekistan (since February 2021) are not available (yet).

1 Like