A multi‐source seeder for digital releases

Tags: #<Tag:0x00007f9ada949018> #<Tag:0x00007f9ada948eb0> #<Tag:0x00007f9ada948d48>

Hello, everyone:

I have made a multi‐source seeder for digital releases, written in PHP. It started as a personal tool (as I expended too much time trying to find accurate information for digital releases). It is available for testing at https://etc.marlonob.info/atisket/

Motivation

To automate and make more accurate the importing of digital releases by providing a way to link, merge, and validate different vendors’ data

How it works?

By querying vendor APIs (currently, Deezer, Spotify, and iTunes). A release is considered the same if, and only if it shares the same UPC across vendors.

The entry point for the search can be a UPC, Deezer album ID, Spotify album ID, a pair of iTunes ID and country, or a combination of the four. If no UPC is provided, the one indicated by the first vendor queried would be used to try to find the respective releases in other vendors (if any) (there’s a not‐always‐available way of getting a UPC from iTunes, via its cover art URL).

Main features

Detects the UPC from some iTunes releases: Sometimes the publisher includes the UPC of the release as part of the cover image name. If this info is available, then is is tested against the API to ensure that it’s accurate.

Find MB releases that already have some of the data (vendor IDs, UPC) and links to them to prevent duplicate entries.

Fill up the artist’s MBID if it can link them via URL relationships.

Offer possible RG to which the relase may belong.

Links to high‐resolution images for the cover art. This vastly depend on what the vendors offer.

Get the ISRC of the tracks and show them. Currently, it can’t submit them.

Get all available release countries from Spotify; optionally, it can query all iTunes stores for availability. If the countries where the release is available are all where Spotify has presence, the release event is set to [worldwide] and the list is added in the annotation (the user can switch this, i.e., put the countries as release events and remove them from the annotation). If at least one country is restricted, all countries where available are added as release events (again, this can be reversed if the user wants it). If the restricted countries are less than fifteen, then a list of them will be added to the annotation.

Cache the API response from Spotify: Given that the Spotify API is not open, the response for the album is cached, and made available at least for nine days so it can be consulted for review purposes.

Unicode replacements for quotation marks, apostrophe, and hyphen (when it’s between two letters). The duration is added up to the milliseconds (for iTunes and Spotify only).

Pending

Bandcamp support: I haven’t been able to find useful documentation for its API.

ISRC submitting: I don’t really would know how to implement it; if someone can guide me through the process for PHP/JavaScript, that would be very helpful.

Disclaimers

  • The Spotify API is queried using my personal key and token. I don’t anticipate any restriction for overuse, but that may be a possibility.
  • Sometimes, the APIs return no result, even though the album is available, this may be overcome by manually adding the vendor ID, but do this only if you are certain that the UPC is the same (in theory, the script would automatically detect the discrepancy in UPCs and disable submit, but I feel that I have not tested all the possible scenarios).
  • Querying for all iTunes stores delays the response significantly. I don’t think that the traffic generated would be clasiffied as spam, but, again, that may be possible.
  • The script is considered to be in beta fase, and as such is provided as is.

Final notes

If you feel like it, you may test it and share your feedback via this thread. I don’t have much time for this project so the revisions in the code may take some time, but not more than a week, I hope.

English is not my mother language, so any grammatical and spelling corrections are welcomed.

If someone wants it, I will share the code. I don’t know the first thing about git, so I can’t use it. Beware: a) The code is barely commented, and b) I’m not a programmer (over a half of the writing of this script was used looking up at documentation).

The code is hosted in a (shared) server that I have for personal use, so, it may not be the fastest.

If someone more experienced wants to host it in other server, or become the maintainer in github or other git server, I will be very glad to help in what I can.

16 Likes

Well, I’m speechless! Thank you so much for sharing this tool with us!

I will definitely test it as soon as I have a non-bandcamp release that I want to add.

2 Likes

I added a release using the Spotify as the primary source, and no data was found for iTunes and Deezer.

After adding, I did another search using iTunes as the primary source, which detected the release on Deezer and Spotify, with the UPC matching up with the release that I had added.


1 Like

I’m not sure what happened. If I try https://etc.marlonob.info/atisket/?spf_id=6PmJnFiiGkOof3aicZYS6j it gives me information for the three of them.

Did you, by any chance, save the URL of the first case? or, could you try to replicate the problem and share the URL? Thank you.

I just tried to replicate the problem and it’s picking up the iTunes and Deezer data now.

1 Like

Simply WoW! I’m amazed by your tool. Thank you !

Can you add (if possible)
https://music.youtube.com (streaming service like spotify and deezer with a “free” tier)
https://play.google.com/store/music (popular store)
https://www.bandcamp.com/ (popular store)
https://www.amazon.com/ (digital music section)

3 Likes

This is most excellent. :slight_smile:
If for no other reason than knowledge acquisition, I’d love to have a look at what how that all works. But I’m sure it will bear some fruit on my end too.

…and as for Git, I’m happy to make it available and even merge in your submissions if you send me only the changed files each/anytime.