ROpdebee's userscripts support thread

ROpdebee · September 22, 2021, 12:21am

Consider this thread a support thread for any userscript created by me. Questions, feedback, problems and bug reports most welcome here!

For reference:
The userscript repository
Issue tracker

ROpdebee · September 22, 2021, 4:00pm

@nadl40: Responding here to keep the noise in the other thread to a minimum.
I’ve looked into Qobuz a bit, getting the covers should be easy enough since the path to the full res cover merely depends on the album ID. For booklets though, I presume you need an account to download those? If so, that won’t be supported for two reasons: 1) Doing anything with accounts, passwords, API keys, etc. in a userscript is pretty difficult to do in a userscript, and 2) I’d rather not publish a script that automatically uploads content that’s only available with a paid subscription, for obvious reasons If booklets can be retrieved without an account or API key or anything like that, I’d be happy to see what I can do.

Could you perhaps provide some Qobuz links, maybe even a MB release which has one attached, so I can figure out what these links look like? It looks like it’s https://www.qobuz.com/[language]/album/[name]/[id], with language being optional, but maybe there are other link formats?

crayonbro · September 22, 2021, 4:42pm

this is a dream come true for me. thank you so much this saves us a lot of time. i would suggest, if possible, to add vgmdb as well

nadl40 · September 22, 2021, 6:48pm

I’ve checked my script and unfortunately you need API authentication… for a while I thought this is not required as once you get a hash of album metadata then download links do not require authentication

george@ideapad:~/scripts/qobuz$ ./get_qobuz_metadata_only.pl --get 7619990103283 | grep pdf
‘original_url’ => ‘https://static.qobuz.com/goodies/63/000082836.pdf’,
‘url’ => ‘https://static.qobuz.com/goodies/63/000082836.pdf’

Here is a qobuz link to a release with booklet

This release is same as the booklet links above but to get the id of the booklet, one needs to retrieve the hash for an album for which you need authentication… too bad. It’s a good source for booklets and pretty bad metadata

tigerman325 · September 22, 2021, 8:36pm

Yeah, I’d love a way to know barcodes on the new Qobuz releases since they seem to have removed them from their URLs. I haven’t found any API that’s open to the public.

nadl40 · September 22, 2021, 8:56pm

The endpoint is http://www.qobuz.com/api.json/0.2 and it still works.

I’ve used it maybe 3 or 4 years ago and the documentation was freely available but looks like it’s removed by now, they had a github and a website.

You had to get app id and of course have a subscription.

tigerman325 · September 22, 2021, 9:16pm

Yeah, I’ve always had roadblocks because I don’t pay for the service or understand how to get keys etc. I always have to rely on public APIs like Spotify, Deezer, HDtracks or 7digital.

nadl40 · September 22, 2021, 10:03pm

Found it (and bookmarked it ) here qobuz API ver 0.2

I will go over it as I’m not 100% sure the album call needs to be authenticated, they had number of non authenticated calls for searches.

ROpdebee · September 22, 2021, 10:09pm

Unfortunately it seems that only the main page of that repo is archived, pages deeper in the docs aren’t archived

I’ve noticed it mentions “back cover” as well, are those publicly accessible? If so, do you have an example? Might be interesting to consider (although a digital release with a back cover seems a bit weird to me)

nadl40 · September 22, 2021, 10:41pm

No, I have not seen backover yet, album cover json looks like that

       'image' => {

                    'thumbnail' => 'https://static.qobuz.com/images/covers/83/32/7619990103283_50.jpg',
                    'large' => 'https://static.qobuz.com/images/covers/83/32/7619990103283_600.jpg',
                    'back' => undef,
                    'small' => 'https://static.qobuz.com/images/covers/83/32/7619990103283_230.jpg'
                  },

Also no luck getting the album json without authentication. This could be done as I have a working script but user would need to provide credentials to get authorization token and then album json can be returned.

As you said before, it does not fit well as a user script.

ROpdebee · September 22, 2021, 10:53pm

Alright, if you come across one with a back cover that can be accessed without authentication, let me know and I’ll see whether it can be integrated.

I’ve just pushed 2021.9.23 which adds support for Amazon, Amazon Music, and Qobuz. Thanks for the suggestion @nadl40!

@crayonbro: VGMdb seems like a great provider to add, I’ve created a ticket to track it: #36. Extracting the images should be very easy, but I’d like to translate its captions to MB artwork types and comments too, so it’s going to take a little while longer to add.

aerozol · September 23, 2021, 1:33am

This script has been absolutely incredible. Clicking the Discogs or Bandcamp button is a HEAVENLY experience. VGMDB will be the icing

Does anyone else get a double button for releases with 1 Bandcamp link?

Might be on purpose, but the second button here doesn’t return anything so seems like something might be broken? Note that it has a ‘stream at’ and ‘purchase’ relationship for the same link. It doesn’t happen for this release which only has 1 relationship.

Hmmm I was just going to mention that if a release doesn’t have art it would be cool if we didn’t have to click through an empty cover art page to submit (should really be a MB request but you are faster ahaha), but I saw that the mb. INLINE STUFF script is already doing that for me

Not sure why I didn’t notice it yesterday…

ROpdebee · September 23, 2021, 6:57am

That’s not intentional and was due to the script not properly deduplicating URL ARs. It should’ve been fixed in 2021.9.22 or 2021.9.22.2 yesterday.

On the topic of updates, is everyone receiving automatic updates now that the script location has changed? If you’re on any version above 2021.9.21, then automatic updates are working. Keep in mind that the first update has only been out for less than 24h though, depending on your update frequency, your userscript engine might not have checked yet.

Zas · September 23, 2021, 9:15am

@ROpdebee thanks for this great script, I think you can improve the edit note:

Bandcamp URLs have a comma appended when an attribute is added: https://bandcamp.url, maximized. It would be better if there was a whitespace at the end of url to ease cut’n’paste: https://bandcamp.url (maximized)
the actual source URL of the image should be provided too (in Bandcamp case, f4.bcbits... url)
userscript signature should be at the end, and sources before it

Something like:

https://avecaesar.bandcamp.com/album/split (maximised)
https://f4.bcbits.com/img/a0032185859_0.jpg
–
MB: Enhanced Cover Art Uploads 2021.9.23

nadl40 · September 22, 2021, 2:54pm

I wonder if you could add qobuz.com

Good cover art but most importantly they also have booklets when labels provide it.
I have some perl scripts that can provide a url for both, might you save time figuring out qobuz API.

Please let me know if interested.

chaban · September 23, 2021, 11:14am

Enhanced Cover Art Uploads

~~Buttons are duplicated for same URL with different relationship types~~
An option to abort and/or add only front/first cover. Especially Discogs can have quite a lot of images.
To save time and accommodate for people who don’t have as much bandwidth (admittedly for Discogs the latter is not much of an issue right now )
Bandcamp provider:
- Grab track images. (there’s a userscript that automatically displays it when playing)
  - Quarantine Pixel Party | Nyokeë
- Consider also grabbing the image as presented on the page
  - Edit #66611500 - MusicBrainz
  - Release “The Bridge” by Multinational Synth Corp - Cover Art - MusicBrainz
VGMdb provider:
- To see all images one needs to be logged in.
  FITHOS LUSEC WECOS VINOSEC: FINAL FANTASY VIII [Limited Edition] | SSCX-10037 - VGMdb has 18 images
Display hint about cover provider quality (ideally they can be compared)
Some services upscale and/or filter images, others like Bandcamp generally have original files:
- https://musicbrainz.org/edit/77578034 (SoundCloud)
- Edit #81901201 - MusicBrainz (iTunes, Deezer)
- Edit #81653603 - MusicBrainz (Bandcamp)

Paste multiple external links at once

Can it be made to run on add and create pages too? Especially in the release when having to add new artists would be nice.
Bug? When pasting multiple URLs, some of which are already present it eventually no longer splits and the "don't split" option disappears. Also misses paypal URL on first try
1. https://musicbrainz.org/artist/b719e113-3113-4f71-b768-afe8a5cdaf04/edit
```
https://open.spotify.com/artist/2V7GcwRUXpWdgYSyPXv870
https://www.instagram.com/jera.isidore/
https://twitter.com/JeraIsidore
https://www.youtube.com/channel/UCPHrO3_NSeOZ3ONErcBMZLQ
https://www.facebook.com/Jera-Isidore-104861881873537/
https://paypal.me/JeraIsidore
```
2. https://musicbrainz.org/artist/20fa23d9-bb5d-4bea-8bfe-72e6939be6c7/edit
```
https://www.facebook.com/smartstephanie
https://www.youtube.com/user/SmartStephanieTV
```
  After pasting remove any of the two new links.

mattgoldspink · September 23, 2021, 12:12pm

Personally I think we shouldn’t do that. This script has saved me so much time! I’m super happy to just hit and keep it running in a tab while I add other date and check if it’s done a minute or two later. It feels to me better to have those cover artworks in CAA & MB than to miss out on them.

That would be great to know the dimensions of the cover coming through. Sometimes the ones on discogs are so tiny there’s no readable text on it, but you can’t see that until you click through. I’m not a fan of having those uploaded, but if they were at least sucked in to the upload page first I can then quickly scan and delete the ones that don’t meet my personal cut off.

ROpdebee · September 23, 2021, 3:25pm

2021.9.23.2 is out, with two enhancements:

Full VGMdb support (thanks @crayonbro for the suggestion). Types and comments are extracted on a best-effort basis, so make sure to double-check the pre-filled data to make sure its correct and well-formed.
Improved edit notes (thanks @zas for the suggestions). Source URLs now go at the top, URLs extracted from a page are included, as are the non-maximised URLs for provenance. Those are truncated to three URLs to prevent too much noise in the edit history for releases with a lot of covers (imagine if all URLs were shown on all edits after importing all the images from this Discogs release ) Ideally each edit would only include the URLs relevant to the single image in the edit, but that’d require hooking very deeply into the upload process, and I’d likely break stuff. Example of how it is now:

https://vgmdb.net/album/86043
 * https://media.vgm.io/albums/34/86043/86043-1555603051.jpg
 * https://media.vgm.io/albums/34/86043/86043-1560159976.jpg
 * https://media.vgm.io/albums/34/86043/86043-1560159983.jpg
 * …and 1 additional images
https://www.amazon.co.jp/gp/product/B07QTNG28W
 * https://images-na.ssl-images-amazon.com/images/I/51Avck-4usL.jpg
   Maximised from https://images-na.ssl-images-amazon.com/images/I/51Avck-4usL._AC_US40_.jpg
–
MB: Enhanced Cover Art Uploads 2021.9.23.2
https://github.com/ROpdebee/mb-userscripts

@chaban: It’ll take me a while to fully triage all of that (update from the future: It took 2 hours), but first impressions:

An option to abort and/or add only front/first cover.

Aborting while it’s fetching images would be quite tricky, and I generally agree with @mattgoldspink here. I’m concerned that allowing an abort might lead to partial cover sets, and that wouldn’t be ideal. OTOH, having the option to fetch only the first image (presumably the front cover) would be a good compromise, IMO. #37

In general, I think it’d be nice if it first grabbed and displayed small thumbnails for all images, and only then fetches and queues the full image. That’s a non-trivial change to make though.

Bandcamp provider: Grab track images

Good idea, definitely feasible and worth it. #38

Bandcamp provider: Consider also grabbing the image as presented on the page

That should also be feasible. Off the top of my head, I’d say 1) grab the original image since we’ll definitely need it, 2) check whether its dimensions are non-square, 3) grab the thumbnail if it is. I’d argue that there should be some allowance of non-squareness though, there’s not much point in doing that if the original is 1000x1010. So maybe anything within a 1.0 ± 0.05 ratio could be considered square (So 1000x1050 would be kept as-is, 1000x1100 would get the square thumbnail)? #40

VGMdb provider: To see all images one needs to be logged in.

The VGMdb provider added in the most recent update uses the vgmdb.info 3rd-party open API, which seems to return all covers without requiring any account or API token.

Display hint about cover provider quality (ideally they can be compared)

That would be a good addition for sure, but I’m a bit stumped on how to best display it. Adding a markdown file in the repo and linking to it (like what’s being done for the supported providers) would be easiest, would that be an acceptable solution? To summarise the edit notes:

SoundCloud has bad compression
iTunes PNGs are often useless unless the source image is a PNG
Deezer is upscaled and badly compressed (and IIRC they used to/still do apply color filters on some covers?)
Bandcamp is mostly fine
Spotify is always 640x640

I don’t think there will be any reliable way to always grab the “best” image (and “best” is often subjective anyway), like e.g. the Bandcamp example that’s upscaled from a 400x400 image, I don’t believe there’s any way that could be detected automatically without significantly hampering UX and hogging browser resources. For Apple Music, at first glance it does seem possible to automatically do the substitution of the file extension (there are some exceptions, notably URLs given by a-tisket, which use the iTunes URLs instead of Apple Music, and those don’t include the original file name). However, are we sure that those JPEGs aren’t lossily re-encoded versions of a lossy source JPEG?

In general, I think three things are needed:

A central overview of caveats of cover art providers.
A way of previewing the queued images in full size (like the artwork dialogs on the cover art page itself)
Basic properties of the image (dimensions, size, file type)

Points 2 and 3 would be feasible to do once the CAA Dimensions and Supercharged CAA Edits scripts are ported to TypeScript. It’d also be nice if all of the image properties would be calculated for all images prior to actually fetching it, but I’m concerned that that would be too taxing on the network connection.

I’ve created #41 to track, but I have to work out the details a lot more.

Paste multiple external links at once

#42 and #43.

Adding it on more pages should be possible. The other thing is indeed a bug which I had encountered already, in short, the script isn’t listening for URL removals so it’s also not adding itself to the new input boxes.

chaban · September 24, 2021, 5:57pm

Possible bug: Enhanced Cover Art Uploads (2021.9.23.3) using Tampermonkey adds undefined instead of the homepageURL in edit note footer. Works fine using Violentmonkey though.

kellnerd · September 24, 2021, 6:38pm

Thanks, there is an inconsistency between the different userscript managers:

console.log(GM_info.script);

{ // Violentmonkey
  ...
  "namespace": "https://github.com/ROpdebee/mb-userscripts",
  "homepageURL": "https://github.com/ROpdebee/mb-userscripts",
  // "homepage" is undefined
  ...
}

{ // Tampermonkey
  ...
  "namespace": "https://github.com/ROpdebee/mb-userscripts",
  "homepage": "https://github.com/ROpdebee/mb-userscripts",
  // "homepageURL" is undefined
  ...
}

// Greasemonkey: script is not running at all

Edit: Fix has been tested and published in version 2021.9.24.