Hy,
Actually my Picard is running over 80k mp3’s and by accident I sent the machine to hibernate (Ubuntu) and now it sits there and does look up the same file since hours, so i think a thread died. It might also be, that my internet connection went away (as mine is not very stable) and came back.
Anyway, it seems to loop over nothing right now, only thing thats changing is memory allocation (ist down to 460KB right now, where it was ~2GB), so I think it might be nice to have a “stop tasks” button, so i could save the tags already downloaded.
thx
To be honest, Picard just needs a warning when you try to run too many files at once, because this is a bad idea to start with - even if it seems convenient!
I’m not sure exactly how Picard works, but is it trying to store 80k files worth of cover images etc into memory?
But anyway, yes a stop button would be good, I agree, but something to warn people from running too many files wouldn’t hurt either 
5 Likes
Thing is i got one folder with all album folders in it - so i dont have another possiblity than loading all in one. other thing is, my music folder is on a nas, which is quite slow (poor processor). this combination is nasty, as picard i.e. does not respond for ~20 seconds to a mouseclick while writing to files. i think picard is already using different threads, but that might be the optimum to do under python (?). but the stop button would make bad decissions revertible without loosing too much time. i.e. if i load all the files this lasts several hours already. if i then select too many files to scan and want to go to hibernation, i have to re-do the loading, beacuse the scan does not come back after hibernation, while with a stop button i could just stop the scanning but retain the loaded files and continue after hibernation.
now i do it that way:
-load all files
-select a bunch of, scan them
-save the results,
-remove the written files
1 Like
Again, agreed that a stop button would be useful, but surely it would be easier for you to click on ‘add files’ in Picard, go into your music folder, just select all the folders that start with ‘A’ (or whatever selection), do them and then move on to the next.
This would solve your problems of having to wait to load everything and then do it all over again.
If Picard is indeed loading information on every album into memory I’m not sure if a stop button is going to free that space up again anyway, until the files are removed, but again, not sure.
1 Like
Fucking pissed at this garbage.
I like the program, but at some things it was so poorly thought out.
Today stupid me had the terrible idea of trying to scan 7.5K files, the garbage totally stalled,
ran very slow, ran for almost 12 hours when I finally had to shut it down, after waiting 12 long
hours. I’m so fucking pissed.
Why are computers and programs so fucking stupid and dumb?
4 Likes
do they have a 64 bit version?
1 Like
1 Like
wow, that was so rude, hahaha
This is a feature I feel should be present also.
Sometimes you might need it when things are taking very long or seem stuck in general, but it would also be very useful when you e.g. notice that Picard is matching some compilation albums to many separate ‘original’ albums. (edit; it’s usually the other way around)
Especially when using a plugin such as Wikidata Genre, when you notice that happening, you know you are in for a world of pain and a very, very long and useless wait. (and a burden on MusicBrainz’ server)
Something like ‘End all tasks’ seems a pretty essential feature to me.
There are some performance improvements in the next release and big libraries shouldn’t be an issue anymore (unless you hit an unknown bug and the program crashes, or use the following genre plugins). Loading, clustering, moving and saving should be a bit faster.
Operations that involve network requests are bottlenecked by them. Based on my testing, Wikidata Genre makes things 5x slower than standalone lookup/scan, while AB Mood Genre makes things 3x slower. I’ve tested with a 1.5k file library that originally took 10min to scan, while Wikidata/AB Genre plugins took 50min/30min. The number of network requests increases with the same ratio of the slowdown.
Cancelling network requests is fairly simple, but not sure on how to trace back to their sources and cancel the tasks themselves.
2 Likes
zas and I had a short talk about this recently. Another aspect of the inability to abort requests is that requests also continue running even if one has removed e.g. the album for which the requests are running, see e.g. PICARD-1857
The basic idea was to extend the existing task handling in the webservice module to keep track of source objects. I think for most network requests this would even be enough, as cancelling would cause an error and that should be handled already.
But once we have cancelling network requests we for sure want to cancel also other background tasks (fingerprinting, loading a file, saving a file etc.). Currently we lack the information what is really happening. We just have the number of “pending files” (but don’t know why they are pending) and network requests (but don’t know what they are for). We also have the thread pools, but not all threads there can be considered safe to cancel, not all are about what the user would consider background processing of files…
What we came up with as an idea for a more thorough solution: Have dedicated Task objects as an abstract representation of a background task. A Task object can be linked to a source (or sources?), maybe has a description (so we can do more in UI with it) and can be cancelled. As concrete implementations of Task there might be at least a ThreadTask and NetworkRequestTask.
Tasks are handled by a TaskQueue, which keeps track of running the tasks. You might cancel individual Tasks or ask the TaskQueue to cancel all. If a running task can be cancelled or not might depend on the implementation, but in any case you can cancel pending Tasks.
As a start I would focus on the network request side of things, as I think it is easier to implement. But once we have this other background tasks could be moved to this system step by step.
Nice thing is that we would also know more about what is going on and could visualize this better for the user. E.g. I have been toying with an implementation of showing a (hideable) progress dialog, that shows up when a lot of work is happening. You can at any time hide it to the status bar and show it to see what is going on. The UI was working, but with the current limitations in information what is happening in the background it was not working very well. I think the task implementation above would be able to solve it.
Now just someone needs to actually start working on it 
4 Likes
Both plugins add multiple additional requests per album, so this is not much surprising. AcousticBrainz adds one request per track, while Wikidata can do multiple requests per track depending on the settings. But it tries to be clever and only request data for the same entity once.
But looking at the code I see that the Wikidata plugin does not configure the rate limiting, which means it runs with the default 1 request per second limit. Reading API:Etiquette - MediaWiki I think it will be ok for us to disable this for Wikidata.
1 Like
Would the tasks be separated per target (file/track/cluster/album) or per pipeline (add/remove/load/save/clustering/scan/lookup)?
Not sure I understand the question. A task would be a single action, such as e.g. saving a file, or doing a network request for an album.
1 Like
Took a look at this, and yes, we should indeed disable the rate limit for Wikidata. But it will not make a huge difference. The requests I looked at took between 700 and 1200 ms, so the 1s rate limit does not have such a big impact here. Still, will save some time in some cases.
1 Like
Hmm, just thinking on how many tasks would be listed on the task progress panel and cancelling them individually would be kinda hard. But there’s nothing preventing them to be grouped per type, with stats for waiting/working/finished.
1 Like