Picard 2.3 Beta 1

I prefer having a whisky and drink it :wink:

2 Likes

Not heard of Pavlov whisky before? Just her cakes.

2 Likes

Always need good cakes when testing software.

2 Likes

Another bug undocumented feature

I keep coming back to this AcoustID thing as there is clearly something weird here.

Generate AcoustID data is not being saved.

I have just loaded my tagged FLAC files back up and can’t find the AcoustID data.

I look online and it is also not there in the Release in MB - even though I have submitted it a few times now.

Maybe I am going mad, so we’ll try again slowly…

  • I have my Tagged by Picard files. All 94 of them.

  • Quick independent check in MP3TAG and I see MBIDs and other data, but not AcoustID. Weird.

  • Start Picard, drag folder of all 94 files in, they automatically jump to the right due to the MBIDs

  • And the “Thumbprint status” column is empty. Weird.

  • Drag to the left, recluster, press Generate AcoustID

  • Wait and watch all 94 tracks get their little thumb logo in the new column…

  • Drag cluster back to the right. See all 94/96 light up again. All with little thumbs in the columns.

  • BUT look at the bottom pane with the before\after tag data. Fail to find AcoustID listed at all!

  • Press Submit AcoustID (thumbs now change red to grey)

  • Press SAVE

  • Still no AcoustID showing in that lower pane.

  • Still have thumbs next to all the tracks, but no way to see my AcoustID.

  • Double check in MP3TAG - and no AcoustIDs are being written to the files.

  • Fire up Picard again, drop files in, Drag to the left, now try the old method of SCAN

  • NOW we have the new AcoustID data appearing in the bottom pane.

  • Hitting SAVE now correctly writes the data to the files.

  • Now close and reopen Picard and the AcoustIDs are shown

  • The old method is still good.

Assumption is that when hitting Generate AcoustID this data is not correctly being copied to the lower pane/data structure. So it is not there when I hit Save. Therefore never making it to the files.

I also notice when the files are on the left and I hit Generate AcoustID I see nothing change in the bottom pane. The separate icons light up as each are generated in the list, but it feels as if the actual value is not getting stored\displayed.

When I hit SCAN i see the bottom pane add an AcoustID field for each track in turn…

Okay, getting a bit tired now, but maybe another point to help narrow this down.

Edit: There is another timing issue here. So this really needs LONG lists of fat flacs to get time to see it happen while tracks move left to right.

Tagged files with no AcoustIDs ready to go on left side.

Cursor on left side, highlight the release. When I hit SCAN I get no AcoustIDs displayed in the bottom pane.

Click over to the right pane and it shows the AcoustIDs.

Click back to the left, no AcoustIDs being displayed. The bottom pane shrinks and removes the row.

I am clicking back and forth while the tracks are still being scanned and matched. The AcoustID row appears and hides depending on the side I am on.

Once all the tracks are scanned and moved to the right, I can then drag them back to the left and NOW they all show their AcoustIDs on the left.

Sorry, I’ll turn the computer off now before I break anything else.

Sorry, I’m just a bit of an awkward tester - got another oddity caused by the large 94 file package. :hammer:

It is possible to LOOSE files from the interface and my pack of 94 shrunk.

This is all about being impatient and not having a stop action button. I know I am dragging stuff around while work is in progress. That is the point here. :wink:

  • Open Picard.
  • Throw 94 files release in that has MBIDs already tagged
  • They jump to the right and start downloading the 8 items of JPG artwork for the Release (I always download all available art)
  • I quickly grab the folder of Unmatched Files and throw it back to the left into Unclusterd while Picard is still downloading image files (important step) .
  • Files start reappearing under unclustered files as all 94 slowly get written down the page…
  • …but that list never gets to the bottom because…
  • Suddenly the artwork has finished downloading and the files jump back to the right. (I assume it is completing the “match the files” step from the initial shuffle to the right.)
  • But what has happened?
  • Eeek! now I only have 62 files and can’t work out where the rest have gone. LOL

Second repeat run of this test I only ended up with 4 files. :rofl:

A few more runs and I have worked out what happens. I think the trick to trip the bug is to still have the files moving from right (unmatched files) to left (unclustered files) at the moment the art stops downloading.

If the files are still reappearing on the left, building the list, at the exact moment they are then called back to the right. Crunch - files lost in limbo somewhere.

Yeah, I know I am giving the code a kicking. But you wanted proper tests. 11 disc box set tests of evilness. Mwahahahahahaha…

To reproduce have a LONG list of flac files in the release, and lots of artwork to download.

The artwork has to finish downloading while the list on the left is still being slowly written. I expect you can get the timing even better if you have loads of columns enabled on the left.

(tip - Tori Amos has some HUGE images at 20MB per image. Good for this kind of test. There are also some huge Pink Floyd boxsets like the Discovery compilation box with tons of files and artwork. It will be tricky to get the timings correct as you have different network speeds, etc…)

And I have seen this issue appear in v2.2xx. I never really worked out why I could sometime loose files when moving right to left, but I think this could be part of it. Timing issues caused by images still downloading on the right when I am dragging back to the left. I trigger this more often due to by tendency of getting all artwork.

BTW - my testing is only helping you get perfection. I know you guys have worked hard on many areas of this program and it shows. I’m only trying to kick out a few unknown bugs. The laughing is due to enjoying helping out by finding weird things… There is clearly lots of good new stuff in this release. Thanks for that

1 Like

@IvanDobsky would you mind create tickets for bugs you detected and for features/improvements you suggest. It helps us a lot to have someone who’s testing the app in depth, but we need to keep track of things to do.

You’re right about the lack of feedback, we tried to improve things over years, but to improve things significantly in this field we’ll need to rewrite a lot of code, that’s planned.

All suggestions about UI improvements are very welcome, please add screenshots/mockups to tickets, so we can see immediatly what is expected.

Thanks for testing and helping improving Picard!

8 Likes

I think there is some confusion here. First of, it’s not really that “Scan” is the old function and it gets replaced with new “Generate AcoustID fingerprints” button. Both serve different purposes.

“Generate AcoustID fingerprints” does calculate the fingerprint (using the fpcalc utility) for the files. It does not do any requests to the AcoustID server. It was added to address the use case that people wanted to submit data (fingerprints) to AcoustID but did not want to use AcoustID to identify their files.

The fingerprint is different from the AcoustID. The fingerprint is directly generated from the audio data using a mathematical algorithm. It currently is not stored in the files, as it is easy and reasonable fast to regenerate. So if you load back a file into Picard and want to submit its fingerprint to AcoustID you need to generate that fingerprint again. But you really don’t need this fingerprint permanently, it is only of use for querying the AcoustID server for matches or submitting it to AcoustID.

The fingerprint itself was basically invisible in Picard before. Once generated Picard kept it around as long as the file was loaded, but there was no indication whether a file had a fingerprint generated or if it was already submitted to AcoustID or not. Hence the new “Fingerprint status” column.

Now there is that magical AcoustID. The AcoustID is kind of a grouping of similar fingerprints. One important feature of the AcoustID fingerprints is that they can be compared for similarity. So while the fingerprint for e.g. a lossy file might be slightly different (due to differences in audio) it will be very similar to other fingerprints of the same recording. To get the AcoustID one needs to query the AcoustID server using the fingerprint. AcoustID will search its database for AcoustID with matching fingerprints attached and will return a search result. This result can also include details about matching MB releases as the AcoustID serves as the link between fingerprints and MusicBrainz data. It’s important to note that there is no guarantee the AcoustID will return any results, it is possible there is no match for this fingerprint. It’s also possible there is an AcoustID, but no linked MusicBrainz data.

The “Generate AcoustID fingerprints” action currently does not query the AcoustID server for matches, hence there is no AcoustID in return. I intentionally left this out. You don’t need this to submit the fingerprint to AcoustID but it slows down the entire process by a factor of 2.5 - 4 (or even more, depending on your internet speed and the actual response time of the server).

We could of course do this lookup to get the AcoustID just for the sake of it. But not sure how to call this then without raising wrong expectations. I intentionally called the action “Generate AcoustID fingerprints”, because this is what it does and what you can actually “generate” (aka calculate). You cannot “generate” AcoustIDs, you can search for matching AcoustIDs. But you probably won’t get one.

There is no need for neither dragging nor clustering, you can generate the fingerprints directly on the right. Actually this is kind of the idea here.

When you hit scan for files on the left which don’t have any AcoustID tag yet you will basically never see any AcoustIDs on the left. That’s because as soon as the response from the AcoustID server comes the file will be moved to the right. The only exception to this should be the rare case where there is an AcoustID for the file but no MB recording linked to it, then the file would stay on the left.

7 Likes

@outsidecontext sorry for the confusion again. So a “fingerprint” does not come up with the same numeric string as the AcoustID? This is why it is thrown away?

And it is not “quick” when you are working with 94 files, hence the timing bug I thought I had spotted.

I made the wrong guess on this stuff. I didn’t read all the development threads so was not aware of this difference.

I was one of those people who asked for a Generate AcoustID button, but I thought it was going to actually fully Generate a usable AcoustID that could be stored in the file without looking up a match on the MusicBrainz release database. I now better understand what it is and will have to go back to work out how I’d get it properly into my workflow.

@Zas you can see by the reply from @outsidecontext that there is not really any point in me putting tickets in for this stuff as I just misunderstood everything again. Looks like nothing I saw was bugs, just misunderstood features.

Sorry for wasting everyone’s time.

This is why I try and stay away from the tickets side. I can’t talk your language.

(Sorry, having a really bad day here. Apologies for being so negative.)

1 Like

Not mine :wink:

I would probably never have learned or understood the workings of these features if you hadn’t brought it up.
(and if the coder-wizards hadn’t given these very informative and eloquent answers of course…)

1 Like

I’m glad someone got something from it. :slight_smile: It is frustrating for me to have wasted so many hours over a complete misunderstanding of a feature. Next time I’ll leave it until things are better documented so I can test in a way that helps people.

Is the OS X minimum version bumped with this release? I’ve having trouble getting it to run on 10.12. If I run it from the DMG, it quits immediately. If I copy it elsewhere to run, I get a message that it “is damaged and can’t be opened. You should move it to the Trash.”

Running from the command line gets me:

[53512] Error loading Python lib ‘/Volumes/MusicBrainz Picard 2.3.0b1/MusicBrainz Picard.app/Contents/MacOS/Python’: dlopen: dlopen(/Volumes/MusicBrainz Picard 2.3.0b1/MusicBrainz Picard.app/Contents/MacOS/Python, 10): Symbol not found: _futimens
Referenced from: /Volumes/MusicBrainz Picard 2.3.0b1/MusicBrainz Picard.app/Contents/MacOS/Python (which was built for Mac OS X 10.13)
Expected in: /usr/lib/libSystem.B.dylib
in /Volumes/MusicBrainz Picard 2.3.0b1/MusicBrainz Picard.app/Contents/MacOS/Python

Which certainly looks like the minimum is now 10.13, so I guess my question is if that is known / intentional - I didn’t see it in the changelog

1 Like

I’m just trying out the portable beta - I just wanted to say thank you for the fingerprinting context menu! I always used to cluster the album and lookup, save, then drag it back and scan. So much faster now for that!

1 Like

I am sorry my answers gave you these feelings, that was definitely not my intention :frowning: Actually I value your feedback very much. I wouldn’t have written a lengthy reply trying to explain the fingerprinting otherwise.

I know the fingerprinting is complicated, I needed a while to wrap my head around it as well. The problem is that it is primarily a tool meant to be used by software for identifying audio files. But technical details about it surface at various points to the user. I find it difficult to balance the user expectations generated by this with a usable and functional UI.

Yes, the fingerprint is quite different, look at this textual representation of an AcoustID fingerprint. Another way to present those fingerprints is graphical. The text representation is not really suited for display. I experimented with displaying this fingerprint in the tooltip for the fingerprint status, but it is not really helpful. Actually there is a defined tag in Picard acoustid_fingerprint to store this fingerprint, and Picard even can us the fingerprint from it when you run analyze. But it does not save it. I think the original idea was to cache fingerprints in the files, but AFAIK luks decided that generating a fingerprint is working fast enough to not rely on cached data from the files. But I’m considering adding an option to store the fingerprint anyway.

About your concern: So for now this “Generate AcoustID fingerprints” is really meant as a solution to submit data to AcoustID for already matched files. If your primary concern is to get the AcoustID tag filled this would need to work differently. My main problem with a button to get the AcoustID is again managing user expectations. Users expect to press this button and get an AcoustID in 100% of the cases. If not they report a bug. But actually that’s not how it works, AcoustID might very well not return a result for this fingerprint. In this case you need to match the file first, submit the fingerprint, wait a while until the server has processed the request, then try again to get an AcoustID. If not the server’s submission queue is probably longer, so wait a bit longer, try again.

I have added a ticket for this at PICARD-1732. So far I could not reproduce it.

But it would be really great if you could report things like this directly to the bug tracker. The forums are great for discussions of ideas and questions how things work, but issues reported here will quickly get lost.

4 Likes

That’s unintentional. But we changed the build system this release cycle, and this likely caused the issue (we had to update the build to address requirements of macOS 10.15, and also we unified a lot of the packaging and deployment, which previously was handled by two separate services for Windows and macOS). As the limiting factor seems to be the Python install I will see whether we can fix this. We should be able to use the official Python binaries which should run on macOS 10.12 still.

My main problem is that I actually can’t test on macOS 10.12. I only have a single mac mini which currently runs 10.15. I probably could try to install a virtualized macOS 10.12 there, but that machine is already struggling with performance, I doubt it would work well :frowning:

Update: Tracked at PICARD-1733

1 Like

It is not your answers that did that. I am more annoyed at wasting my own evening on a total misunderstanding of how it worked. Thank you for your time to explain it. When time allows I’ll try and look again at it.

I also realise I am confusing the names of “fingerprint” and “AcoustID” in my descriptions above. Better than when I first joined and spent six months calling it “AcousticID”. Even allowing for the WrongSpeak there are still issues described in there.

To reproduce it is about timing and working with 100 large FLAC files. So go to a slower PC to test. Or put a big load onto the computer at the same time - CPU and network (run fifteen YouTube videos in the background or similar?). One trouble with Dev PCs is they are often powerful machines, and then the home user runs it on a slower PC.

If I can find any time in coming days I’ll see if I can make a mini video of this in action. The main step is having a long list of FLAC files being fingerprinted and lots of art being downloaded. Same with the report of pressing Fingerprint then Lookup on 100files. If lookup gets that match too quick, the fingerprint generation gets abandoned.

When the files are lost dragging from right to left I guess a memory leak will build up due to the way the list shrinks.

Sorry about the Tickets. Personal thing. I would get too depressed watching tickets get argued over and ignored for years. Maybe another month I’ll have a better head and able to make yet another account up and try and get my head around that Ticket system.

1 Like

I just made the same mistake as @IvanDobsky - confusing the fingerprint feature with getting the AcoustIDs set in my tracks :confused:. I was looking forward to quickly updating the AcoustIDs on my tracks, but the fingerprinting feature does nothing for that. It is useful when creating new AcoustID records in the MB database, but not helpful for backfilling existing AcoustIDs.

3 Likes

@arturus I fixed the build and also got access to a test macOS 10.12. Could you try the package from https://keybase.pub/phwolfer/musicbrainz/macos-app/MusicBrainz-Picard-2.3.0.beta1%2B6937.ffd9f70f.20200206160451.dmg ? It should work.

1 Like

Works great! Thank you. Had a minor visual glitch:

That went away after I loaded files, and did not return when I quit and started the program again, so really minor, but I felt like I should note it.

2 Likes