When trying to save an album, some files are not getting saved. And I see the following error in the log.
I’ve traced the error to the calculated offset being 1 byte larger that the measured filesize
Note this happens to just some files within an album. I am on MacOS and saving to a NTFS external disk.
That seems to be basically the same issue as reported for mutagen (the tag reading component Picard uses) at https://github.com/quodlibet/mutagen/issues/496
That issue is about WAVE files, but AIFF and WAVE files share the same basic structure and they share some code in mutagen.
In my (phw on Github) comments there I mention that probably mutagen should just fail more cleanly. That would be the easiest solution, but would not change much for your situation with Picard. The files are in a way broken and loading would fail.
But maybe we could detect cases like this, where the internal structure and the sizes noted there are fine but the file is missing some bytes at the end, simply assume the missing bytes as null bytes and add them. I haven’t looked into this in detail yet, though.
Thanks for your response, since I figured out how to run from source and can now debug the problem I will see if can work up a solution. I need to see if the issue is always just the size of the offset being one byte larger than the filesize.
I have been ripping a few of the Backstreet Boy albums and I noticed on a few of the albums I keep running into the same issue even with re-ripping the CD in itunes. For the album black and blue, 5 of the 13 songs have the red circle with exclamation mark as soon as I hit save and then I tried re-ripping it and deleting the old files, and it keeps happening on the same songs. I’ve tried using Musicbrainz Picard in admin mode, but same thing still happens. I have no idea what’s going on here. I have included some imgur links so you can see a visual of the error log and what keeps happening.
That’s the issue reported at Cannot write tags to certain WAV files · Issue #496 · quodlibet/mutagen · GitHub .
@outsidecontext Hmm - I just read the issue you linked to and I see that your fix PR for this issue has been awaiting merging now for almost 2 years without review. I really don’t understand this because this repo is not abandoned (regular commits as recent as 2 days ago) and you are a highly respected audio-code developer.
Perhaps Picard should “temporarily” use your fork of mutagen until such time as required fixes get merged into the root repo.
Sounds like an immediate solution for @orbit317, if wanted, would be to change format to FLAC for now.
I think the problem with the pull request review is that someone really needs to deep dive into the format details and how the affected files differ from the expectations.
I actually had also thought about just adding this patch to Picard as well. But I personally would really have liked to have my changes thoroughly reviewed and verified by someone else before putting them into production use, though.
The current bug comes from some misunderstanding (or at least different understanding) on how a valid IFF file needs to be structured (essentially trailing empty data is often omitted by other tools, even though the sizes recorded in the file structure indicates it is there). But while the issue is annoying and a possible deal breaker for affected users it at least completely fails to read such files and refuses to do any changes to them.
The changed code tries to handle such files anyway. But if something is wrong with this code the consequences are more serious, potentially corrupting the files.
But agreed, this finally needs to be fixed. Maybe I should try to get in touch with Christoph from mutagen again.
I am not knowledgeable about the internals of the WAV file and tag structures, so I am not going to be able to help here. However given the variable quality of audio file structure and tagging standards, the scope for different interpretations and the likelihood that different tagging and player software works differently, in general terms these sorts of issues feel to me to be problematic. Consequently I have no personal views on whether your solution is correct or whether others would consider it correct or what the real consequences on the audio files would be if you did this. That said…
It seems to me that sufficient analysis has been done to know that a mismatch between the actual file length and the internally stated length is the cause for mutagen crashing. The question is what to do when you discover this:
If you need to read the missing bytes, assume that they are zeros. If you need to write the file, only add missing bytes if you need to write something into them or after them. This would be the least intrusive fix as it wouldn’t change the existing file. This would mean a fix that avoids the crash rather than fixing the file.
As above, but whenever you write to the file, extend it with zero bytes to make the lengths match. Optionally you could decide to use sparse writes so that the file has the correct length, but doesn’t use additional disk space - or you could write actual zero bytes.
I have no idea which option would be best - not the least because of the unknown impact on all the various bits of software on various o/s and hardware that read (and play) these files. But from a quick code review it seems that both #515 and #517 fit option 2 rather than option 1.
I think it might also be worth trying to find a bunch of wav files that have this issue, try out a “fixed” version of mutagen on these files and then see if we can get as many people to try the old and new version of these files on as many different pieces of hardware and software as possible to check that the files are still playable afterwards. (There might be copyright issues with this suggestion, in which case we might need to limited examples to CC audio or create artificial examples of corrupt files to test.)
That’s essentially exactly what the patch achieves.
I’m pretty confident in this patch, really. It worked well with all the different error files I received for various occurrences of the issue, and I don’t really think it can break other files.
That was discussed in my first approach in a separate PR and was considered dangerous, maybe even a potential security issue.
@orbit317: I build a portable install of Picard with the fixes to handle such files included. Could you do me the favor of testing this with your files? Please make a backup of your files before you do so!
The grab the portable build from https://transfer.sh/bj9wJp/MusicBrainz-Picard-2.9.0.alpha1+1901.65ad6f5d.20230214083151.exe and try to load and tag files. Does it work without issues?
@outsidecontext Hey, just wanted to get back to you about trying out your portable build. A bit of what you guys are talking about is over my head, but wanted to report back. I used the portable build, and it worked! All files were saved perfectly without error! Is there any other things you’d like me to try?
Thanks a lot. That’s really great to hear, much appreciated. Your results are all the information I needed. I also have a couple of test files here I had from similar reports, and they are also all working fine with the changes.
I also have good news as I have now the possibility to get the changes into the mutagen project (which is the software library used by Picard to read and write all the different tagging formats), and that means we’ll get the bugfix finally out. I’ll also make sure the next Picard release will contain all the needed changes.
Thanks again for your support here and sorry for the trouble.
Just wanted to let you know…The program you work on is wonderful! Because of this program I was able to tag so much of music library (which I’ve been working on sifting through 1200 songs little by little for weeks). It really is an amazing program and without it, I think I would have just accepted defeat. Anyway, thank you for everything!
Do you still have this file available? the link here is dead. I was having the same issues mentioned in this thread with .wav files, however the 2.9.0a1 pre-release on github does not fix the problem. I tried both the installer version and the portable version, and no dice.
I can trigger building it again. I’ll do this later today and I’ll upload it somewhere more permanent. Thanks in advance for testing.
Here are fresh builds of latest Picard development version with patched mutagen 1.46:
Windows portable: https://s3.eu-central-1.amazonaws.com/artifacts.picard.uploadedlobster.com/MusicBrainz-Picard-2.9.0.alpha2%2B1933.e9d2bda3.20230518083456.exe
Please let me know if this handles your files properly. Please keep a copy of your original files before you manipulate them with Picard just to be safe nothing breaks.
Just wanted to let you know, so far so good! tested every album I’d had issues with in the past and they all saved without problem. Let me know if there’s anything in particular you want me to try.
Thanks a lot. When loading the files, saving the tags works and when the files afterwards load and play in other tools without issue that’s all I need to know.
Overall from all my own testing I’m very confident these fixes work as intended 
