Picard doesn't recognize some non-Latin characters?

MusicBrainz Picard
, ,
1

It seems that Picard has issues recognizing file/folder names that have non-Latin characters in them. For example:

While attempting to run this Classical album through Picard, it choked and could only barely find the correct album because it recognized 1 of the 14 tracks. I had to manually drag the other 13 into their correct locations. While it was able to save that one, the rest of the tracks would not save, giving me the dreaded “no” or “error” symbol (red circle with a horizontal line in the middle) next to them and turning their rows red.

Here are some screenshots of the process.

First, when adding the folder, before clustering:

Second, after clustering and running the “Scan” button (“Lookup” would not work), then dragging over the remaining unmatched 13 tracks:

1054×507

Third, after manually dragging the tracks from “Unmatched Files” to their correct “slots”:

1051×496

Fourth, after hitting the “Save” button in the toolbar:

1053×395

And here’s how the details of a selected track (#1 in this case) look:

1933×404

For the curious, the folder name where all of these files reside is:
Wolfgang Amadeus Mozart, Slovak Philharmonic Orchestra, Slovak Philharmonic Chorus, Zdeněk Košler - Requiem (1989) [CD - MP3 - 320]
and the first track (one of the 13 that Picard didn’t recognize) has the filename:
01 - Wolfgang Amadeus Mozart - Requiem in D minor, K. 626 (Süßmayr completion)- I. Introitus- ''Requiem aeternam''.mp3

When I renamed the folder to just:
Requiem (1989) [CD - MP3 - 320]
and removed the offending parts of the filenames, so the first track for example looks like:
01 - Requiem in D minor, K. 626 - I. Introitus- ''Requiem aeternam'',
then Picard was able to detect the files without any trouble at all.

Thus, it appears that this is some sort of bug with Picard and various international (non-Latin) characters, such as ě, š, ü, and ß.

2

The red icon on your first screenshot indicates an error loading the files. Please take a look at Help > “View debug/error log” to get the error details.

Also please give some details of your operating system, on what kind of file system the files are located and about the locale / language settings of your OS.

1 Like
3

More likely you have gone over the maximum 260char file name length for the OS. Notice how the only one that has matched is the one with the shortest filename. Just adding that Album name and Track name together is a 250 characters without knowing what folder it is sitting in.

Wolfgang Amadeus Mozart, Slovak Philharmonic Orchestra, Slovak Philharmonic Chorus, Zdeněk Košler - Requiem (1989) [CD - MP3 - 320]\01 - Wolfgang Amadeus Mozart - Requiem in D minor, K. 626 (Süßmayr completion)- I. Introitus- ''Requiem aeternam''.mp3

3 Likes
4

Thanks so much to both of you for replying! :pray:t2:

@outsidecontext: It turns out that Ivan’s answer is the correct one. Nevertheless, here’s the first error in the log (username obscured) in case it helps for any other reason:

E: 23:34:27,816 pluginmanager.plugin_error:184: Plugin 'classical_extras'
Traceback (most recent call last):
  File "picard\pluginmanager.py", line 268, in _load_plugin_from_directory
  File "C:\Users\<username>\AppData\Local\MusicBrainz\Picard\plugins\classical_extras.zip\classical_extras\__init__.py", line 121, in <module>
    PRESERVE = [x.strip() for x in config.setting["preserved_tags"].split(',')]
AttributeError: 'list' object has no attribute 'split'
E: 23:34:46,970 util.thread.run:64: Traceback (most recent call last):
  File "site-packages\mutagen\_util.py", line 251, in _openfile
FileNotFoundError: [Errno 2] No such file or directory: "X:\\downloads\\music\\--temp\\Wolfgang Amadeus Mozart, Slovak Philharmonic Orchestra, Slovak Philharmonic Chorus, Zdeněk Košler - Requiem (1989) [CD - MP3 - 320]\\01 - Wolfgang Amadeus Mozart - Requiem in D minor, K. 626 (Süßmayr completion)- I. Introitus- ''Requiem aeternam''.mp3"

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "picard\util\thread.py", line 60, in run
  File "picard\file.py", line 171, in _load_check
  File "picard\formats\id3.py", line 259, in _load
  File "picard\formats\id3.py", line 682, in _get_file
  File "site-packages\mutagen\_file.py", line 48, in __init__
  File "site-packages\mutagen\_util.py", line 155, in wrapper
  File "contextlib.py", line 112, in __enter__
  File "site-packages\mutagen\_util.py", line 272, in _openfile
mutagen.MutagenError: [Errno 2] No such file or directory: "X:\\downloads\\music\\--temp\\Wolfgang Amadeus Mozart, Slovak Philharmonic Orchestra, Slovak Philharmonic Chorus, Zdeněk Košler - Requiem (1989) [CD - MP3 - 320]\\01 - Wolfgang Amadeus Mozart - Requiem in D minor, K. 626 (Süßmayr completion)- I. Introitus- ''Requiem aeternam''.mp3"

@IvanDobsky: That was it; thank you. I was completely unaware of the 260-character limit, but it does make sense that a limit exists; I just figured it would be larger than 260. As you can see from the error log above, the first track wound up being 281 characters long. The full path to track #11 in that same folder is 257; just under the limit.

Again, thanks for figuring this out, Ivan! :smiley:

2 Likes
5

@SturmB - glad to help out. It is an ancient Windows limitation. Your initial test when you renamed the folder to Requiem (1989) [CD - MP3 - 320] is what sent you down the wrong path. That took you down under the 260 limitation.

I spotted it because of track 11 being the shortest on screen and the only one to work. I also had a mate phone me up last week with the exact same problem in MP3tag.

1 Like
6

Starting with Windows 10 version 1607 that limitation is mostly gone.

2 Likes
7

@chaban You are right, but

There is one caveat. This new setting won’t necessarily work with every application out there, but it will work with most. Specifically, any modern applications should be fine, as should all 64-bit applications. Older 32-bit applications need to be manifested in order to work, which really just means that the developer has indicated in the application’s manifest file that the application supports longer paths. Most popular 32-bit apps should experience no problem. Still, you don’t risk anything by trying the setting out. If an application doesn’t work, the only thing that will happen is that it won’t be able to open or save files that are saved in places where the full path exceeds 260 characters.

Source: https://www.howtogeek.com/266621/how-to-make-windows-10-accept-file-paths-over-260-characters/

AFAIK The built-in Microsoft Windows File Explorer (and also Office 365) is still not capable to work with paths over 260 characters, or I’m wrong?

1 Like
8

I think with this Windows 10 change mentioned above even the Explorer can finally handle those paths, but I’m not sure. It still requires a registry change, though.

Overall this has been quite a patch work in the past and Microsoft has been struggling with lifting this limit while staying backward compatible. All the support before has been kind of a hack, only with Windows 10 they start fulling supporting it. Having the Explorer struggle with those files tells a lot.

When I tested this with Picard a while back I was able to load and save files with long path names when I opened the files via the file dialog, but when drag and dropping those files Picard does not even notice (dropping files with long names is handled completely different in Windows than short file names). I have not yet tested with long filename support enabled in Windows 10, though.

1 Like
9

In a freshly installed Windows 10 Pro, Build 2004, fully patched, you can create folders and subfolders with about max. 400 characters (after changing the registry with the LongPathsEnabled entry and device reboot), but this are only 150 characters more as usual :woozy_face: :face_with_monocle:

If you try to create a folder beyond this limit, you still get the usual warning screen:
Path too long
If you try to create a new textfile, you get a “Neues Textdokument.txt” but you are unable to rename it :exploding_head:

So I would say: NO, the Windows File Explorer is still not capable. Unfortunately.
It seems, that even the newest available Excel in Office 365 is not capable, at least not in daily practice.

3 Likes
10

@SturmB I tested this again on an upt-to-date Windows 10 system, both with the registry key for long file support enabled and without. You might be able to work partially with long paths in Picard.

So in general you have three ways to load files into Picard:

  1. Via Add Files / Add Folders dialog: This actually works for me, no matter if long file format is enabled or not (1). So this might be a solution if you want to handle your files with Picard. But be aware that currently Picard is hard coded to shorten file paths to 260 characters if it renames files and generates folders.
  2. Via drag and drop from the integrated file browser. This is what you have done, and it results in the error you have posted. This does not work at all, no matter if long file name support is enabled. I did not know about this issue, but I will debug it.
  3. Via drag and drop from outside, e.g. from Explorer. Files dropped on Picard this way by default are not even recognized by Picard, Picard does not even notice something got dropped (2).

For case 3) however there is a way forward: If long path support is enabled in registry and the application manifest (picard.exe.manifest) also indicates long file support (3), then drag and drop also works. Which is great. Nope, unfortunately not true. In my test Explorer was actually passing the shortened DOS compatible file name for the long path (something like C:\Users\Developer\Music\LONGPA~1\WOLFGA~1\01-WOL~1.INT, another legacy thing Windows still handles), hence it worked. Tested on a different path where Explorer uses the \\?\ prefix and drag and drop does not work. So this needs to be handled separately.

I think going forward we should extend Picard in two ways:

  1. Extend the application manifest by default to indicate long file support
  2. If long path support is enabled in the registry on Windows 10 then don’t enforce the 260 character path limit. But maybe allow the user to enable it again. Not everybody wants to end with paths the Explorer cannot handle properly.
  3. Fix the issue with the internal file browser.
  4. We should also have a FAQ entry about enabling long file support.

Long paths issues are actually tracked in the following ticket:

(1) This is handled by special file paths: Windows at some time extended the classic Windows API so that file paths starting with the special character sequence “\?” can exceed the path limit. This really is kind of a hack, but it allows applications to access those paths. If you open a long path via the dialogs the application receives a path with this prefix (e.g. \\?\C:\A Very Long Path With More Than 260 Characters ...\Some File.mp3)
(2) This is because WIndows handles drag and drop differently in this case, and this is not supported by Qt5, the GUI library Picard uses. A manual implementation of this might be possible.
(3) See https://docs.microsoft.com/en-us/windows/win32/fileio/naming-a-file#enable-long-paths-in-windows-10-version-1607-and-later

5 Likes
11

Normal users will not know to add a registry switch. The average user doesn’t even realise there is a limit in the OS. Otherwise they wouldn’t try and make such insanely long folder names. :slight_smile:

IMHO there are more important issues to spend time on in Picard. Allowing people to create insanely long paths will just catch the User out when the files are used elsewhere.

2 Likes
12

I mostly agree, that’s why we maintained that path limit on Windows. Even if you can create longer paths it is not helpful if you than cannot access or modify those files with other software, including the default file browser.

But it still would be great if Picard would not fail loading those paths. I think getting the basic loading working isn’t too difficult (but still there are likely a lot of cases to consider for full support). But the drag and drop issue is tricky. I looked at that recently, implementing this requires some effort. It really is not high on my priority list.

Claiming that Windows supports long paths is unfortunately still not entirely true. What is true is that MS is working on it and there is partial support.

3 Likes