File metadata standards and Picard's implementation of these

[thwaller]

@outsidecontext

That Picard ads empty IPLS or TIPL and TMCL frames is IMHO a bug,

Ok, and yes, that is the initial post and query which opened up the looking that resulted in the rest.

@outsidecontext

It is just an extra empty frame, so a very minor bug of “wasting” a few bytes in some cases.

I spend a good deal of time making sure tags are correct and proper… that is just me though. So when something alters tags in a way that is unexpected or unknown to me, that is a major problem… but a minor problem in the scope from your side, I agree.

@outsidecontext

If you are referring to Picard writing TSOP (and some other v2.4 frames) to v2.3 headers, yes, this is an intentional violation of the standard.

This is just my view, and I can understand yours. If I wanted to write TSOP headers and they are not available in 2.4, then I use 2.3. And if I set a software to write 2.3 or 2.4 tags, I expect the result to be 2.3 or 2.4 tags, respectively… unless there is some indicator of deviation or overriding options that I have set. Deviating creates a hybrid standard that requires others to follow suit or segregation lines are created (ie… Apple and the rest of the world).

@outsidecontext

Probably because disabling this would effectively make the files less compatible and so far I haven’t heard about any issue that change was causing in other software.

I see your point, but personally differ. You are never less compatible by sticking to the standards. I think what you are meaning (don’t mean to speak for you) is that you are unable to use the “enhanced” functionality that others use without making the same deviations they have. To that I agree. I see this a lot with MS Word, Adobe InDesign, etc on a business level.

@outsidecontext

We are talking solely about ID3 tags here and MP4 uses a different tagging format.

Yep, just making sure and making sure it is clear, the software product and not the music product…
…made by Apple, to be non-compliant with standards. This should be expected as it is Apple, and their entire business model centers on incompatibility with others.

So it is clear, I do see your points and am only sharing my views on it as appropriate. I understand (I think) that the tag version crossing issue was/is centered on iTunes. If I may speak of opinion…no intent on debating anything but just explaining my view on this so it is understood vs seen as just arguing or complaining at nothing.

I see this as the users problem. If you use an Apple product, you made that choice… the choice to use a product(s) from a company that intentionally defies standards and causes difficulty in the marketplace for many. So if you opt to be an Apple user, it is on you to make changes to conform when sharing with others, not mine or other non users. I mean not to offend Apple users.

This issue is existent in many places… Microsoft Office, Adobe Creative Suite, programming for internet use, file systems, character sets, audio file containers and formats, etc. But in all those cases, there is a simple solution for compatibility… sticking to the standards and avoiding the proprietary and other forms of product specific enhancements.

I think the issue of the improper tag writing is “solved” as it relates to the 2.3/2.4 portion. I appreciate your explanation and have provided mine… and I cannot really say that either of us is more right or wrong than the other.

A side question… I reviewed the Picard mappings (Appendix B: Tag Mapping — MusicBrainz Picard v2.11.0rc1 documentation). When you state “iTunes MP4”, are you meaning/implying QTFF? I assume this, but QTFF and MP4 are different standards, but very closely related… and if I recall the MP4 standard was based on QTFF. The other option would be you are combining the QTFF and MP4 into the same, similar to how ID3 v2.3 and v2.4 are combined as ID3v2 and noted where different. Note that I do understand the common misconception is that the mp4 is an Apple format.

[outsidecontext]

@thwaller

Deviating creates a hybrid standard that requires others to follow suit or segregation lines are created (ie… Apple and the rest of the world).

Yes, that’s the status quo actually. And to be honest the “standardization” of media file metadata never was in a very good shape. It is often more driven by usage of big players, namely iTunes and Windows Media Player. Developers of other soft- and hardware often cared more about compatibility with those then any existing spec (which were never formally standardized anyway).

If you have to decide for your tagging software what tag / frame / atom name to use for some information you should really look at how other software is doing it, because there is no use of writing information nobody else can read. There is a reason why the ID3 developer site lists Unofficial Frames Seen in the Wild (including TSOP, TCMP, TSO2, which are all used by Picard). There are multiple examples of deviation from the specs, e.g. the use of TPE2 (Band/orchestra/accompaniment) as Album Artist. This is not following the words of the spec, but it is how virtually all music players and taggers treat it if they support the concept of Album Artist (and it was probably started by one of the usual suspects I listed above). Also rating via POPM never really got used as intended by its inventors, which wouls allow multiple ratings for different people in the same file.

Update: And since you mentioned just using ID3 v2.4 instead of v2.3: The very same software that is responsible for some of the current tag usage used to be notoriously bad at supporting ID3 v2.4, so this was not an option for many users. Overall v2.3 is still the more compatible, although I would recommend using v2.4 unless you have some component that doesn’t support it properly.

@thwaller

When you state “iTunes MP4”, are you meaning/implying QTFF? I assume this, but QTFF and MP4 are different standards, but very closely related… and if I recall the MP4 standard was based on QTFF.

Yes, the same tagging is used for Quicktime files, too. To my knowledge the underlying metadata structure is technical identical. In 2008 Apple published a document called “iTunes Metadata Format Specification” (it used to be available via their developer site after registration, unfortunately the last time I looked for it it seemed to be no longer available) that documented the use of metadata atoms for MP4 files. Picards usage of tags is in large parts based on this.

At the moment if you use Picard to tag Quicktime files they will use exactly the same tags. I think this is currently what most software does (if it supports older quicktime files at all). There might have been other tag names in use before Apple formalized the tag names a bit more with the metadata spec. But I never heard of anyone requesting something special only for Quicktime.

[thwaller]

@outsidecontext

The very same software that is responsible for some of the current tag usage used to be notoriously bad at supporting ID3 v2.4, so this was not an option for many users. Overall v2.3 is still the more compatible, although I would recommend using v2.4 unless you have some component that doesn’t support it properly.

Absolutely agree. I prefer 2.4 for a large reason that it is UTF8. If 2.3 allowed for this, I would be a user of 2.3 despite the other reasons to use 2.4.

@outsidecontext

And to be honest the “standardization” of media file metadata never was in a very good shape. It is often more driven by usage of big players, namely iTunes and Windows Media Player.

I also agree with this statement, but just not personally, but me not liking it does not change the reasonable reality. Winamp is another for that list, and in fact, I still see today Winamp specific tags being used. It is very well known that for MP4 (m4a) that Apple just does what it wants, and most will use what they do as the “standard”. Although the mp4 is not an Apple format, they are the ones (almost only ones) that use it on a large level.

@outsidecontext

In 2008 Apple published a document called “iTunes Metadata Format Specification” (it used to be available via their developer site after registration, unfortunately the last time I looked for it it seemed to be no longer available) that documented the use of metadata atoms for MP4 files. Picards usage of tags is in large parts based on this.

Yes, I have this document actually. Do you need it? I am happy to share, but unsure if I just want to post publicly. Apple might make me disappear for sharing something that tells others detail of what they do . LOL! I also have the QTFF (QuickTime File Format) document. I am not aware that iTunes (the m4a music distribution) has anything published currently aside from the QTFF specs that it follows, but if there, I would like to find it if you are aware of such a thing.

As a side comment should it be helpful in any way, there is also some deviation as it relates to TKEY. The “Camelot System” is often used which populates that frame with technically invalid data, like 12B, 1A, 12A.
RE: In http://id3.org/id3v2.4.0-frames, section 4.2.3.
TKEY
The ‘Initial key’ frame contains the musical key in which the sound
starts. It is represented as a string with a maximum length of three
characters. The ground keys are represented with “A”,“B”,“C”,“D”,“E”,
“F” and “G” and halfkeys represented with “b” and “#”. Minor is
represented as “m”, e.g. “Dbm” $00. Off key is represented with an
“o” only.

[outsidecontext]

@thwaller

Yes, I have this document actually. Do you need it? I am happy to share, but unsure if I just want to post publicly. Apple might make me disappear for sharing something that tells others detail of what they do . LOL!

Thanks, but I have it here

@thwaller

As a side comment should it be helpful in any way, there is also some deviation as it relates to TKEY. The “Camelot System” is often used which populates that frame with technically invalid data, like 12B, 1A, 12A.
RE: In http://id3.org/id3v2.4.0-frames, section 4.2.3.

Picard actually does know this tag [1], but doesn’t fill it by itself, so it doesn’t matter too much right now.

But that very well illustrates another issues with tagging specs: Often the tags are very loosely defined and open for interpretation. While for ID3 at least here is some effort for defining the values, it still gets ignored by implementors. And others might have a similar tag, but without proper value specification. E.g. Microsoft defines for ASF / WMA metadata a tag “WM/InitialKey”, but does not specify any format of the values. From their spec:

This attribute value could be: “A sharp (minor)”.

Well yeah, could be. Or something else, however you please

[1] I just noticed the key tag is missing from the tag mapping table, I submitted a pull request to fix this for the website.

[thwaller]

@outsidecontext

But that very well illustrates another issues with tagging specs:

The only part that differs to me is on one side you have misusing a frame with either data that represents a different but related purpose (Album Artist for example) or the initial key using a different standard than the spec uses (but still appropriate to the frame)… where on the other hand there is a frame that is not even in spec at all, or should be added as a user defined tag… a frame that should not exist at all vs illformed frame content. But to your point, a deviation is a deviation.

iTunes actually defies their own specification, for example the ©day atom. Their spec states “YYYY-MM-DD format string (may be incomplete, i.e. only year)”, while I can pull this from a m4a file:
Atom “©day” contains: 2018-06-08T07:00:00Z
But at least the atom itself is expected.

I think the option to turn off such things would be great, but at the same time, I would favor the opinions of tagging users over mine. I would have liked to have been aware of this though vs it being done without my knowledge.

[outsidecontext]

@thwaller Can I ask you on your opinion on the use of TPE2 frame as “Album Artist” (as discussed in PICARD-479). There was an option for this in very early Picard, but it was not considered worthwhile since no software used this field as intended by the standard.

Do you know of any software that uses TPE2 and does not misuse it as “Album Artist”? Or at least any use case where it would at least be useful not to store the album artist there?

[thwaller]

All softwares I have seen and used use TPE2 in the same as aART. However, I see this only in the implementation of the end user final product. Meaning this … mutagen uses TPE2 as Band/Orchestra/Accompaniment. However, in the implementations I have seen, the software GUI has labeled said field to Album Artist, which yes, is misused,

[outsidecontext]

Thanks for the feedback, much appreciated. I think we should keep it as is for now until someone comes up with an actual use case for having it different. No point in having an option that basically nobody will want to use.

A note about mutagen: In generall mutagen just provides access to tags, it does not define their use. But for ID3 there is even a simplified easyid3 implementation, and that also maps TPE2 to albumartist