Compability between different DocTypeVersions

[mkver]

a) This issue is inspired by PR #278 of the Matroska repo. I am wondering what happens when a file with a certain DocType and DocTypeVersion x is read by a reader for that DocType that supports a higher DocTypeVersion y (with x<y) in case that the DocTypeVersion y has a new mandatory element with default value that is not part of DocTypeVersion x. MAY the reader infer that the element is present with its default value? Or is it even a SHOULD or a MUST? Or a MUST NOT?

The specs currently only say that both the EBMLVersion and EBMLDocTypeVersion are meant to be backward compatible and that a parser for a higher version can read lower versions. But it is not really clear how this is achieved:

• If it is a MUST, SHOULD or MAY, then the backward compability is a constraint on the designers of the EBML Schema: If they add new mandatory elements with default values whose parents are already part of a lower EBMLDocTypeVersion then they have to make sure that nothing bad happens when a reader for the higher DocTypeVersion reads a file with a lower DocTypeVersion and infers the newer elements that are not present.

  This is the approach that Matroska uses: E.g. when FlagEnabled was introduced with version 2, the default value was chosen to mean "enabled"; for "FlagInterlaced" it means "undetermined" etc.

• If it is a MUST NOT, then a reader for a higher version not only needs to know the elements up to the version the reader supports, but also needs to know the level of said elements in order to make the right choices for situations like the one outlined here. This puts the onus on the implementators.

I want to add that I don't see a technical reason why every EBML Schema should make the same choice between these alternatives. As has already been said, Matroska de facto adopted the MUST/SHOULD/MAY approach, but other users may choose differently (except when a constraint to EBML is added). Still, the choice for the elements in the EBML Header should be unique (and documented in the EBML specifications).

Should an EBML Schema be allowed to answer this for itself, then they should be mandated to document their answer.

b) If a reader for version y is supposed to be able to read all files with DocTypeReadVersion <= y, then this means that deprecated elements are just illegal to use in files with big enough DocTypeVersion, but that readers have to support them for all eternity? That would be a pretty hefty requirement.

c) And what is if version x had a mandatory element with default value, but said element has been deprecated in version y (the version the reader implements)? If readers really have to support deprecated elements for all eternity as the current spec seems to imply then the reader has to know about this element and MUST infer it (i.e. the reader has to play the file as if the reader were a reader for version x)?

[robUx4]

Should we have attributes like default_version to define since what version of the DocType the default attribute exists ? With its value being the same as minver in most cases ?

[robUx4]

As for deprecated values, if it's not clear already, the content should be discarded if found in a file with a newer DocType. But they are meant to be interpreted for the versions they are legit.