Subject: [Cellar] [WIP] XML Schema for EBML Schema

From: Dave Rice <[email-masked]>
Date: Mon, 17 Oct 2016 22:46:23 -0400

Hi all,

I posted a draft XML Schema in this pull request https://github.com/Matroska-Org/ebml-specification/pull/115/files. This xsd allows an EBML Schema to be validated. For instance using the EBMLSchema.xsd from the PR [1] and the ebml_matroska.xsd (expression of an EBML Schema) from the matroska-specification repo [2], one can validate ebml_matroska.xml (a draft EBML Schema) via:  

xmllint --noout --schema EBMLSchema.xsd ebml_matroska.xml

I’ve checked for other RFCs that incorporate XML Schemas and see a few such as RFC5168 and RFC5935 that do. I think an XML Schema can help the working group internally by giving us a method to verify that Matroska’s EBML Schema (ebml_matroska.xsd) stays valid during updates by the working group, though it could also be incorporated into the specification for EBML. Is putting an XML Schema in an RFC advisable or should this stay external to the RFC?

From using the draft EBMLSchema.xsd on ebml_matroska.xml, it highlights a few issues with the current ebml_matroska.xml.

- un-standardized attributes

Matroska’s EBML Schema includes unstandardized attributes such as ‘cppname’, ‘divx’, and ‘webm’. Should the EBML Schema allow unknown attributes or should these three be removed or incorporated? Feasible we could have a separate ebml_webm.xml document since it’s a separate EBML Document Type. Also the ‘divx’ elements could be removed and put into an EBML Schema extension (though we’d have to define how EBML Document Type Extensions should work).

- HTML embedded in <documentation>

The definitions in the <documentation> element include HTML tags: <a>, <br>, <del> and <strong>. Should HTML tags be permitted in EBML Schema definitions? Possibly we could reduce the need for many(all?) of them by extending EBML Schema with additional elements such as methods to handle enumeration and references (see https://github.com/Matroska-Org/ebml-specification/issues/114).

- maxOccurs is not defined as integer only

In the recent updates to the EBML Schema, maxOccurs used to be an integer or the strings ‘unbounded’ or ‘identical’, but is now integer only. However in ebml_matroska.xml we have 43 elements defined with `maxOccurs=‘unbounded’`. Should we change maxOccurs back or find some method of expressing an unbounded occurrence limit with an integer?

Dave Rice

[1]: https://raw.githubusercontent.com/Matroska-Org/ebml-specification/dc1803dfbc23102a26b1c13ebd0e1c52ca630703/EBMLSchema.xsd
[2]: https://raw.githubusercontent.com/Matroska-Org/matroska-specification/gh-pages/ebml_matroska.xml

---

From: Steve Lhomme <[email-masked]>
Date: Sun, 30 Oct 2016 09:06:47 +0100

> xmllint --noout --schema EBMLSchema.xsd ebml_matroska.xml

When running that with the linked files I get a lot of validation
failure (cppname and webm to name a few).

For cppname it was an internal thing that was useful to generate the
semantic C and C++ files used by libmatroska and libmatroska2. I will
update my tools to add those specific changes locally from the
official Matroska Schema.

For webm that should probably be stripped as well from the official
Schema. Google changes their support as they see fit. They may create
an RFC stating which elements are supported and even provide their own
Webm Schema. Also our Schema will not cover the "webm" DocType.

> I’ve checked for other RFCs that incorporate XML Schemas and see a few such as RFC5168 and RFC5935 that do. I think an XML Schema can help the working group internally by giving us a method to veri…

IMO it's good as an annex.

> Matroska’s EBML Schema includes unstandardized attributes such as ‘cppname’, ‘divx’, and ‘webm’. Should the EBML Schema allow unknown attributes or should these three be removed or incorporated? Fe…

(I should read the rest of an email before starting a reply).

Yes the DivX stuff should also be stripped. It falls in the category
of adding new elements from another RFC/source. And yes we need to
know how to "append" elements to the official/main/v1 Matroska Schema.

> The definitions in the <documentation> element include HTML tags: <a>, <br>, <del> and <strong>. Should HTML tags be permitted in EBML Schema definitions? Possibly we could reduce the need for many…

How about Markdown syntax instead ? It doesn't collide with XML. (sorry Reto)

> In the recent updates to the EBML Schema, maxOccurs used to be an integer or the strings ‘unbounded’ or ‘identical’, but is now integer only. However in ebml_matroska.xml we have 43 elements define…

The doc for the EBMLMaxOccurrence path element says:
If `EBMLMaxOccurrence` is not present then that `EBML Element` is
considered to have an unbounded `EBMLMaxOccurrence` value.

That's consistent with ABNF patterns. I suppose that's also how XML
maxOccurs work? So I suppose we should update the maxOccurs elements.
Adding 1 where it's missing, and removing elements that had unbounded.
I'll do a PR that way.

> _______________________________________________
> Cellar mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/cellar

-- 
Steve Lhomme
Matroska association Chairman

---

From: Steve Lhomme <[email-masked]>
Date: Sun, 30 Oct 2016 09:40:26 +0100

> Adding 1 where it's missing, and removing elements that had unbounded.

https://github.com/Matroska-Org/ebml-specification/pull/124
https://github.com/Matroska-Org/matroska-specification/pull/64

The latter is manually edited so there might be issues.

What about bytesize ? The validator says it's not valid but it seems
to me it should be valid.

---

From: Dave Rice <[email-masked]>
Date: Sun, 30 Oct 2016 07:53:05 -0400

> On Oct 30, 2016, at 04:40, Steve Lhomme <[email-masked]> wrote:
> 

IIIRC bytesize should be changed to size. There may have been a PR for that change.
Dave

---

From: Steve Lhomme <[email-masked]>
Date: Sun, 30 Oct 2016 14:01:40 +0100

Le 30 oct. 2016 12:53, "Dave Rice" <[email-masked]> a écrit :
> >>> I posted a draft XML Schema in this pull request
https://github.com/Matroska-Org/ebml-specification/pull/115/files. This xsd
allows an EBML Schema to be validated. For instance using the
EBMLSchema.xsd from the PR [1] and the ebml_matroska.xsd (expression of an
EBML Schema) from the matroska-specification repo [2], one can validate
ebml_matroska.xml (a draft EBML Schema) via:
> >>> I’ve checked for other RFCs that incorporate XML Schemas and see a
few such as RFC5168 and RFC5935 that do. I think an XML Schema can help the
working group internally by giving us a method to verify that Matroska’s
EBML Schema (ebml_matroska.xsd) stays valid during updates by the working
group, though it could also be incorporated into the specification for
EBML. Is putting an XML Schema in an RFC advisable or should this stay
external to the RFC?
> >>> From using the draft EBMLSchema.xsd on ebml_matroska.xml, it
highlights a few issues with the current ebml_matroska.xml.
> >>> Matroska’s EBML Schema includes unstandardized attributes such as
‘cppname’, ‘divx’, and ‘webm’. Should the EBML Schema allow unknown
attributes or should these three be removed or incorporated? Feasible we
could have a separate ebml_webm.xml document since it’s a separate EBML
Document Type. Also the ‘divx’ elements could be removed and put into an
EBML Schema extension (though we’d have to define how EBML Document Type
Extensions should work).
> >>> The definitions in the <documentation> element include HTML tags:
<a>, <br>, <del> and <strong>. Should HTML tags be permitted in EBML Schema
definitions? Possibly we could reduce the need for many(all?) of them by
extending EBML Schema with additional elements such as methods to handle
enumeration and references (see
https://github.com/Matroska-Org/ebml-specification/issues/114).
> >>> In the recent updates to the EBML Schema, maxOccurs used to be an
integer or the strings ‘unbounded’ or ‘identical’, but is now integer only.
However in ebml_matroska.xml we have 43 elements defined with
`maxOccurs=‘unbounded’`. Should we change maxOccurs back or find some
method of expressing an unbounded occurrence limit with an integer?
> IIIRC bytesize should be changed to size. There may have been a PR for
that change.

Yes, I saw it afterwards and merged it.

> >>> [1]:
https://raw.githubusercontent.com/Matroska-Org/ebml-specification/dc1803dfbc23102a26b1c13ebd0e1c52ca630703/EBMLSchema.xsd
> >>> [2]:
https://raw.githubusercontent.com/Matroska-Org/matroska-specification/gh-pages/ebml_matroska.xml

---

From: Dave Rice <[email-masked]>
Date: Sun, 30 Oct 2016 09:19:14 -0400

> On Oct 30, 2016, at 4:06 AM, Steve Lhomme <[email-masked]> wrote:
> 

Actually the EBML specification at https://github.com/Matroska-Org/ebml-specification/blob/fc7bcba4fc85fe96346860ed3afb23528dada5a7/specification.markdown#-element-1 currently says

"EBML Schemas MAY contain additional attributes to extend the semantics but MUST NOT conflict with the definitions of the <element> attributes defined within this document."

So either, the XML Schema should be adjusted to allow ANY attribute within <element> or we should remove this provision. I prefer the latter but that would mean cleaning out the cppname and webm attributes.