Subject: [Cellar] Matroska Interlacement proposal draft

From: Dave Rice <[email-masked]>
Date: Sun, 10 Jan 2016 15:08:06 -0500

Hi Cellar,

This proposal continues from earlier conversation on matroska-devel: http://lists.matroska.org/pipermail/matroska-devel/2015-October/004808.html.

Currently Matroska only expresses video interlacement in the FlagInterlaced Element, which is defined like this (see: http://matroska.org/technical/specs/index.html#FlagInterlaced):

Element Name: FlagInterlaced
Level:        4
ID:           [9A]
Mandatory:    mand.
Multiple:     -
Range:	0-1
Default:      0
Type:         u
Description:  Set if the video is interlaced. (1 bit)

An issue here is that there isn’t a distinction between unknown and progressive interlacement. Also if the video is interlaced there is no clarification if it is top or bottom field first (or one of the progressive segmented frame options). While some video encodings (such as DV and FFV1) can contain their own information about interlacement, many (especially uncompressed formats) can not.

I propose the values are expanded such as:

0	undetermined
1	interlaced
2	progressive

or

0	progressive
1	interlaced
2	undetermined

I am unclear if FlagInterlaced=0 has been traditionally interpreted as undetermined or progressive. Advice?

Then a second element can optionally contain field ordering metadata.

Element Name: FieldOrdering
Level:        4
ID:           [9D]
Mandatory:    -
Multiple:     -
Range:	0-2
Default:      0
Type:         u
Description:  Declare the field ordering of the video. If FlagInterlaced is not set to 1, this Element MUST be ignored.

0	undetermined
1	interlaced (bottom field first)
2	interlaced (top field first)

To do:
- consider progressive segmented fields
- consider noting field storage order in addition to field display order

I did consider that bff and tff values could be added as new values to FlagInterlaced, so that only one element would be needed. However it may affect reverse compatibility since some muxer may interpret only a value of FlagInterlaced=1 as interlaced.

Dave Rice

---

From: Steve Lhomme <[email-masked]>
Date: Mon, 11 Jan 2016 11:12:42 +0100

> I am unclear if FlagInterlaced=0 has been traditionally interpreted as undetermined or progressive. Advice?

Progressive.

> I did consider that bff and tff values could be added as new values to FlagInterlaced, so that only one element would be needed. However it may affect reverse compatibility since some muxer may int…

Not sure any player/demuxer/muxer would make much use of that flag as
it is now (VLC ignores the data). So it's ok to break backward
compatibility.

> Dave Rice
> _______________________________________________
> Cellar mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/cellar

-- 
Steve Lhomme
Matroska association Chairman

---

From: Dave Rice <[email-masked]>
Date: Mon, 11 Jan 2016 11:24:29 -0500

> On Jan 11, 2016, at 5:12 AM, Steve Lhomme <[email-masked]> wrote:
> 

It does seem unconventional to have the default value be progressive rather than undetermined. From the prior definition of FlagInterlaced, "Set if the video is interlaced. (1 bit)”, it doesn’t not indicate whether a zero value means progressive or undetermined. For instance, if I create a file under the rules below and want to express certainty that the file is progressive I can set FlagInterlaced to 0; however that file is no different than an older file which has FlagInterlaced set to 0 because there is no certainly about that quality at all.

I updated and merging the concepts of FlagInterlaced and FieldOrdering into only FlagInterlaced. I suggest considering 0 as undetermined and 2 as progressive as I think this more clearly reflects the level of certainty used in past files. Also note that I left FlagInterlaced=1 to still be considered as interlaced although the field dominance is not clear as this seems semantically equivalent to the current specs, though maybe its usage should not be recommended.

Update FlagInterlaced  version 2
=============
Element Name: FlagInterlaced
Level:        4
ID:           [9A]
Mandatory:    mand.
Multiple:     -
Range:  0-4
Default:      0
Type:         u
Description: 	A flag to declare is the video is known to be progressive or interlaced and if applicable to declare details about the interlacement.
			0: progressive
			1: interlacement (unknown field ordering)
			2: undetermined
			3: interlaced (bottom field first)
			4: interlaced (top field first)

Dave Rice

---

From: Jerome Martinez <[email-masked]>
Date: Mon, 11 Jan 2016 17:40:20 +0100

> It does seem unconventional to have the default value be progressive rather than undetermined. From the prior definition of FlagInterlaced, "Set if the video is interlaced. (1 bit)”, it doesn’t not…

+1 (we have no way to know if the muxer wanted to say "undetermined" or 
"progressive", and I prefer "undetermined" in that case)

> I updated and merging the concepts of FlagInterlaced and FieldOrdering into only FlagInterlaced. I suggest considering 0 as undetermined and 2 as progressive as I think this more clearly reflects t…

-1.
 From my point of view, they are 2 different pieces of metadata, so I 
prefer to have 2 Matroska elements, with 2 different lists we could 
expand in the future independently.
BTW, what about 3:2 pulldown? set as "undetermined" or a specific item? 
This is an example of the reason I prefer 2 Matroska elements: let's 
imagine that we don't put a 3:2 pulldown specific item now (set as 
"undetermined field ordering"), but we decide later to add it (let say 
5). with merged concepts, an old reader would lose the interlace 
information despite the fact stream has not changed but the element has 
changed from 1 ("interlaced undetermined") to 5 ("interlaced 3:2 
pulldown", but "unknown for this old reader). With separate concepts, 
the old reader would still know that the stream is interlaced.

---

From: Dave Rice <[email-masked]>
Date: Mon, 11 Jan 2016 12:08:01 -0500

> On Jan 11, 2016, at 11:40 AM, Jerome Martinez <[email-masked]> wrote:
> 
> +1 (we have no way to know if the muxer wanted to say "undetermined" or "progressive", and I prefer "undetermined" in that case)
> 
>> 
> From my point of view, they are 2 different pieces of metadata, so I prefer to have 2 Matroska elements, with 2 different lists we could expand in the future independently.
> BTW, what about 3:2 pulldown? set as "undetermined" or a specific item? This is an example of the reason I prefer 2 Matroska elements: let's imagine that we don't put a 3:2 pulldown specific item now (set as "undetermined field ordering"), but we decide later to add it (let say 5). with merged concepts, an old reader would lose the interlace information despite the fact stream has not changed but the element has changed from 1 ("interlaced undetermined") to 5 ("interlaced 3:2 pulldown", but "unknown for this old reader). With separate concepts, the old reader would still know that the stream is interlaced.

Update FlagInterlaced  version 3
=============
Element Name: FlagInterlaced
Level:        4
ID:           [9A]
Mandatory:    mand.
Multiple:     -
Range:  0-2
Default:      0
Type:         u
Description: 	A flag to declare is the video is known to be progressive or interlaced and if applicable to declare details about the interlacement.
			0: undetermined
			1: interlacement (unknown field ordering)
			2: progressive

0       undetermined

For "interlaced 3:2 pulldown”, wouldn’t such as expression require additional information to align the ordered frames with the pattern of the pulldown? Additionally if the content is edited the application of the pulldown could be offset within the timeline. Perhaps we should consider extending FieldOrdering to encompass "interlaced 3:2 pulldown" as a separate issue and focus the thread on clarifying progressive, interlaced, or undetermined along with field ordering information.

Dave Rice

---

From: Jerome Martinez <[email-masked]>
Date: Mon, 11 Jan 2016 18:12:03 +0100

Le 11/01/2016 18:08, Dave Rice a écrit :
> [...]
> For "interlaced 3:2 pulldown”, wouldn’t such as expression require 
> additional information to align the ordered frames with the pattern of 
> the pulldown? Additionally if the content is edited the application of 
> the pulldown could be offset within the timeline. Perhaps we should 
> consider extending FieldOrdering to encompass "interlaced 3:2 
> pulldown" as a separate issue and focus the thread on clarifying 
> progressive, interlaced, or undetermined along with field ordering 
> information.

"3:2 pulldown" was mostly an example about the reason I prefer 2 
separate elements, I am OK for postponing this subject (such stream is 
currently not handled by current Matroska, so not a problem, it is just 
"undetermined" for the moment).

---

From: "Sebastian G. <bastik>" <[email-masked]>
Date: Mon, 11 Jan 2016 19:21:33 +0100

> Update FlagInterlaced  version 3

'(unknown field order)' is not required if it is split into to elements.
I have no preference for or against splitting it up.

> 2       interlaced (top field first)

I think that using '0' as no statement is a good decision.

(I always wondered why two ways of interlacement exist as it makes it
more complex than it has to be. I'd like to have full frames, anyway.)

--
Sebastian

---

From: Dave Rice <[email-masked]>
Date: Mon, 11 Jan 2016 15:42:23 -0500

> On Jan 11, 2016, at 1:21 PM, Sebastian G. <bastik> wrote:
> 

[…]

Based on comments, I made the following patch. I also renamed the proposed “FieldOrdering” element to “Field Order” and change the meaning of FlagInterlaced=1 to simply “interlaced”.

Creating the patch for specdata.xml brought up two new issues:

The format of specdata.xml includes an attribute called ‘minver’ to refer to which version of Matroska began supporting the element. According to matroska.org "Version 4 is currently work in progress. There may be further additions to v4.”; however there has been no distinction within specdata.xml between ‘in progress’ and official. Perhaps Matroska should mark version 4 as official as-is and the ongoing work within CELLAR would work towards a release of version 5. In the patch below I used minver=“5RC” as (version 5 release candidate) in infer that it is not official (yet).

Also specdata.xml includes a boolean attribute to say if the element is supported by webm. FlagInterlaced is supported by webm (as correlates to http://www.webmproject.org/docs/container/), but I’m leaving webm=“1” off of the new element “FieldOrder” (as I’m not on that team).

From 5d7f7b8122232c20a3e39de70d587f684b48cd2f Mon Sep 17 00:00:00 2001
From: dericed <[email-masked]>
Date: Mon, 11 Jan 2016 15:11:26 -0500
Subject: [PATCH] clarify FlagInterlaced and add FieldOrder element

---
 spectool/specdata.xml | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/spectool/specdata.xml b/spectool/specdata.xml
index 57abb97..2167e15 100644
--- a/spectool/specdata.xml
+++ b/spectool/specdata.xml
@@ -109,7 +109,8 @@ between two successive fields at the output of the decoding process (see <a href
   <element name="TrackTranslateCodec" level="4" id="0x66BF" type="uinteger" mandatory="1" minver="1" webm="0">The <a href="http://www.matroska.org/technical/specs/index.html#ChapProcessCodecID">chapter codec</a> using this ID (0: Matroska Script, 1: DVD-menu).</element>
   <element name="TrackTranslateTrackID" level="4" id="0x66A5" type="binary" mandatory="1" minver="1" webm="0">The binary value used to represent this track in the chapter codec data. The format depends on the <a href="http://www.matroska.org/technical/specs/index.html#ChapProcessCodecID">ChapProcessCodecID</a> used.</element>
   <element name="Video" cppname="TrackVideo" level="3" id="0xE0" type="master" minver="1">Video settings.</element>
-  <element name="FlagInterlaced" cppname="VideoFlagInterlaced" level="4" id="0x9A" type="uinteger" mandatory="1" minver="2" webm="1" default="0" range="0-1">Set if the video is interlaced. (1 bit)</element>
+  <element name="FlagInterlaced" cppname="VideoFlagInterlaced" level="4" id="0x9A" type="uinteger" mandatory="1" minver="2" webm="1" default="0" range="0-2">A flag to declare is the video is known to be progressive or interlaced and if applicable to declare details about the interlacement. (0: undetermined, 1: interlaced, 2: progressive)</element>
+  <element name="FieldOrder" cppname="VideoFieldOrder" level="4" id="0x9D" type="uinteger" mandatory="1" minver="5RC" default="0" range="0-2">Declare the field ordering of the video. If FlagInterlaced is not set to 1, this Element MUST be ignored. (0: undetermined, 1: interlaced with bottom field first, 2: interlaced with top field first)</element>
   <element name="StereoMode" cppname="VideoStereoMode" level="4" id="0x53B8" type="uinteger" minver="3" webm="1" default="0">Stereo-3D video mode (0: mono, 1: side by side (left eye is first), 2: top-bottom (right eye is first), 3: top-bottom (left eye is first), 4: checkboard (right is first), 5: checkboard (left is first), 6: row interleaved (right is first), 7: row interleaved (left is first), 8: column interleaved (right is first), 9: column interleaved (left is first), 10: anaglyph (cyan/red), 11: side by side (right eye is first), 12: anaglyph (green/magenta), 13 both eyes laced in one Block (left eye is first), 14 both eyes laced in one Block (right eye is first)) . There are some more details on <a href="http://www.matroska.org/technical/specs/notes.html#3D">3D support in the Specification Notes</a>.</element>
   <element name="AlphaMode" cppname="VideoAlphaMode" level="4" id="0x53C0" type="uinteger" minver="3" webm="1" default="0">Alpha Video Mode. Presence of this element indicates that the BlockAdditional element could contain Alpha data.</element>  <element name="OldStereoMode" level="4" id="0x53B9" type="uinteger" maxver="0" webm="0" divx="0">DEPRECATED, DO NOT USE. Bogus StereoMode value used in old versions of libmatroska. (0: mono, 1: right eye, 2: left eye, 3: both eyes).</element>
   <element name="PixelWidth" cppname="VideoPixelWidth" level="4" id="0xB0" type="uinteger" mandatory="1" minver="1" range="not 0">Width of the encoded video frames in pixels.</element>
-- 
2.6.4

Best Regards,
Dave Rice

---

From: Steve Lhomme <[email-masked]>
Date: Thu, 14 Jan 2016 11:56:04 +0100

> Update FlagInterlaced  version 3

Given that element is just informational (only the codec can actually
decide how to handle the data), I'm fine with this change. It won't
change playback of anything. I don't think raw video was stored with a
codec that doesn't store this information.

---

From: Steve Lhomme <[email-masked]>
Date: Thu, 14 Jan 2016 13:13:29 +0100

> +  <element name="FieldOrder" cppname="VideoFieldOrder" level="4" id="0x9D" type="uinteger" mandatory="1" minver="5RC" default="0" range="0-2">Declare the field ordering of the video. If FlagInterl…

I'll have to check if the spec generating tools supports minver as a
non integer before merging this.

---

From: Dave Rice <[email-masked]>
Date: Fri, 22 Jan 2016 00:26:05 -0500

> On Jan 14, 2016, at 5:56 AM, Steve Lhomme <[email-masked]> wrote:
> 

But doesn't 'raw video' imply that the interlacement information is not stored within the encoding. For instance v210 or uyvy422 or yuvy422 in Matroska is only sample values (some null bit padding with v210 as well), but there's no metadata about interlacement, so such encodings are wholly dependent on the container to clarify such information.

Another example was the use of FFV1 version 1 in Matroska. Neither FFV1 nor Matroksa declared field order so it was unknown. FFV1 version 3 has since added interlacement data to the encoding.

Dave Rice

---

From: Dave Rice <[email-masked]>
Date: Fri, 22 Jan 2016 00:29:45 -0500

> On Jan 14, 2016, at 7:13 AM, Steve Lhomme <[email-masked]> wrote:
> 

So perhaps we should have a separate xml for drafting while the work on the version is underway. Such as:

- specdata.xml (as the official version 4 EBML Schema)
- specdata_v5_draft.xml as a working document to patch to until Matroska version 5 is deemed official

---

From: Steve Lhomme <[email-masked]>
Date: Tue, 26 Jan 2016 12:48:05 +0100

> But doesn't 'raw video' imply that the interlacement information is not stored within the encoding. For instance v210 or uyvy422 or yuvy422 in Matroska is only sample values (some null bit padding…

Correct, this is the same reason there are fields for audio as well.
It's informational when contained in the codec, because the codec
should know better its encoding. In that case it should be moderately
trusted. When there's no other source for that information, then
that's the main source.

---

From: Steve Lhomme <[email-masked]>
Date: Tue, 26 Jan 2016 12:53:10 +0100

> - specdata_v5_draft.xml as a working document to patch to until Matroska version 5 is deemed official

Sounds like a good plan.