Subject: Re: [Cellar] Matroska Info section questions

From: Dave Rice <[email-masked]>
Date: Sat, 2 Jul 2016 01:13:52 -0400

Hi all,

> On Jun 30, 2016, at 3:13 PM, Sebastian G. <bastik> wrote:
> 
> 30.06.2016, 20:13 hubblec4:
>> Hi Sebastian
> 
> Hi Martin
> 
>> 
>> Am 30.06.2016 um 19:28 schrieb Sebastian G. <bastik>:
>>> 
>>>>> IIUC, chained Segments represent content that should be presented
>>>>> sequentially (ie. play a.mkv then b.mkv then c.mkv), but are there
>>>>> other kinds of relationships between chained Segments? Such as a
>>>>> video-only mkv and an audio-only mka chained together to play
>>>>> simultaneously?
>>>> Is that really possible to combine a video-only and an audio-only mkv to
>>>> play simultaneously? Sounds for me not really as a "chaining", more like
>>>> a "combining".
>>>> Again a question of the splitter/player.
>>> Wouldn't anyone mux a audio-only and a video only file to get a mkv that
>>> contains both, the video stream and the audio stream. Players are
>>> supposed to handle such files as they are ordinary files. Otherwise it
>>> might be hugely complex. (Don't get me wrong there are video players
>>> that allow to load a different audio file to play it along with the
>>> video.)
>> 
>> You are right. MPC-HC and other good players allows to load externel
>> audio and subtitle files
>> and play them simultaneously.
>> 
>>> I'm not sure if the intro/ending case is still valid for current seasons
>>> of any series, but the potential use was to encode the intro and the
>>> ending once. So that one has:
>>> 
>>> a.mkv (intro) then b.mkv (episode1) then c.mkv (ending)
>>> a.mkv (intro) then d.mkv (episode2) then c.mkv (ending)
>>> a.mkv (intro) then e.mkv (episode3) then c.mkv (ending)
>> And this is the next disadvantage of File-Segement-Linking.
>> Intro und ending are separted files.
>> With Chapter-Segment-Linking(CSL) looks your example like this:
>> 
>> a.mkv (intro + episode1 + ending)
>> b.mkv (CSL to a.mkv(intro) + episode2 + CSL to a.mkv(ending))
>> c.mkv (CSL to a.mkv(intro) + episode3 + CSL to a.mkv(ending))
>> b and c mkv consist of the episode only.
>> 
> 
> This is indeed more sophisticated (I'm expressing this as positive).
>> From an outside perspective the relation between those files is unclear.
> E.g. a user that removes a.mkv  after watching b.mkv might be surprised
> that c.mkv does not display correctly. Nevertheless CSL is something good.

I incorporated most of Martin and Sebastian comments in work on a pull request at https://github.com/Matroska-Org/matroska-specification/pull/24/files.

Some further questions though:

The DateUTC definition needs some help. It is currently: "Date of the origin of timestamp (value 0), i.e. production date.” Origin of timestamp? Isn’t this Element used to record the timestamp of the muxing of the Segment?

There’s a discrepancy were PrevFilename and NextFilename should be escaped but that’s not mentioned for SegmentFilename. I changed SegmentFilename in the PR to be escaped as well, but I don’t understand why a filename stored in a utf-8 element should be escaped.

The changes in the PR for the element definitions can be seen here: https://github.com/Matroska-Org/matroska-specification/pull/24/files#diff-a8ca7ff6e14a2888ab22d3cafb032f0d

I don’t quite understand ChapterTranslate and its sub-elements yet if someone wants to review/edit these.

I’m still unclear about when SegmentFamily is used. It seems Hard Linking may be done with only Prev/Next/Segment UID. Do any splitter/players use SegmentFamily for something?

For reviewability in the listserv, here is a squashed patch of the pull request.

---
 ebml_matroska.xml | 30 +++++++++++++++++++-----------
 notes.md          | 14 +++++---------
 2 files changed, 24 insertions(+), 20 deletions(-)

diff --git a/ebml_matroska.xml b/ebml_matroska.xml
index f41048c..50ca192 100644
--- a/ebml_matroska.xml
+++ b/ebml_matroska.xml
@@ -15,27 +15,33 @@
       </element>
     </element>
     <element name="Info" level="1" id="0x1549A966" type="master" minOccurs="1" maxOccurs="unbounded" minver="1">
-      <documentation lang="en">Contains miscellaneous general information and statistics on the file.</documentation>
+      <documentation lang="en" type="definition">Contains general information about the Segment.</documentation>
       <element name="SegmentUID" level="2" id="0x73A4" type="binary" minver="1" webm="0" range="not 0" bytesize="16">
-        <documentation lang="en">A randomly generated unique ID to identify the current Segment between many others (128 bits).</documentation>
+        <documentation lang="en" type="definition">A randomly generated unique ID to identify the Segment amongst many others (128 bits).</documentation>
+        <documentation lang="en" type="usage notes">If the Segment is a part of a Linked Segment then this Element is required.</documentation>
       </element>
       <element name="SegmentFilename" level="2" id="0x7384" type="utf-8" minver="1" webm="0">
-        <documentation lang="en">A filename corresponding to this Segment.</documentation>
+        <documentation lang="en" type="definition">An escaped filename corresponding to this Segment.</documentation>
       </element>
       <element name="PrevUID" level="2" id="0x3CB923" type="binary" minver="1" webm="0" bytesize="16">
-        <documentation lang="en">A unique ID to identify the previous chained Segment (128 bits).</documentation>
+        <documentation lang="en" type="definition">A unique ID to identify the previous Linked Segment (128 bits).</documentation>
+        <documentation lang="en" type="usage notes">If the Segment is a part of a Linked Segment then either the PrevUID or the NextUID Element is required. If a Segment contains a PrevUID but not a NextUID then it may be considered as the last Segment of the Linked Segment. The PrevUID must not be equal to the SegmentUID.</documentation>
       </element>
       <element name="PrevFilename" level="2" id="0x3C83AB" type="utf-8" minver="1" webm="0">
-        <documentation lang="en">An escaped filename corresponding to the previous Segment.</documentation>
+        <documentation lang="en" type="definition">An escaped filename corresponding to the file of the previous Linked Segment.</documentation>
+        <documentation lang="en" type="usage notes">Provision of the previous filename is for display convenience, but PrevUID should be considered authoritative for identifying the Previous Segment.</documentation>
       </element>
       <element name="NextUID" level="2" id="0x3EB923" type="binary" minver="1" webm="0" bytesize="16">
-        <documentation lang="en">A unique ID to identify the next chained Segment (128 bits).</documentation>
+        <documentation lang="en" type="definition">A unique ID to identify the next Linked Segment (128 bits).</documentation>
+        <documentation lang="en" type="usage notes">If the Segment is a part of a Linked Segment then either the PrevUID or the NextUID Element is required. If a Segment contains a NextUID but not a PrevUID then it may be considered as the first Segment of the Linked Segment. The NextUID must not be equal to the SegmentUID.</documentation>
       </element>
       <element name="NextFilename" level="2" id="0x3E83BB" type="utf-8" minver="1" webm="0">
-        <documentation lang="en">An escaped filename corresponding to the next Segment.</documentation>
+        <documentation lang="en" type="definition">An escaped filename corresponding to the file of the next Linked Segment.</documentation>
+        <documentation lang="en" type="usage notes">Provision of the next filename is for display convenience, but NextUID should be considered authoritative for identifying the Next Segment.</documentation>
       </element>
       <element name="SegmentFamily" level="2" id="0x4444" type="binary" maxOccurs="unbounded" minver="1" webm="0" bytesize="16">
-        <documentation lang="en">A randomly generated unique ID that all Segments related to each other must use (128 bits).</documentation>
+        <documentation lang="en" type="definition">A randomly generated unique ID that all Segments of a Linked Segment must share (128 bits).</documentation>
+        <documentation lang="en" type="usage notes">If the Segment is a part of a Linked Segment then this Element is required.</documentation>
       </element>
       <element name="ChapterTranslate" level="2" id="0x6924" type="master" maxOccurs="unbounded" minver="1" webm="0">
         <documentation lang="en">A tuple of corresponding ID used by chapter codecs to represent this Segment.</documentation>
@@ -57,7 +63,7 @@
       </element>
       TimecodeScale When combined with <a href="http://www.matroska.org/technical/specs/index.html#TimecodeScaleDenominator">TimecodeScaleDenominator</a> the Timestamp scale is given by the fraction TimecodeScale/TimecodeScaleDenominator in seconds.-->
       <element name="Duration" level="2" id="0x4489" type="float" minver="1" range="&gt; 0x0p+0">
-        <documentation lang="en">Duration of the Segment (based on TimecodeScale).</documentation>
+        <documentation lang="en" type="definition">Duration of the Segment measured in nanoseconds when scaled by TimecodeScale.</documentation>
       </element>
       <element name="DateUTC" level="2" id="0x4461" type="date" minver="1">
         <documentation lang="en">Date of the origin of timestamp (value 0), i.e. production date.</documentation>
@@ -66,10 +72,12 @@
         <documentation lang="en">General name of the Segment.</documentation>
       </element>
       <element name="MuxingApp" level="2" id="0x4D80" type="utf-8" minOccurs="1" minver="1">
-        <documentation lang="en">Muxing application or library ("libmatroska-0.4.3").</documentation>
+        <documentation lang="en" type="definition">Muxing application or library (example: "libmatroska-0.4.3").</documentation>
+        <documentation lang="en" type="usage notes">Include the full name of the application or library followed by the version number.</documentation>
       </element>
       <element name="WritingApp" level="2" id="0x5741" type="utf-8" minOccurs="1" minver="1">
-        <documentation lang="en">Writing application ("mkvmerge-0.3.3").</documentation>
+        <documentation lang="en" type="definition">Writing application (example: "mkvmerge-0.3.3").</documentation>
+        <documentation lang="en" type="usage notes">Include the full name of the application followed by the version number.</documentation>
       </element>
     </element>
     <element name="Cluster" level="1" id="0x1F43B675" type="master" maxOccurs="unbounded" minver="1">
diff --git a/notes.md b/notes.md
index 2f532a8..62d9bf8 100644
--- a/notes.md
+++ b/notes.md
@@ -101,23 +101,19 @@ The position in some Elements refers to the position, in octets, from the beginn
 
 The exact time of an object represented in nanoseconds. To find out a Block's Raw Timecode, you need the Block's timecode, the [Cluster's]({{site.baseurl}}/index.html#Cluster) [Timecode]({{site.baseurl}}/index.html#Timecode), and the TimecodeScale. For calculation, please see the see the <a href="notes.html#TimecodeScale">TimecodeScale notes.</a>
 
-### Segment linking
+### Linked Segments
 
-#### Hard linking
+#### Hard Linking
 
-This linking can also be called splitting. It's the operation of cutting one Segment in several parts. The resulting parts should play as if it was just one part (the original Segment). That means the timecode of each part follows the ones from the previous parts. The track numbers are the same. The chapters only match the current Segment (unless the edition is ordered, where all parts should be in each Segment). And most important, the NextUID and PrevUID points the respective Segment UIDs.
+Hard Linking (also called splitting) is the operation of cutting one Segment in several smaller linked Segments. The resulting Linked Segments should play as if it was the original Segment. Within a Linked Segment the timecode of each Segment MUST follow consequtively and the track numbers and timescale MUST be the same amongst the Segments. The chapters only match the current Segment (unless the Edition is ordered, where all parts should be in each Segment). Within a Linked Segment the NextUID and PrevUID MUST reference the respective Segment UIDs of the next and previous Segments.
 
 #### Soft linking
 
-Soft linking is used by codec chapters. They can reference another Segment and jump on that Segment. The way the Segments are described are internal to the chapter codec and unknown to the Matroska level. But there are Elements in the Segment Information (such as `ChapterTranslate`) that can translate a value representing a Segment in the chapter codec and to the current Segment UID. All Segments that could be used in a file/segment this way should be marked as members of the same family (SegmentFamily), so that the player can quickly switch from one to the other.
+Soft Linking is used by codec chapters. They can reference another Segment and jump on that Segment. The way the Segments are described are internal to the chapter codec and unknown to the Matroska level. But there are Elements in the Segment Information (such as `ChapterTranslate`) that can translate a value representing a Segment in the chapter codec and to the current Segment UID. All Segments that could be used in a Linked Segment in this way SHOULD be marked as members of the same family via the SegmentFamily Element, so that the player can quickly switch from one to the other.
 
 #### Medium linking
 
-This kind of linking is a mix between hard and soft linking. Each Segment linked is independant from the other (standalone, unlike hard linked ones). But it should be treated as a hard-link by the player. Medium linking is done through chapters using the `ChapterSegmentUID` Element and only makes sense for ordered editions. The Segment matching the UID should be played as if it was part of the original Segment (Segment it's linked from) and then resume playback in the original Segment. That means the timecodes of the following content should be shifted by the duration of the linked Segment. As for hard-linking, the resulting Segment edition should be played and considered as one.
-
-### SegmentUID
-
-The 128 bits UIDs must be as unique as possible. It is suggested to compute the MD5 sum of some data parts of the file (the checksum of the Cluster level if you use one).
+This kind of linking is a mix between Hard Linking and Soft Linking. Each Segment linked is independant from the others (standalone, unlike hard linked ones). But it should be treated as a hard-link by the player. Medium linking is done through chapters using the `ChapterSegmentUID` Element and only makes sense for ordered editions. The Segment matching the UID should be played as if it was part of the original Segment (Segment it's linked from) and then resume playback in the original Segment. That means the timecodes of the following content should be shifted by the duration of the linked Segment. As for hard-linking, the resulting Segment edition should be played and considered as one.
 
 ### Table Columns
 The columns from the specifications table have these meanings.
-- 
2.9.0

Best Regards,
Dave Rice

---

From: Moritz Bunkus <[email-masked]>
Date: Sat, 2 Jul 2016 09:03:37 +0200

Hey,

> The DateUTC definition needs some help. It is currently: "Date of the
> origin of timestamp (value 0), i.e. production date.” Origin of
> timestamp? Isn’t this Element used to record the timestamp of the
> muxing of the Segment?

Given its ambiguous definition and that it wasn't ever defined clearly
that's exactly how mkvmerge (and other tools) use it at the moment.

> There’s a discrepancy were PrevFilename and NextFilename should be
> escaped but that’s not mentioned for SegmentFilename. I changed
> SegmentFilename in the PR to be escaped as well, but I don’t
> understand why a filename stored in a utf-8 element should be escaped.

To be honest I don't even remember what that "escaping" was supposed to
look like, never mind why we would need to escape it in the first
place. Steve? Care to shed some light?

All uses I've seen so far simply put the file name in there and don't
bother with that "escaped" part. mkvmerge for example doesn't mangle the
name in any way, it relies on the user providing the correct value to
set that element to.

> someone wants to review/edit these.

I think those come from the days of trying to convert DVD menus to
Matroska structure. Again Steve would be the one to be able to offer
some insight.

Kind regards,
mosu

---

From: hubblec4 <[email-masked]>
Date: Sat, 2 Jul 2016 11:27:54 +0200

Hi all

> The DateUTC definition needs some help. It is currently: "Date of the origin of timestamp (value 0), i.e. production date.” Origin of timestamp? Isn’t this Element used to record the timestamp of t…

DateUTC should be used for the muxing date. When mkvmerge mux a 
segement/mkv then is this the date to be stored.

> I don’t quite understand ChapterTranslate and its sub-elements yet if someone wants to review/edit these.

This element and his sub-elements are needed for the Matroska 
Menu(DVD-Menu). I have never seen this elements in a "normal" muxed mkv. 
ChapterTranslate has only one task -> to identify an edition in a 
menu-structure.
A chapter to identify in a menu-struture is simple, cause chapters has 
elements for doing this
(ChapterProcessPrivate).
The big problem is, there are no splitters/players are able to handle 
this elements.

> I’m still unclear about when SegmentFamily is used. It seems Hard Linking may be done with only Prev/Next/Segment UID. Do any splitter/players use SegmentFamily for something?

I don't understand too for what and when and in which kind this element 
is used.
SegmentFamily is used by Soft-Linking, I see this element in 
"menu-structured" outputs of DVDMenuXtraxtor.
Steve Lhomme could help us maybe.
At the moment there are no splitter/player that use this element.

Best Regards,
Martin Below

---

From: Dave Rice <[email-masked]>
Date: Sat, 2 Jul 2016 16:05:11 -0400

> On Jul 2, 2016, at 3:03 AM, Moritz Bunkus <[email-masked]> wrote:
> 
> Hey,
> 

Updated locally to: "The date and time that the Segment was created by the muxing application or library"

> place. Steve? Care to shed some light?

I removed the reference to escaping in the three filename elements. Steve?

> Matroska structure. Again Steve would be the one to be able to offer

For now I didn’t do too much there and mostly updated Hard and Medium Linking. Please review the pull request for the latest draft. https://github.com/Matroska-Org/matroska-specification/pull/24

[…]

Dave Rice

---

From: hubblec4 <[email-masked]>
Date: Sun, 3 Jul 2016 10:40:18 +0200

Hi all

I still have a question about Segment-Linking.
What is with linked segments that contains different stream contents?

Normally all stream contents should be the same, that is what 
LAV-splitter can handle.
Example:
video a.mkv (1.stream - h264; 2.stream - AAC-6Channels; 3.stream - Subtitle)
video b.mkv (1.stream - h264; 2.stream - AAC-2Channels; 3.stream - Subtitle)
video c.mkv (1.stream - MPEG2; 2.stream - AC3-6Channels; 3.stream - 
Subtitle)

A linking between a.mkv and b.mkv is possible and works good.
But a linking a/b.mkv with c.mkv doesn't work correctly.
LAV-splitter will generate a correct new timeline, but when c.mkv is 
reached then stops/hang the playback.

Is this technically possible to link segments with different stream 
contents?
Should the player/splitter have an abillity to do this (maybe a new 
stream check/init when reached a new segment)?

Best Regards,
Martin Below

---

From: Dave Rice <[email-masked]>
Date: Sun, 3 Jul 2016 18:05:54 -0400

Hi Martin,
Moritz and I worked on the pull request more at https://github.com/Matroska-Org/matroska-specification/pull/24 and it is now merged.

> On Jul 3, 2016, at 4:40 AM, hubblec4 <[email-masked]> wrote:
> 
> Hi all
> 
> Normally all stream contents should be the same, that is what LAV-splitter can handle.
> Example:
> video c.mkv (1.stream - MPEG2; 2.stream - AC3-6Channels; 3.stream - Subtitle)

The current constraint is only that the Segments must share track numbers and timescale. See https://github.com/Matroska-Org/matroska-specification/commit/01eaa6a3b2e4fa05fe9edc2caecbfbdd2aeecfe2#diff-db235fc914867ffea6b540c8732499d5R106. 

IMHO this constraint should be more specific. Does 'share track numbers' mean that the all `TrackNumber` values of all `TrackEntry` elements must be the same amongst all Segments that are linked?

If 'timescale' means the `TimecodeScale` Element then it should be named as such.

> LAV-splitter will generate a correct new timeline, but when c.mkv is reached then stops/hang the playback.

I'm unclear if this is because the spec should not support this case or if the player does not accept this case.

> Is this technically possible to link segments with different stream contents?

I think the spec should be adjusted to make the answer to this clear. I'm not saying if that answer is yes or no, but the spec currently seems to imply that a link between a/b.mkv and c.mkv is okay, though I'm not sure this is a worthwhile feature.

> Should the player/splitter have an abillity to do this (maybe a new stream check/init when reached a new segment)?

This remind me that I'm seen in some forums a reference to a consideration from the ffmpeg developers that Linked Segments are a security concern. If so, we should detail that more in the specification as the IETF recommends that security considerations are clarified in rfc docs.
Dave Rice

---

From: hubblec4 <[email-masked]>
Date: Mon, 4 Jul 2016 12:47:41 +0200

Hi Dave, Hi all

> IMHO this constraint should be more specific. Does 'share track numbers' mean that the all `TrackNumber` values of all `TrackEntry` elements must be the same amongst all Segments that are linked?

In my Linking-tests was it possible to link segments  which have the 
same CodecID(audio) but different channel settings.
A FLAC audio with 6 channels and another one with 2 channels.
The player/splitter has no problems to play such a linking.
So I would say NO, that ALL "TrackEntry-values" must be the same.
Important at the moment is, that a splitter don't have to change the 
audio-codec while demuxing/playing.
The same works for the video content, a MPEG2-video from a DVD can 
linked with a MPEG2-video from Bluray.
(different resolutions)

> I think the spec should be adjusted to make the answer to this clear. I'm not saying if that answer is yes or no, but the spec currently seems to imply that a link between a/b.mkv and c.mkv is okay…

The whole problem is, that we have not a really good Matroska-Splitter. 
Im sure it should not a problem for a splitter to re-init the decoders, 
if the contents of  a linked segment are changed.
For the moment: The specs should say: Linking with the same contents!
Need we sample files for this? I could generate some.

Best Regards,
Martin Below