Subject: Re: [Cellar] [Matroska-devel] Colour Format proposal

From: Steve Lhomme <[email-masked]>
Date: Mon, 18 Apr 2016 10:29:34 +0200

Following the Videolan Colorimetry workshop from this week-end, I have
a better understanding of this whole subject.

I think it's good if we can have as much information in Matroska as we
can. But they probably should be split between elements that are
necessary for accurate playback (list to be defined) and the other
information like how the content was captured on camera. The first
group should be in the TrackInfo while the others should be just tags
(so with string names) that target the whole file or some chapters
(each scene probably has different capture characteristics).

>From Frank's list I would put those in the TrackInfo:
- MatrixCoefficients
- Range
- TransferFunction
- Primaries

- BitsPerChannel
- ChromaSubsamplingHorz / ChromaSubsamplingVert
- CbSubsamplingHorz / CbSubsamplingVert

I'm not sure about the Luminance min/max but maybe it's needed for HDR ?
Not sure either about arbitrary RGB chromacity values of the
primaries. Are people deviating from the few norms that are available
? And have tools for playback ? That seems to provide the same
information as the "Primaries" field and it's never good to have 2
different set of values describing the same thing. You never know if
you have to write both and which to use for playback especially when
you need 8 values to make one set to compare...

Are MaxCLL and MaxFALL exclusive ? Are they needed for HDR playback ?

I think MasteringMetadata is not needed for playback.

For list of possible values, we shouldn't follow lists from standards
that are not freely available. We should also have the default value
be 0 when possible. And no "reserved" values.

2016-03-29 19:12 GMT+02:00 Hendrik Leppkes via Matroska-devel
<[email-masked]>:
> On Tue, Mar 29, 2016 at 6:11 PM, Frank Galligan via Matroska-devel
> <[email-masked]> wrote:
>>
>>
>> On Mon, Mar 28, 2016 at 5:05 AM, Steve Lhomme <[email-masked]> wrote:
>>>
>>> 2016-03-17 5:46 GMT+01:00 Frank Galligan <[email-masked]>:
>>> > OK really no comments for a long time.
>>> >
>>> > On Fri, Feb 19, 2016 at 1:45 PM, Michael Niedermayer
>>> > <[email-masked]> wrote:
>>> >>
>>> >> Hi
>>> >>
>>> >> On Thu, Feb 18, 2016 at 11:50:27AM -0800, Frank Galligan wrote:
>>> >> > Here is the current proposal, minus the reference to the 265 doc.
>>> >> >
>>> >> > The parent element would be Video [E0].
>>> >> >
>>> >> >
>>> >> > Element Name: Colour
>>> >> >
>>> >> > Level:        4
>>> >> >
>>> >> > ID:           [55][B0]
>>> >> >
>>> >> > Mandatory:    -
>>> >> >
>>> >> > Multiple:     -
>>> >> >
>>> >> > Default:      -
>>> >> >
>>> >> > Type:         m
>>> >> >
>>> >> > Description:  Settings describing the colour format.
>>> >> >
>>> >> >
>>> >> > Element Name: MatrixCoefficients
>>> >> >
>>> >> > Level:        5
>>> >> >
>>> >> > ID:           [55][B1]
>>> >> >
>>> >> > Mandatory:    -
>>> >> >
>>> >> > Multiple:     -
>>> >> >
>>> >> > Default:      2
>>> >> >
>>> >> > Type:         u
>>> >> >
>>> >> > Description:  The Matrix Coefficients of the video used to derive
>>> >> > luma
>>> >> > and
>>> >> >
>>> >> >              chroma values from reg, green, and blue color primaries.
>>> >> > For
>>> >> >
>>> >> >              clarity, the value and meanings for MatrixCoefficients
>>> >> > are
>>> >> > adopted
>>> >> >
>>> >> >              from Table 4 of ISO/IEC 23001-8:2013/DCOR1. (0:GBR, 1:
>>> >> > BT709,
>>> >> >
>>> >> >              2: Unspecified, 3: Reserved, 4: FCC, 5: BT470BG, 6:
>>> >> > SMPTE
>>> >> > 170M,
>>> >> >
>>> >> >              7: SMPTE 240M, 8: YCOCG, 9: BT2020 Non-constant
>>> >> > Luminance,
>>> >> >
>>> >> >              10: BT2020 Constant Luminance)
>>> >> >
>>> >> >
>>> >>
>>> >> > Element Name: BitsPerChannel
>>> >> >
>>> >> > Level:        5
>>> >> >
>>> >> > ID:           [55][B2]
>>> >> >
>>> >> > Mandatory:    -
>>> >> >
>>> >> > Multiple:     -
>>> >> >
>>> >> > Default:      0
>>> >> >
>>> >> > Type:         u
>>> >> >
>>> >> > Description:  Number of decoded bits per channel. A value of 0
>>> >> > indicates
>>> >> > that
>>> >> >
>>> >> >              the BitsPerChannel is unspecified.
>>> >> >
>>> >>
>>> >> what would this be set to for old 16bit rgb, that is 5 bit red
>>> >> 6 bit green, 5 bit blue rawvideo.
>>> >> This maybe does not matter and iam not strongly suggesting to add it,
>>> >> rather i want to point it out so its not unintentionally forgotten
>>> >
>>> >
>>> > I didn't get into RGB here. As you said 565, 551, there are a good
>>> > amount of
>>> > combinations. I think DirectShow (or maybe it was DirectDraw) that had
>>> > R,G,B, and A masks to show which bits belonged to which channel. I also
>>> > worked with formats like RGBBGR repeating.
>>> >>
>>> >>
>>> >>
>>> >> [...]
>>> >>
>>> >> > Element Name: CbSubsamplingHorz
>>> >> >
>>> >> > Level:        5
>>> >> >
>>> >> > ID:           [55][B5]
>>> >> >
>>> >> > Mandatory:    -
>>> >> >
>>> >> > Multiple:     -
>>> >> >
>>> >> > Default:      -
>>> >> >
>>> >> > Type:         u
>>> >> >
>>> >> > Description:  The amount of pixels to remove in the Cb channel for
>>> >> > every
>>> >> > pixel
>>> >> >
>>> >> >              not removed horizontally. This is additive with
>>> >> >
>>> >> >              ChromaSubsamplingHorz. Example: For video with 4:2:1
>>> >> > chroma
>>> >> >
>>> >> >              subsampling, the ChromaSubsamplingHorz should be set to
>>> >> > 1
>>> >> > and
>>> >> >
>>> >> >              CbSubsamplingHorz should be set to 1.
>>> >> >
>>> >> >
>>> >> > Element Name: CbSubsamplingVert
>>> >> >
>>> >> > Level:        5
>>> >> >
>>> >> > ID:           [55][B6]
>>> >> >
>>> >> > Mandatory:    -
>>> >> >
>>> >> > Multiple:     -
>>> >> >
>>> >> > Default:      -
>>> >> >
>>> >> > Type:         u
>>> >> >
>>> >> > Description:  The amount of pixels to remove in the Cb channel for
>>> >> > every
>>> >> > pixel
>>> >> >
>>> >> >              not removed vertically. This is additive with
>>> >> >
>>> >> >              ChromaSubsamplingVert.
>>> >>
>>> >> What if Cr is subsampled more than Cb ?
>>> >> That too is rather obscure, but theres code in FFmpeg to handle such
>>> >> jpegs, so i suspect this case while very rare is not entirely non
>>> >> existent ...
>>> >
>>> >
>>> > I guess we can add that as well if more people really want it. Actually
>>> > I
>>> > didn't even have CbSubsampling* elements at first. I only added that to
>>> > support the 4:2:1 format that was defined in the first enum.
>>> >>
>>> >>
>>> >>
>>> >> >
>>> >> >
>>> >> > Element Name: ChromaSitingHorz
>>> >> >
>>> >> > Level:        5
>>> >> >
>>> >> > ID:           [55][B7]
>>> >> >
>>> >> > Mandatory:    -
>>> >> >
>>> >> > Multiple:     -
>>> >> >
>>> >> > Default:      0
>>> >> >
>>> >> > Type:         u
>>> >> >
>>> >> > Description:  How Chroma is subsampled horizontally. (0: Unspecified,
>>> >> > 1:
>>> >> > Left
>>> >> >
>>> >> >              collocated , 2: Half)
>>> >> >
>>> >> > Element Name: ChromaSitingVert
>>> >> >
>>> >> > Level:        5
>>> >> >
>>> >> > ID:           [55][B8]
>>> >> >
>>> >> > Mandatory:    -
>>> >> >
>>> >> > Multiple:     -
>>> >> >
>>> >> > Default:      0
>>> >> >
>>> >> > Type:         u
>>> >> >
>>> >> > Description:  How Chroma is subsampled vertically. (0: Unspecified,
>>> >> > 1:
>>> >> > Top
>>> >> >
>>> >> >              collocated , 2: Half)
>>> >> >
>>> >>
>>> >> iam not sure this is enough to specify all variants
>>> >> for 4:2:0 alone there are a few different variants
>>> >> theres mpeg1 style
>>> >> mpeg2 progressive and interlaced
>>> >> the mpeg2/mpeg4 style also differs from itself if the image is fliped
>>> >> right-left
>>> >> cropping 1 or 2 lines of the top of mpeg2 yuv420 also results in
>>> >> different variants
>>> >
>>> >
>>> >
>>> > I also didn't get into interlaced.
>>> >
>>> >
>>> >
>>> > I don't think we should add enough elements to support every format that
>>> > was
>>> > ever produced. Opinions?
>>>
>>> For archival (one of the main goal here) I think we should.
>>>
>>> > I think we should probably strive to support 99% of what is currently
>>> > produced today. Opinions?
>>>
>>> The problem is that when you leave 1% out, you need to be sure it's
>>> possible to integrate it later. Usually the best approach is to know
>>> beforehand how you're going to do it. So in the end it's just like
>>> covering it. That's just a general remark though.
>>
>> I don't want to exclude anything in the future, but I also don't want to
>> make something overly complex/over specified to support a format that will
>> barely see any use.
>>
>> I want to move forward with this, so how about we make a list and then vote
>> on what we think should be added to the sepc?
>> 1. Raw RGB
>> 2. Raw YUV
>> 3. Interlaced
>> 4. mpeg1 style
>> 5. mpeg2 pogessive
>> 6. mpeg2 interlaced
>> 7. image flipped right-left
>> 8. jpeg Cr subsampled more than Cb
>> 9. non-linear color values (DPX scans)
>>
>> Anything else we should add?
>>
>
> Just my 2 cents, but if you want to support everything and the kitchen
> sink, you should use string fields that can be arbitrarily expanded,
> instead of integer enumerations.
>
> - Hendrik
> _______________________________________________
> Matroska-devel mailing list
> [email-masked]
> http://lists.matroska.org/cgi-bin/mailman/listinfo/matroska-devel
> Read Matroska-Devel on GMane: http://dir.gmane.org/gmane.comp.multimedia.matroska.devel

-- 
Steve Lhomme
Matroska association Chairman