Subject: [Cellar] [PATCH] ffv1.md: Add support for more planes beyond RGBA / YCbCrA

From: Michael Niedermayer <[email-masked]>
Date: Fri, 12 Jan 2018 17:57:48 +0100

Idea-from: https://github.com/FFmpeg/FFV1/issues/47

Signed-off-by: Michael Niedermayer <[email-masked]>
---
 ffv1.md | 29 ++++++++++++++++++++++++++---
 1 file changed, 26 insertions(+), 3 deletions(-)

diff --git a/ffv1.md b/ffv1.md
index e57ab3e..e8c2f22 100644
--- a/ffv1.md
+++ b/ffv1.md
@@ -297,7 +297,7 @@ In YCbCr color space, the Cb and Cr planes are optional, but if used then MUST b
 - Y, Cb, Cr
 - Y, Cb, Cr, Alpha
 
-When FFV1 uses the YCbCr color space, the Y plane MUST be coded first. If the Cb and Cr planes are used then they MUST be coded after the Y plane. If an Alpha (transparency) plane is used, then it MUST be coded last.
+When FFV1 uses the YCbCr color space, the Y plane MUST be coded first. If the Cb and Cr planes are used then they MUST be coded after the Y plane. If an Alpha (transparency) plane is used, then it MUST be coded after them. Any additional planes are coded after that.
 
 ### JPEG2000-RCT
 
@@ -374,6 +374,8 @@ An FFV1 `Frame` using JPEG2000-RCT MUST use one of the following arrangements:
 - Y, Cb, Cr
 - Y, Cb, Cr, Alpha
 
+After these additional components can be encoded.
+
 When FFV1 uses the JPEG2000-RCT color space, the horizontal lines are interleaved to improve caching efficiency since it is most likely that the RCT will immediately be converted to RGB during decoding. The interleaved coding order is also Y, then Cb, then Cr, and then if used Alpha.
 
 As an example, a `Frame` that is two pixels wide and two pixels high, could be comprised of the following structure:
@@ -836,7 +838,7 @@ Inferred to be 1 if not present.
 
 ### quant_table_set_index_count
 
-`quant_table_set_index_count` is defined as `1 + ( ( chroma_planes || version \<= 3 ) ? 1 : 0 ) + ( alpha_plane ? 1 : 0 )`.
+`quant_table_set_index_count` is defined as `primary_color_count - ( chroma_planes ? 2 : 0 ) + ( ( chroma_planes || version \<= 3 ) ? 1 : 0 )`.
 
 ### quant_table_set_index
 
@@ -904,7 +906,9 @@ SliceContent( ) {                                             |
 
 ### primary_color_count
 
-`primary_color_count` is defined as 1 + ( chroma_planes ? 2 : 0 ) + ( alpha_plane ? 1 : 0 ).
+`primary_color_count` is either stored or if not, it is defined as
+`1 + ( chroma_planes ? 2 : 0 ) + ( alpha_plane ? 1 : 0 )`.
+If it is stored it must be larger or equal to the default.
 
 ### plane_pixel_height
 
@@ -1036,6 +1040,14 @@ Parameters( ) {                                               |
         ec                                                    | ur
         intra                                                 | ur
     }                                                         |
+    if (version >= 4) {                                       |
+        primary_color_count                                   | ur
+        for( i = 0; i < primary_color_count; i++ ) {          |
+            frequency[i]                                      | ur
+            bandwidth[i]                                      | ur
+            gamma[i]                                          | ur
+        }                                                     |
+    }                                                         |
 }                                                             |
 ```
 
@@ -1147,6 +1159,17 @@ RFC:`log2_v_chroma_subsample` indicates the subsample factor, stored in powers t
 | 0     | transparency plane is not present|
 | 1     | transparency plane is present    |
 
+### frequency
+`frequency` is the middle frequency of the given component in herz.
+It is 0 if not applicable, for example for an alpha component or if it is unknown.
+For Cb the blue channel shall be used for frequency and bandwidth.
+For Cr the red channel shall be used for frequency and bandwidth.
+For Y in YCbCr the green channel shall be used for frequency and bandwidth.
+
+### bandwidth
+`bandwidth` is the bandwidth of of the given component in herz.
+It is 0 if not applicable, for example for an alpha plane.
+
 ### num_h_slices
 
 `num_h_slices` indicates the number of horizontal elements of the slice raster.  
-- 
2.15.1

---

From: Jerome Martinez <[email-masked]>
Date: Fri, 12 Jan 2018 21:02:36 +0100

On 12/01/2018 17:57, Michael Niedermayer wrote:
> [...]

As we can break everything thanks to version bump, I suggest to simplify 
it and separate completely data from each plane, so:

+`quant_table_set_index_count` is defined as `1 + ( ( chroma_planes || version \<= 3 ) ? 1 : 0 ) + ( alpha_plane ? 1 : 0 )` if versions <= 3, else  as `
primary_color_count`

> +        for( i = 0; i < primary_color_count; i++ ) {          |

As we can break everything thanks to version bump, I suggest to simplify 
it and separate completely data from each plane, and put a "if (version 
 >= 4) sooner, e.g. after colorspace_type. It would also the advantage 
to avoid conflicting content (e.g. chroma_plane not present and alpha 
plane present but frequency is about blue channel, what to prioritize?)
I had in mind, after v3 finalized, to suggest that (modified in order to 
comply with your patch new fields):
just after colorspace and if version can, we could:
- trash unique 
bits_per_raw_sample/chroma_plane/log2_h_chroma_subsample/log2_v_chroma_subsample/alpha_plane,
- add primary_color_count, and for each primary color add 
bits_per_raw_sample/x_subsample/y_subsample/frequency/bandwidth/gamma

Background: we have some files with different bit depth per component. A 
public file is 
https://github.com/openexr/openexr-images/blob/master/v2/LeftView/Balls.exr 
(16-bit for 4 first channels and 32-bit for 5th channel) but I see in 
H.265 that there is the possibility to have a different bit depth per 
component so I prefer to be future proof even if we don't use all 
possibilities now. Note that I am fully aware that they are corner cases 
but I think we could have a spec for such corner case even if the most 
used encoder/decoder does not support them (and is not requested to 
support them).

> +It is 0 if not applicable, for example for an alpha plane.

I practice, what are the expected values for legacy RGBA or YCbCrA?
I don't get how to do the difference between Alpha and something else 
not capable to be represented as frequency/bandwidth/gamma, if we have 
both in same file.
For instance, I need (in a not so late future, in order to compress in a 
lossless manner some uncommon files) to store the following "colors":
AR, AG, AB: Red,  green  and  blue  alpha/opacity,  for  colored mattes  
(required  to  composite images of objects like colored glass correctly).
Z: Distance of the front of a sample from the viewer.
Z Back: Distance of the back of a sample from the viewer.
Source: http://www.openexr.com/TechnicalIntroduction.pdf
Note: the source document talks also about numerical identifier or 
nested components e.g. “light1.R”, “light1.G”, “light1.B”, “light2.R”, 
“light2.G”, “light2.B” but I never saw such file (and it would become 
hard to catch for applying JPEG2000-RCT)
Maybe a choice with a "standard form" like what you suggest + a "non 
standard" form with free text?

Jérôme