Subject: [Cellar] [FFV1] Adding Bayer support

From: Jerome Martinez <[email-masked]>
Date: Fri, 12 Jan 2018 22:11:38 +0100

I added a pull request on GitHub:
https://github.com/FFmpeg/FFV1/pull/100
Suggesting to use DgCoCg-R transformation before transformation for 
Bayer "color space".
This is a new colorspace_type value so we could put it in FFV1 v0 to v3 
without conflict with decoders in the wild.

I did not send the patch sooner, waiting for another patch to be 
accepted and waiting for finishing my corresponding encoder (so it is 
currently theory), but as we are talking about adding more planes and 
the corresponding metadata, I would like to be sure we handle this case 
in FFV1 v4.

Jérôme

---

From: Michael Niedermayer <[email-masked]>
Date: Sat, 13 Jan 2018 12:17:34 +0100

On Fri, Jan 12, 2018 at 10:11:38PM +0100, Jerome Martinez wrote:
> Suggesting to use DgCoCg-R transformation before transformation for Bayer
> "color space".
> I did not send the patch sooner, waiting for another patch to be accepted
> and waiting for finishing my corresponding encoder (so it is currently
> theory), but as we are talking about adding more planes and the
> corresponding metadata, I would like to be sure we handle this case in FFV1
> v4.

Is there some existing comparissions of YDgCoCg-R

I found only "Custom Lossless Compression and High-Quality
Lossy Compression of White Blood Cell Microscopy
Images for Display and Machine Learning
Applications"
http://lup.lub.lu.se/luur/download?func=downloadFile&recordOId=3810913&fileOId=3810914

which in figure 4.1 compares 8 lossless compression strategies
YDgCoCg in that is at the 3rd worst place, only 2 are worse

Now looking at YDgCoCg specifically, and asking, are there obvious changes
worth trying ?
i see 2 immedeatly
first the green channel has twice as many samples as the others, YDgCoCg works
around this by spliting it into 2 channels using a very simple diagonal 
transform. This should theoretically not be needed and would preserve more 
spatial correlation if its not done.
To show what i mean, please look at the green array
G   G   G   G
  G   G   G   G
G   G   G   G
  G   G   G   G

If you rotate the axis by 45° you will see that there are no more missing samples
and you have with the exception of the border a full 2d array to compress.
The boarder samples are comparably few.
I guess and thats a pure guess that this would compress better than spliting
this in 2 arrays and compressing them independant.

The 2nd obvious improvment to try is that the tranform transforms samples
that are spatially shifted relative to each other. I suspect if this is 
compensated for my some means the seperation would work better.
  
with these suggestions you could still have a sepearte transform to do before
using the existing compression algorithm

How can one try other transforms than YDgCoCg ?
do you have some patch for the ffmpeg ffv1 implementation that one could
experiment with to try different strategies ?

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

In fact, the RIAA has been known to suggest that students drop out
of college or go to community college in order to be able to afford
settlements. -- The RIAA

---

From: Jerome Martinez <[email-masked]>
Date: Sat, 13 Jan 2018 13:01:57 +0100

> YDgCoCg in that is at the 3rd worst place, only 2 are worse

If I understand well the report after a quick review (I'll read it more 
a bit later), all but 1 have a similar compression ratio (+/- 1%), to be 
compared to affinity with current FFV1 bitstream (just changing 
JPEG2000-RCT to the suggested RCT). So there would be only 1 interesting 
method to check (9% better), and this is a completely different one 
(wavelet) with some risk about patents. Would it worth it? IMO it is not.

> this in 2 arrays and compressing them independant.

I don't get "compressing them independent": the suggested RCT is focused 
on having a green difference, and I don't see how to do without that, as 
we would have a Green plane 2x bigger than the others so JPEG2000-RCT is 
not applicable.

> experiment with to try different strategies ?

It is difficult for me to understand the suggested other methods.

I suggest the following steps:
- I try to find enough real world sample streams I can publicly share, 
and set up a test platform, for easy comparisons.
- I implement YDgCoCg-R support and I run tests on it
- Anyone having an idea for better compression implements it and runs it 
against the same sample streams
- Based on the comparison (compression performance / speed performance / 
complexity), we pick one of the suggested methods.

Jérôme

---

From: Michael Niedermayer <[email-masked]>
Date: Sat, 13 Jan 2018 13:09:33 +0100

[...]
> - I try to find enough real world sample streams I can publicly share, and
> set up a test platform, for easy comparisons.

thats very close to what i had in mind, i think this is a great plan

about samples, please include both very sharp and more blurry (at pixel level)
samples. These 2 should behave quite different compression wise with sparse
sampling of colors. Also high noise and low noise content should be included
high ISO low light would produce very noisy content, low ISO bright light
should produce low noise.

[...]

What does censorship reveal? It reveals fear. -- Julian Assange