Subject: [Cellar] FFV1 Version 4

From: Kieran O Leary <[email-masked]>
Date: Tue, 5 Jan 2016 14:22:46 +0000

Hello,

I saw on your charter that a milestone for September 2016 is "Submit
specification for FFV1 video codec version 4 to IESG (Standards Track)".
https://datatracker.ietf.org/wg/cellar/charter/

Is there any information or roadmap available about proposed changes from
version 3 to version 4? I'm asking as the added features of Version 3 were
significant for moving image archives, and I'm curious to know what version
4 may look like.

Best regards,

Kieran O'Leary.

---

From: Dave Rice <[email-masked]>
Date: Tue, 5 Jan 2016 11:08:16 -0500

> On Jan 5, 2016, at 9:22 AM, Kieran O Leary <[email-masked]> wrote:
> 
> Hello,
> 
> I saw on your charter that a milestone for September 2016 is "Submit specification for FFV1 video codec version 4 to IESG (Standards Track)". https://datatracker.ietf.org/wg/cellar/charter/ <https://datatracker.ietf.org/wg/cellar/charter/>
> 
> Is there any information or roadmap available about proposed changes from version 3 to version 4? I'm asking as the added features of Version 3 were significant for moving image archives, and I'm curious to know what version 4 may look like.

This is the place to provide proposals. There has been a little discussion on FFmpeg-devel but none yet on CELLAR. IMO I'd like to consider the following for FFV1:

Increased Self-Description of Intended Decoding
- documentation of color matrix (i.e. bt.601, bt.709, etc)
- documentation of sample range (i.e. full range vs broadcast range)

Increased Self-Description of the Encoding
- storage of encoding parameters (I see this features in h264, are these called SEI messages?)

Extending Pixel Format Support
- support for Bayer pixel formats

Some Header Adjustments
- currently some values such as bit depth, display aspect ratio, and chroma subsampling are set per slice rather than per frame. Perhaps there is a reason I am not considering, but possibly some adjustment could improve the logic of the frame and slice headers.

Looking forward to discussion on this topic.

Best Regards,
Dave Rice

---

From: Michael Niedermayer <[email-masked]>
Date: Tue, 5 Jan 2016 17:09:40 +0100

Hi

> significant for moving image archives, and I'm curious to know what version

i dont know about roundmaps but what might be in the next version
i guess depends largly on what volunteers do (spec, implementation
and testing wise)
support for bayer "pixel" formats and maybe paletted formats could
be in it. Some compression improvments by using inter frame prediction
would be an idea too. But i think 2016-sep is too soon for complex
additions like inter frames. All these things require trying multiple
solutions and testing/evaluating how they perform comression, speed
and complexity wise to see what makes sense to add to the spec

> _______________________________________________
> Cellar mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/cellar

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

When you are offended at any man's fault, turn to yourself and study your
own failings. Then you will forget your anger. -- Epictetus

---

From: Michael Niedermayer <[email-masked]>
Date: Tue, 5 Jan 2016 17:30:58 +0100

> and complexity wise to see what makes sense to add to the spec

also theres a raw slice coding mode and a more flexible RCT in the
implementation and partly in the draft spec, these likely will be in
version 4 (depening on peoples oppinions i assume)

the "raw" coding mode for slices is needed so a tight upper bound
can be put on the compressed size of slices, otherwise its possible
to construct a sequence of frames where one slice has a compressed
size that is very much larger than its raw size, this would likely
never occur with any real video but contructed videos that push
the range coder statistics one direction and then sharply switch
everything to have completely opposite statistics would result in
a locally bloated up bitstream.
Such bloated up slices would annoyingly require larger buffers to be
allocated to allow encoding them, the raw coding mode puts a
constraint on this so that a buffer that is about as large as the raw
pixels is gurateed to be large enough for coding a slice in some mode.
Reducing the needed memory and complexity of an encoder implementation
and also reducing the worst case local bitrate requirement for
anything else in the chain

[...]

No snowflake in an avalanche ever feels responsible. -- Voltaire

---

From: Michael Niedermayer <[email-masked]>
Date: Tue, 5 Jan 2016 17:34:39 +0100

On Tue, Jan 05, 2016 at 11:08:16AM -0500, Dave Rice wrote:
[...]

> - currently some values such as bit depth, display aspect ratio, and chroma subsampling are set per slice rather than per frame. Perhaps there is a reason I am not considering, but possibly some ad…

the reason behind these being in slices is that slices should be
independantly decodable
If these would be in a single frame header and that frame header
is lost then none of the slices would be decodable (unless the
information is not needed to decode them)

[...]

You can kill me, but you cannot change the truth.

---

From: Jerome Martinez <[email-masked]>
Date: Tue, 5 Jan 2016 17:55:55 +0100

> information is not needed to decode them)

We have problem about resilience in:
- the ConfigurationRecord(). Very important, and not duplicated. So if 
we want resilience, we should first focus in it.
- if the slice_size of the last slice is corrupted, we lose all slices 
(ok, we could decode each slice, let say slice_sizeof last slice and 
first byte of first slice). So there is not a so good resilience anyway.
So I am not sure it helps so much to have display aspect ratio, and 
picture_structure in a slice header, without more work on resilience 
(slice synchro? something else?). If keep them per slice, maybe we 
should say in spec that all slices must have the exact same display 
aspect ratio and picture_structure per frame (does the decoder support 
different picture_structures in a frame?)

Additionally, picture_structure, sar_num, sar_dem (and other?) do not 
use to change (or they change as often as  chroma_planes, 
h_chroma_subsample, alpha_plane...), so we may consider to put them in 
the ConfigurationRecord() too (and we may need to take care of 
duplicating it in the container?)

I guess such question should have its own topic.

PS: Dave, I see  colorspace_type, bits_per_raw_sample, chroma_plane,  
h_chroma_subsample, v_chroma_subsample, so bit depth and chroma 
subsampling in the ConfigurationRecord(), so per stream. Where so you 
see them per slice? I only see display aspect ratio per slice.

---

From: Michael Niedermayer <[email-masked]>
Date: Tue, 5 Jan 2016 18:44:45 +0100

On Tue, Jan 05, 2016 at 05:55:55PM +0100, Jerome Martinez wrote:
> - the ConfigurationRecord(). Very important, and not duplicated. So

the global header / extradata / ConfigurationRecord can and should
be duplicated at the container level.
repeating it at the codec level would make finding a duplicate element
relativly hard as one probably would have to scan the whole file.
A container could contain something like an index to point to
redundant copies, while at codec level this would not work so easily

> if we want resilience, we should first focus in it.
> - if the slice_size of the last slice is corrupted, we lose all
> slices (ok, we could decode each slice, let say slice_sizeof last
> slice and first byte of first slice). So there is not a so good
> resilience anyway.

its not hard to scan for slices with valid CRCs without knowing their
start/end. for each assumed end the size is just stored in 3 bytes
there which gives the start, and the crc is there easy to check
(Checking all these potential slice crcs can be done efficiently if
 iam not missing something)

> (slice synchro? something else?).

> If keep them per slice, maybe we
> aspect ratio and picture_structure per frame

agree (per frame all must match)

> (does the decoder
> support different picture_structures in a frame?)

different picture_structures in 1 frame dont make sense or do i
misunderstand ?

> 
> Additionally, picture_structure, sar_num, sar_dem (and other?) do
> not use to change (or they change as often as  chroma_planes,
> h_chroma_subsample, alpha_plane...), so we may consider to put them
> in the ConfigurationRecord() too (and we may need to take care of

they could optionally be put in the global ConfigurationRecord
but it should stay possible for them to change if thats what the
source video we encode is doing

[...]

Good people do not need laws to tell them to act responsibly, while bad
people will find a way around the laws. -- Plato

---

From: Jerome Martinez <[email-masked]>
Date: Tue, 5 Jan 2016 19:01:58 +0100

Le 05/01/2016 18:44, Michael Niedermayer a écrit :
> the global header / extradata / ConfigurationRecord can and should be 
> duplicated at the container level. repeating it at the codec level 
> would make finding a duplicate element relativly hard as one probably 
> would have to scan the whole file. A container could contain something 
> like an index to point to redundant copies, while at codec level this 
> would not work so easily 

As far as I know, I never seen duplicated ConfigurationRecord.
I guess we should add such option (e.g. in Matroska specs, there is 
already a discussion about duplicated parts) for people wanting to 
secure this important part.5

> different picture_structures in 1 frame dont make sense or do i

Some people may be crazy enough for wanting and trying e.g.:
- slice 0,0 top field first
- slice 0,0 (the same as the line above) bottom field first
- slice 0,1 progressive
- slice 1,0 bottom field first
- slice 1,0 (the same as the line above) top field first
- slice 1,1 progressive
(we have MABFF feature in AVC/H264, so I don't invent too much)

and for the moment, I see nothing in the FFV1 specification forbidding 
that (did I miss something?)
So either we are clear it is authorized (and the reference decoder can 
deal with it), or we explicitly forbid it (picture_structure is same in 
all slices of a frame)

> source video we encode is doing

I like the idea to have such optional possibility. this is not a lot, 
but always some bits of compression.
A candidate for FFV1 version 4 feature!

Jérôme

---

From: Kieran O Leary <[email-masked]>
Date: Wed, 6 Jan 2016 10:11:55 +0000

Hi guys,

I'd like to propose the inclusion of Logarithmic information within the
FFV1 codec when transcoding from LOG RGB DPX.

This could be very beneficial to moving image archives who want to
losslessly compress their LOG DPX scans to FFV1. Currently if the FFV1
transcode is converted back to DPX, it is with Linear values, making it
unsuitable for preservation purposes.

It appears that ffmpeg doesn't support LOG in general (or maybe this lack
of support is just in DPX), from what I've gleaned from the mailing list,
so perhaps the LOG 'flag' within the FFV1 bitstream is only half the
battle? This conversation between Carl Eugen Hoyos and Dave Rice is my
source for this info, so apologies if I'm taking them up the wrong way :
https://ffmpeg.org/pipermail/ffmpeg-user/2015-March/025634.html

I would love to know if this is possible, or if anyone else would be
interested in this feature.
Here's some ffmpeg-user threads about the subject from the past year:
https://ffmpeg.org/pipermail/ffmpeg-user/2015-March/025625.html &
https://lists.mplayerhq.hu/pipermail/ffmpeg-user/2015-December/029536.html

Best regards,

Kieran O'Leary

On Tue, Jan 5, 2016 at 6:01 PM, Jerome Martinez <[email-masked]>
wrote:

>> duplicated at the container level. repeating it at the codec level would
>> make finding a duplicate element relativly hard as one probably would have
>> to scan the whole file. A container could contain something like an index
>> to point to redundant copies, while at codec level this would not work so
>> easily
>>
>
> already a discussion about duplicated parts) for people wanting to secure
> this important part.5
>
> [...]
>>
> deal with it), or we explicitly forbid it (picture_structure is same in all
> slices of a frame)
>
>
>
> I like the idea to have such optional possibility. this is not a lot, but
> always some bits of compression.

---

From: "Peter B." <[email-masked]>
Date: Thu, 07 Jan 2016 19:00:50 +0100

On 01/06/2016 11:11 AM, Kieran O Leary wrote:
> FFV1 codec when transcoding from LOG RGB DPX

I just wanted to mention that as well.
Thanks Kieran :)

I was already asked several times (mainly by film archives) if that is
possible.
Currently, "logarithmic-DPX-to-FFV1-and-back" is reported to be doable,
but requires to manually set the output DPX to be non-linear...

I'm not too experienced with non-linear color encoding, but in a recent
presentation by Reto Kromer, he mentioned that "logarithmic" often just
means "logarithmic-ish".

Therefore it might be good (or necessary) to also store and handle
information about what type of non-linear color values are used?

Kind regards,
Pb

---

From: Jerome Martinez <[email-masked]>
Date: Thu, 7 Jan 2016 19:34:43 +0100

> information about what type of non-linear color values are used?

I am not an expert, I just see that "Logarithmic" is in the list of DPX 
transfer characteristics. so it looks like it is an unique value to store.

On my side, I tried to map DPX transfer characteristics to MPEG transfer 
characteristics, but it is not obvious.
For example, DPX transfer characteristics 5 is "SMPTE 274M", 6 is "BT.709"
They are actually same, and can be mapped to MPEG transfer 
characteristics 1, 6, 14 or 15 (several values for the same function)

List from each spec is not bidirectional, this is the reason I am 
thinking to have a code for the source of the value + the value from 
this source.
This is the only possibility if we want to be let people keep the exact 
metadata from the source (and being able to "revert" this compression).

We could do our own list for Matroska, but in that case we need to 
maintain it (for example, SMPTE added a new transfer characteristic 
value "ADX" in 2014 so we need to add a value in our list too) and take 
values from different specs in order to have an exhaustive list.
(and "reverting" it will not be with the exact same value: DPX 5 or 6 
will be mapped to MKV value X and "revert" will have to decide to put 
value 5 or  arbitrary).

I guess we first need to be clear about what we want to do (keep the 
exact value from the source or not, maintain a list...)

PS: just to be clear, when you speak about "logarithmic", you speak 
about DPX transfer characteristic value 3 "Logarithmic [to be defined by 
SMPTE I23 Technology Committee,
sub-group on “Transfer Characteristics”]", right?
(note: the "to be defined" is in the spec from 2003 as well from the 
spec from 2014 :), without further description)

Jérôme

---

From: "Peter B." <[email-masked]>
Date: Thu, 07 Jan 2016 23:09:40 +0100

On 01/07/2016 07:34 PM, Jerome Martinez wrote:
> about DPX transfer characteristic value 3 "Logarithmic [to be defined
> by SMPTE I23 Technology Committee,

Well, personally I'd say I'm speaking about "color values that are
non-linear".
In practice, I've only encountered "quasi-log(TM)" (aka "logarithmic) -
and DPX being the most popular case for these non-linear color values.

So in order to stick to real-world use-cases I mentioned
"logarithmic-DPX-to-FFV1-and-back".
Could also be "Non-linear-Foo-to-FFV1-and-back" ;)

Maybe it would make sense to collect what kind of non-linear color
interpretations appear in the real world and derive some way of handling
them.
At least the most prominent ones?

For example, like the ones Reto mentioned:
// -------------------
- log RGB (= "quasi-log", sometimes also called "pseudo-log") encodings:
FilmStream (log60), ARRI log F, Panalog, Sony-log (= S-log), REDLOG, SI
log90
- log neg encodings: Cineon printing density (CPD), "DPX", ARRI log C
// -------------------

Regards,
Pb

PS: Maybe something like a CLUT could be stored as a self-contained way
of interpreting the non-linearity of the color values?