Subject: Re: [Sframe] Proposal: SFrame && SPacket && !SIDU

From: Justin Uberti <[email-masked]>
Date: Tue, 16 Nov 2021 10:50:51 -0800

Richard, when you talk about "units" here, are you referring to things
coming out of the codec? or the RTP packetizer for said codec?

The recent IETF meeting ended with some confusion as to this specific
question and it would be good for us on this list to make sure we're all in
alignment.

On Tue, Sep 7, 2021 at 12:46 PM Richard Barnes <[email-masked]> wrote:

> I'm not totally sure what you mean by "spatial frames" here; looks like
> the notion varies some by codec.  Assuming "spatial frames" are frames in
> the sense of "something that comes out of depacketizater", I think we're
> probably on the same page.
>
> Maybe it would be clearer to state the proposal as: Encryption is done in
> units of whole RTP payloads.  One payload for SPacket; all payloads in the
> frame for SFrame.  We would not support cases where an encrypted unit would
> span a partial RTP payload.  For example:
>
> NOT OK
> Payload 1: First part of SFrame unit A
> Payload 2: Second part of A, first part of B
> Payload 3: Second part of B
>
> OK
> Payload 1: SFrame unit A
> Payload 2: First part of Sframe unit B
> Payload 3: Second part of B
>
> Does that line up with what you have in mind?
>
> --Richard
>
> On Mon, Sep 6, 2021 at 2:53 PM Sergio Garcia Murillo <
> [email-masked]> wrote:
>
>> Hi Richard,
>>
>> Just one minor comment, the "media frames", should be spatial frames if
>> SVC is used so an SFU is able to not forward all spatial layers to the
>> receiver. Apart from that, I am ok with removing "SIDUs" from the scope.
>>
>> Best regards
>> Sergio
>>
>> On Thu, Sep 2, 2021 at 10:20 PM Richard Barnes <[email-masked]> wrote:
>>
>>> Hi folks,
>>>
>>> I'd like to see if we can agree to limit our scope a bit.  There has
>>> been discussion of multiple levels at which SFrame could be applied:
>>>
>>> * "SFrame" - Whole media frames
>>> * "SPacket" - Individual RTP media payloads
>>> * "SIDU" - Some codec-specific "independently decodeable units"
>>>
>>> I'd like to propose that we take "SIDU" off of the table, but keep the
>>> other two.
>>>
>>> It seems like SFrame and SPacket both have strengths in certain
>>> scenarios (depending on how you want to trade off bandwidth vs.
>>> complexity), but they're both pretty straightforward in terms of how they
>>> integrate with the media processing pipeline.  In particular, they don't
>>> have any codec-dependence.  SIDU, by contrast, seems like a world of pain.
>>> It requires that the encryption scheme be entangled with the codec, which
>>> requires special per-codec integration logic in the specifications.  And
>>> the bandwidth gains for all that complexity are not that large.
>>>
>>> Is there anyone here who feels that SIDU is something this group needs
>>> to spend time specifying?
>>>
>>> --Richard
>>> --
>>> Sframe mailing list
>>> [email-masked]
>>> https://www.ietf.org/mailman/listinfo/sframe
>>>
>> --
> Sframe mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sframe
>

---

From: Richard Barnes <[email-masked]>
Date: Tue, 16 Nov 2021 16:33:07 -0500

Specifically, I mean the things that are AEAD-encrypted.

My mental model is that there are two moving parts here:
1. A general SFrame encapsulation, defined in this group, that takes a
"unit" of media data and encrypts it with an AEAD cipher
2. One or more ways to apply that encapsulation to a real-time media stream.

Since AEADs deal with discrete quantities of data and not streams, one of
the things (2) has to define is how to divide a media stream up into units
that are fed into the SFrame encap/decap algorithms.  (In other words,
between which layers in the media stack you put in the SFrame layer.)  And
on decode, you need a way for the receiver to recognize when SFrame
ciphertexts begin and end.

For SFrame and SPacket, those answers are pretty clear:
* For SPacket: SFrame goes between packetization and SRTP; ciphertext
begin/end = packet begin/end
* For SFrame: SFrame goes between encoding and packetization; ciphertext
begin/end = packet begin/end (with generic depacketizer)

Those answers would be appreciably more complicated and codec-dependent if
we try to develop an SIDU approach as well as SFrame/SPacket.  Thus my hope
of ruling it out of scope.

--RLB

---

From: Richard Barnes <[email-masked]>
Date: Tue, 16 Nov 2021 16:35:26 -0500

BTW, if there are going to be SFrame-relevant discussions at other WGs, it
would be helpful if folks could post notice on this list.  Maybe I'm the
only one not really following other media WGs, but I missed whatever
discussions at the recent IETF you're referencing.

On Tue, Nov 16, 2021 at 4:33 PM Richard Barnes <[email-masked]> wrote:

>>> Maybe it would be clearer to state the proposal as: Encryption is done
>>> in units of whole RTP payloads.  One payload for SPacket; all payloads in
>>> the frame for SFrame.  We would not support cases where an encrypted unit
>>> would span a partial RTP payload.  For example:
>>>
>>> NOT OK

---

From: Justin Uberti <[email-masked]>
Date: Tue, 16 Nov 2021 14:21:40 -0800

> begin/end = packet begin/end (with generic depacketizer)

Makes sense, but these are two very different things (although I suppose
they may largely reduce to the same thing in trivial cases such as G.711).
For one, SFrame requires defining some sort of generic packetization,
whereas SPacket seems like it would reuse existing RTP packetization and
dependencies.

> of ruling it out of scope.

While I appreciate the desire to cut scope, to me SIDU is just a slight
refinement of SFrame. IDUs are frames, after all.

---

From: Justin Uberti <[email-masked]>
Date: Tue, 16 Nov 2021 14:22:29 -0800

Here I am referring to the discussion in avtcore surrounding
https://datatracker.ietf.org/doc/html/draft-murillo-avtcore-multi-codec-payload-format
.

On Tue, Nov 16, 2021 at 1:35 PM Richard Barnes <[email-masked]> wrote:

>> Those answers would be appreciably more complicated and codec-dependent
>> if we try to develop an SIDU approach as well as SFrame/SPacket.  Thus my
>> hope of ruling it out of scope.
>>
>> --RLB
>>
>>>>> Just one minor comment, the "media frames", should be spatial frames
>>>>> if SVC is used so an SFU is able to not forward all spatial layers to the
>>>>>> I'd like to propose that we take "SIDU" off of the table, but keep
>>>>>> the other two.
>>>>>>
>>>>>> Is there anyone here who feels that SIDU is something this group
>>>>>> needs to spend time specifying?
>>>>>>
>>>>>> --Richard
>>>>>> --
>>>>>> Sframe mailing list
>>>>>> [email-masked]