Subject: [dispatch] Magnus Westerlund's Block on charter-ietf-sframe-00-00: (with BLOCK and COMMENT)

From: Magnus Westerlund via Datatracker <[email-masked]>
Date: Mon, 07 Sep 2020 09:42:15 -0700

Magnus Westerlund has entered the following ballot position for
charter-ietf-sframe-00-00: Block

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/charter-ietf-sframe/

----------------------------------------------------------------------
BLOCK:
----------------------------------------------------------------------

I know we have had discussion touching on this before. But post vacation and
looking on this charter again I think we need to have some additional
discussion of the goals and how the charter describes them in relation to
encoder sub-streams and identification of what is encapsulated.

In regards to the below:

This working group will not specify the signaling required to configure SFrame
encryption.  In particular, considerations related to SIP or SDP are out of
scope.  This is because SFrame is intended to be applied as an additional layer
on top of the base levels of protection that these protocols provide.  This
working group will, however, define how SFrame interacts with RTP (e.g., with
regard to packetization, depacketization, and recovery algorithms) to ensure
that it can be used in environments such as WebRTC.

I think there exist a conflict in the above paragraph in relation to stated
goals of the work. With the following earlier sentence: " It may also be
desirable to encrypt units of intermediate size (e.g., H.264 NALUs or AV1 OBUs)
to allow partial frames to be usable." in mind creating an RTP payload format
that is capable of carrying SFRAMEs that contains these units will require some
interaction with the signalling. Even without these sub-stream SFRAMEs there
exist a description capability that needs to exist in an RTP payload format for
the end-consumer to correctly be able to route the protected data after
decapsulation and that the end-point having that capability.

If the goal here when it comes to RTP is simply to be able to treat SFRAME as
CODEC in WebRTC and thus use WebRTC InsertableStreams as a receiver of the
decrypted media ADUs. Require the use of the WebRTC application to have a
proprietary signalling to know what this ADU is and then route it to a media
decoder? I can see that working in the WebRTC only context. However, I would
prefer if some thought was spent on at least having a model for what
information may be needed to be able to handle the media streams. Considering
RFC 7656 (https://datatracker.ietf.org/doc/rfc7656/) and the work that was
needed for us to up-level how RTP worked and even discuss this so that we
understood each other. I think SFRAME needs to discuss how it is going to
handle identification of the data encapsulated by SFRAMEs for media. A single
media source can be encoded in multiple formats. Each format may produce one or
more sub-streams of encoding for scalability or robustness and this needs to
conveyed.

So looking at the above challenges in the context of SFRAME over RTP. So a
possibility here is to say that the SSRC represents either just a media source.
The RTP payload format provides only fragmentation of the SFRAME across
multiple RTP packets and the RTP timestamp can be used to indicate its
belonging in the timeline of the encoding. That puts a lot of the
identification on the SFRAME layer, but its minimizes the signalling
interactions related to RTP. However it creates limitation about what the SFU
can do, especially when it comes to repair. Switching can be done based on
Frame-marker extension header. However, layer related loss detection becomes
impossible without additional information, or use of multiple SSRCs.

Thus, I think the charter as currently written are uncertain if it can be
executed on with stated goals.

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

* Information to form a unique nonce within the scope of the key

Is this really "Information to form a" the best formulation. I am uncertain if
the goal is to have a specification for how to generate unique Nonce values
within the context of a particular key, or if it is related to which
information sources that should be used when creating a nonce?

---

From: Sergio Garcia Murillo <[email-masked]>
Date: Tue, 8 Sep 2020 01:40:06 +0200

On 07/09/2020 18:42, Magnus Westerlund via Datatracker wrote:

> However, I would

SFRAME content should be treated just as an opaque payload and when used 
within RTP, there should no additional identification inside the SFRAME 
payload itself (except the ones strictly needed for encrypting/decrypting).

Any further identification or metadata required must be performed at RTP 
level:

This working group will, however, define how SFrame interacts with RTP (e.g., with
regard to packetization, depacketization, and recovery algorithms) to ensure
that it can be used in environments such as WebRTC.

What I think it is missing in the charter is define what will happen if 
we find that there is any piece missing in RTP to be able to use SFRAME 
with it. Should we produce them within this group or just write the 
requirements and liaise with the appropriate ietf group so they are 
completed there?

> possibility here is to say that the SSRC represents either just a media source.

In no way SFRAME should be required to have knowledge about SSRCS.

> impossible without additional information, or use of multiple SSRCs.

There are several ways of performing that without imposing any 
limitations on the SFU. Some of which would require extra specification 
work, but I would not like to jump directly into the technical proposals 
while still discussing about the charter.

Best regards

Sergio

---

From: Richard Barnes <[email-masked]>
Date: Thu, 10 Sep 2020 10:18:58 -0400

Hi Magnus,

I agree with Sergio here.  The whole idea of SFrame is to have an
encryption layer that doesn't care about any of the RTP nuances you talk
about -- all of the details of how what decoder you send the plaintext to,
substreams, etc. are handled by whatever RTP configuration mechanism you've
set up.  SFrame is just a translation layer between what you send on the
wire and what you do all the other stuff with.

Clearly, the nonce formation text is not clear.  I've made some edits.

https://docs.google.com/document/d/10rG8nAR0U6cBBPffzXnLaPPYL4uzxYViAvgiSezoa7o/edit?usp=sharing

--Richard

On Mon, Sep 7, 2020 at 12:42 PM Magnus Westerlund via Datatracker <
[email-masked]> wrote:

> --
> Sframe mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sframe
>

---

From: westhawk <[email-masked]>
Date: Thu, 10 Sep 2020 15:34:12 +0100

> On 7 Sep 2020, at 17:42, Magnus Westerlund via Datatracker <[email-masked]> wrote:
> 
> Even without these sub-stream SFRAMEs there

I think the assumption here (perhaps understated) is that both ends are running compatible 
sframe implementations. In webRTC this is done by them both loading the same javascript or app version.
In a more heterogeneous situation some signalling may be needed, but that’s explicitly out of scope.

> However it creates limitation about what the SFU
> can do, especially when it comes to repair. 

I think that’s an inevitable (and acceptable) consequence of e2e encryption.

T.

---

From: Magnus Westerlund <[email-masked]>
Date: Thu, 10 Sep 2020 15:26:05 +0000

Hi,

I have now read Sergio and Bernards discussion on the Dispatch list. By dropping
it down to only dispatch I missed that discussion. 

On Thu, 2020-09-10 at 10:18 -0400, Richard Barnes wrote:
> Hi Magnus,
> 
> I agree with Sergio here.  The whole idea of SFrame is to have an encryption
> layer that doesn't care about any of the RTP nuances you talk about -- all of
> the details of how what decoder you send the plaintext to, substreams, etc.
> are handled by whatever RTP configuration mechanism you've set up.  SFrame is
> just a translation layer between what you send on the wire and what you do all
> the other stuff with.

Let me clarify what I see as the issue with the charter. 

So the whole point is to ensure that you can move SFRAMEs without any RTP. Thus,
what I was trying to exemplify through the previous work in RTP is to be able to
do this SFRAME do require meta data that enables a handling function to seperate
layers and dependecies. To support this I think you end up i the need for
extensible meta data, data that is unencrypted but strong considerd to avoid
information leakage and have good security properties. Taking the regular RTP
payload format headers straight may not be the best fit for many codecs.  If
that is going to be generic you end up in a naming and strucutre discussion like
what ended up in the Taxonomy RFC and like for the frame marking RTP header
extension.

However, the issue I really reacted to here is the fact that the charter is
stating that it should do an RTP payload format. When doing an RTP payload
format I think it is pre-mature of the charter to say that the signalling
required for the RTP payload foramt is out of scope. I think that needs to be
part of the process to discuss in the RTP payload format how an RTP middlebox
can use the payload format, and any RTP extension like frame marking to figure
out what to do when switching. That RTP payload format will require some
configuration capability to be able to work. 

So what I am really asking for is the removable of the restriction that prevents
a regular RTP payload format design work. And stating that SDP is out of scope
implies that to me in the direct context of the payload format. 

I think the edits are an improvement solving several unclarities around the
other issues I raised.

Maybe one way forward is to break the paragraph talking about signalling and RTP
payload format into two different paragraph. Where the RTP payload format need
for signalling is not associated with the higher level need for SFRAME
signalling over for example a SIP system. Which I am fine with excluding
explicitly. 

So just talking about the RTP Payload format. I think this do need a model for
how an RTP SFU is going to be able to do switching on it. Shouldn't that be in
scope? 

Cheers

Magnus Westerlund 

----------------------------------------------------------------------
Networks, Ericsson Research
----------------------------------------------------------------------
Ericsson AB                 | Phone  +46 10 7148287
Torshamnsgatan 23           | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden | mailto: [email-masked]
----------------------------------------------------------------------

---

From: Sergio Garcia Murillo <[email-masked]>
Date: Fri, 11 Sep 2020 04:27:05 +0200

On 10/09/2020 17:26, Magnus Westerlund wrote:

> [...] However, the issue I really reacted to here is the fact that the 
> charter is

I agree. As I read the charter, SDP is out of the scope for 
negotiating/signaling the usage of SFrame and the parameters that may be 
used for encryption/decryption.

However, if the encrypted output is going to be transmitted over RTP 
using a new packetization format, we need to address how to negotiate 
that in the SDP.

Note that this new packetization format will not only be alid for 
transporting encrypted frames (SFrame or not) but even any other 
audio/video frames.

> [...]

Note that the RTP payload format may not be the one providing the 
switching information for an SFU. It is much cleaner to carry that 
information in a new (hop-by-hop encrypted) header extension so the 
payload does not need to be parser by the SFU at all, so the 
packetization can be "payload-agnostic".

Best regards

Sergio

---

From: Magnus Westerlund <[email-masked]>
Date: Fri, 11 Sep 2020 07:39:34 +0000

Hi,

I think we are making good progress here. 

> used for encryption/decryption.

The signalling for what is in the SFRAME is out of scope also. 

So do I assume correctly that the idea is that the application layer using
SFRAME if it haves multiple independent or sets of dependent streams there will
be no support for that aspect in SFRAME layer? Instead it is up to the
application to put such information to either put it inside the encapsualted
part of the SFRAME or map it to the lower transport layer, like to RTP SSRCs and
extension information. 

> using a new packetization format, we need to address how to negotiate 

Good

> transporting encrypted frames (SFrame or not) but even any other 

I understand the potential exists. However, the recommendation for RTP has been
to try to consider Application Level Framing. In the case of SFRAME that
consideration moves up above SFRAME in the choice of how data is split into
SFRAMEs. However, having an RTP paylaod format for SFRAME, then that should
carry SFRAMEs. If you starts putting in other binary objects into it, then you
create a new demultiplexing point for format between the RTP payload format and
the SFRAME processing. That appears unmotivated. 

> packetization can be "payload-agnostic".

Yes, I agree that putting the layer dependency information into an payload
external header extension etc is very reasonable. However, the RTP payload
format will need to discuss this need and potentially recommend a particular
method for carrying the dependency information within RTP.

So to conlude I think there is one charter related question here.

Should the RTP payload format aspect of the charter be more explicit in that it
needs to disucss the general model of how to use SFRAME and how an application
can use the facilities of RTP to get good performance from RTP mechanism like
FEC and retransmission? 

Cheers

Magnus Westerlund

---

From: Magnus Westerlund <[email-masked]>
Date: Fri, 11 Sep 2020 09:09:03 +0000

Hi,

I did forgot one aspect that I should have picked up earlier.

The relationship to AVTCORE in this work. Can we be expicit if the SFRAME WG do
the RTP Payload format and then cordinate with AVTCORE and have joint WG Last
call or if the RTP payload format is done in AVTCORE? I am fine with the first.

Cheers

Magnus

On Fri, 2020-09-11 at 07:39 +0000, Magnus Westerlund wrote:
> > > Maybe one way forward is to break the paragraph talking about signalling
> > > and
> > > RTP
> > > how an RTP SFU is going to be able to do switching on it. Shouldn't that
> > > be
> > > in
> > > scope?
> > 
-- 
Cheers

Magnus Westerlund

---

From: Sergio Garcia Murillo <[email-masked]>
Date: Fri, 11 Sep 2020 12:04:03 +0200

> part of the SFRAME or map it to the lower transport layer, like to RTP SSRCs and

If i understood you correctly, SFrame is kind of agnostic to the media 
streams. Currently it has a single global frame counter for all the 
transport, so the number of streams/dependency between them is not 
known/needed by SFrame.

> the SFRAME processing. That appears unmotivated.

Note that SFRAME is not the only encryption possible with w3c insertable 
streams, it is up to the application to define its own crypto if they 
want. So the payload must be able to transport non-SFRAME opaque blobs.

> can use the facilities of RTP to get good performance from RTP mechanism like

I don't think so, RTX and FEC frames MUST not be modified/affected by 
SFRAME/the new packetization format, so they work out of the box. We can 
be explicit about that in the chapter as a requirement.

Best regards

Sergio

---

From: Magnus Westerlund <[email-masked]>
Date: Fri, 11 Sep 2020 11:19:26 +0000

Hi,

> > So do I assume correctly that the idea is that the application layer
> > using SFRAME if it haves multiple independent or sets of dependent
> > streams there will be no support for that aspect in SFRAME layer?
> > Instead it is up to the application to put such information to either
> > put it inside the encapsualted part of the SFRAME or map it to the
> > lower transport layer, like to RTP SSRCs and extension information.
> 
> 
transport, so
> the number of streams/dependency between them is not known/needed
> by SFrame.

Yes, and when using a multi-stream application with layered encoding being
protected by SFRAME and transported over RTP with repair functions the high
layer application will need to have a model for how it maps individual
SFRAMEs to RTP layer functions to enable them to do their work. You get a
very limited functionality if in an SFU system tries to send all over a
single RTP SSRC. So this becomes a discussion in the RTP Payload format
context when carrying SFRAMEs. 

> > I understand the potential exists. However, the recommendation for RTP
> > has been to try to consider Application Level Framing. In the case of
> > SFRAME that consideration moves up above SFRAME in the choice of how
> > data is split into SFRAMEs. However, having an RTP paylaod format for
> > SFRAME, then that should carry SFRAMEs. If you starts putting in other
> > binary objects into it, then you create a new demultiplexing point for
> > format between the RTP payload format and the SFRAME processing. That
> appears unmotivated.
> 
> 
want. So
> the payload must be able to transport non-SFRAME opaque blobs.

So I would consider this a misuse of the RTP payload format. The media type
will be for SFRAMEs. I understand that you might not be able to prevent this
in WebRTC. However, from an interoperability point of view you will be
stating that this is SFRAMEs. RTP will not be able to tell a difference. But
I don't see that this is will explicitly support carrying other things than
SFRAME. Because that means bringing in other consideration including the
security model for the E2E usage. 

> >
> > Should the RTP payload format aspect of the charter be more explicit
> > in that it needs to disucss the general model of how to use SFRAME and
> > how an application can use the facilities of RTP to get good
> > performance from RTP mechanism like FEC and retransmission?
> 
> 

So I am not saying that RTX or FEC shall be modified. What I am saying is
that the SFRAME payload format should discuss the impact on the performance
of these functions depending on how you structure the SFRAMES across SSRCs.
The most simple example is the one where you put all layers for a video
encoding in a single SSRC. In such a stream you loose a single RTP packet
between the transmitting end-point and the SFU. Now the SFU is only
forwarding the base layer and not the enhancement layers. If base layer and
enhancement layer share RTP stream, the SFU can't determine if the missing
packet contains an base layer data or enhancement layer data. If the base
layer and enhancement layer would be on different SSRC, the fact that there
is a missing packet on the enhancement layer RTP stream means that the SFU
can ignore that as it anyway are not forwarding it. This same argument can
be applied between media sources. So how you map your application level
structures onto RTP do matter for the resulting performance of RTP. Putting
all on a single SSRC is not a path to good transport performance. 

Cheers

Magnus Westerlund

---

From: Sergio Garcia Murillo <[email-masked]>
Date: Fri, 11 Sep 2020 17:32:07 +0200

> context when carrying SFRAMEs.

I don't really understand your point, let's put an example vp9 svc.

Chrome will encode each picture from the video stream and pass it to the 
vp9 svc encoder. The encoder will produce n-frames (one per spatial 
layer) for a given picture.

Chrome will get each frame with its metadata (spatial/layer id + layer 
structure info + previous frame dependencies + future frame depencies) 
and pass it over to the insertable streams.

The payload will go to javascript, in where SFRAME will encrypt it and 
returned as a binary blob.

The browser will get the binary blob, packetize it in several rtp 
packets according to the new packetization format and add the metadata 
as a header extension.

Is this process what you are describing or is there anything missing?

> security model for the E2E usage.

Not sure if ignoring the reality on how the packetization format is 
going to be used within insertable streams is a good idea.

> all on a single SSRC is not a path to good transport performance.

But that is already the case for SVC codecs, so there is nothing new to 
specify in that regard.

Best regards

Sergio

---

From: Magnus Westerlund <[email-masked]>
Date: Mon, 14 Sep 2020 07:29:54 +0000

Hi,

Please see inline

> Is this process what you are describing or is there anything missing?

Then the client will send these RTP packets to an SFU that will need to make
decision on which of the packets to forward for which layers to a particular
receiver. To be able to do this there are first the aspect of that the meta data
needs to be included so that the SFU can make decision based on the packets it
recieves. In RTP context this likely are a more generic structure like the RTP
header extension for Frame marking (draft-ietf-avtext-framemarking). 

For packet it doesn't receive it lacks the meta data. Thus, for a more efficient
repair of packets that are missing and to repair only packets that the SFU
actually needs, then you actually have to map the layered structure into RTP
level structures so that the SFU can determine that the missing packet belongs
to a layer that it actually intends to forward. Putting everything in a single
SSRC requires the SFU to repair all losses independent of it needing the packet
or not. 

A even more extreme example is if the client has 3 video cameras and produce 3
video encodings. In the SFRAME context it is possible to take the video frames
from all these threee encodings and put them in a single RTP stream (SSRC) with
SFRAMEs from that end-point. However, that would force your meta data to contain
also source identification rather than to using different SSRCs. Putting
multiple encoding on an single RTP stream would deprive the SFU of the
possibility to do rate control per encoding (RFC 5104 - TMMBR) as well as pause
streams it doesn't forward at all (RFC 7728 - Stream PAUSE) because all the
existing RTP mechanism that are included in WebRTC works on the RTP streams, not
sub-streams that doesn't have an identifier in the RTP layer. In addition to
having to repair any loss for the aggregated stream of the three encodings.

> going to be used within insertable streams is a good idea.

If you are passing other things than SFRAME you are also sending them without
end-to-end protection. Why is this happening, if the information was intended
for the SFU, then I think we should identify it as such, rather than having the
SFU have to hunt for it. If not, why are you sending it outside of the SFRAME
envelope? 

> But that is already the case for SVC codecs, so there is nothing new to 

So for SVC and other scalable codes there exists options for how to do this. And
where you can write and RTP packetizer for SVC that takes basicaly a mode and a
layer structure configuration as input, that is not possible for SFRAME as that
information is not visible in the RTP payload information. Instead it must come
with each SFRAME to packetize how to map it to the RTP layer. 

Thus, in my view the SFRAME RTP Payload format needs to discuss the core of
these issues to make it clear to the application both its options as well as
point to the impact of those options. 

Cheers

Magnus Westerlund

---

From: Sergio Garcia Murillo <[email-masked]>
Date: Mon, 14 Sep 2020 11:39:15 +0200

> having to repair any loss for the aggregated stream of the three encodings.

No one has spoken of putting every on a single SSRC, the new rtp 
packetization would change the way the payload is packetized into the 
RTP packets, but not the rtp/ssrc/mid mapping of the media streams which 
would be the same as if not SFRAME is use. So this is a non-issue.

> SFU have to hunt for it. If not, why are you sending it outside of the SFRAME

This is an application concern. I don't know why they would choose to do 
it, but the fact is that they can do it.

> point to the impact of those options.

This is already possible for any SVC codec re-using the Dependency 
Descriptor header extension:

https://aomediacodec.github.io/av1-rtp-spec/#a1-introduction

If you read my example above, this information is know before the frame 
goes to the Insertable Stream process, so it will be available for the 
rtp packetization without going via SFRAME.

Best regards

Sergio

---

From: Magnus Westerlund <[email-masked]>
Date: Mon, 14 Sep 2020 14:12:27 +0000

Sergio,

Having 20 years of experience with RTP payload format designs some of your
formulations made me a bit worried. After this discussion I think we are much
closer on a common understanding of the high level functionality necessary. 

However, there will be questions about where we document that interaction and
what dependencies that do exist on other specificaitons for scalable codecs that
are generic and is desirable to require from RTP middleboxes. In addition even
without RTP there will be need from the application to consider how it can
utilize one SFRAME key and its IVs to carry multiple application sub-streams and
the need to expose that to lower layer transport function to achiveve
performance goals. 

So I think some clarification on the RTP payload format work, and it that is
going to happen in the SFRAME WG. If happening in SFRAME WG I would like to have
a joint WG Last call with AVTCORE WG. 

> Descriptor header extension:
https://protect2.fireeye.com/v1/url?k=6efce3ab-305c2002-6efca330-86959e472243-20fbdf13d7e98258&q=1&e=d4082b1d-c5ac-4946-be12-6a70a4838aa1&u=https%3A%2F%2Faomediacodec.github.io%2Fav1-rtp-spec%2F%23a1-introduction

So should defining the dependency descriptor in IETF be part of the SFRAME WG,
or is this a normative dependency? Currently this is a industry forum header
extension not an IETF product. And what short commings of Frame Marking is it
addressing. 

Cheers

Magnus Westerlund 

----------------------------------------------------------------------
Networks, Ericsson Research
----------------------------------------------------------------------
Ericsson AB                 | Mobile +46 73 0949079
Torshamnsgatan 23           |
SE-164 80 Stockholm, Sweden | mailto: [email-masked]
----------------------------------------------------------------------

---

From: Sergio Garcia Murillo <[email-masked]>
Date: Mon, 14 Sep 2020 19:27:48 +0200

> the need to expose that to lower layer transport function to achiveve

What I fail to understand is the nature of your concerns and how should 
we address them on the charter:

  * Is it something that we plan to do that is not covered in the charter?
  * Is it something that we don't plan to do but the charter is wide
    enough to allow it?
  * Is it something that we plan to do and it is covered in the charter
    but you don't agree to it?
  * Is it something that we don't plan to do and is neither covered by
    the charter and we should add it?

> a joint WG Last call with AVTCORE WG.

I raised that question already a couple of times. I am not sure what is 
the role of the SFRAME WG regarding specifying a new RTP codec-agnostic 
payload.

IMHO this WG should collect the requirements, match them to current 
specifications, and if anything is missing, liaise with the appropriate 
groups in order to produce the specs based on our requirements.

Best regards

Sergio

---

From: Bernard Aboba <[email-masked]>
Date: Mon, 14 Sep 2020 11:45:49 -0700

Sergio said:

"No one has spoken of putting everything on a single SSRC, the new RTP
packetization would change the way the payload is packetized into the
RTP packets, but not the rtp/ssrc/mid mapping of the media streams which
would be the same as if SFRAME is not in use. So this is a non-issue."

[BA] Right.  SFRAME doesn't change the SSRC allocation scheme, nor does it
prevent the SFU from modifying the SSRC.
If the application was sending simulcast with multiple SSRCs/RIDs, SFRAME
won't affect that.  Similarly with SVC, if the application was sending all
layers on a single SSRC, SFRAME won't change that either.

"If you read my example above, this information is known before the frame
goes to the Insertable Stream process, so it will be available for the RTP
packetization without going via SFRAME."

[BA] As you note, the metadata contains information about each frame, and
the Insertable Streams implementation assumes that the application
modifications don't invalidate the metadata, which currently contains info
such as frame type, frameId, dependencies, width, height, LID, TID, SSRC
and CSRCs.

Most of the RTP header extension fields are the same for each packet within
a frame; only a few will change between packets within a frame, such as the
Start and End bits.

The exercise is to make sure that the metadata is sufficient to enable the
generic RTP packetizer to fill in all the RTP header and RTP header
extension fields.  For the RTP header extension fields, this includes info
such as Discardable, Switch, Required, Base Layer Sync, etc.  Today it
isn't clear to me that this info is completely determinable from the
Insertable Streams metadata.

Given that we don't have a lot of deployment of frame forwarding RTP header
extensions yet, it's probably best to avoid tying SFRAME  to any particular
design.  For example, you wouldn't want a design that could only function
with the framemarking RTP header extension or the Generic Frame Descriptor
or the Dependency Descriptor, so that a new specification is needed when
the N+1 frame forwarding technology comes along.

---

From: Sergio Garcia Murillo <[email-masked]>
Date: Mon, 14 Sep 2020 21:43:09 +0200

On 14/09/2020 20:45, Bernard Aboba wrote:
>
> Given that we don't have a lot of deployment of frame forwarding RTP 
> header extensions yet, it's probably best to avoid tying SFRAME  to 
> any particular design.  For example, you wouldn't want a design that 
> could only function with the framemarking RTP header extension or the 
> Generic Frame Descriptor or the Dependency Descriptor, so that a new 
> specification is needed when the N+1 frame forwarding technology comes 
> along.
>
I agree.

Best regards

Sergio

---

From: Magnus Westerlund <[email-masked]>
Date: Tue, 15 Sep 2020 09:29:12 +0000

> > specification is needed when the N+1 frame forwarding technology comes 

For this all points to that SFRAME WG shall not do the RTP Payload format,
instead that work shall be done in AVTCORE WG. Thus, everything for SFRAME that
has to do with RTP will be dealt with in AVTCORE and to my knowledge this is
within the charter of AVTCORE. However, maybe the SFRAME charter should be
explicit about that this shuold be noted in a paragraph in the SFRAME WG charter
that the WG will collaborate on this aspect. Because I will assume that there
will be some aspects that will flow from the RTP Payload format work towards the
SFRAME format. 

Cheers

Magnus Westerlund

---

From: Magnus Westerlund <[email-masked]>
Date: Tue, 15 Sep 2020 09:46:55 +0000

On Mon, 2020-09-14 at 19:27 +0200, Sergio Garcia Murillo wrote:
> What I fail to understand is the nature of your concerns and how should we
> address them on the charter:
> Is it something that we plan to do that is not covered in the charter?
> Is it something that we don't plan to do but the charter is wide enough to
> allow it?
> Is it something that we plan to do and it is covered in the charter but you
> don't agree to it?
> Is it something that we don't plan to do and is neither covered by the charter
> and we should add it?

I think with the resolution of the RTP payload format there is only the question
if there needs to exist an informational discussion towards those who intended
to apply the SFRAME format in their application the difference between
application streams or strucutres and the SFRAME sequence and their keys and the
requirement that puts on meta data and mapping to lower layer transport
depending on application. I do note that a pure store and forward application
like email has its set of needs that is quite different from real-time media. 

I personally would like to see it written into the charter that this needs to be
considered. In the current draft update I think there wording that would imply
that this needs to happen. However, depending on how the below change are done
this might get lost. 

An alternative for the WG to actually ensure that they have a solid description
and components that solve the issue would be to write an non-standard
description of how SFRAME would be used in WebRTC multi-media multi-party with
SFU conference application  and what is needed to make it work. There one could
discuss the WebRTC applications control over how the media streams SFRAME
encapsualtion and mapping onto RTP streams impact the underlying repair tools
and SFUs of WebRTC conference application. 

If things are done correctly all the components, SFRAME, RTP Payload format, and
meta deta transport would exist in specifications of their own, so that it could
be that high level description of how an WebRTC application maker would get this
to work avoiding those pitfalls that will exist.

> I raised that question already a couple of times. I am not sure what is the
> role of the SFRAME WG regarding specifying a new RTP codec-agnostic payload.
> specifications, and if anything is missing, liaise with the appropriate groups
> in order to produce the specs based on our       requirements.

Fine, lets make it explicit in a paragraph in the SFRAME WG charter that AVTCORE
WG will be the place to do the work on the RTP Payload format and the WGs will
colaborate on this. 
 
Cheers

Magnus Westerlund

---

From: Emad Omara <[email-masked]>
Date: Mon, 28 Sep 2020 15:30:01 -0700

Hi Magnus,

Jumping late to this thread, just want to summarize the discussion and
proposed changes to make sure we are all on the same page.

   1. AVTCORE is the place to address any WebRTC changes (ie payload type,
   frame metadata ,packetization, etc)
   2. SFrame charter needs to be changed to explicitly mention #1
   3. SFrame doc needs to describe the integration point and changes with
   WebRTC in high level language as a guidance for the ATCORE work (I suggest
   doing so in a separate draft)

Would these address your concerns so we can move forward? Please let me
know if there is anything else I missed.

Thanks
Emad

On Tue, Sep 15, 2020 at 2:47 AM Magnus Westerlund <magnus.westerlund=
[email-masked]> wrote:

> > > formulations made me a bit worried. After this discussion I think we
> are
> > > much
> > > However, there will be questions about where we document that
> interaction
> > > and
> > > what dependencies that do exist on other specificaitons for scalable
> codecs
> > > that
> > > are generic and is desirable to require from RTP middleboxes. In
> addition
> > > even
> > > utilize one SFRAME key and its IVs to carry multiple application
> sub-streams
> > > and
> > > going to happen in the SFRAME WG. If happening in SFRAME WG I would
> like to
> > > have
> the
> groups
> <+46%2073%20094%2090%2079>
> Torshamnsgatan 23           |

---

From: Magnus Westerlund <[email-masked]>
Date: Tue, 29 Sep 2020 12:58:04 +0000

On Mon, 2020-09-28 at 15:30 -0700, Emad Omara wrote:
> AVTCORE is the place to address any WebRTC changes (ie payload type, frame
> metadata ,packetization, etc)

I wouldn't necessary say that all type of WebRTC changes are in scope for
AVTCORE. I was primarily thinking about the RTP payload format and potentially a
new meta data header extension for the switching decision for RTP packets with
SFRAMEs in them. 

> SFrame charter needs to be changed to explicitly mention #1
Yes that would be good. 

> SFrame doc needs to describe the integration point and changes with WebRTC in
> high level language as a guidance for the ATCORE work (I suggest doing so in a
> separate draft)

Yes, I think this is more a high level use-case / architectural informational
document so that the one can ensure that the technical solution do support known
and desired uses cases of SFRAME.

> Would these address your concerns so we can move forward? Please let me know
> if there is anything else I missed.

Yes, I belief so. 

Cheers

Magnus Westerlund

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 29 Sep 2020 07:37:49 -0700

> > if there is anything else I missed.

OK, then please let me know when the github version has been updated, and
I'll copy it into the datatracker for the IESG to see.

-MSK