From nobody Wed Nov 4 11:41:39 2020 Return-Path: X-Original-To: sframe@ietf.org Delivered-To: sframe@ietfa.amsl.com Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 751ED3A0FEB; Wed, 4 Nov 2020 11:41:29 -0800 (PST) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: Alvaro Retana via Datatracker To: "The IESG" Cc: sframe-chairs@ietf.org, sframe@ietf.org, dispatch@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 7.21.0 Auto-Submitted: auto-generated Precedence: bulk Reply-To: Alvaro Retana Message-ID: <160451888939.14817.13295583498248182378@ietfa.amsl.com> Date: Wed, 04 Nov 2020 11:41:29 -0800 Archived-At: Subject: [Sframe] Alvaro Retana's No Objection on charter-ietf-sframe-00-02: (with COMMENT) X-BeenThere: sframe@ietf.org X-Mailman-Version: 2.1.29 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Nov 2020 19:41:30 -0000 Alvaro Retana has entered the following ballot position for charter-ietf-sframe-00-02: No Objection When responding, please keep the subject line intact and reply to all email addresses included in the To and CC lines. (Feel free to cut this introductory paragraph, however.) The document, along with other ballot positions, can be found here: https://datatracker.ietf.org/doc/charter-ietf-sframe/ ---------------------------------------------------------------------- COMMENT: ---------------------------------------------------------------------- It's not clear to me what exactly this last piece of text means: Input to the WG Proposals already existing relating to this charter proposal: https://datatracker.ietf.org/doc/html/draft-omara-sframe-00 It is just a pointer to existing work? Is it a statement that the WG should use (or maybe will use) this document as the base for discussion? Given that the document was the source of the discussion that led to this WG, I assume it's the latter. Please be clear. From nobody Thu Nov 5 08:03:09 2020 Return-Path: X-Original-To: sframe@ietf.org Delivered-To: sframe@ietfa.amsl.com Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 0BF263A13D2; Thu, 5 Nov 2020 08:03:00 -0800 (PST) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: Benjamin Kaduk via Datatracker To: "The IESG" Cc: sframe-chairs@ietf.org, sframe@ietf.org, dispatch@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 7.21.0 Auto-Submitted: auto-generated Precedence: bulk Reply-To: Benjamin Kaduk Message-ID: <160459217995.25275.15999743679552040874@ietfa.amsl.com> Date: Thu, 05 Nov 2020 08:02:59 -0800 Archived-At: Subject: [Sframe] Benjamin Kaduk's No Objection on charter-ietf-sframe-00-02: (with COMMENT) X-BeenThere: sframe@ietf.org X-Mailman-Version: 2.1.29 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Nov 2020 16:03:00 -0000 Benjamin Kaduk has entered the following ballot position for charter-ietf-sframe-00-02: No Objection When responding, please keep the subject line intact and reply to all email addresses included in the To and CC lines. (Feel free to cut this introductory paragraph, however.) The document, along with other ballot positions, can be found here: https://datatracker.ietf.org/doc/charter-ietf-sframe/ ---------------------------------------------------------------------- COMMENT: ---------------------------------------------------------------------- This working group, however, will not specify the signaling required to configure SFrame encryption. In particular, considerations related to SIP or SDP are out of scope. This is because SFrame is intended to be applied as an additional layer on top of the base levels of protection that these protocols provide. [...] This text reads like it is saying that SIP and SDP provide "base levels of protection", which is not necessarily the case in the context of the protection that SFrame provides. It is anticipated that several use cases of SFrame will involve its use with keys derived from the MLS group key exchange protocol. The working group will define a mechanism for doing SFrame encryption using keys from MLS, including, for example, the derivation of SFrame keys per MLS epoch and per sender. It may be worth adding another sentence here to reiterate that just because MLS is complicated and may need special integration, that does not preclude using SFrame with other sources of key material. From nobody Fri Nov 6 11:09:07 2020 Return-Path: X-Original-To: sframe@ietf.org Delivered-To: sframe@ietfa.amsl.com Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id A294A3A0B5C; Fri, 6 Nov 2020 11:08:54 -0800 (PST) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit From: The IESG To: "IETF-Announce" X-Test-IDTracker: no X-IETF-IDTracker: 7.21.0 Auto-Submitted: auto-generated Precedence: bulk Cc: sframe-chairs@ietf.org, The IESG , dispatch@ietf.org, sframe@ietf.org MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit Message-ID: <160468973464.30968.15143783219177718112@ietfa.amsl.com> Date: Fri, 06 Nov 2020 11:08:54 -0800 Archived-At: Subject: [Sframe] WG Action: Formed Secure Media Frames (sframe) X-BeenThere: sframe@ietf.org X-Mailman-Version: 2.1.29 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2020 19:08:55 -0000 A new IETF WG has been formed in the Applications and Real-Time Area. For additional information, please contact the Area Directors or the WG Chairs. Secure Media Frames (sframe) ----------------------------------------------------------------------- Current status: Proposed WG Chairs: Bobo Bose-Kolanu Martin Thomson Assigned Area Director: Murray Kucherawy Applications and Real-Time Area Directors: Barry Leiba Murray Kucherawy Mailing list: Address: sframe@ietf.org To subscribe: https://www.ietf.org/mailman/listinfo/sframe Archive: https://mailarchive.ietf.org/arch/browse/sframe/ Group page: https://datatracker.ietf.org/group/sframe/ Charter: https://datatracker.ietf.org/doc/charter-ietf-sframe/ Real-time conferencing sessions increasingly require end-to-end protections that prevent intermediary servers from decrypting real-time media. The PERC WG developed a “double encryption” scheme for end-to-end encryption that was deeply tied to SRTP as its underlying transport. This entanglement has prevented widespread deployment. This working group will define the SFrame secure encapsulation to provide authenticated encryption for real-time media content that is independent of the underlying transport. The encapsulation will provide the following information to drive the authenticated encryption for each encryption operation: * Selection among multiple encryption keys in use during a real-time session * An algorithm for forming a unique nonce within the scope of the key based on information in the encapsulation framing The SFrame specification will detail the specific security properties that the encapsulation provides, and discuss their implications under common usage scenarios / threat models. The transport-independence of this encapsulation means that it can be applied at a higher level than individual RTP payloads. For example, it may be desirable to encrypt whole frames that span multiple packets in order to amortize the overhead from framing and authentication tags. It may also be desirable to encrypt units of intermediate size (e.g., H.264 NALUs or AV1 OBUs) to allow partial frames to be usable. The working group will choose what levels of granularity can be selected in the protocol. An application using SFrame will need to choose several aspects of its operation, for example: * Selecting whether SFrame is to be used for a given media flow * Specifying which encryption algorithm should be used * Provisioning keys and key identifiers to endpoints * Selecting the granularity at which SFrame encryption is applied (if multiple options are available) This working group, however, will not specify the signaling required to arrange SFrame encryption. In particular, considerations related to SIP or SDP are out of scope. This is because SFrame is intended to be applied as an additional layer on top of the base levels of protection that these protocols provide. This working group will, however, define the guidance for how SFrame interacts with RTP (e.g., with regard to packetization, depacketization, and recovery algorithms) to ensure that it can be used in environments such as WebRTC. Other WebRTC changes such as the payload format and metadata format will be addressed by the AVTCORE working group. It is anticipated that several use cases of SFrame will involve its use with keys derived from the MLS group key exchange protocol. The working group will define a mechanism for doing SFrame encryption using keys from MLS, including, for example, the derivation of SFrame keys per MLS epoch and per sender. The availability of this mechanism for using keys from MLS does not preclude the use of other sources of key material. Milestones: Jun 2021 - Submit SFrame specification to IESG (Standards Track) From nobody Sun Nov 8 15:33:23 2020 Return-Path: X-Original-To: sframe@ietfa.amsl.com Delivered-To: sframe@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 39A8E3A0FD2 for ; Sun, 8 Nov 2020 15:33:21 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.101 X-Spam-Level: X-Spam-Status: No, score=-2.101 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=lowentropy.net header.b=Bq6flvSo; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=WL1y9mng Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1HUYODMuVafY for ; Sun, 8 Nov 2020 15:33:19 -0800 (PST) Received: from wout5-smtp.messagingengine.com (wout5-smtp.messagingengine.com [64.147.123.21]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6CABB3A0FD1 for ; Sun, 8 Nov 2020 15:33:16 -0800 (PST) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id 5F8DAEBC for ; Sun, 8 Nov 2020 18:33:15 -0500 (EST) Received: from imap10 ([10.202.2.60]) by compute1.internal (MEProxy); Sun, 08 Nov 2020 18:33:15 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lowentropy.net; h=mime-version:message-id:in-reply-to:references:date:from:to :subject:content-type:content-transfer-encoding; s=fm3; bh=l69Ch qJNOjGBCSke63X8ZXtr9TdopGP+lJotjqOLzvk=; b=Bq6flvSogE2/NYK9To0ya 7H1BlQ/WByGdPDVNbJuzjAf1QgwZ/ULDd1bFlNx6wcu/UjtfqmiaNlnEBWUS8ohQ 3LM70mmVFw40EsG84t/cgEiONRjvlNjC760Um0qc145g73Nnf46Zr/glkuRmLGaw RvDhTj7wYZxgXgSksRUfr9LlONK/qaVSll149YFrk1MHCe5+SySmlG68Oa4yckEg uDa91yrv0QPR4zTBb+FeHNMXrRwT+oEQslJYb2VMrl/DKAqZkRVn9zBw7h3SKEGc BgxtjmLTor8ExJD8wXHgsXAZhJniHFPV9F7dDJW5fEDLzJyy3Ha5v3oiNvpKZHkC Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=l69ChqJNOjGBCSke63X8ZXtr9TdopGP+lJotjqOLz vk=; b=WL1y9mngT5nXMoVpDRDT3T4P0Zs/e3AmIIEyOQYW6GwmbcmgndBYvhr4G rCQZaBi753Z70d2WJsIpSctb7jYNxDhHdkrsS/rHHJaAJIgHe9UtT19FDm9NzOHA inmhmcanVNtR6q0bQjTmHWwIvkLD0A+oqc0mnnnZZBT2Rx9gVefdXJalDYZWVigt e8a5DuKbSKlBssUFaFJVzj+52FG7KkUW7IMv/uUwADFDffs2KPw4nBbiB3pfEOG1 AT+/4ULIW2oJKef1ilI7qvcEQ/WGqIUc7ebqzUG3gZJ3VIoFuFIXPAHcf/jAFARd 6Hvvx3vuNFTUR0aHYLwVxWlqDsb8Q== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedruddugedgudduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtgfesth hqredtreerjeenucfhrhhomhepfdforghrthhinhcuvfhhohhmshhonhdfuceomhhtsehl ohifvghnthhrohhphidrnhgvtheqnecuggftrfgrthhtvghrnhepvedtveejudeigffhue eguedvtdfhffeitdekgeeuheffleevjeduvdekfefhieegnecuffhomhgrihhnpehivght fhdrohhrghdpmhgvvghtvggthhhordgtohhmpdhgihhthhhusgdrtghomhenucevlhhush htvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmtheslhhofigvnhht rhhophihrdhnvght X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id 847F420093; Sun, 8 Nov 2020 18:33:14 -0500 (EST) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.3.0-570-gba0a262-fm-20201106.001-gba0a2623 Mime-Version: 1.0 Message-Id: In-Reply-To: <160468973464.30968.15143783219177718112@ietfa.amsl.com> References: <160468973464.30968.15143783219177718112@ietfa.amsl.com> Date: Mon, 09 Nov 2020 10:32:55 +1100 From: "Martin Thomson" To: sframe@ietf.org Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: quoted-printable Archived-At: Subject: Re: [Sframe] WG Action: Formed Secure Media Frames (sframe) X-BeenThere: sframe@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Nov 2020 23:33:21 -0000 Hey everyone, After consultation with a few people, I've posted an initial agenda (inc= luded below). If you have any thoughts on this and how we might better = use the two hours we have, any input is welcome. See also https://datatracker.ietf.org/doc/agenda-109-sframe/ ## Meeting Details Tuesday Session III (16:00-18:00 UTC+7, 09:00-11:00 UTC) Chairs: Bobo Bose-Kolanu, Martin Thomson Resources: * [Minutes](https://codimd.ietf.org/notes-ietf-109-sframe) * [Chat](xmpp:sframe@jabber.ietf.org?join) * [Video Stream](https://meetings.conf.meetecho.com/ietf109/?group=3Dsfr= ame&short=3D&item=3D1) * [Calendar](https://datatracker.ietf.org/meeting/109/session/28496.ics)= * [Materials](https://github.com/sframe-wg/wg-materials) ## Agenda | Item | People | Time | | :--------------- | :--------------------- | ---: | | Introduction | Chairs | 5 | | Charter Review | Chairs | 10 | | Use Cases | | | | - Conferencing | Emad Omara | 10 | | - Streaming | Dr. Alex Gouaillard | 10 | | - WebRTC | Youenn Fablet | 10 | | Protection | Emad Omara | 30 | | - Video Payloads | Sergio Garcia Murrillo | 20 | | MLS Integration | Richard Barnes | 20 | | AOB | - | ~ | ## Documents [draft-omara-sframe](https://datatracker.ietf.org/doc/html/draft-omara-s= frame-00) On Sat, Nov 7, 2020, at 06:08, The IESG wrote: > A new IETF WG has been formed in the Applications and Real-Time Area. = For > additional information, please contact the Area Directors or the WG Ch= airs. >=20 > Secure Media Frames (sframe) > ----------------------------------------------------------------------= - > Current status: Proposed WG >=20 > Chairs: > Bobo Bose-Kolanu > Martin Thomson >=20 > Assigned Area Director: > Murray Kucherawy >=20 > Applications and Real-Time Area Directors: > Barry Leiba > Murray Kucherawy >=20 > Mailing list: > Address: sframe@ietf.org > To subscribe: https://www.ietf.org/mailman/listinfo/sframe > Archive: https://mailarchive.ietf.org/arch/browse/sframe/ >=20 > Group page: https://datatracker.ietf.org/group/sframe/ >=20 > Charter: https://datatracker.ietf.org/doc/charter-ietf-sframe/ >=20 > Real-time conferencing sessions increasingly require end-to-end protec= tions > that prevent intermediary servers from decrypting real-time media. Th= e PERC > WG developed a =E2=80=9Cdouble encryption=E2=80=9D scheme for end-to-e= nd encryption that was > deeply tied to SRTP as its underlying transport. This entanglement ha= s > prevented widespread deployment. >=20 > This working group will define the SFrame secure encapsulation to prov= ide > authenticated encryption for real-time media content that is independe= nt of > the underlying transport. The encapsulation will provide the followin= g > information to drive the authenticated encryption for each encryption > operation: >=20 > * Selection among multiple encryption keys in use during a real-time s= ession >=20 > * An algorithm for forming a unique nonce within the scope of the key = based > on information in the encapsulation framing >=20 > The SFrame specification will detail the specific security properties = that > the encapsulation provides, and discuss their implications under commo= n usage > scenarios / threat models. >=20 > The transport-independence of this encapsulation means that it can be = applied > at a higher level than individual RTP payloads. For example, it may b= e > desirable to encrypt whole frames that span multiple packets in order = to > amortize the overhead from framing and authentication tags. It may al= so be > desirable to encrypt units of intermediate size (e.g., H.264 NALUs or = AV1 > OBUs) to allow partial frames to be usable. The working group will ch= oose > what levels of granularity can be selected in the protocol. >=20 > An application using SFrame will need to choose several aspects of its= > operation, for example: >=20 > * Selecting whether SFrame is to be used for a given media flow >=20 > * Specifying which encryption algorithm should be used >=20 > * Provisioning keys and key identifiers to endpoints >=20 > * Selecting the granularity at which SFrame encryption is applied (if > multiple options are available) >=20 > This working group, however, will not specify the signaling required t= o > arrange SFrame encryption. In particular, considerations related to S= IP or > SDP are out of scope. This is because SFrame is intended to be applie= d as an > additional layer on top of the base levels of protection that these pr= otocols > provide. This working group will, however, define the guidance for ho= w > SFrame interacts with RTP (e.g., with regard to packetization, > depacketization, and recovery algorithms) to ensure that it can be use= d in > environments such as WebRTC. Other WebRTC changes such as the payload= format > and metadata format will be addressed by the AVTCORE working group. >=20 > It is anticipated that several use cases of SFrame will involve its us= e with > keys derived from the MLS group key exchange protocol. The working gr= oup > will define a mechanism for doing SFrame encryption using keys from ML= S, > including, for example, the derivation of SFrame keys per MLS epoch an= d per > sender. The availability of this mechanism for using keys from MLS do= es not > preclude the use of other sources of key material. >=20 > Milestones: >=20 > Jun 2021 - Submit SFrame specification to IESG (Standards Track) >=20 >=20 >=20 > From nobody Sun Nov 15 22:24:41 2020 Return-Path: X-Original-To: sframe@ietfa.amsl.com Delivered-To: sframe@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 361DC3A1409 for ; Sun, 15 Nov 2020 22:24:40 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.896 X-Spam-Level: X-Spam-Status: No, score=-1.896 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=ipv-sx.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kV25rOf_ijhW for ; Sun, 15 Nov 2020 22:24:38 -0800 (PST) Received: from mail-qk1-x72e.google.com (mail-qk1-x72e.google.com [IPv6:2607:f8b0:4864:20::72e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id AE16E3A1405 for ; Sun, 15 Nov 2020 22:24:38 -0800 (PST) Received: by mail-qk1-x72e.google.com with SMTP id k4so5994970qko.13 for ; Sun, 15 Nov 2020 22:24:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ipv-sx.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=HfV6kCOoAPu1bfI6OnSPTZ8iE1QFoWNlsZx6xbfZOj0=; b=fa0LkNTFeuVBJ32sbjfnvUNc21MWw1rZnu1CF0rpzOfEmQ36B+W6nFYkCAcx9V+8pc LCrG2AzMNQXMKNPzBv3SvC9LcxIGey9B1XxXK06j85cZ0kVUZC94zBb3ZUpRO1PfLsp2 7Pf53dL3qO5Lra5NwN4ZKm5tAJoMGs72od8qQL9XOg640aov9ozP2hOsHThVAI4+De34 Tf+UXwJBVLlekcSl5vhP90N7LckUZuQyJ6o/mOU17ctx3zq//mfHEUSoeyU1vPP0UW0b 0dWG28ec47aW9R/SI7z9hJrq8I01+PXIsij2UrX+Tgmui8nwZiQ0ECcrRaBcc7BCexpP 4r6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=HfV6kCOoAPu1bfI6OnSPTZ8iE1QFoWNlsZx6xbfZOj0=; b=DkLA2uZWBy/Jq2c0LfJuYCXt+dDkpZqHCrEhY0x40JqgkuaWwY/+d6FiBZzdBLoAcA Tl8BkIK5XvI7byLqQp1MX6oJwD/80byiS3dQY2MdZL5ayWRkyxaujC0bPxhyOBxe+KFf EX1qwnOJaZxuZfAvndTMB9mvLKtd5YPmHNRD9HBWhzoF+M5f/im4MNL5goA7efOqAfjb mpeQcszZ+kmwGIMRw6Vbs+Oww2CQAUOaZ6jy4KBjFg6B9z9I4zLNf+2DhMQeksX0+SCd L9jHe33WT/Lb+OeSpDfWEenlQqnTDAou0GvubSVjGztFmzv2p1UuQIavasGvYYrjM3wP qiQg== X-Gm-Message-State: AOAM531xBjiYyHKhhipe/yHlcKBfNxbEph8mBJBmexFGx2KULU8cnq79 TjghdVAFbgXhU45zX499CLUQzoF3/8h+mp79Cw8zajCke0s= X-Google-Smtp-Source: ABdhPJwDWUKqv2djTB5pnuO2i7QUCd3k7t7cJfEvpAhulhippDaqFdAvp3DZuTJzRsYjVrnBv6KVLDAIJj1UYnkdJig= X-Received: by 2002:a37:4d13:: with SMTP id a19mr13143469qkb.159.1605507877273; Sun, 15 Nov 2020 22:24:37 -0800 (PST) MIME-Version: 1.0 References: <160550774779.2160.13722712942589938652@ietfa.amsl.com> In-Reply-To: <160550774779.2160.13722712942589938652@ietfa.amsl.com> From: Richard Barnes Date: Mon, 16 Nov 2020 01:24:24 -0500 Message-ID: To: sframe@ietf.org Content-Type: multipart/alternative; boundary="000000000000b59af905b4336ee6" Archived-At: Subject: [Sframe] Fwd: Confirm submission of I-D draft-barnes-sframe-mls X-BeenThere: sframe@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Nov 2020 06:24:40 -0000 --000000000000b59af905b4336ee6 Content-Type: text/plain; charset="UTF-8" Hi SFrame folks, FYI, I have just submitted the following draft, which describes a scheme for generating keys for SFrame out of MLS. There is some time for discussion on the agenda for tomorrow, and of course discussion is welcome here. --Richard ---------- Forwarded message --------- From: IETF I-D Submission Tool Date: Mon, Nov 16, 2020 at 1:22 AM Subject: Confirm submission of I-D draft-barnes-sframe-mls To: Raphael Robert , Richard Barnes Hi, The IETF datatracker draft submission service has received your draft draft-barnes-sframe-mls-00, and requires a confirmation step in order to be able to complete the posting of the draft. Please follow this link to the page where you can confirm the posting: https://datatracker.ietf.org/submit/status/115847/confirm/5a9663259a7f5c60d8b744b43fd32ec8/ Best regards, The IETF Secretariat through the draft submission service --000000000000b59af905b4336ee6 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi SFrame folks,

FYI, I have= just submitted the following draft, which describes a scheme for generatin= g keys for SFrame out of MLS.=C2=A0 There is some time for discussion on th= e agenda for tomorrow, and of course discussion is welcome here.
=
--Richard

---------- Forwarded message ---------
= From: IETF I-D Submission Tool <idsubmis= sion@ietf.org>
Date: Mon, Nov 16, 2020 at 1:22 AM
Subje= ct: Confirm submission of I-D draft-barnes-sframe-mls
To: Raphael Robert= <raphael@wire.com>, Richard = Barnes <rlb@ipv.sx>


Hi,

The IETF datatracker draft submission service has received your draft
draft-barnes-sframe-mls-00, and requires a
confirmation step in order to be able to complete the posting of
the draft.
Please follow this link to the page where you can confirm the posting:

https://d= atatracker.ietf.org/submit/status/115847/confirm/5a9663259a7f5c60d8b744b43f= d32ec8/


Best regards,

=C2=A0 =C2=A0 =C2=A0 =C2=A0 The IETF Secretariat
=C2=A0 =C2=A0 =C2=A0 =C2=A0 through the draft submission service



--000000000000b59af905b4336ee6-- From nobody Tue Nov 17 00:36:00 2020 Return-Path: X-Original-To: sframe@ietfa.amsl.com Delivered-To: sframe@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1052A3A098D for ; Tue, 17 Nov 2020 00:35:58 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.098 X-Spam-Level: X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=lowentropy.net header.b=CtnR1Tfe; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=kSSAc6PS Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id quhBDLlkpL7B for ; Tue, 17 Nov 2020 00:35:56 -0800 (PST) Received: from wout4-smtp.messagingengine.com (wout4-smtp.messagingengine.com [64.147.123.20]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 010C53A0975 for ; Tue, 17 Nov 2020 00:35:36 -0800 (PST) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id 59B67EC4 for ; Tue, 17 Nov 2020 03:35:36 -0500 (EST) Received: from imap10 ([10.202.2.60]) by compute1.internal (MEProxy); Tue, 17 Nov 2020 03:35:36 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lowentropy.net; h=mime-version:message-id:date:from:to:subject:content-type; s= fm3; bh=926QSDqy53VIryQC/9ID0HEO3o8Ae1J7fUFA5sYWEoA=; b=CtnR1Tfe iEq+Uel6BhXxcVD7Y9G3EmQUEz3IazFRHTT2gYIcgRBCuJYhm3H92zct0Trg0aS1 nrnUErMo/gOvK11u5zP73SRpCwD0EsHelkF5x8cZtp4kqkdK1Pdyqq7SrmsDKRwn AHcEO9XCySO+SU1sRJKkYBj/ARCeadCFOUHPYavqrSjYUz8yGYIiX8YW5oXv6cUg d+bU4ZknmXm4z+ZO/JvMt6m3FBpQdqSdfceWRZHh7zbXxGM2e8tpLG8VuW/Ty2vu Bb5knelJw4uUGQQZCnYA2MutJyE+gMql3m8Aqp0yU8L+DP7AwWTj4oqG3ykws28F 1/LBQZZftxUtxw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:message-id :mime-version:subject:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm1; bh=926QSDqy53VIryQC/9ID0HEO3o8Ae 1J7fUFA5sYWEoA=; b=kSSAc6PS+3GGZJHiksLVyVmA7fBUnVUB4qM1SqFTInhh6 qgkGpwyJj04bE1XXXinGNlJjMVWmxAeNW1TaCZj/gVBnPr7bUhiO1uP57mdFqh+w Zs9B83zzQQ0JKT9wCiFiMzmhW5n7sNOHi8I0dt6APHydBuTCqrSPpWk6JGHodqq/ NsaQsvZ8NTdReqHnnkh6H2ji1M4nyoe6rhiQS4NViYYebCugba3+/yWd7vQUh7Hf KpAUX2jn3qvu6Nw2yF23b/ROHqlKdFHRj73UPgpndNPyJcDF4UPFkqVGnq6sJ5sX iPcNMvcDsBs3WtbaYZMZ1nEb9hhodPiY/7OLoqKLA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrudefvddgudduudcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhepofgfggfkfffhvffutgesthdtre dtreertdenucfhrhhomhepfdforghrthhinhcuvfhhohhmshhonhdfuceomhhtsehlohif vghnthhrohhphidrnhgvtheqnecuggftrfgrthhtvghrnhepvefguddtgfejfeduueehke ehkeduueegheefgeevkeekgeelveevffeuudffheehnecuvehluhhsthgvrhfuihiivgep tdenucfrrghrrghmpehmrghilhhfrhhomhepmhhtsehlohifvghnthhrohhphidrnhgvth X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id BDA15200BC; Tue, 17 Nov 2020 03:35:35 -0500 (EST) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.3.0-622-g4a97c0b-fm-20201115.001-g4a97c0b3 Mime-Version: 1.0 Message-Id: <5c3fa9e5-ae74-41f5-b82b-4361ba3ff404@www.fastmail.com> Date: Tue, 17 Nov 2020 19:35:14 +1100 From: "Martin Thomson" To: sframe@ietf.org Content-Type: text/plain Archived-At: Subject: [Sframe] Seeking backup X-BeenThere: sframe@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2020 08:35:58 -0000 I have tried, unsuccessfully, to debug meetecho problems for the last hour. There is no guarantee I will be able to do anything for the upcoming meeting. Anyone who can drive slides, please contact me. From nobody Tue Nov 17 00:55:56 2020 Return-Path: X-Original-To: sframe@ietfa.amsl.com Delivered-To: sframe@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BF9123A0A07 for ; Tue, 17 Nov 2020 00:55:54 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.896 X-Spam-Level: X-Spam-Status: No, score=-1.896 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=ipv-sx.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nGgiumNYiknT for ; Tue, 17 Nov 2020 00:55:53 -0800 (PST) Received: from mail-qk1-x72d.google.com (mail-qk1-x72d.google.com [IPv6:2607:f8b0:4864:20::72d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4E5243A09FC for ; Tue, 17 Nov 2020 00:55:52 -0800 (PST) Received: by mail-qk1-x72d.google.com with SMTP id 11so19681324qkd.5 for ; Tue, 17 Nov 2020 00:55:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ipv-sx.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=0g8mMGZyRigAOXAdnV9XYLvP3kKSPRXM3hg9bDR1bpc=; b=kVlRB4Eb7a4s9Ui+YpVBBMZY0o7xlFV9GbLAg4RuNPDC/ykYfN04bey+eMTP+DcKHD 9bxXbPb4Xw+dDlrPz7Bj1Qt3EjKlgnXa37EGtPfT2HidtLadaNRBkagUWM78kyFn9PDZ Wqry4wibsp4B8JxGkQ9yRbru+azoJA9CwaL6tK5SegFRovjGeTYFTWbe5woDVJ7g9/+B WEXD2GMGm7q74C3WP3cae3kPJROlUHCaYrkzUpa6Go5GlGZQLjg7SmJ5hyXZC7+zKqwi 7TzQyynzQs+RZj4KKSfqBLKprru/cdySRxA1n5Jbf0/sn/2hYnwjG/nAEAamu2iL1+Ee v87w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=0g8mMGZyRigAOXAdnV9XYLvP3kKSPRXM3hg9bDR1bpc=; b=rLKFuSUTZF/codvLbaT+9M/1IQR3iKV+UoBLjgpSd/ThEkFn0nqo4Scagd46/VbQff TA8r7kPOOviyZVE5LoSCU3o2Ntr9Kk1xSQKIh46X97z4cxHaUqB92iyNRPYDgDo8WL4a GPMhEh70aRIT/M30u8sjntqSNBdZprRFF9VOR1kjY2N80r7S0wFNhoODl2cVxDcRRd3g BMDWr13v2+QS6gF2iasziP+AHi7+XoTxCsv72ogrK4CHtfwkfx3D5aAQOXd0AZrPqmxt Rk2VB87dPMJanxHZUtNd7ez8T/wLT9TofP2RWeiXa6nVcqTsSIyBS9tgWU7+n5Q1WLC/ MHnw== X-Gm-Message-State: AOAM533ftGDParZexDusdIxip+SOiS/pWuRxr+aI02Fy0jEP2CkimUrD z9XPiJLEQ7OlRAN5R35DOshXJQOYTySGiR57EGB4S2/blfo= X-Google-Smtp-Source: ABdhPJwlIdA+v4uB3E2TTiy2fj20fGXeSTcCLwt3ELsEjAORXzw4vOmylpPvw7sf4nTS/7Dae/KZWK4uHiS0ywSHvkw= X-Received: by 2002:a37:a855:: with SMTP id r82mr12555848qke.132.1605603351808; Tue, 17 Nov 2020 00:55:51 -0800 (PST) MIME-Version: 1.0 References: <5c3fa9e5-ae74-41f5-b82b-4361ba3ff404@www.fastmail.com> In-Reply-To: <5c3fa9e5-ae74-41f5-b82b-4361ba3ff404@www.fastmail.com> From: Richard Barnes Date: Tue, 17 Nov 2020 03:55:36 -0500 Message-ID: To: Martin Thomson Cc: sframe@ietf.org Content-Type: multipart/alternative; boundary="0000000000006f67e105b449a9f2" Archived-At: Subject: Re: [Sframe] Seeking backup X-BeenThere: sframe@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2020 08:55:55 -0000 --0000000000006f67e105b449a9f2 Content-Type: text/plain; charset="UTF-8" I can drive, if things are in the meeting materials manager. On Tue, Nov 17, 2020 at 3:36 AM Martin Thomson wrote: > I have tried, unsuccessfully, to debug meetecho problems for the last > hour. There is no guarantee I will be able to do anything for the upcoming > meeting. Anyone who can drive slides, please contact me. > > -- > Sframe mailing list > Sframe@ietf.org > https://www.ietf.org/mailman/listinfo/sframe > --0000000000006f67e105b449a9f2 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I can drive, if things are in the meeting materials manage= r.

On Tue, Nov 17, 2020 at 3:36 AM Martin Thomson <mt@lowentropy.net> wrote:
I have tried, unsuccessfully, to debug = meetecho problems for the last hour.=C2=A0 There is no guarantee I will be = able to do anything for the upcoming meeting.=C2=A0 Anyone who can drive sl= ides, please contact me.

--
Sframe mailing list
Sframe@ietf.org https://www.ietf.org/mailman/listinfo/sframe
--0000000000006f67e105b449a9f2-- From nobody Tue Nov 17 13:44:24 2020 Return-Path: X-Original-To: sframe@ietfa.amsl.com Delivered-To: sframe@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 782123A0A3D for ; Tue, 17 Nov 2020 13:44:23 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.098 X-Spam-Level: X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=lowentropy.net header.b=O+2nLKQ1; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=Y4V4Zo+d Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 92khh-4bYtJr for ; Tue, 17 Nov 2020 13:44:22 -0800 (PST) Received: from wout4-smtp.messagingengine.com (wout4-smtp.messagingengine.com [64.147.123.20]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4F5D63A07C0 for ; Tue, 17 Nov 2020 13:44:22 -0800 (PST) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id A54E6E83 for ; Tue, 17 Nov 2020 16:44:21 -0500 (EST) Received: from imap10 ([10.202.2.60]) by compute1.internal (MEProxy); Tue, 17 Nov 2020 16:44:21 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lowentropy.net; h=mime-version:message-id:date:from:to:subject:content-type; s= fm3; bh=or1mW+f+wBQwjyvUbD1EAwjjWqL0WbKjMO9AxRijHRo=; b=O+2nLKQ1 BDx5L+lECoHOIZcmtDOlO+UcRYZTVFJuFahP1Q7XbSARQl/1hQLF6oNrYirZ9cvS CIYqb5EPERdFe1PVyZocMaQ/IYZCTQqj/7UutMu/PY6/+XsOCtmPuk1N5JigRrDF lWsSwd+1ym2NSm65bNB88cEGMCWHjNTVjF003sM6bfk4rxd7NNu2eRj+P/jsSonz MxEo2c/dH9FRAqsY59Hx1BXDQnrUjSRbACvRIsWDjLmU6TM3g9TBJnwShznJLX0B Ru9Eh90uuuezYNCNShm9h8L+qKub9mLNR03Cstzua9GTRlDS4jgJYU3FjtGq+XRG GNmhJ9YlLq+Alg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:message-id :mime-version:subject:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm1; bh=or1mW+f+wBQwjyvUbD1EAwjjWqL0W bKjMO9AxRijHRo=; b=Y4V4Zo+dQkpLqyLrbt2qBMTG6g1h3eWidan8Sew23J17w L4WaBHgOAZ46jCH4NTygNFjAuBwJliFY7I0s+yjpDQ0M3ScKRadppy7Yetoh37mr 5QB0vgLkwNS+Hx3AEPwVDnZeSbCbgGP34if+bl/BDk29GG5+AfgP7qSKuZkO9hbC JXQbJlVodnKhDC6vYEvIvjee2OT/B85HQ0MJHdkS+UxSIFwmXpggaUPsXnYXiDCw kE32ROyTBVVSSvPHmpL2TKCRCTlU//Hho4Zs5O9WUCZKxBGsaSb2FluLM0mk10AX u8sceGzQ50lnEV7YeQXc1PyfwkB10y84G8SWqellw== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrudeffedgudehfecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecuogfuuhhsphgvtghtffhomhgrihhnucdlgeelmd enucfjughrpefofgggkfffhffvufgtsehttdertderredtnecuhfhrohhmpedfofgrrhht ihhnucfvhhhomhhsohhnfdcuoehmtheslhhofigvnhhtrhhophihrdhnvghtqeenucggtf frrghtthgvrhhnpeffgeegfeeutdegleehgfdtieffieefteevheelieeliedtgeeuleev uedtteduvdenucffohhmrghinhepihgvthhfrdhorhhgpdhgihhthhhusgdrihhonecuve hluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepmhhtsehlohif vghnthhrohhphidrnhgvth X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id 0E50D200AD; Tue, 17 Nov 2020 16:44:20 -0500 (EST) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.3.0-622-g4a97c0b-fm-20201115.001-g4a97c0b3 Mime-Version: 1.0 Message-Id: Date: Wed, 18 Nov 2020 08:44:00 +1100 From: "Martin Thomson" To: sframe@ietf.org Content-Type: text/plain Archived-At: Subject: [Sframe] Minutes uploaded X-BeenThere: sframe@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2020 21:44:24 -0000 Thanks again to Richard for stepping in and covering. https://datatracker.ietf.org/meeting/109/session/sframe contains the official materials. https://sframe-wg.github.io/wg-materials/ hosts the materials in a nicer format. If you have any corrections to make, please use the interface GitHub provides. From nobody Thu Nov 19 13:40:35 2020 Return-Path: X-Original-To: sframe@ietfa.amsl.com Delivered-To: sframe@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 102373A1296 for ; Thu, 19 Nov 2020 13:40:34 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.097 X-Spam-Level: X-Spam-Status: No, score=-2.097 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GLZK5xNkNP91 for ; Thu, 19 Nov 2020 13:40:32 -0800 (PST) Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B69583A1297 for ; Thu, 19 Nov 2020 13:40:00 -0800 (PST) Received: by mail-wm1-x32b.google.com with SMTP id p22so8536838wmg.3 for ; Thu, 19 Nov 2020 13:40:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=to:from:subject:message-id:date:user-agent:mime-version :content-language; bh=qaCVQds58ter7U1M3kDtKOwWpN3ByRL3nsBoOVXPyPk=; b=SgSfxWnGusttR7cqNQ7G4dL0mGY19il9THTRrAsDs5CrTfcuvA/fsvmyVO9fezKGBW M5iSBbxmJIWr/LaILRC+1ubYiO8i4sigEvkxpDAwihTq4NE94dHLVS5SfT82CRZaGtvW WAUEx7dLqWzd1fpaxTpdr2QnAoJNYIHQD9sx4iAIgOuFo2s4+GIpXHGiuX5MqJQs26Vr On+oJvJwkOZVPq68NE/qjKfXc9UgJcFj6RpCdiMRJyExuSjf4GumJiZpsN5+YUFQKOZ7 29xfwXMkk4VSmt1W0N5PEmpNDRb7WxJPfoLYFO7lnZvpxXmFR/I/GHh+RwPNeAokGo4d dCBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:from:subject:message-id:date:user-agent :mime-version:content-language; bh=qaCVQds58ter7U1M3kDtKOwWpN3ByRL3nsBoOVXPyPk=; b=H5+bxM4B0aI454liAXHYqQfJQybSFRVO2c5G/fWccM70djXOHTUcypb+LRSnGJ98xi /7yaHbioLyWHIXIIAJgc3yk46zr2Dc7SIx7+IANzECX+fPMAx1qxIqs6cxAl/6k426NK avlhF2sNr+6nP5wNXL3cAyIpwRBtw7cjAjRaMSO59dRMDqBC1jgrMDaIefUB6q2Pt3Qb zmwuJsXcnbb4HRIKOv66xOJ08IrO0UG47Q+ysjGB6l9Yh727ngrxlfWIHDD8Zql1G2ut P0iRM5fJ+8mAYFwxk6cFN7/+gwPii8WExUrz2muegW9w3KUfV9+dO1J32uS7X1LUmR5t gPdA== X-Gm-Message-State: AOAM531T00mfHYAEuVwf1Wrd+Bvtp27/rVqwnL6EB2pzG5BFPgKc3XHV HsfW1HxeTm1HV0aJSXG4tuqjhX5XI9UUtw== X-Google-Smtp-Source: ABdhPJxI45+4OaptL6x71/FOTe0Zh12lPDbd4Z/8i7gM1vtykcC7KDuzRrjyvlILfzSuq46LNLfQLA== X-Received: by 2002:a1c:6002:: with SMTP id u2mr6107233wmb.29.1605821998822; Thu, 19 Nov 2020 13:39:58 -0800 (PST) Received: from [192.168.0.11] (79.108.125.160.dyn.user.ono.com. [79.108.125.160]) by smtp.googlemail.com with ESMTPSA id u16sm1762078wrn.55.2020.11.19.13.39.58 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 19 Nov 2020 13:39:58 -0800 (PST) To: "sframe@ietf.org" From: Sergio Garcia Murillo Message-ID: Date: Thu, 19 Nov 2020 22:39:57 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.4.3 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="------------01A91898AB6973D182B02591" Content-Language: en-US Archived-At: Subject: [Sframe] Partial decodability and IDUs X-BeenThere: sframe@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2020 21:40:34 -0000 This is a multi-part message in MIME format. --------------01A91898AB6973D182B02591 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Hi all, As most of you already know, this morning I made a presentation in AVTCORE introducing the topic about the need to specify an agnostic video codec packetization format. https://datatracker.ietf.org/meeting/109/materials/slides-109-avtcore-sframe-rtp-encapsulation-00 I got an AP for creating an initial draft so it could be reviewed and accepted. However, there were two main concerns that we should address in this this group: * Historically, avtcore has explicitly designed not to be payload agnostic and  declined to standardized codec agnostic payload formats in  number of cases.  If that is to be changed, needs to be done deliberately. * Need to define the "minimum decoding unit" or "independently decodable unit", that SFrame will work with. Regarding the second one * Full video frames (just use whatever is the encoder output) * Spatial layer frames * "independend decodable subframes" like h264 slices, vp8 partitions or av1 tiles which allows partial decodability which is mainly aimed for enhancing packet loss resilience. Spatial layer frames is the minimum we should target as if not it will just prevent SFUs for using SVC codecs. So the question is if we should go deeper and implement lower partitions of the frames or not. AFAIK, currently, libwertc does not support partial decodability and I personally haven't seen any practical usage of this in the RTC world (while it makes a lot of sense in streaming/broadcasting world), but would like to hear what is the view and experience of the other members of this group. Also note that if we are going to support them on SFrame this will require a greater effort because we will need to explicitly define how the frames must be split before being encrypted y SFrame for *each* possible video codec (h264,h265,vp8,vp9,av1,...). There was also the question about how/if we should support other codec features like DON/interleaved mode for h264, which I also think we should not support mainly because we are not currently using it on webrtc implementations. What do you think? Best regards Sergio --------------01A91898AB6973D182B02591 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit

Hi all,


As most of you already know, this morning I made a presentation in AVTCORE introducing the topic about the need to specify an agnostic video codec packetization format.

https://datatracker.ietf.org/meeting/109/materials/slides-109-avtcore-sframe-rtp-encapsulation-00


I got an AP for creating an initial draft so it could be reviewed and accepted.

However, there were two main concerns that we should address in this this group:

  • Historically, avtcore has explicitly designed not to be payload agnostic and  declined to standardized codec agnostic payload formats in  number of cases.  If that is to be changed, needs to be done deliberately.
  • Need to define the "minimum decoding unit" or "independently decodable unit", that SFrame will work with.


Regarding the second one

  • Full video frames (just use whatever is the encoder output)
  • Spatial layer frames
  • "independend decodable subframes" like h264 slices, vp8 partitions or av1 tiles which allows partial decodability which is mainly aimed for enhancing packet loss resilience.


Spatial layer frames is the minimum we should target as if not it will just prevent SFUs for using SVC codecs. So the question is if we should go deeper and implement lower partitions of the frames or not.


AFAIK, currently, libwertc does not support partial decodability and I personally haven't seen any practical usage of this in the RTC world (while it makes a lot of sense in streaming/broadcasting world), but would like to hear what is the view and experience of the other members of this group. Also note that if we are going to support them on SFrame this will require a greater effort because we will need to explicitly define how the frames must be split before being encrypted y SFrame for *each* possible video codec (h264,h265,vp8,vp9,av1,...).


There was also the question about how/if we should support other codec features like DON/interleaved mode for h264, which I also think we should not support mainly because we are not currently using it on webrtc implementations.


What do you think?


Best regards

Sergio


--------------01A91898AB6973D182B02591-- From nobody Thu Nov 19 13:54:16 2020 Return-Path: X-Original-To: sframe@ietfa.amsl.com Delivered-To: sframe@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9C97A3A12B4 for ; Thu, 19 Nov 2020 13:54:14 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -17.599 X-Spam-Level: X-Spam-Status: No, score=-17.599 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, ENV_AND_HDR_SPF_MATCH=-0.5, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_DEF_SPF_WL=-7.5] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HXkIsYWN7tyy for ; Thu, 19 Nov 2020 13:54:13 -0800 (PST) Received: from mail-il1-x12a.google.com (mail-il1-x12a.google.com [IPv6:2607:f8b0:4864:20::12a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0FEAA3A12B2 for ; Thu, 19 Nov 2020 13:54:13 -0800 (PST) Received: by mail-il1-x12a.google.com with SMTP id y18so6717716ilp.13 for ; Thu, 19 Nov 2020 13:54:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=CWoLRhMx5J6RkzmPehLs20k14J18ngoZZ0g9R4fBSWA=; b=t7/8cO3TCk4aSOXUA10IlIYocJSgAsfU9fL1aq5JEGNb64QShCYRkyA4fzTIYWhtE5 Xbd4/DX3WKh9P99s9abMtV0ugMXxI2M4ZuieUA/afHekI5RGq68a98mjYQdpdzPUHg1v L73zwlDfzcH01eIjC8evhBm09NwGDepUiMYmT1ZlwIH0sVeLVeY5s9wDHYMxfKukvbjf Ko15vsf8WidyiReWotj1CbISvH8cVurtTzhPusindJHM81OJqz541Q7QDwhsEnHIu9Tw j59UngxTlJSTs7z3GjuRHWfrzCZLZ+zXaS63lTlOOTn+bXDL/9yqlgUA7vJIv6bdULvs M6Bw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=CWoLRhMx5J6RkzmPehLs20k14J18ngoZZ0g9R4fBSWA=; b=MUI2xhRY5Z8NaY8HSls8WpSduE+L7otwPla//QWttfkTcidi7jQvNWKpnLpwp62Trl /Zic10aR3EIo8/RlJv7/Zc8FTzOnxOUY8hJ1YHAhCQpkZwlposKPZp33yp5Olylkc08t MSa5oRbDS3AV0PxVqIlzz1Ztio+mAXyJAuohOu1v5aHjCOr4hGU2wiJN6H+U0G/oSXsN B6P3SYYW/6jDAZj0EnUc9zykdgtmyHiHQyLPi0WtqQ4CIAK0XsyNm2qXMxWdkA92PJHi OXp/pNymupg3seeTCikXEVGRrES7jBMxa5q4I67JW/AeQ8XsFV/gqwRAykBBQZAuSbCa DVVw== X-Gm-Message-State: AOAM530kzj/PFSLOSkBIaMyFsb0lwfMw3L5NvN2pPR8IcOgsPlhcre2W k4yIKcioqtzvS4P5WiKqtLML0ETg6Z9V1TfVE0S6zw== X-Google-Smtp-Source: ABdhPJz6fMallAmuk5pnGoQv6uvOAMFGUmuFPM/rfY3Loclt2nTdt0ybAKs8CXOBHWTnOJp6h+ZjHPgRqD1V+aAUAvg= X-Received: by 2002:a92:db0b:: with SMTP id b11mr14196161iln.55.1605822851977; Thu, 19 Nov 2020 13:54:11 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Justin Uberti Date: Thu, 19 Nov 2020 13:53:57 -0800 Message-ID: To: Sergio Garcia Murillo Cc: "sframe@ietf.org" Content-Type: multipart/alternative; boundary="000000000000aa796c05b47cc4dc" Archived-At: Subject: Re: [Sframe] Partial decodability and IDUs X-BeenThere: sframe@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2020 21:54:15 -0000 --000000000000aa796c05b47cc4dc Content-Type: text/plain; charset="UTF-8" The encoder needs to be aware of any mechanism to generate IDUs (e.g., slices), and typically each of these IDUs will be handed up to the consumer individually. So the SFRAME layer doesn't need to do any splitting, it just knows that it should treat each IDU as something it needs to individually SFRAME and packetize. On Thu, Nov 19, 2020 at 1:40 PM Sergio Garcia Murillo < sergio.garcia.murillo@gmail.com> wrote: > Hi all, > > > As most of you already know, this morning I made a presentation in AVTCORE > introducing the topic about the need to specify an agnostic video codec > packetization format. > > > https://datatracker.ietf.org/meeting/109/materials/slides-109-avtcore-sframe-rtp-encapsulation-00 > > > I got an AP for creating an initial draft so it could be reviewed and > accepted. > > However, there were two main concerns that we should address in this this > group: > > - Historically, avtcore has explicitly designed not to be payload > agnostic and declined to standardized codec agnostic payload formats in > number of cases. If that is to be changed, needs to be done deliberately. > - Need to define the "minimum decoding unit" or "independently > decodable unit", that SFrame will work with. > > > Regarding the second one > > - Full video frames (just use whatever is the encoder output) > - Spatial layer frames > - "independend decodable subframes" like h264 slices, vp8 partitions > or av1 tiles which allows partial decodability which is mainly aimed for > enhancing packet loss resilience. > > > Spatial layer frames is the minimum we should target as if not it will > just prevent SFUs for using SVC codecs. So the question is if we should go > deeper and implement lower partitions of the frames or not. > > > AFAIK, currently, libwertc does not support partial decodability and I > personally haven't seen any practical usage of this in the RTC world (while > it makes a lot of sense in streaming/broadcasting world), but would like to > hear what is the view and experience of the other members of this group. > Also note that if we are going to support them on SFrame this will require > a greater effort because we will need to explicitly define how the frames > must be split before being encrypted y SFrame for *each* possible video > codec (h264,h265,vp8,vp9,av1,...). > > > There was also the question about how/if we should support other codec > features like DON/interleaved mode for h264, which I also think we should > not support mainly because we are not currently using it on webrtc > implementations. > > > What do you think? > > > Best regards > > Sergio > > > -- > Sframe mailing list > Sframe@ietf.org > https://www.ietf.org/mailman/listinfo/sframe > --000000000000aa796c05b47cc4dc Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
The encoder needs to be aware of any mechanism to generate= IDUs (e.g., slices), and typically each of these IDUs will be handed up to= the consumer individually. So the SFRAME layer doesn't need to do any = splitting, it just knows that it should treat each IDU as something it need= s to individually SFRAME and packetize.

On Thu, Nov 19, 2020 at 1:40 PM Serg= io Garcia Murillo <se= rgio.garcia.murillo@gmail.com> wrote:
=20 =20 =20

Hi all,


As most of you already know, this morning I made a presentation in AVTCORE introducing the topic about the need to specify an agnostic video codec packetization format.

https://datatra= cker.ietf.org/meeting/109/materials/slides-109-avtcore-sframe-rtp-encapsula= tion-00


I got an AP for creating an initial draft so it could be reviewed and accepted.

However, there were two main concerns that we should address in this this group:

  • Historically, avtcore has explicitly designed not to be payload agnostic and=C2=A0 declined to standardized codec agnostic payload formats in=C2=A0 number of cases.=C2=A0 If that is to be ch= anged, needs to be done deliberately.
  • Need to define the "minimum decoding unit" or "ind= ependently decodable unit", that SFrame will work with.


Regarding the second one

  • Full video frames (just use whatever is the encoder output)
  • Spatial layer frames
  • "independend decodable subframes" like h264 slices, vp8 partitions or av1 tiles which allows partial decodability which is mainly aimed for enhancing packet loss resilience.


Spatial layer frames is the minimum we should target as if not it will just prevent SFUs for using SVC codecs. So the question is if we should go deeper and implement lower partitions of the frames or not.


AFAIK, currently, libwertc does not support partial decodability and I personally haven't seen any practical usage of this in the RTC world (while it makes a lot of sense in streaming/broadcasting world), but would like to hear what is the view and experience of the other members of this group. Also note that if we are going to support them on SFrame this will require a greater effort because we will need to explicitly define how the frames must be split before being encrypted y SFrame for *each* possible video codec (h264,h265,vp8,vp9,av1,...).


There was also the question about how/if we should support other codec features like DON/interleaved mode for h264, which I also think we should not support mainly because we are not currently using it on webrtc implementations.


What do you think?


Best regards

Sergio


--
Sframe mailing list
Sframe@ietf.org https://www.ietf.org/mailman/listinfo/sframe
--000000000000aa796c05b47cc4dc-- From nobody Thu Nov 19 14:11:24 2020 Return-Path: X-Original-To: sframe@ietfa.amsl.com Delivered-To: sframe@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 10E483A0B18 for ; Thu, 19 Nov 2020 14:11:23 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.897 X-Spam-Level: X-Spam-Status: No, score=-1.897 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, NICE_REPLY_A=-0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=cosmosoftware-io.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id q_plGTGNlPsO for ; Thu, 19 Nov 2020 14:11:21 -0800 (PST) Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DA0603A0DAB for ; Thu, 19 Nov 2020 14:11:20 -0800 (PST) Received: by mail-wr1-x42f.google.com with SMTP id j7so8066647wrp.3 for ; Thu, 19 Nov 2020 14:11:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cosmosoftware-io.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-language; bh=I05X5lXdPkG6iHplTGlEGBpUW7HxQdHH0NLlBhSnxYI=; b=x6zkupURyGiSgL6Xezd8L0GMJREvt4VjfaYYog2JQ+Td5drPUjpVb0AUzCzItoUTae X+PHV2K9vyXl5AGjw2K6PQGA2OF4vlhNUrtgSVzQdP/XIg2/T7kzlkFMw4HMMbZ7orC7 nCRrs0YKocpzIxDQT0npcoHnWgjskZUtOozOkvODPCV8w8frGSqhb99oO6USpdHuaUgi XHjTr3W2v4Yc6N2HYgKm12LHuM9L/6iAePWe+TcEX5RVWtJWVOdNgj3+jXs8AbvOfYEE A5PBmyXoYBXjxzUySYvd+zjI97yq665VDCp6LXVK8ZjlnFwRs94q4F88mDPVteyes4ey XSrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language; bh=I05X5lXdPkG6iHplTGlEGBpUW7HxQdHH0NLlBhSnxYI=; b=LoxfdVdJ5mkrDQADob0Ql02XHtMAEUUK95rSz6LI9pIlvxSSOdvVFNe9Ra/U1A3qcE xKQ0wvlWZdrfBXfyXKW+tQUNVFumcw6wBb/dhnS0/eFwYgpmgWFAVG5iSRqakrny9V+c rnBSISI94qdP4mPiNqkYzr31JVNHhE9G6vcYE00nLWKpL2rYFf0rO0sdixEYFTtt/K4+ U1Qgo+FNhkpO8mIYHm1KL58aicwpUyXS8qyE6WeVaupw1G1ZTEy8wQTkYQazKtUoQKjS sIv0ez6vV0aykyRgykhDRSA3lcUYExE5i6tvW3vXa1K1djAA3eCKtULfKjszbGXSVpaq Q+Xg== X-Gm-Message-State: AOAM532AQBsp8RCfFBrX0dFgyDg+8HOo6pCySyE82z8xWBEh4+mu+Gn9 BSPWxukLRs/2HMMYCtTG7iJhrulISj3WMg== X-Google-Smtp-Source: ABdhPJwmFecvg4x8+hXJpUY6fIJPQpP1UvAzcLbXBfIlz++uD3V3XALZJZvx1GIrD9F6N5yUz1aFRA== X-Received: by 2002:a5d:4cca:: with SMTP id c10mr13118210wrt.372.1605823878755; Thu, 19 Nov 2020 14:11:18 -0800 (PST) Received: from [192.168.0.11] (79.108.125.160.dyn.user.ono.com. [79.108.125.160]) by smtp.gmail.com with ESMTPSA id y4sm1916733wmj.2.2020.11.19.14.11.17 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 19 Nov 2020 14:11:18 -0800 (PST) To: sframe@ietf.org References: From: Sergio Garcia Murillo Message-ID: <7eadfc25-d326-3a02-f88f-bfdc02b9a42c@cosmosoftware.io> Date: Thu, 19 Nov 2020 23:11:16 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.4.3 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/alternative; boundary="------------6C2FD14850A9AA83381C2325" Content-Language: en-US Archived-At: Subject: Re: [Sframe] Partial decodability and IDUs X-BeenThere: sframe@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2020 22:11:23 -0000 This is a multi-part message in MIME format. --------------6C2FD14850A9AA83381C2325 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit You are right regarding that the SFrame layer does not need to know what is feed in for encryption, but in order to be able to have a working end to end solution for webrtc, someone will need to define what and how this IDUs are generated and reassembled for each codec if we want to have interoperable implementations in different devices. That process is codec-dependant and I would require quite a lot of effort (and also supporting it on the agnostic packetization), so I would prefer to have strong arguments in favor of doing it. On 19/11/2020 22:53, Justin Uberti wrote: > The encoder needs to be aware of any mechanism to generate IDUs (e.g., > slices), and typically each of these IDUs will be handed up to the > consumer individually. So the SFRAME layer doesn't need to do any > splitting, it just knows that it should treat each IDU as something it > needs to individually SFRAME and packetize. > > On Thu, Nov 19, 2020 at 1:40 PM Sergio Garcia Murillo > > wrote: > > Hi all, > > > As most of you already know, this morning I made a presentation in > AVTCORE introducing the topic about the need to specify an > agnostic video codec packetization format. > > https://datatracker.ietf.org/meeting/109/materials/slides-109-avtcore-sframe-rtp-encapsulation-00 > > > > I got an AP for creating an initial draft so it could be reviewed > and accepted. > > However, there were two main concerns that we should address in > this this group: > > * Historically, avtcore has explicitly designed not to be > payload agnostic and  declined to standardized codec agnostic > payload formats in  number of cases.  If that is to be > changed, needs to be done deliberately. > * Need to define the "minimum decoding unit" or "independently > decodable unit", that SFrame will work with. > > > Regarding the second one > > * Full video frames (just use whatever is the encoder output) > * Spatial layer frames > * "independend decodable subframes" like h264 slices, vp8 > partitions or av1 tiles which allows partial decodability > which is mainly aimed for enhancing packet loss resilience. > > > Spatial layer frames is the minimum we should target as if not it > will just prevent SFUs for using SVC codecs. So the question is if > we should go deeper and implement lower partitions of the frames > or not. > > > AFAIK, currently, libwertc does not support partial decodability > and I personally haven't seen any practical usage of this in the > RTC world (while it makes a lot of sense in streaming/broadcasting > world), but would like to hear what is the view and experience of > the other members of this group. Also note that if we are going to > support them on SFrame this will require a greater effort because > we will need to explicitly define how the frames must be split > before being encrypted y SFrame for *each* possible video codec > (h264,h265,vp8,vp9,av1,...). > > > There was also the question about how/if we should support other > codec features like DON/interleaved mode for h264, which I also > think we should not support mainly because we are not currently > using it on webrtc implementations. > > > What do you think? > > > Best regards > > Sergio > > > -- > Sframe mailing list > Sframe@ietf.org > https://www.ietf.org/mailman/listinfo/sframe > > > --------------6C2FD14850A9AA83381C2325 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit

You are right regarding that the SFrame layer does not need to know what is feed in for encryption, but in order to be able to have a working end to end solution for webrtc, someone will need to define what and how this IDUs are generated and reassembled for each codec if we want to have interoperable implementations in different devices.

That process is codec-dependant and I would require quite a lot of effort (and also supporting it on the agnostic packetization), so I would prefer to have strong arguments in favor of doing it.



On 19/11/2020 22:53, Justin Uberti wrote:
The encoder needs to be aware of any mechanism to generate IDUs (e.g., slices), and typically each of these IDUs will be handed up to the consumer individually. So the SFRAME layer doesn't need to do any splitting, it just knows that it should treat each IDU as something it needs to individually SFRAME and packetize.

On Thu, Nov 19, 2020 at 1:40 PM Sergio Garcia Murillo <sergio.garcia.murillo@gmail.com> wrote:

Hi all,


As most of you already know, this morning I made a presentation in AVTCORE introducing the topic about the need to specify an agnostic video codec packetization format.

https://datatracker.ietf.org/meeting/109/materials/slides-109-avtcore-sframe-rtp-encapsulation-00


I got an AP for creating an initial draft so it could be reviewed and accepted.

However, there were two main concerns that we should address in this this group:

  • Historically, avtcore has explicitly designed not to be payload agnostic and  declined to standardized codec agnostic payload formats in  number of cases.  If that is to be changed, needs to be done deliberately.
  • Need to define the "minimum decoding unit" or "independently decodable unit", that SFrame will work with.


Regarding the second one

  • Full video frames (just use whatever is the encoder output)
  • Spatial layer frames
  • "independend decodable subframes" like h264 slices, vp8 partitions or av1 tiles which allows partial decodability which is mainly aimed for enhancing packet loss resilience.


Spatial layer frames is the minimum we should target as if not it will just prevent SFUs for using SVC codecs. So the question is if we should go deeper and implement lower partitions of the frames or not.


AFAIK, currently, libwertc does not support partial decodability and I personally haven't seen any practical usage of this in the RTC world (while it makes a lot of sense in streaming/broadcasting world), but would like to hear what is the view and experience of the other members of this group. Also note that if we are going to support them on SFrame this will require a greater effort because we will need to explicitly define how the frames must be split before being encrypted y SFrame for *each* possible video codec (h264,h265,vp8,vp9,av1,...).


There was also the question about how/if we should support other codec features like DON/interleaved mode for h264, which I also think we should not support mainly because we are not currently using it on webrtc implementations.


What do you think?


Best regards

Sergio


--
Sframe mailing list
Sframe@ietf.org
https://www.ietf.org/mailman/listinfo/sframe

--------------6C2FD14850A9AA83381C2325-- From nobody Thu Nov 19 14:55:04 2020 Return-Path: X-Original-To: sframe@ietfa.amsl.com Delivered-To: sframe@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 925153A133E for ; Thu, 19 Nov 2020 14:55:02 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.896 X-Spam-Level: X-Spam-Status: No, score=-1.896 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=ipv-sx.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fBGN1i7s0dXW for ; Thu, 19 Nov 2020 14:55:00 -0800 (PST) Received: from mail-qk1-x72f.google.com (mail-qk1-x72f.google.com [IPv6:2607:f8b0:4864:20::72f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 40DC23A133D for ; Thu, 19 Nov 2020 14:54:59 -0800 (PST) Received: by mail-qk1-x72f.google.com with SMTP id a13so7229790qkl.4 for ; Thu, 19 Nov 2020 14:54:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ipv-sx.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=uHJLjOycj5tJMROqfDIntvDsuGTYRVwC7AbU7PwE72o=; b=oIix/dNDVoYY2mryth7HCczZO+PhQl54lI/Ws9pcLkXzuAiA1dnSoA6/wkQZL5dpoz i+YocEabxo+PgNQ7ffMnUc3ptAlurn6rfolDNDd38XWU7mow88tLFex9V6WOf/XV7ZL4 te210DYKWlJlgCJfUx/6hJu9WWsvTL+PBVJWMl+kfNK6+xxkbtvAGcoKjK29s+HwjNPz z6YHIneaK99RPfj+SMmU09UOOR9eOLbHuc3TQHVOW1sq+U+Uwexo5tH/Cva1MShefccP KvCs+qUZuH3gjOXph+GTxef0hoHfA/CiGb8zux7l9FZdOLaA/6f6o/l6w/ykDagwp7mz 6BiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=uHJLjOycj5tJMROqfDIntvDsuGTYRVwC7AbU7PwE72o=; b=nr3s/4rAof20aJzYrC4turXhE5YDww8PqplTBuVq9MiwktfrxbpCCRYSfQvY+Wc3wl klr0KxoEhSo8zZUUEJz1JY6wqucERilWLvv1CIMaI86cS+3URfZTaUf5l/4cbg5PdDmr R8t7nUA4bwiEQbelorNImnnwnhCe0dfjNQei2lkjyfNCq9qHmBIFvUQ6oubJ75lgLwZI kTm7pYSvDU4nLmEw3grwnjZyWKXOmOrN0+7A6xgyXxhSwL6yLM76lLfxxjGBE0MEKnYX dohBBQ9BUops6vk944myukO30ZDCcqd3Kf1hRDmydBftxJFo0pzxi8WECO/oE2Pl+fhO gywA== X-Gm-Message-State: AOAM532J9Iy+lhTWtWpAbP/U5lbhhQjcCWjC9mKRXcXCXRoUNGMlg2BK 5kto57nL1Jg89pCb4aY3wTZ/ajJqx747TBnuPeezcRicDy0= X-Google-Smtp-Source: ABdhPJyO3uOXZ/Hy/+xqln27uhVoKso2jheNFYUk9pdncnzni/Y141Td3ti1diY32drqh1rMGypWowZNNxgaVSlWSUM= X-Received: by 2002:a37:6143:: with SMTP id v64mr13502836qkb.490.1605826498777; Thu, 19 Nov 2020 14:54:58 -0800 (PST) MIME-Version: 1.0 References: <7eadfc25-d326-3a02-f88f-bfdc02b9a42c@cosmosoftware.io> In-Reply-To: <7eadfc25-d326-3a02-f88f-bfdc02b9a42c@cosmosoftware.io> From: Richard Barnes Date: Thu, 19 Nov 2020 17:54:46 -0500 Message-ID: To: Sergio Garcia Murillo Cc: sframe@ietf.org Content-Type: multipart/alternative; boundary="000000000000087b3405b47d9ed2" Archived-At: Subject: Re: [Sframe] Partial decodability and IDUs X-BeenThere: sframe@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2020 22:55:03 -0000 --000000000000087b3405b47d9ed2 Content-Type: text/plain; charset="UTF-8" This may be too simplistic, but there's also a non-codec-specific approach here that occurs to me: Just have the SFrame header have a length field. I'm imagining a setup where we have the SFrame layer between the encode and packetization layers, and: - The encoding/decoding layer produces consumes each frame as a sequence of IDUs - The SFrame layer translates between IDUs and SFrame encrypted enits (SEUs? in any case, each SEU is an encrypted IDU) - The packetization / depacketization layer packs SEUs into packets The only thing you need to make that work is (1) a mechanism for the receiver to understand what chunks of the SEU sequence he has (e.g., fixing reordering), and (2) a way to unpack SEUs if there can be multiple in a packet. It seems like (1) could mostly be a transport assumption. For (2), you would just need something like a length field. As Sergio points out, there is a need for someone to know where the IDU boundaries are, either at the SFrame layer (if the input is a whole frame), or at the encode layer (if the encode-SFrame API can talk in terms of IDUs). But especially in the latter case, this framework keeps the codec-specific stuff local to the encode layer, which is codec-specific in any case. There is some trade-off here, in that this framework doesn't expose any encoded information to the SFU. But ISTM that if you want that functionality, then (a) you're going to have to have codec awareness sprinkled all through the stack and (b) you're going to have to be really careful designing the which-parts-to-encrypt scheme to avoid undermining your security guarantees. So I'm generally inclined toward the cleaner abstraction here. --Richard On Thu, Nov 19, 2020 at 5:11 PM Sergio Garcia Murillo < sergio.garcia.murillo@cosmosoftware.io> wrote: > You are right regarding that the SFrame layer does not need to know what > is feed in for encryption, but in order to be able to have a working end to > end solution for webrtc, someone will need to define what and how this IDUs > are generated and reassembled for each codec if we want to have > interoperable implementations in different devices. > > That process is codec-dependant and I would require quite a lot of effort > (and also supporting it on the agnostic packetization), so I would prefer > to have strong arguments in favor of doing it. > > > > On 19/11/2020 22:53, Justin Uberti wrote: > > The encoder needs to be aware of any mechanism to generate IDUs (e.g., > slices), and typically each of these IDUs will be handed up to the consumer > individually. So the SFRAME layer doesn't need to do any splitting, it just > knows that it should treat each IDU as something it needs to individually > SFRAME and packetize. > > On Thu, Nov 19, 2020 at 1:40 PM Sergio Garcia Murillo < > sergio.garcia.murillo@gmail.com> wrote: > >> Hi all, >> >> >> As most of you already know, this morning I made a presentation in >> AVTCORE introducing the topic about the need to specify an agnostic video >> codec packetization format. >> >> >> https://datatracker.ietf.org/meeting/109/materials/slides-109-avtcore-sframe-rtp-encapsulation-00 >> >> >> I got an AP for creating an initial draft so it could be reviewed and >> accepted. >> >> However, there were two main concerns that we should address in this this >> group: >> >> - Historically, avtcore has explicitly designed not to be payload >> agnostic and declined to standardized codec agnostic payload formats in >> number of cases. If that is to be changed, needs to be done deliberately. >> - Need to define the "minimum decoding unit" or "independently >> decodable unit", that SFrame will work with. >> >> >> Regarding the second one >> >> - Full video frames (just use whatever is the encoder output) >> - Spatial layer frames >> - "independend decodable subframes" like h264 slices, vp8 partitions >> or av1 tiles which allows partial decodability which is mainly aimed for >> enhancing packet loss resilience. >> >> >> Spatial layer frames is the minimum we should target as if not it will >> just prevent SFUs for using SVC codecs. So the question is if we should go >> deeper and implement lower partitions of the frames or not. >> >> >> AFAIK, currently, libwertc does not support partial decodability and I >> personally haven't seen any practical usage of this in the RTC world (while >> it makes a lot of sense in streaming/broadcasting world), but would like to >> hear what is the view and experience of the other members of this group. >> Also note that if we are going to support them on SFrame this will require >> a greater effort because we will need to explicitly define how the frames >> must be split before being encrypted y SFrame for *each* possible video >> codec (h264,h265,vp8,vp9,av1,...). >> >> >> There was also the question about how/if we should support other codec >> features like DON/interleaved mode for h264, which I also think we should >> not support mainly because we are not currently using it on webrtc >> implementations. >> >> >> What do you think? >> >> >> Best regards >> >> Sergio >> >> >> -- >> Sframe mailing list >> Sframe@ietf.org >> https://www.ietf.org/mailman/listinfo/sframe >> > > -- > Sframe mailing list > Sframe@ietf.org > https://www.ietf.org/mailman/listinfo/sframe > --000000000000087b3405b47d9ed2 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
This may be too simplistic, but there's also a no= n-codec-specific approach here that occurs to me: Just have the SFrame head= er have a length field.=C2=A0

I'm imagini= ng a setup where we have the SFrame layer between the encode and packetizat= ion layers, and:
- The encoding/decoding layer produces consumes = each frame as a sequence of IDUs
- The SFrame layer translates be= tween IDUs and SFrame encrypted enits (SEUs? in any case, each SEU is an en= crypted IDU)
- The packetization / depacketization layer packs SE= Us into packets

The only thing you need to mak= e that work is (1) a mechanism for the receiver to understand what chunks o= f the SEU sequence he has (e.g., fixing reordering), and (2) a way to unpac= k SEUs if there can be multiple in a packet.=C2=A0 It seems like (1) could = mostly be a transport assumption.=C2=A0 For (2), you would just need someth= ing like a length field.

As Sergio points out, the= re is a need for someone to know where the IDU boundaries are, either at th= e SFrame layer (if the input is a whole frame), or at the encode layer (if = the encode-SFrame API can talk in terms of IDUs).=C2=A0 But especially in t= he latter case, this framework keeps the codec-specific stuff local to the = encode layer, which is codec-specific in any case.

There is some trade-off here, in that this framework doesn't expose an= y encoded information to the SFU.=C2=A0 But ISTM that if you want that func= tionality, then (a) you're going to have to have codec awareness sprink= led all through the stack and (b) you're going to have to be really car= eful designing the which-parts-to-encrypt scheme to avoid undermining your = security guarantees.=C2=A0 So I'm generally inclined toward the cleaner= abstraction here.

--Richard

On Thu, Nov 19, 20= 20 at 5:11 PM Sergio Garcia Murillo <sergio.garcia.murillo@cosmosoftware.io> wrote= :
=20 =20 =20

You are right regarding that the SFrame layer does not need to know what is feed in for encryption, but in order to be able to have a working end to end solution for webrtc, someone will need to define what and how this IDUs are generated and reassembled for each codec if we want to have interoperable implementations in different devices.

That process is codec-dependant and I would require quite a lot of effort (and also supporting it on the agnostic packetization), so I would prefer to have strong arguments in favor of doing it.



On 19/11/2020 22:53, Justin Uberti wrote:
=20
The encoder needs to be aware of any mechanism to generate IDUs (e.g., slices), and typically each of these IDUs will be handed up to the consumer individually. So the SFRAME layer doesn't need to do any splitting, it just knows that it should treat each IDU as something it needs to individually SFRAME and packetize.

On Thu, Nov 19, 2020 at 1:40 PM Sergio Garcia Murillo <sergio.garcia.murillo@gmail.com> wrote:

Hi all,


As most of you already know, this morning I made a presentation in AVTCORE introducing the topic about the need to specify an agnostic video codec packetization format.

https:/= /datatracker.ietf.org/meeting/109/materials/slides-109-avtcore-sframe-rtp-e= ncapsulation-00


I got an AP for creating an initial draft so it could be reviewed and accepted.

However, there were two main concerns that we should address in this this group:

  • Historically, avtcore has explicitly designed not to be payload agnostic and=C2=A0 declined to standardized code= c agnostic payload formats in=C2=A0 number of cases.=C2=A0 If= that is to be changed, needs to be done deliberately.
  • Need to define the "minimum decoding unit" or "independently decodable unit", that SFrame will = work with.


Regarding the second one

  • Full video frames (just use whatever is the encoder output)
  • Spatial layer frames
  • "independend decodable subframes" like h264 sli= ces, vp8 partitions or av1 tiles which allows partial decodability which is mainly aimed for enhancing packet loss resilience.


Spatial layer frames is the minimum we should target as if not it will just prevent SFUs for using SVC codecs. So the question is if we should go deeper and implement lower partitions of the frames or not.


AFAIK, currently, libwertc does not support partial decodability and I personally haven't seen any practical usage of this in the RTC world (while it makes a lot of sense in streaming/broadcasting world), but would like to hear what is the view and experience of the other members of this group. Also note that if we are going to support them on SFrame this will require a greater effort because we will need to explicitly define how the frames must be split before being encrypted y SFrame for *each* possible video codec (h264,h265,vp8,vp9,av1,...).


There was also the question about how/if we should support other codec features like DON/interleaved mode for h264, which I also think we should not support mainly because we are not currently using it on webrtc implementations.


What do you think?


Best regards

Sergio


--
Sframe mailing list
Sframe@ietf.= org
https://www.ietf.org/mailman/listinfo/sframe

--
Sframe mailing list
Sframe@ietf.org https://www.ietf.org/mailman/listinfo/sframe
--000000000000087b3405b47d9ed2-- From nobody Thu Nov 19 15:13:37 2020 Return-Path: X-Original-To: sframe@ietfa.amsl.com Delivered-To: sframe@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 404533A1376 for ; Thu, 19 Nov 2020 15:13:36 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.097 X-Spam-Level: X-Spam-Status: No, score=-2.097 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SkO0p4MBjzly for ; Thu, 19 Nov 2020 15:13:34 -0800 (PST) Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com [IPv6:2a00:1450:4864:20::12f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 27F393A1375 for ; Thu, 19 Nov 2020 15:13:34 -0800 (PST) Received: by mail-lf1-x12f.google.com with SMTP id 74so10755469lfo.5 for ; Thu, 19 Nov 2020 15:13:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=lEQXersisikeWf7a8vfHCsWPtd8K/ADuREFV9scYyW4=; b=UIxkK4n+jwk6rI7dZk/oxvg07Q3cQme777uNOQIbzUDghbX3sy1gerST4QzoIW1IEf j1zskaCTv1HgwPmYzGs2eGIKs9Y0djk/ffVPZPy6/pCUnAl0H0bez+NR/D7zi03ZeLV9 Vt2dBq5t9SQ/bxF6VI8+XpiXWZM3JsGkFoTzsgDjTfQUUJooRnOv4iUkc8FLavvDdi5t fHajsaYed/O/x6+HIIj33Z54OlKVTqfrB1XaBItkL0HLIZXjMRZnRHReBTdi/wKMxGgz p9stts/AS6LKK0B7DTDbtPRxLemPe6lTlufKECShTtKPvw/SvfROoBk6DHAM7GwiF/wQ SKrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=lEQXersisikeWf7a8vfHCsWPtd8K/ADuREFV9scYyW4=; b=ENh3OUnlcQczE7B7/wtpGpmyPqIzzhgS+306Njz9uC2sn1B4IGSIGAxe8lyN0TqJDp UZNEkR4eOVz9gFUX51f/XLVoSJUBP1pMo78cJ/egffoL3TcADJGD8EY8FdVBaro/6JYK qBG85fv4vp1LLu8PbBpo2DIM6XXp6xZvSwPn1zhFfO3zyKw/tLp6vIZFp7PpKgVBC1tY 0v1rrzgcKqHX1A3pOrG24PpeNxL+GjgJgi3wBgW5ulCbLNNkonETN50ACgltQsIjw+vx Vd65ZZx0DuvF570TiFiFXKIzG1E/Ms15YIQ1nis7qizHvCqyip0fWZ/+TlkIckYDEJzl +IJA== X-Gm-Message-State: AOAM533DmOq8wzijHzjNNdif5FLKWzTs0BhRqKSpLyUx1cTp66E31R9v bmcF61patrKiGAFCWBGMpJuwFfX4jFhqZIuIuBb0g859U9oQBg== X-Google-Smtp-Source: ABdhPJwa+ugSGT4MsNze1c3XkehyP5ZtVTD5ilE+jvXZlUwNqf4mPKyvjhcAZ26IQkSBO6AhPGP5CuA4Y6aCAkB7Jfg= X-Received: by 2002:a19:88c:: with SMTP id 134mr6309928lfi.560.1605827612103; Thu, 19 Nov 2020 15:13:32 -0800 (PST) MIME-Version: 1.0 References: <7eadfc25-d326-3a02-f88f-bfdc02b9a42c@cosmosoftware.io> In-Reply-To: <7eadfc25-d326-3a02-f88f-bfdc02b9a42c@cosmosoftware.io> From: Bernard Aboba Date: Thu, 19 Nov 2020 15:13:22 -0800 Message-ID: To: Sergio Garcia Murillo Cc: sframe@ietf.org Content-Type: multipart/alternative; boundary="00000000000063d9e305b47de0b1" Archived-At: Subject: Re: [Sframe] Partial decodability and IDUs X-BeenThere: sframe@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2020 23:13:36 -0000 --00000000000063d9e305b47de0b1 Content-Type: text/plain; charset="UTF-8" Sergio said: "You are right regarding that the SFrame layer does not need to know what is feed in for encryption, but in order to be able to have a working end to end solution for webrtc, someone will need to define what and how this IDUs are generated and reassembled for each codec if we want to have interoperable implementations in different devices." [BA] The job of an SFrame sender is to encrypt and packetize the bitstream provided by the encoder. For SFU to act on the packetization in an optimal way, some rules have to be followed (such as not packetizing frames from different layers in the same packet). So the sender/packetizer will have codec-specific logic, so it can parse the bitstream for meta-data, and figure out how to do the packetization in a manner appropriate for that codec. This could include separately packetizing IDUs (slices/tiles), but I'm not clear this needs to be required for all use cases. The SFM does not peer into the payload, it only acts on the meta-data included in RTP header extensions, so the recovery/forwarding/dropping decision can be largely codec-independent. The receiver decrypts and de-packetizes the bitstream, then provides it to the decoder. The recovery/decryption/de-packetizion process should also be codec-independent. On Thu, Nov 19, 2020 at 2:11 PM Sergio Garcia Murillo < sergio.garcia.murillo@cosmosoftware.io> wrote: > You are right regarding that the SFrame layer does not need to know what > is feed in for encryption, but in order to be able to have a working end to > end solution for webrtc, someone will need to define what and how this IDUs > are generated and reassembled for each codec if we want to have > interoperable implementations in different devices. > > That process is codec-dependant and I would require quite a lot of effort > (and also supporting it on the agnostic packetization), so I would prefer > to have strong arguments in favor of doing it. > > > > On 19/11/2020 22:53, Justin Uberti wrote: > > The encoder needs to be aware of any mechanism to generate IDUs (e.g., > slices), and typically each of these IDUs will be handed up to the consumer > individually. So the SFRAME layer doesn't need to do any splitting, it just > knows that it should treat each IDU as something it needs to individually > SFRAME and packetize. > > On Thu, Nov 19, 2020 at 1:40 PM Sergio Garcia Murillo < > sergio.garcia.murillo@gmail.com> wrote: > >> Hi all, >> >> >> As most of you already know, this morning I made a presentation in >> AVTCORE introducing the topic about the need to specify an agnostic video >> codec packetization format. >> >> >> https://datatracker.ietf.org/meeting/109/materials/slides-109-avtcore-sframe-rtp-encapsulation-00 >> >> >> I got an AP for creating an initial draft so it could be reviewed and >> accepted. >> >> However, there were two main concerns that we should address in this this >> group: >> >> - Historically, avtcore has explicitly designed not to be payload >> agnostic and declined to standardized codec agnostic payload formats in >> number of cases. If that is to be changed, needs to be done deliberately. >> - Need to define the "minimum decoding unit" or "independently >> decodable unit", that SFrame will work with. >> >> >> Regarding the second one >> >> - Full video frames (just use whatever is the encoder output) >> - Spatial layer frames >> - "independend decodable subframes" like h264 slices, vp8 partitions >> or av1 tiles which allows partial decodability which is mainly aimed for >> enhancing packet loss resilience. >> >> >> Spatial layer frames is the minimum we should target as if not it will >> just prevent SFUs for using SVC codecs. So the question is if we should go >> deeper and implement lower partitions of the frames or not. >> >> >> AFAIK, currently, libwertc does not support partial decodability and I >> personally haven't seen any practical usage of this in the RTC world (while >> it makes a lot of sense in streaming/broadcasting world), but would like to >> hear what is the view and experience of the other members of this group. >> Also note that if we are going to support them on SFrame this will require >> a greater effort because we will need to explicitly define how the frames >> must be split before being encrypted y SFrame for *each* possible video >> codec (h264,h265,vp8,vp9,av1,...). >> >> >> There was also the question about how/if we should support other codec >> features like DON/interleaved mode for h264, which I also think we should >> not support mainly because we are not currently using it on webrtc >> implementations. >> >> >> What do you think? >> >> >> Best regards >> >> Sergio >> >> >> -- >> Sframe mailing list >> Sframe@ietf.org >> https://www.ietf.org/mailman/listinfo/sframe >> > > -- > Sframe mailing list > Sframe@ietf.org > https://www.ietf.org/mailman/listinfo/sframe > --00000000000063d9e305b47de0b1 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Sergio said:=C2=A0

&qu= ot;You are right regarding that the SFrame layer does not need to know what= is feed in for encryption, but in order to be able to have a working end t= o end solution for webrtc, someone will need to define what and how this ID= Us are generated and reassembled for each codec if we want to have interope= rable implementations in different devices."

= [BA] The job of an SFrame sender is to encrypt and packetize the bitstream = provided by the encoder.=C2=A0 For SFU to act on the packetization in an op= timal way, some rules have to be followed (such as not packetizing frames f= rom different layers in the same packet). So the sender/packetizer will hav= e codec-specific logic,=C2=A0so it can parse the bitstream for meta-data, a= nd figure out how to do the packetization in a manner appropriate for that = codec.=C2=A0 This could include separately packetizing IDUs (slices/tiles),= but I'm not clear this needs to be required for all use cases.=C2=A0

The SFM does not peer into the payload, it only act= s on the meta-data included in RTP header extensions, so the recovery/forwa= rding/dropping decision can be largely codec-independent.=C2=A0
<= br>
The receiver decrypts and de-packetizes the bitstream, then p= rovides it to the decoder.=C2=A0 The recovery/decryption/de-packetizion pro= cess should also be codec-independent.

On Thu, Nov 19, 2020 at 2= :11 PM Sergio Garcia Murillo <sergio.garcia.murillo@cosmosoftware.io> wrote:
=20 =20 =20

You are right regarding that the SFrame layer does not need to know what is feed in for encryption, but in order to be able to have a working end to end solution for webrtc, someone will need to define what and how this IDUs are generated and reassembled for each codec if we want to have interoperable implementations in different devices.

That process is codec-dependant and I would require quite a lot of effort (and also supporting it on the agnostic packetization), so I would prefer to have strong arguments in favor of doing it.



On 19/11/2020 22:53, Justin Uberti wrote:
=20
The encoder needs to be aware of any mechanism to generate IDUs (e.g., slices), and typically each of these IDUs will be handed up to the consumer individually. So the SFRAME layer doesn't need to do any splitting, it just knows that it should treat each IDU as something it needs to individually SFRAME and packetize.

On Thu, Nov 19, 2020 at 1:40 PM Sergio Garcia Murillo <sergio.garcia.murillo@gmail.com> wrote:

Hi all,


As most of you already know, this morning I made a presentation in AVTCORE introducing the topic about the need to specify an agnostic video codec packetization format.

https:/= /datatracker.ietf.org/meeting/109/materials/slides-109-avtcore-sframe-rtp-e= ncapsulation-00


I got an AP for creating an initial draft so it could be reviewed and accepted.

However, there were two main concerns that we should address in this this group:

  • Historically, avtcore has explicitly designed not to be payload agnostic and=C2=A0 declined to standardized code= c agnostic payload formats in=C2=A0 number of cases.=C2=A0 If= that is to be changed, needs to be done deliberately.
  • Need to define the "minimum decoding unit" or "independently decodable unit", that SFrame will = work with.


Regarding the second one

  • Full video frames (just use whatever is the encoder output)
  • Spatial layer frames
  • "independend decodable subframes" like h264 sli= ces, vp8 partitions or av1 tiles which allows partial decodability which is mainly aimed for enhancing packet loss resilience.


Spatial layer frames is the minimum we should target as if not it will just prevent SFUs for using SVC codecs. So the question is if we should go deeper and implement lower partitions of the frames or not.


AFAIK, currently, libwertc does not support partial decodability and I personally haven't seen any practical usage of this in the RTC world (while it makes a lot of sense in streaming/broadcasting world), but would like to hear what is the view and experience of the other members of this group. Also note that if we are going to support them on SFrame this will require a greater effort because we will need to explicitly define how the frames must be split before being encrypted y SFrame for *each* possible video codec (h264,h265,vp8,vp9,av1,...).


There was also the question about how/if we should support other codec features like DON/interleaved mode for h264, which I also think we should not support mainly because we are not currently using it on webrtc implementations.


What do you think?


Best regards

Sergio


--
Sframe mailing list
Sframe@ietf.= org
https://www.ietf.org/mailman/listinfo/sframe

--
Sframe mailing list
Sframe@ietf.org https://www.ietf.org/mailman/listinfo/sframe
--00000000000063d9e305b47de0b1-- From nobody Thu Nov 19 15:15:36 2020 Return-Path: X-Original-To: sframe@ietfa.amsl.com Delivered-To: sframe@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8E9223A0DEE for ; Thu, 19 Nov 2020 15:15:34 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.897 X-Spam-Level: X-Spam-Status: No, score=-1.897 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, NICE_REPLY_A=-0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=cosmosoftware-io.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7fkp7LU3wmSg for ; Thu, 19 Nov 2020 15:15:32 -0800 (PST) Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0B9BB3A0DEB for ; Thu, 19 Nov 2020 15:15:31 -0800 (PST) Received: by mail-wr1-x434.google.com with SMTP id s8so8131630wrw.10 for ; Thu, 19 Nov 2020 15:15:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cosmosoftware-io.20150623.gappssmtp.com; s=20150623; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=KGVYiuoEmNn5Imd6lMpehmCoCKDBARJgPsH95LgFkTs=; b=nXjyRx9KgMr2LNXqulTHaPau5ev/uSUhE/7EL/zqQJmLg8Z/ytERjkVAvkDpygOcPJ u/vDDrL2LYiNlEIECAjUb/NsSBAlyMwwwLT1l+uk1ejf8/LThudDR0llc9Ut7RHy06hC GJ43JmkWb+X1rHmBtKlMIOs1BHNj4VCaHwYUhkV7fkkLCr6HUEFicgkNtZARhnPaoYzs zaFg8XYLDVqp6oG8WC+aPZhWerWL1mL+3ThLosYjPUaWwZ01dleKQI/ftkJTd9to/2Yw oM3oafY+xbIWqoPySHL7J8vrD0OAUypOOoagOV0Xz8ui7KoJ4ttq1EN8sZP0Bv557hsO i+QQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language; bh=KGVYiuoEmNn5Imd6lMpehmCoCKDBARJgPsH95LgFkTs=; b=o7Yhnb+nr7H4ww3L1E5D/bjPav7HouPsdIuj5xaTI4cOXVhhlMmEqnFU7WsHtU+tzD 0nS/gFFltSoq/252Qdnkz23ZOZ4CB0k+74PsgdhfOHxJAGV9e75vDnOE8n3E+urBZg9b K58tM6CUqAjv5YGKXoGaZ8Dmil/PO5C6mXxacC9MJ8+UGt5EbZJ5fXclpPSbMkcVFYJE Gck0FVR4FFHPUBSYobx44QPrZbq6fOXhoOSDVIR1gRZVyLKd3M8uvUeMDLgpjeLfO+AK C4SCWYpq8pjzB8YHjmqF5kEjcI1YHjox7Z7HjtBDM/COWA5FAOc6kZdQG/wd2o8uDmH7 a+QQ== X-Gm-Message-State: AOAM530e0eUBekS0qDWGHmXPNRxtpcunkvkPDEcdFjygtMSQtB2FHiv8 mfaK7X5dfKKerZghR+JUQ/85FvuvqXqRJA== X-Google-Smtp-Source: ABdhPJxgYw05HRYu7ocKZUwyRGDnNTbPdnMYG3qjEkGWHg9tg+HDDi74PLfG9387dvjY8cgtvkJD5w== X-Received: by 2002:a5d:6046:: with SMTP id j6mr12783089wrt.317.1605827730016; Thu, 19 Nov 2020 15:15:30 -0800 (PST) Received: from [192.168.0.11] (79.108.125.160.dyn.user.ono.com. [79.108.125.160]) by smtp.gmail.com with ESMTPSA id n23sm2022986wmk.24.2020.11.19.15.15.29 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 19 Nov 2020 15:15:29 -0800 (PST) To: Richard Barnes Cc: sframe@ietf.org References: <7eadfc25-d326-3a02-f88f-bfdc02b9a42c@cosmosoftware.io> From: Sergio Garcia Murillo Message-ID: <2752b6f0-ec61-6a19-e17c-0347c451a17c@cosmosoftware.io> Date: Fri, 20 Nov 2020 00:15:28 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.4.3 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/alternative; boundary="------------921F768AA26BAFA08AB9EF0B" Content-Language: en-US Archived-At: Subject: Re: [Sframe] Partial decodability and IDUs X-BeenThere: sframe@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2020 23:15:34 -0000 This is a multi-part message in MIME format. --------------921F768AA26BAFA08AB9EF0B Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit How to actually packetize the IDUs is not a big deal, you only need to have a start and end of IDU bits on the header and rely on increasing rtp cseq nums (i.e. no IDU interleaving), but at the end of the day someone will have to specify what an IDU in h264 means and what is its format (a group of nal units?). I am afraid that we end up another doing a per-codec packetizer, which would be almost the same as just using current packetization formats and explain how to use encrypted data with them. And all for a feature that we are not current using in webrtc. Best regards Sergio On 19/11/2020 23:54, Richard Barnes wrote: > This may be too simplistic, but there's also a non-codec-specific > approach here that occurs to me: Just have the SFrame header have a > length field. > > I'm imagining a setup where we have the SFrame layer between the > encode and packetization layers, and: > - The encoding/decoding layer produces consumes each frame as a > sequence of IDUs > - The SFrame layer translates between IDUs and SFrame encrypted enits > (SEUs? in any case, each SEU is an encrypted IDU) > - The packetization / depacketization layer packs SEUs into packets > > The only thing you need to make that work is (1) a mechanism for the > receiver to understand what chunks of the SEU sequence he has (e.g., > fixing reordering), and (2) a way to unpack SEUs if there can be > multiple in a packet.  It seems like (1) could mostly be a transport > assumption.  For (2), you would just need something like a length field. > > As Sergio points out, there is a need for someone to know where the > IDU boundaries are, either at the SFrame layer (if the input is a > whole frame), or at the encode layer (if the encode-SFrame API can > talk in terms of IDUs).  But especially in the latter case, this > framework keeps the codec-specific stuff local to the encode layer, > which is codec-specific in any case. > > There is some trade-off here, in that this framework doesn't expose > any encoded information to the SFU.  But ISTM that if you want that > functionality, then (a) you're going to have to have codec awareness > sprinkled all through the stack and (b) you're going to have to be > really careful designing the which-parts-to-encrypt scheme to avoid > undermining your security guarantees.  So I'm generally inclined > toward the cleaner abstraction here. > > --Richard > > On Thu, Nov 19, 2020 at 5:11 PM Sergio Garcia Murillo > > wrote: > > You are right regarding that the SFrame layer does not need to > know what is feed in for encryption, but in order to be able to > have a working end to end solution for webrtc, someone will need > to define what and how this IDUs are generated and reassembled for > each codec if we want to have interoperable implementations in > different devices. > > That process is codec-dependant and I would require quite a lot of > effort (and also supporting it on the agnostic packetization), so > I would prefer to have strong arguments in favor of doing it. > > > > On 19/11/2020 22:53, Justin Uberti wrote: >> The encoder needs to be aware of any mechanism to generate IDUs >> (e.g., slices), and typically each of these IDUs will be handed >> up to the consumer individually. So the SFRAME layer doesn't need >> to do any splitting, it just knows that it should treat each IDU >> as something it needs to individually SFRAME and packetize. >> >> On Thu, Nov 19, 2020 at 1:40 PM Sergio Garcia Murillo >> > > wrote: >> >> Hi all, >> >> >> As most of you already know, this morning I made a >> presentation in AVTCORE introducing the topic about the need >> to specify an agnostic video codec packetization format. >> >> https://datatracker.ietf.org/meeting/109/materials/slides-109-avtcore-sframe-rtp-encapsulation-00 >> >> >> >> I got an AP for creating an initial draft so it could be >> reviewed and accepted. >> >> However, there were two main concerns that we should address >> in this this group: >> >> * Historically, avtcore has explicitly designed not to be >> payload agnostic and declined to standardized codec >> agnostic payload formats in  number of cases.  If that is >> to be changed, needs to be done deliberately. >> * Need to define the "minimum decoding unit" or >> "independently decodable unit", that SFrame will work with. >> >> >> Regarding the second one >> >> * Full video frames (just use whatever is the encoder output) >> * Spatial layer frames >> * "independend decodable subframes" like h264 slices, vp8 >> partitions or av1 tiles which allows partial decodability >> which is mainly aimed for enhancing packet loss resilience. >> >> >> Spatial layer frames is the minimum we should target as if >> not it will just prevent SFUs for using SVC codecs. So the >> question is if we should go deeper and implement lower >> partitions of the frames or not. >> >> >> AFAIK, currently, libwertc does not support partial >> decodability and I personally haven't seen any practical >> usage of this in the RTC world (while it makes a lot of sense >> in streaming/broadcasting world), but would like to hear what >> is the view and experience of the other members of this >> group. Also note that if we are going to support them on >> SFrame this will require a greater effort because we will >> need to explicitly define how the frames must be split before >> being encrypted y SFrame for *each* possible video codec >> (h264,h265,vp8,vp9,av1,...). >> >> >> There was also the question about how/if we should support >> other codec features like DON/interleaved mode for h264, >> which I also think we should not support mainly because we >> are not currently using it on webrtc implementations. >> >> >> What do you think? >> >> >> Best regards >> >> Sergio >> >> >> -- >> Sframe mailing list >> Sframe@ietf.org >> https://www.ietf.org/mailman/listinfo/sframe >> >> >> > -- > Sframe mailing list > Sframe@ietf.org > https://www.ietf.org/mailman/listinfo/sframe > > --------------921F768AA26BAFA08AB9EF0B Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit

How to actually packetize the IDUs is not a big deal, you only need to have a start and end of IDU bits on the header and rely on increasing rtp cseq nums (i.e. no IDU interleaving), but at the end of the day someone will have to specify what an IDU in h264 means and what is its format (a group of nal units?).

I am afraid that we end up another doing a per-codec packetizer, which would be almost the same as just using current packetization formats and explain how to use encrypted data with them. And all for a feature that we are not current using in webrtc.

Best regards

Sergio


On 19/11/2020 23:54, Richard Barnes wrote:
This may be too simplistic, but there's also a non-codec-specific approach here that occurs to me: Just have the SFrame header have a length field. 

I'm imagining a setup where we have the SFrame layer between the encode and packetization layers, and:
- The encoding/decoding layer produces consumes each frame as a sequence of IDUs
- The SFrame layer translates between IDUs and SFrame encrypted enits (SEUs? in any case, each SEU is an encrypted IDU)
- The packetization / depacketization layer packs SEUs into packets

The only thing you need to make that work is (1) a mechanism for the receiver to understand what chunks of the SEU sequence he has (e.g., fixing reordering), and (2) a way to unpack SEUs if there can be multiple in a packet.  It seems like (1) could mostly be a transport assumption.  For (2), you would just need something like a length field.

As Sergio points out, there is a need for someone to know where the IDU boundaries are, either at the SFrame layer (if the input is a whole frame), or at the encode layer (if the encode-SFrame API can talk in terms of IDUs).  But especially in the latter case, this framework keeps the codec-specific stuff local to the encode layer, which is codec-specific in any case.

There is some trade-off here, in that this framework doesn't expose any encoded information to the SFU.  But ISTM that if you want that functionality, then (a) you're going to have to have codec awareness sprinkled all through the stack and (b) you're going to have to be really careful designing the which-parts-to-encrypt scheme to avoid undermining your security guarantees.  So I'm generally inclined toward the cleaner abstraction here.

--Richard

On Thu, Nov 19, 2020 at 5:11 PM Sergio Garcia Murillo <sergio.garcia.murillo@cosmosoftware.io> wrote:

You are right regarding that the SFrame layer does not need to know what is feed in for encryption, but in order to be able to have a working end to end solution for webrtc, someone will need to define what and how this IDUs are generated and reassembled for each codec if we want to have interoperable implementations in different devices.

That process is codec-dependant and I would require quite a lot of effort (and also supporting it on the agnostic packetization), so I would prefer to have strong arguments in favor of doing it.



On 19/11/2020 22:53, Justin Uberti wrote:
The encoder needs to be aware of any mechanism to generate IDUs (e.g., slices), and typically each of these IDUs will be handed up to the consumer individually. So the SFRAME layer doesn't need to do any splitting, it just knows that it should treat each IDU as something it needs to individually SFRAME and packetize.

On Thu, Nov 19, 2020 at 1:40 PM Sergio Garcia Murillo <sergio.garcia.murillo@gmail.com> wrote:

Hi all,


As most of you already know, this morning I made a presentation in AVTCORE introducing the topic about the need to specify an agnostic video codec packetization format.

https://datatracker.ietf.org/meeting/109/materials/slides-109-avtcore-sframe-rtp-encapsulation-00


I got an AP for creating an initial draft so it could be reviewed and accepted.

However, there were two main concerns that we should address in this this group:

  • Historically, avtcore has explicitly designed not to be payload agnostic and  declined to standardized codec agnostic payload formats in  number of cases.  If that is to be changed, needs to be done deliberately.
  • Need to define the "minimum decoding unit" or "independently decodable unit", that SFrame will work with.


Regarding the second one

  • Full video frames (just use whatever is the encoder output)
  • Spatial layer frames
  • "independend decodable subframes" like h264 slices, vp8 partitions or av1 tiles which allows partial decodability which is mainly aimed for enhancing packet loss resilience.


Spatial layer frames is the minimum we should target as if not it will just prevent SFUs for using SVC codecs. So the question is if we should go deeper and implement lower partitions of the frames or not.


AFAIK, currently, libwertc does not support partial decodability and I personally haven't seen any practical usage of this in the RTC world (while it makes a lot of sense in streaming/broadcasting world), but would like to hear what is the view and experience of the other members of this group. Also note that if we are going to support them on SFrame this will require a greater effort because we will need to explicitly define how the frames must be split before being encrypted y SFrame for *each* possible video codec (h264,h265,vp8,vp9,av1,...).


There was also the question about how/if we should support other codec features like DON/interleaved mode for h264, which I also think we should not support mainly because we are not currently using it on webrtc implementations.


What do you think?


Best regards

Sergio


--
Sframe mailing list
Sframe@ietf.org
https://www.ietf.org/mailman/listinfo/sframe

--
Sframe mailing list
Sframe@ietf.org
https://www.ietf.org/mailman/listinfo/sframe
--------------921F768AA26BAFA08AB9EF0B-- From nobody Thu Nov 19 15:23:02 2020 Return-Path: X-Original-To: sframe@ietfa.amsl.com Delivered-To: sframe@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3B7403A0E10 for ; Thu, 19 Nov 2020 15:23:01 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.097 X-Spam-Level: X-Spam-Status: No, score=-2.097 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gUuwUYludwIZ for ; Thu, 19 Nov 2020 15:22:59 -0800 (PST) Received: from mail-lj1-x233.google.com (mail-lj1-x233.google.com [IPv6:2a00:1450:4864:20::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A9ADA3A0E0B for ; Thu, 19 Nov 2020 15:22:58 -0800 (PST) Received: by mail-lj1-x233.google.com with SMTP id x9so8096991ljc.7 for ; Thu, 19 Nov 2020 15:22:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=GRf2Iz21NxVY5Iv5Y6MiKsaozcvRmPOPu3AutcbEJXA=; b=mk7JL/UkGxd3xbZesob+X4NZMjdHlInhk3/muYsRkWI6aRjhkLTSeoMqPuR+pEkO2t JVhqGQQAjgLcHNZyQHB2YUcbWEGs/ntaaEGaTutbeKLpAQSLC/SJUwAIyxaKTHp8zM7W CFve+GE2G77foeXKQy5/uo71klAj7gu68IlUm8hEkcMZCVdu+jg+BFx/eU5PYGcsr3Fj PGmsh0rlI0kJg16EhAdEPa8kSvkqeVlJKir21xOKZvlMCSSCaFc+N+GHyvL9DXJwRnXJ mvQLvpbjHQdiDsd3cLVCHrMeo3JiDmOL03zrM65GW/6LOh3uVShZI51qCMXEJ65w5pwK z5Wg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=GRf2Iz21NxVY5Iv5Y6MiKsaozcvRmPOPu3AutcbEJXA=; b=OfSei+s1ndpckeeHGf8xBi1Wb69jm/hS54lHP7jXHUn3ET6ncTqlO+xld3MNmD03zx 7vTwc1J5/39+nGZX9PsaITNpD201NjEmsWTEuZtJ0qdPFoW3lZbwiovT+b+fBVcarzH5 980WS4Y/+YmeqANc6VR6CUFuwqcZumtst1H3iPNPrFWIUc1DROMIIQ4476lv+nnw3Zr/ BZRkoS6Wm59T9t7aQSUACfd4UbsSwIhl58ZnpJCjeIMKBKsXl0MKwkBjQplE7hObHzVn tTfusUhIjJ1VXpwPmKvTEQuYoe+4zvsX2zNfIN0DphvgAhFk+GhfEmd0REN6oEABzc7M FBEg== X-Gm-Message-State: AOAM532j1VZSG0cwRzRe+h8Lzhnab9cG8nDsJ25ynp8nmnBhMt69P/Hv aHvEXbntb0t3eNkAYXXEAUjmTWvjPOB6uOS4QAs= X-Google-Smtp-Source: ABdhPJwJEHdIfc3mu3C0xITIwdf2KwblMJ0Vgm6ycEjwkqdIyYdaZ2os+2TALlTn2TgOt9pCKdEI0n2nFFw6o2IK+ww= X-Received: by 2002:a05:651c:1246:: with SMTP id h6mr7168949ljh.187.1605828176430; Thu, 19 Nov 2020 15:22:56 -0800 (PST) MIME-Version: 1.0 References: <7eadfc25-d326-3a02-f88f-bfdc02b9a42c@cosmosoftware.io> <2752b6f0-ec61-6a19-e17c-0347c451a17c@cosmosoftware.io> In-Reply-To: <2752b6f0-ec61-6a19-e17c-0347c451a17c@cosmosoftware.io> From: Bernard Aboba Date: Thu, 19 Nov 2020 15:22:46 -0800 Message-ID: To: Sergio Garcia Murillo Cc: Richard Barnes , sframe@ietf.org Content-Type: multipart/alternative; boundary="00000000000006cc2e05b47e0270" Archived-At: Subject: Re: [Sframe] Partial decodability and IDUs X-BeenThere: sframe@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2020 23:23:01 -0000 --00000000000006cc2e05b47e0270 Content-Type: text/plain; charset="UTF-8" Sergio said: "I am afraid that we end up another doing a per-codec packetizer, which would be almost the same as just using current packetization formats and explain how to use encrypted data with them. And all for a feature that we are not current using in webrtc." [BA] Some per-codec logic is required. How else can the metadata be obtained from the bitstream? But the question is whether it is *required* to packetize IDUs separately, so they don't share fate. I'd say this would depend on the use case. If WebRTC didn't care enough about this to implement it previously, why should SFrame make a difference? On Thu, Nov 19, 2020 at 3:15 PM Sergio Garcia Murillo < sergio.garcia.murillo@cosmosoftware.io> wrote: > How to actually packetize the IDUs is not a big deal, you only need to > have a start and end of IDU bits on the header and rely on increasing rtp > cseq nums (i.e. no IDU interleaving), but at the end of the day someone > will have to specify what an IDU in h264 means and what is its format (a > group of nal units?). > > I am afraid that we end up another doing a per-codec packetizer, which > would be almost the same as just using current packetization formats and > explain how to use encrypted data with them. And all for a feature that we > are not current using in webrtc. > > Best regards > > Sergio > > > On 19/11/2020 23:54, Richard Barnes wrote: > > This may be too simplistic, but there's also a non-codec-specific approach > here that occurs to me: Just have the SFrame header have a length field. > > I'm imagining a setup where we have the SFrame layer between the encode > and packetization layers, and: > - The encoding/decoding layer produces consumes each frame as a sequence > of IDUs > - The SFrame layer translates between IDUs and SFrame encrypted enits > (SEUs? in any case, each SEU is an encrypted IDU) > - The packetization / depacketization layer packs SEUs into packets > > The only thing you need to make that work is (1) a mechanism for the > receiver to understand what chunks of the SEU sequence he has (e.g., fixing > reordering), and (2) a way to unpack SEUs if there can be multiple in a > packet. It seems like (1) could mostly be a transport assumption. For > (2), you would just need something like a length field. > > As Sergio points out, there is a need for someone to know where the IDU > boundaries are, either at the SFrame layer (if the input is a whole frame), > or at the encode layer (if the encode-SFrame API can talk in terms of > IDUs). But especially in the latter case, this framework keeps the > codec-specific stuff local to the encode layer, which is codec-specific in > any case. > > There is some trade-off here, in that this framework doesn't expose any > encoded information to the SFU. But ISTM that if you want that > functionality, then (a) you're going to have to have codec awareness > sprinkled all through the stack and (b) you're going to have to be really > careful designing the which-parts-to-encrypt scheme to avoid undermining > your security guarantees. So I'm generally inclined toward the cleaner > abstraction here. > > --Richard > > On Thu, Nov 19, 2020 at 5:11 PM Sergio Garcia Murillo < > sergio.garcia.murillo@cosmosoftware.io> wrote: > >> You are right regarding that the SFrame layer does not need to know what >> is feed in for encryption, but in order to be able to have a working end to >> end solution for webrtc, someone will need to define what and how this IDUs >> are generated and reassembled for each codec if we want to have >> interoperable implementations in different devices. >> >> That process is codec-dependant and I would require quite a lot of effort >> (and also supporting it on the agnostic packetization), so I would prefer >> to have strong arguments in favor of doing it. >> >> >> >> On 19/11/2020 22:53, Justin Uberti wrote: >> >> The encoder needs to be aware of any mechanism to generate IDUs (e.g., >> slices), and typically each of these IDUs will be handed up to the consumer >> individually. So the SFRAME layer doesn't need to do any splitting, it just >> knows that it should treat each IDU as something it needs to individually >> SFRAME and packetize. >> >> On Thu, Nov 19, 2020 at 1:40 PM Sergio Garcia Murillo < >> sergio.garcia.murillo@gmail.com> wrote: >> >>> Hi all, >>> >>> >>> As most of you already know, this morning I made a presentation in >>> AVTCORE introducing the topic about the need to specify an agnostic video >>> codec packetization format. >>> >>> >>> https://datatracker.ietf.org/meeting/109/materials/slides-109-avtcore-sframe-rtp-encapsulation-00 >>> >>> >>> I got an AP for creating an initial draft so it could be reviewed and >>> accepted. >>> >>> However, there were two main concerns that we should address in this >>> this group: >>> >>> - Historically, avtcore has explicitly designed not to be payload >>> agnostic and declined to standardized codec agnostic payload formats in >>> number of cases. If that is to be changed, needs to be done deliberately. >>> - Need to define the "minimum decoding unit" or "independently >>> decodable unit", that SFrame will work with. >>> >>> >>> Regarding the second one >>> >>> - Full video frames (just use whatever is the encoder output) >>> - Spatial layer frames >>> - "independend decodable subframes" like h264 slices, vp8 partitions >>> or av1 tiles which allows partial decodability which is mainly aimed for >>> enhancing packet loss resilience. >>> >>> >>> Spatial layer frames is the minimum we should target as if not it will >>> just prevent SFUs for using SVC codecs. So the question is if we should go >>> deeper and implement lower partitions of the frames or not. >>> >>> >>> AFAIK, currently, libwertc does not support partial decodability and I >>> personally haven't seen any practical usage of this in the RTC world (while >>> it makes a lot of sense in streaming/broadcasting world), but would like to >>> hear what is the view and experience of the other members of this group. >>> Also note that if we are going to support them on SFrame this will require >>> a greater effort because we will need to explicitly define how the frames >>> must be split before being encrypted y SFrame for *each* possible video >>> codec (h264,h265,vp8,vp9,av1,...). >>> >>> >>> There was also the question about how/if we should support other codec >>> features like DON/interleaved mode for h264, which I also think we should >>> not support mainly because we are not currently using it on webrtc >>> implementations. >>> >>> >>> What do you think? >>> >>> >>> Best regards >>> >>> Sergio >>> >>> >>> -- >>> Sframe mailing list >>> Sframe@ietf.org >>> https://www.ietf.org/mailman/listinfo/sframe >>> >> >> -- >> Sframe mailing list >> Sframe@ietf.org >> https://www.ietf.org/mailman/listinfo/sframe >> > -- > Sframe mailing list > Sframe@ietf.org > https://www.ietf.org/mailman/listinfo/sframe > --00000000000006cc2e05b47e0270 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Sergio said:=C2=A0

"I am afraid that we end up a= nother doing a per-codec packetizer, which would be almost the same as just= using current packetization formats and explain how to use encrypted data = with them. And all for a feature that we are not current using in webrtc.&q= uot;

[BA] Some per-codec logic is requ= ired.=C2=A0 How else can the metadata=C2=A0be obtained from the bitstream? = But the question is whether it is *required* to packetize IDUs separately, = so they don't share fate.=C2=A0 I'd say this would depend on the us= e case. If WebRTC didn't care enough about this to implement it previou= sly, why should SFrame make a difference?=C2=A0


On Thu, Nov 1= 9, 2020 at 3:15 PM Sergio Garcia Murillo <sergio.garcia.murillo@cosmosoftware.io> = wrote:
=20 =20 =20

How to actually packetize the IDUs is not a big deal, you only need to have a start and end of IDU bits on the header and rely on increasing rtp cseq nums (i.e. no IDU interleaving), but at the end of the day someone will have to specify what an IDU in h264 means and what is its format (a group of nal units?).

I am afraid that we end up another doing a per-codec packetizer, which would be almost the same as just using current packetization formats and explain how to use encrypted data with them. And all for a feature that we are not current using in webrtc.

Best regards

Sergio


On 19/11/2020 23:54, Richard Barnes wrote:
=20
This may be too simplistic, but there's also a non-codec-specific approach here that occurs to me: Just have the SFrame header have a length field.=C2=A0

I'm imagining a setup where we have the SFrame layer between the encode and packetization layers, and:
- The encoding/decoding layer produces consumes each frame as a sequence of IDUs
- The SFrame layer translates between IDUs and SFrame encrypted enits (SEUs? in any case, each SEU is an encrypted IDU)
- The packetization / depacketization layer packs SEUs into packets

The only thing you need to make that work is (1) a mechanism for the receiver to understand what chunks of the SEU sequence he has (e.g., fixing reordering), and (2) a way to unpack SEUs if there can be multiple in a packet.=C2=A0 It see= ms like (1) could mostly be a transport assumption.=C2=A0 For (2), y= ou would just need something like a length field.

As Sergio points out, there is a need for someone to know where the IDU boundaries are, either at the SFrame layer (if the input is a whole frame), or at the encode layer (if the encode-SFrame API can talk in terms of IDUs).=C2=A0 But especiall= y in the latter case, this framework keeps the codec-specific stuff local to the encode layer, which is codec-specific in any case.

There is some trade-off here, in that this framework doesn't expose any encoded information to the SFU.=C2=A0 But = ISTM that if you want that functionality, then (a) you're going to have to have codec awareness sprinkled all through the stack and (b) you're going to have to be really careful designing the which-parts-to-encrypt scheme to avoid undermining your security guarantees.=C2=A0 So I'm generally inclined toward t= he cleaner abstraction here.

--Richard

On Thu, Nov 19, 2020 at 5:1= 1 PM Sergio Garcia Murillo <sergio.garcia.murillo@cosmosoftwa= re.io> wrote:

You are right regarding that the SFrame layer does not need to know what is feed in for encryption, but in order to be able to have a working end to end solution for webrtc, someone will need to define what and how this IDUs are generated and reassembled for each codec if we want to have interoperable implementations in different devices.

That process is codec-dependant and I would require quite a lot of effort (and also supporting it on the agnostic packetization), so I would prefer to have strong arguments in favor of doing it.



On 19/11/2020 22:53, Justin Uberti wrote:
The encoder needs to be aware of any mechanism to generate IDUs (e.g., slices), and typically each of these IDUs will be handed up to the consumer individually. So the SFRAME layer doesn't need to do any splitting, it just knows that it should treat each IDU as something it needs to individually SFRAME and packetize.

On Thu, Nov 19, 202= 0 at 1:40 PM Sergio Garcia Murillo <sergio.garcia.murillo@gm= ail.com> wrote:

Hi all,


As most of you already know, this morning I made a presentation in AVTCORE introducing the topic about the need to specify an agnostic video codec packetization format.

https://datatracker.ietf.org/meeting/109/materials/slides-109-avtcore-sf= rame-rtp-encapsulation-00


I got an AP for creating an initial draft so it could be reviewed and accepted.

However, there were two main concerns that we should address in this this group:

  • Historically, avtcore has explicitly designed not to be payload agnostic and=C2=A0 declined to standardized codec agnostic payload formats in=C2=A0 number of cases.=C2=A0 I= f that is to be changed, needs to be done deliberately.
  • Need to define the "minimum decoding unit&= quot; or "independently decodable unit", that= SFrame will work with.


Regarding the second one

  • Full video frames (just use whatever is the encoder output)
  • Spatial layer frames
  • "independend decodable subframes" lik= e h264 slices, vp8 partitions or av1 tiles which allows partial decodability which is mainly aimed for enhancing packet loss resilience.


Spatial layer frames is the minimum we should target as if not it will just prevent SFUs for using SVC codecs. So the question is if we should go deeper and implement lower partitions of the frames or not.


AFAIK, currently, libwertc does not support partial decodability and I personally haven't seen any practical usage of this in the RTC world (while it makes a lot of sense in streaming/broadcasting world), but would like to hear what is the view and experience of the other members of this group. Also note that if we are going to support them on SFrame this will require a greater effort because we will need to explicitly define how the frames must be split before being encrypted y SFrame for *each* possible video codec (h264,h265,vp8,vp9,av1,...).


There was also the question about how/if we should support other codec features like DON/interleaved mode for h264, which I also think we should not support mainly because we are not currently using it on webrtc implementations.


What do you think?


Best regards

Sergio


--
Sframe mailing list
Sf= rame@ietf.org
https://www.ietf.org/mailman/listinf= o/sframe

--
Sframe mailing list
Sframe@iet= f.org
https://www.ietf.org/mailman/listinfo/sframe=
--
Sframe mailing list
Sframe@ietf.org https://www.ietf.org/mailman/listinfo/sframe
--00000000000006cc2e05b47e0270-- From nobody Thu Nov 19 15:23:24 2020 Return-Path: X-Original-To: sframe@ietfa.amsl.com Delivered-To: sframe@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5FFEC3A0E10 for ; Thu, 19 Nov 2020 15:23:22 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.897 X-Spam-Level: X-Spam-Status: No, score=-1.897 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, NICE_REPLY_A=-0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=cosmosoftware-io.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1zvUxfzG7A79 for ; Thu, 19 Nov 2020 15:23:20 -0800 (PST) Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id EDDB33A0E0B for ; Thu, 19 Nov 2020 15:23:19 -0800 (PST) Received: by mail-wr1-x42b.google.com with SMTP id j7so8214919wrp.3 for ; Thu, 19 Nov 2020 15:23:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cosmosoftware-io.20150623.gappssmtp.com; s=20150623; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=t3mk4ARrgXe5S+EppYz7HoC57NLg4D5QLc4LLYx0niU=; b=eCdEFABEbIC9xoIRnnV2wxLvlAFAzukSIFx5Jafg56Ptx12mpsDkU7o3w8w5lzIUYj BYxcC9Rd2C+JFE62okW3wBlxawt1sVgB2c5Lgab280rshbg8V/uJ5RG2BTI4J1etFmpu rBWu17ifwMi39+fS8afbqmFmrkWeOSD2ze5rEu/C/NCZKk5A2oakTf/bgiV5g1xgbXTy pndapL5+RO9QGc0lO5w5yFR7cxvXUN/ttsDSBwLWKhWQMZjm8P+H283bbSu1m5RboPRA O+p5EQ5MajtdAiJFKvw1Bb5H8qr9JMczWZ9VhynHbk91ZaJpe5e2pQE1rhKylCLruJax FyJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language; bh=t3mk4ARrgXe5S+EppYz7HoC57NLg4D5QLc4LLYx0niU=; b=bDonTENtn7UTtTSIRRjSw2WjbeoOHMk54/eSz/MJmAg/4bMIk+qF5OIYXgxYiu2CQP c1d+b65Lw7WoTy3neF6ety/yLNfaOen3bX4PfZ2Z5xsuWQRXo2eEcE7g4pLG/4DYzrXw oss0+ba8ekmIB7qwCaN0DUhHAbYRgnKIZMRCAeaZ70pIVJ4QVU8PlWT6C7wqaRRpvHxe eCs/6pCGTmdMYbG1Rp1lpjStZdZxPZPRsBuhEEeAx+S4ewkUXzyhn2grVqDTDywbUWJP teUYBYk9bX+PMUGji2fm7n+C1/dm9ndNuSZesFlD1suMvKSB2Fipy3pzk2FxArI9VlMO LQPA== X-Gm-Message-State: AOAM533ohD/0rMxLSV+npjLVSf4OHMuRL+LoooKhdYFGNkls0OOKox69 HJiDcWu/GtJhJAX2JAGV+RBTKiGkTXMG+Q== X-Google-Smtp-Source: ABdhPJzdq6pztyZ+SgVyWIfWREPQiDv41z7XcltxX0Oj3yMULtvOUb61OOhNc01MFqSrnPI3knHqHw== X-Received: by 2002:a5d:448a:: with SMTP id j10mr11975820wrq.33.1605828198119; Thu, 19 Nov 2020 15:23:18 -0800 (PST) Received: from [192.168.0.11] (79.108.125.160.dyn.user.ono.com. [79.108.125.160]) by smtp.gmail.com with ESMTPSA id f16sm2146375wrp.66.2020.11.19.15.23.17 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 19 Nov 2020 15:23:17 -0800 (PST) To: Bernard Aboba Cc: sframe@ietf.org References: <7eadfc25-d326-3a02-f88f-bfdc02b9a42c@cosmosoftware.io> From: Sergio Garcia Murillo Message-ID: <4d6928e3-fb19-5694-f685-3a42b511581a@cosmosoftware.io> Date: Fri, 20 Nov 2020 00:23:16 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.4.3 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/alternative; boundary="------------654B669104BC318D213FAA90" Content-Language: en-US Archived-At: Subject: Re: [Sframe] Partial decodability and IDUs X-BeenThere: sframe@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2020 23:23:22 -0000 This is a multi-part message in MIME format. --------------654B669104BC318D213FAA90 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit If we are able to say something like: "The encoder is responsible of dividing the encoded byte stream for each spatial video frame in 1 or more byte chunks called IDUs before passing it to the SFrame layer independently, in such a way that if any of the IDUs is not fully received, the rest of the IDUs can be concatenated after decrypting and still be partially decodable by the decoder without doing any further byte stream manipulation." Then I would be fine, but I doubt it could be as easy. Best regards Sergio On 20/11/2020 0:13, Bernard Aboba wrote: > Sergio said: > > "You are right regarding that the SFrame layer does not need to know > what is feed in for encryption, but in order to be able to have a > working end to end solution for webrtc, someone will need to define > what and how this IDUs are generated and reassembled for each codec if > we want to have interoperable implementations in different devices." > > [BA] The job of an SFrame sender is to encrypt and packetize the > bitstream provided by the encoder.  For SFU to act on the > packetization in an optimal way, some rules have to be followed (such > as not packetizing frames from different layers in the same packet). > So the sender/packetizer will have codec-specific logic, so it can > parse the bitstream for meta-data, and figure out how to do the > packetization in a manner appropriate for that codec. This could > include separately packetizing IDUs (slices/tiles), but I'm not clear > this needs to be required for all use cases. > > The SFM does not peer into the payload, it only acts on the meta-data > included in RTP header extensions, so the recovery/forwarding/dropping > decision can be largely codec-independent. > > The receiver decrypts and de-packetizes the bitstream, then provides > it to the decoder.  The recovery/decryption/de-packetizion process > should also be codec-independent. > > On Thu, Nov 19, 2020 at 2:11 PM Sergio Garcia Murillo > > wrote: > > You are right regarding that the SFrame layer does not need to > know what is feed in for encryption, but in order to be able to > have a working end to end solution for webrtc, someone will need > to define what and how this IDUs are generated and reassembled for > each codec if we want to have interoperable implementations in > different devices. > > That process is codec-dependant and I would require quite a lot of > effort (and also supporting it on the agnostic packetization), so > I would prefer to have strong arguments in favor of doing it. > > > > On 19/11/2020 22:53, Justin Uberti wrote: >> The encoder needs to be aware of any mechanism to generate IDUs >> (e.g., slices), and typically each of these IDUs will be handed >> up to the consumer individually. So the SFRAME layer doesn't need >> to do any splitting, it just knows that it should treat each IDU >> as something it needs to individually SFRAME and packetize. >> >> On Thu, Nov 19, 2020 at 1:40 PM Sergio Garcia Murillo >> > > wrote: >> >> Hi all, >> >> >> As most of you already know, this morning I made a >> presentation in AVTCORE introducing the topic about the need >> to specify an agnostic video codec packetization format. >> >> https://datatracker.ietf.org/meeting/109/materials/slides-109-avtcore-sframe-rtp-encapsulation-00 >> >> >> >> I got an AP for creating an initial draft so it could be >> reviewed and accepted. >> >> However, there were two main concerns that we should address >> in this this group: >> >> * Historically, avtcore has explicitly designed not to be >> payload agnostic and  declined to standardized codec >> agnostic payload formats in number of cases.  If that is >> to be changed, needs to be done deliberately. >> * Need to define the "minimum decoding unit" or >> "independently decodable unit", that SFrame will work with. >> >> >> Regarding the second one >> >> * Full video frames (just use whatever is the encoder output) >> * Spatial layer frames >> * "independend decodable subframes" like h264 slices, vp8 >> partitions or av1 tiles which allows partial decodability >> which is mainly aimed for enhancing packet loss resilience. >> >> >> Spatial layer frames is the minimum we should target as if >> not it will just prevent SFUs for using SVC codecs. So the >> question is if we should go deeper and implement lower >> partitions of the frames or not. >> >> >> AFAIK, currently, libwertc does not support partial >> decodability and I personally haven't seen any practical >> usage of this in the RTC world (while it makes a lot of sense >> in streaming/broadcasting world), but would like to hear what >> is the view and experience of the other members of this >> group. Also note that if we are going to support them on >> SFrame this will require a greater effort because we will >> need to explicitly define how the frames must be split before >> being encrypted y SFrame for *each* possible video codec >> (h264,h265,vp8,vp9,av1,...). >> >> >> There was also the question about how/if we should support >> other codec features like DON/interleaved mode for h264, >> which I also think we should not support mainly because we >> are not currently using it on webrtc implementations. >> >> >> What do you think? >> >> >> Best regards >> >> Sergio >> >> >> -- >> Sframe mailing list >> Sframe@ietf.org >> https://www.ietf.org/mailman/listinfo/sframe >> >> >> > -- > Sframe mailing list > Sframe@ietf.org > https://www.ietf.org/mailman/listinfo/sframe > > --------------654B669104BC318D213FAA90 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit

If we are able to say something like:

"The encoder is responsible of dividing the encoded byte stream for each spatial video frame in 1 or more byte chunks called IDUs before passing it to the SFrame layer independently, in such a way that if any of the IDUs is not fully received, the rest of the IDUs can be concatenated after decrypting and still be partially decodable by the decoder without doing any further byte stream manipulation."

Then I would be fine, but I doubt it could be as easy.

Best regards

Sergio

On 20/11/2020 0:13, Bernard Aboba wrote:
Sergio said: 

"You are right regarding that the SFrame layer does not need to know what is feed in for encryption, but in order to be able to have a working end to end solution for webrtc, someone will need to define what and how this IDUs are generated and reassembled for each codec if we want to have interoperable implementations in different devices."

[BA] The job of an SFrame sender is to encrypt and packetize the bitstream provided by the encoder.  For SFU to act on the packetization in an optimal way, some rules have to be followed (such as not packetizing frames from different layers in the same packet). So the sender/packetizer will have codec-specific logic, so it can parse the bitstream for meta-data, and figure out how to do the packetization in a manner appropriate for that codec.  This could include separately packetizing IDUs (slices/tiles), but I'm not clear this needs to be required for all use cases. 

The SFM does not peer into the payload, it only acts on the meta-data included in RTP header extensions, so the recovery/forwarding/dropping decision can be largely codec-independent. 

The receiver decrypts and de-packetizes the bitstream, then provides it to the decoder.  The recovery/decryption/de-packetizion process should also be codec-independent.

On Thu, Nov 19, 2020 at 2:11 PM Sergio Garcia Murillo <sergio.garcia.murillo@cosmosoftware.io> wrote:

You are right regarding that the SFrame layer does not need to know what is feed in for encryption, but in order to be able to have a working end to end solution for webrtc, someone will need to define what and how this IDUs are generated and reassembled for each codec if we want to have interoperable implementations in different devices.

That process is codec-dependant and I would require quite a lot of effort (and also supporting it on the agnostic packetization), so I would prefer to have strong arguments in favor of doing it.



On 19/11/2020 22:53, Justin Uberti wrote:
The encoder needs to be aware of any mechanism to generate IDUs (e.g., slices), and typically each of these IDUs will be handed up to the consumer individually. So the SFRAME layer doesn't need to do any splitting, it just knows that it should treat each IDU as something it needs to individually SFRAME and packetize.

On Thu, Nov 19, 2020 at 1:40 PM Sergio Garcia Murillo <sergio.garcia.murillo@gmail.com> wrote:

Hi all,


As most of you already know, this morning I made a presentation in AVTCORE introducing the topic about the need to specify an agnostic video codec packetization format.

https://datatracker.ietf.org/meeting/109/materials/slides-109-avtcore-sframe-rtp-encapsulation-00


I got an AP for creating an initial draft so it could be reviewed and accepted.

However, there were two main concerns that we should address in this this group:

  • Historically, avtcore has explicitly designed not to be payload agnostic and  declined to standardized codec agnostic payload formats in  number of cases.  If that is to be changed, needs to be done deliberately.
  • Need to define the "minimum decoding unit" or "independently decodable unit", that SFrame will work with.


Regarding the second one

  • Full video frames (just use whatever is the encoder output)
  • Spatial layer frames
  • "independend decodable subframes" like h264 slices, vp8 partitions or av1 tiles which allows partial decodability which is mainly aimed for enhancing packet loss resilience.


Spatial layer frames is the minimum we should target as if not it will just prevent SFUs for using SVC codecs. So the question is if we should go deeper and implement lower partitions of the frames or not.


AFAIK, currently, libwertc does not support partial decodability and I personally haven't seen any practical usage of this in the RTC world (while it makes a lot of sense in streaming/broadcasting world), but would like to hear what is the view and experience of the other members of this group. Also note that if we are going to support them on SFrame this will require a greater effort because we will need to explicitly define how the frames must be split before being encrypted y SFrame for *each* possible video codec (h264,h265,vp8,vp9,av1,...).


There was also the question about how/if we should support other codec features like DON/interleaved mode for h264, which I also think we should not support mainly because we are not currently using it on webrtc implementations.


What do you think?


Best regards

Sergio


--
Sframe mailing list
Sframe@ietf.org
https://www.ietf.org/mailman/listinfo/sframe

--
Sframe mailing list
Sframe@ietf.org
https://www.ietf.org/mailman/listinfo/sframe
--------------654B669104BC318D213FAA90-- From nobody Thu Nov 19 15:27:17 2020 Return-Path: X-Original-To: sframe@ietfa.amsl.com Delivered-To: sframe@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E5FFF3A1389 for ; Thu, 19 Nov 2020 15:27:15 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.899 X-Spam-Level: X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, NICE_REPLY_A=-0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=cosmosoftware-io.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9yiAY71aj4CC for ; Thu, 19 Nov 2020 15:27:14 -0800 (PST) Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 941A73A1386 for ; Thu, 19 Nov 2020 15:27:14 -0800 (PST) Received: by mail-wr1-x434.google.com with SMTP id d12so8118582wrr.13 for ; Thu, 19 Nov 2020 15:27:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cosmosoftware-io.20150623.gappssmtp.com; s=20150623; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding:content-language; bh=NVbk09OprGQOQLKiywjP5kPQKebwGuw7aylVXFoWH/Y=; b=YbEZH7YQdWvG7cP3haNffaXpjqqbAGpgW4J9Sfe8hxTB3qjooQ9lFE537WXYU61WCp CEP2ZC9jUroJZkqgrPmeOWw2SmuZh121f2dJtpD2xuXfYGM7FtOjdlXQNw2DW7YfAcxX LvSaauNgf7MAc4dWxtPS3YRKy79xi1eCL+TYigfFac2YlaF0TuL7CMkgQAOeounxYKcX wwe6MtNhoqRXL9Ohyx0QLLMjfBidVcqnwF2hescDys6zfncgQEuU06KsDbnl9pwrd6cG qW8fxdtIk17wgR4Uroa0hnpSUPi6JZPinxU0z/AmWBfpXy75aMUwHTB6yYT5lqqs6l2y Tpzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=NVbk09OprGQOQLKiywjP5kPQKebwGuw7aylVXFoWH/Y=; b=Ci+bem/Rwvg9PuvMVax2CIt0DRvEPsDoMmJbVv94ccgLb1KzloyF+D/qhHn9NlCrD9 pW6GRzPghvyANaHF9hxMHXGZ4ILbTCTkoUnSmIMckIcHkaRax2bl3dQWjAz1CDyRz/zy xwjd79X2RgIsP65cBPuNMQFNZXTeGJ4UOCEImW7x/JkspnM/izfK3Kjr4nKBt6ncDtLP ahXFGCM0oipZTpxpvJDpGARh0YfvMlGEQ1iwmNConPGEAucSgk8sg84O+Qci9xFu63Zf CogheFNCng6p7TBe10U9PwVmmwq849Q7APY4QsXcK2SOASzLqzo53F7dPwphSQklmkbA U4eQ== X-Gm-Message-State: AOAM532eB2yGC9hNGgP47UYs0jAXHm0xzZo4C2pTfYAwz7pUaWUCxoAQ QgjgBH17wka9vOFIajJ+D/z9bARd0VBq6Q== X-Google-Smtp-Source: ABdhPJzE0kZmpiN61Vkd4ugVP4gMTCIlbVy04fJNawUaIWrBdHtc6nX6oZneGo8YTN0PKwAmajenwQ== X-Received: by 2002:adf:f7ce:: with SMTP id a14mr12765775wrq.294.1605828432612; Thu, 19 Nov 2020 15:27:12 -0800 (PST) Received: from [192.168.0.11] (79.108.125.160.dyn.user.ono.com. [79.108.125.160]) by smtp.gmail.com with ESMTPSA id x13sm2114796wmi.20.2020.11.19.15.27.11 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 19 Nov 2020 15:27:12 -0800 (PST) To: Bernard Aboba Cc: Richard Barnes , sframe@ietf.org References: <7eadfc25-d326-3a02-f88f-bfdc02b9a42c@cosmosoftware.io> <2752b6f0-ec61-6a19-e17c-0347c451a17c@cosmosoftware.io> From: Sergio Garcia Murillo Message-ID: Date: Fri, 20 Nov 2020 00:27:10 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.4.3 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US Archived-At: Subject: Re: [Sframe] Partial decodability and IDUs X-BeenThere: sframe@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2020 23:27:16 -0000 On 20/11/2020 0:22, Bernard Aboba wrote: > Sergio said: > > "I am afraid that we end up another doing a per-codec packetizer, > which would be almost the same as just using current packetization > formats and explain how to use encrypted data with them. And all for a > feature that we are not current using in webrtc." > > [BA] Some per-codec logic is required.  How else can the metadata be > obtained from the bitstream? But the question is whether it is > *required* to packetize IDUs separately, so they don't share fate.  > I'd say this would depend on the use case. If WebRTC didn't care > enough about this to implement it previously, why should SFrame make a > difference? Taking the AV1 dependency descriptor as an example, all of the elements are well defined in a codec agnostic way. Being simplistic in the example, there is no need for describing how to get the width and height of the video frame from an h264 stream, the encoder would be responsible for filling the the correct information. Or at least that was the idea behind the design. Best regards Sergio From nobody Fri Nov 20 07:17:30 2020 Return-Path: X-Original-To: sframe@ietfa.amsl.com Delivered-To: sframe@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9A21C3A0C63 for ; Fri, 20 Nov 2020 07:17:28 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.897 X-Spam-Level: X-Spam-Status: No, score=-1.897 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=ipv-sx.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wRbTnzuFnj27 for ; Fri, 20 Nov 2020 07:17:27 -0800 (PST) Received: from mail-qt1-x836.google.com (mail-qt1-x836.google.com [IPv6:2607:f8b0:4864:20::836]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D60C73A0C62 for ; Fri, 20 Nov 2020 07:17:26 -0800 (PST) Received: by mail-qt1-x836.google.com with SMTP id 7so7327163qtp.1 for ; Fri, 20 Nov 2020 07:17:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ipv-sx.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=qXvz8x6sxfSGTq3t8TlGeYF5nWzCZg2XCNOHq6BAUlg=; b=B7s/lSZRrSep2ftsY05fhgW5PrswGfYEk1YwoCKxSymx6EMsm9fPtRZychCaf8muFL oaFC5z3dirg8M0gmc7Y+SbsZagGFZ4ByFa+y1b173ehwXwcJpEN7QTZwVuTwo8j8r/Em /H9kZc5hO1AVtB2FjKbbbhNUlohYJEkRHvZzo85bPj/5YULt/3y9vyKzKit+IzH1Zk3h KEYBLrtcTFUJl0wC58l/Oazpr+6x9YAxct+2vIaBpuaWUW8+4pgkDS0u/jWvk/hZ9CSQ AaNr/ct7+C31OsG98AOAkoh3Ylgz1dHY5zR2k8hBvthNht2DEN5KL2tz2VdpFPJbpX60 4zfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=qXvz8x6sxfSGTq3t8TlGeYF5nWzCZg2XCNOHq6BAUlg=; b=ACEzSqFvey+Kjs68YNkBdyO44mF85b0wJ/6IXutL8h023QE5/19VniqaIgeJHCeJeb xsTXsX7v5ZATX1dwjNqEGHxtbKeVH8IEnWbJKSiKtvOZtFSMu3j9/13uXSxR7N4R5COY 9RhBW5PVObQRHvfsOUMob8Kfbj/lkguOwatt668VAwXHm94+mO6h0H+kE2YH4kFpHNJX K3nTuTWo1BJ7gVlhwjI2pyZNXjkdcgdR4CqA8aYHRJwo7WXXi/fVOr8uOLiKCUEJpFgL lqX3AS6ZjgBmYLjQO0l/TdmVC5D0DMjxdffe1/IEGfIAwIHglokKH+Ervv3YaiGxZE3X QlbQ== X-Gm-Message-State: AOAM5333aN+J7A0PTrvuzhVHxXaD/hMmTZ6AQ7Z94bP/i2AC2tb+uiHZ 1BgmPZ0+RaZOaoIdJoiy+f4APiTtW0RUgAonWfiVlBfAFTEmUA== X-Google-Smtp-Source: ABdhPJxQpamClVs5jyZm6qg6TbDgmZ1S7PmeO56ZWc8UepOR0JosuGAxdVmyj87kuQa75L25qJYMh79eTU+/VK7+88A= X-Received: by 2002:aed:2661:: with SMTP id z88mr12651565qtc.265.1605885445050; Fri, 20 Nov 2020 07:17:25 -0800 (PST) MIME-Version: 1.0 From: Richard Barnes Date: Fri, 20 Nov 2020 10:17:11 -0500 Message-ID: To: sframe@ietf.org Content-Type: multipart/alternative; boundary="00000000000080add205b48b57fd" Archived-At: Subject: [Sframe] To tree or not to tree? X-BeenThere: sframe@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Nov 2020 15:17:28 -0000 --00000000000080add205b48b57fd Content-Type: text/plain; charset="UTF-8" Hi all, One point that was raised during the SFrame meeting at IETF 109 was whether the MLS-SFrame integration should re-use the MLS "secret tree" structure. For those who might not be deep in MLS, the secret tree provides forward secrecy for MLS messages sent within an epoch, without the need to generate base keys for all participants [1]. In other words, if you have a group where (a) only a few participants are speaking and (b) they ratchet their key after each message, then the secret tree assures that compromise of any group member with current state won't leak the keys for messages that have already been sent/received. Let's consider three possible integrations: 1. SFrame uses the same secret tree as MLS (export at the leaves) 2. SFrame exports a single secret, then makes its own secret tree 3. SFrame exports a single secret, then uses a simpler scheme (like the current one) I would posit that (1) is not workable. That requires a tight coupling between the MLS and SFrame stacks, which will often not be tractable, e.g., in situations where the media logic is in a separate thread or process from the MLS logic. (2) only adds value over (3) if SFrame senders ratchet their keys. Otherwise, there's no forward secrecy boundary; criterion (b) above doesn't apply. The current MLS-SFrame document has no provision for ratcheting within an epoch. We could do it, but it would require more bits of header to send a "generation" as MLS does, to indicate how many times you've ratcheted. It also seems like situations where you have mostly silent participants are more rare in real-time cases than in messaging cases. So my preference would still be for (3), largely because intra-epoch forward secrecy seems like a pretty secondary consideration here. If intra-epoch forward secrecy is a problem people want to solve, then we should do ratcheting, and we should do the secret tree. --Richard [1] https://github.com/mlswg/mls-protocol/blob/master/draft-ietf-mls-protocol.md#secret-tree-secret-tree --00000000000080add205b48b57fd Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi all,

One point that was raised durin= g the SFrame meeting at IETF 109 was whether the MLS-SFrame integration sho= uld re-use the MLS "secret tree" structure.=C2=A0
=
For those who might not be deep in MLS, the secret tree prov= ides forward secrecy for MLS messages sent within an epoch, without the nee= d to generate base keys for all participants [1].=C2=A0 In other words, if = you have a group where (a) only a few participants are speaking and (b) the= y ratchet their key after each message, then the secret tree assures that c= ompromise of any group member with current state won't leak the keys fo= r messages that have already been sent/received.

L= et's consider three possible integrations:

1. = SFrame uses the same secret tree as MLS (export at the leaves)
2. SFrame exports a single secret, then makes its own secret tree
3. SFrame exports a single secret, then uses a simpler scheme (like = the current one)

I would posit that (1) is not= workable.=C2=A0 That requires a tight coupling between the MLS and SFrame = stacks, which will often not be tractable, e.g., in situations where the me= dia logic is in a separate thread or process from the MLS logic.
=
(2) only adds value over (3) if SFrame senders ratchet their= keys.=C2=A0 Otherwise, there's no forward secrecy boundary; criterion = (b) above doesn't apply.=C2=A0 The current MLS-SFrame document has no p= rovision for ratcheting within an epoch.=C2=A0 We could do it, but it would= require more bits of header to send a "generation" as MLS does, = to indicate how many times you've ratcheted.=C2=A0 It also seems like s= ituations where you have mostly silent participants are more rare in real-t= ime cases than in messaging cases.

So my prefe= rence would still be for (3), largely because intra-epoch forward secrecy s= eems like a pretty secondary consideration here.=C2=A0 If intra-epoch forwa= rd secrecy is a problem people want to solve, then we should do ratcheting,= and we should do the secret tree.

--Richard

--00000000000080add205b48b57fd-- From nobody Fri Nov 20 08:01:13 2020 Return-Path: X-Original-To: sframe@ietfa.amsl.com Delivered-To: sframe@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B55803A0D9B for ; Fri, 20 Nov 2020 08:01:11 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.102 X-Spam-Level: X-Spam-Status: No, score=-2.102 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=ericsson.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cIwJrJJOhJd9 for ; Fri, 20 Nov 2020 08:01:09 -0800 (PST) Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-eopbgr150081.outbound.protection.outlook.com [40.107.15.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 66AF23A0D97 for ; Fri, 20 Nov 2020 08:01:08 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jH9AT4anxo+FokbD+HKDVe4ytcwFdQPc/GTU6BkTziTERyU1xGrxYKXLfUmtlY0cf1wH0T9nz9yQYMBL6UuEQWnj02Ish0/gjuT6BqksroQ14Mu+aotLU8kMlKx99/IL+NUw/cZ4uBaecnb9HNOuq6/c7w0Xlwra4kesbIXazANxOvWsgx0XsuvX0HRIzmBfRjT3hcOkSPb/QJlZY/Bt5sMZesPApWKOAJnVAgnv5RVwnG47O+GSD4X02WqVvRqCJRnf/qyfAESilP/HeJeVDoUeAyyd/ROkOvfANvHLiYUEk6vXom/TkJiBWHaK+59XnWmltiC6uYp+QgX+y+rafg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=n/0ilMe9kgZA+y7ser2IvORd44jsBUdPAFg6TtKI1jw=; b=UgoEYtJjyBd4yhq8e9wSbXIyAJ8s25BXDtUXHtvsHBzz8CXogeM/RBl1b587VaDECCnxi2yoId6ISLe+b9X0TnIqQ/Lk+QXrVPGKn+WaKC/Y68SPvflFxqts+JXhon9X6RtgB5xi/NBVGfmWoa9Lfp2R2Z169wZRZpR+PeXSMcd8V8pYOSMBaQkVsAZcJfA+DZK+vP/lpqBmGglX9v9OfmeLYCuWnojy86SCZr/MH7IDZ7q8hKwU54tfT7jSkTHVHUO051unuzowf2yuV3yhvc2VkO1tEWb9Lnybw64WGhXFiHF7UO4le4knxjI/g4+Sp9DnAI6+S4bG37XJeT9tLA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ericsson.com; dmarc=pass action=none header.from=ericsson.com; dkim=pass header.d=ericsson.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=n/0ilMe9kgZA+y7ser2IvORd44jsBUdPAFg6TtKI1jw=; b=DMxyXSXsM/Gt4CUQMMQi1xTOu/YbDJdyHqzNWzX5Z0JpktpYJsPJWiqYy3koShOX/S7C2cjc0nv5BIg3XGB5pHoAGfJD+vn1DkbNa/TnXReWS0yOAWrpBk2Nly7NqEHS6CN5F4Fv3T+4WIkke0i4xSX3UC+GHPMtV2bHFgefTqw= Received: from AM0PR0702MB3764.eurprd07.prod.outlook.com (2603:10a6:208:18::12) by AM0PR07MB5986.eurprd07.prod.outlook.com (2603:10a6:208:110::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3611.9; Fri, 20 Nov 2020 16:01:04 +0000 Received: from AM0PR0702MB3764.eurprd07.prod.outlook.com ([fe80::8d73:a26d:e1bb:8943]) by AM0PR0702MB3764.eurprd07.prod.outlook.com ([fe80::8d73:a26d:e1bb:8943%6]) with mapi id 15.20.3589.016; Fri, 20 Nov 2020 16:01:04 +0000 From: Magnus Westerlund To: "bernard.aboba@gmail.com" , "sergio.garcia.murillo@cosmosoftware.io" CC: "rlb@ipv.sx" , "sframe@ietf.org" Thread-Topic: [Sframe] Partial decodability and IDUs Thread-Index: AQHWvryjarx1ZMYfcUuCgRv1dn/2e6nP/3yAgAAE1wCAAAwnAIAABckAgAACCgCAAAE7AIABFauA Date: Fri, 20 Nov 2020 16:01:04 +0000 Message-ID: References: <7eadfc25-d326-3a02-f88f-bfdc02b9a42c@cosmosoftware.io> <2752b6f0-ec61-6a19-e17c-0347c451a17c@cosmosoftware.io> In-Reply-To: Accept-Language: sv-SE, en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-mailer: Evolution 3.28.5-0ubuntu0.18.04.2 authentication-results: gmail.com; dkim=none (message not signed) header.d=none;gmail.com; dmarc=none action=none header.from=ericsson.com; x-originating-ip: [192.176.1.82] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 3aab7163-9fa3-420b-d40a-08d88d6d7c46 x-ms-traffictypediagnostic: AM0PR07MB5986: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:5236; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: s1u3sL/GWTUlRmLd2A9aT8WlU9I3uh6OMehNa79b+MWeGc8c9fUqzWsDm4BwqfUKjWwgfj2IHw1Pwouv6Qh+0kUokZIue66feFCU5EmC0deQkc8L7eNOSJTPEZvOtzt5JAiBKNRENuVuLw0Tvbf2NPvXD0rqAfp+np9WLMVJaeiM5IfS4RNZRKdpO3UzusANGWhOtuUMM7EAZewim1OV4nRwgUHx+TfXwZw6K4CIPBLH5nzAyTWXb5ZGyxMkG2/A78ihTJWR7asmNdj7K4m//lLXcEub1uAAkmq1DGQE1OIFZD87K3c4nFmwIMRWHj8m2VprK7wMzl8zrVoI9Rlw/mqirqrSLlZZ+tu1RI7IQ8KTjN5Ul0AaiuJV9GyXVk/g x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR0702MB3764.eurprd07.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39860400002)(366004)(376002)(396003)(136003)(346002)(66616009)(86362001)(316002)(99936003)(2616005)(44832011)(71200400001)(2906002)(8936002)(66946007)(6486002)(76116006)(91956017)(66446008)(64756008)(66556008)(66476007)(186003)(83380400001)(36756003)(54906003)(478600001)(8676002)(26005)(110136005)(6512007)(4326008)(5660300002)(6506007)(99106002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?utf-8?B?Rlg2aGFHSERzeWFWaE1RaWcrcktmZTU5WXlRNnB4WE1KczRMNUgxRXA3Z1VS?= =?utf-8?B?K1hXYkFwbmFabDc4ZTA4eXFINmF5dnBITHBpYmlTaVNRY3JlTWJlSU1BeU5x?= =?utf-8?B?MUh0eGxWbVZIa2JvNThxR1ZwaFRzRFdoeUZhRzVnRjBHTkVXdU03REJGQ1pa?= =?utf-8?B?aGFHcGMvUnU5Wjgzb2hOZHI2bWFvZVFYcEo5c2FRMUs5c3VDc1p6U0M2cmNS?= =?utf-8?B?SEtyQ1BRWGliRjE2TGozRm13UzdSSjZ6TnBiK2MzZExobVVvWVNBNllMeTNL?= =?utf-8?B?UGFDRTdDWUFTUG5LaG1WZjNzY0ZQd2tteWp0bGlPWllJbU4xVEdIa2hNSU1B?= =?utf-8?B?OUxMZENzNTd6aFNYVEMyRHEzeDg4R2tTZGZDRUZrNUl0NEhHdWhHd0ZjVUdP?= =?utf-8?B?Q1l0aUlUTFFWYXNwMHN4Rk1iSlAxVTNrSWNjY0xrZFRjNE5sMnoxMlBvUmJw?= =?utf-8?B?c3lmMmZMcmVwT2pNMG9YMUxDNEVCOXBzdk03bDBNNmhlMXExLy9TSU5LWDZ0?= =?utf-8?B?MGtJSUxYV0svTGtrQ1FCblBtQzhhY2NaSWlWc0dBMGpjbHpoM2J4U3BLUzlE?= =?utf-8?B?NDI3QjBnS2xHbWhlOVRXM0JBOHJzOFpFNmZXVkhJYVdES3pPbjFERzR4N2Rs?= =?utf-8?B?VURoV1N4QUVKNHpiaEVnZGNLVVVVK3VJeDVFMVVRcyt1UG9zZW56c0s5dW9D?= =?utf-8?B?QWlLUFVhQytYZmZvTlpoY21VNGJQdm9IcnhKRkMrZEl6UjJ3Y3U0TUFIalVs?= =?utf-8?B?aWFRZEc4NW54NDd4amV1bUlvVU9xZzg5Z2x1dTFKNnVHNUw4UncwWWl3L1Mx?= =?utf-8?B?SWNKN0dmSE1rWGM1eGFjWi8yQ3FHbndTeXRBeUtNSFpEV0xwS1h0VzBPeFgw?= =?utf-8?B?MzJNbEJhdzF0Tm1KYlhJclZZZFk4UXZrL0QyVWp5Z0JQM1ExM0tCQ1dRRm5k?= =?utf-8?B?c084U0FwOXJUQTZxcGdnZzROQ2s2RDZzYStZaFVpWGcxb0NlaDcvLzJiWitD?= =?utf-8?B?RS96UVlyRnlvWU0rZldBZFJHZTFOVVlyaXc1OGVObExKcFpnclhFeCt6dWxp?= =?utf-8?B?bEVHQUx5MktCSEliYW9DMlppWnhTQzdnSE9lemJpT1FDR21yeTNkQ0xIM1h1?= =?utf-8?B?WWN1Q1VjTzl5WE9IWUxwS0NyZ0E3NS9QaFRWRmQxMVBueFNoR0pBazVqbDlY?= =?utf-8?B?eFNFVVNROGtVWXJRQ2tySnJnSDZPSm1QUkNPTzM3KzhraXpFdWx0L2hLRElx?= =?utf-8?B?cSs3UXdhS2tnNmk5WEJYTGpLdmduZmV6bzJjSUZzMFF2OWpUSm11SlQ5bjZO?= =?utf-8?Q?lJg3RV+VIqvmT6yDFbzLofKHr7mSYy0/IQ?= x-ms-exchange-transport-forked: True Content-Type: multipart/signed; micalg="sha-256"; protocol="application/x-pkcs7-signature"; boundary="=-bfgKan+LDXfZqRQSXF7f" MIME-Version: 1.0 X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AM0PR0702MB3764.eurprd07.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3aab7163-9fa3-420b-d40a-08d88d6d7c46 X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Nov 2020 16:01:04.8373 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: f1j5Nzp0OPbS2TQ9u9WBOpFfiQY2JLSjJe1i6OS8vZG8LUiTHsMRJnYZsvwRbuMiloNJY6Jj91ycYJXd2c7RxK9SXbHQ8tLg1JR3t2EBLbA= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR07MB5986 Archived-At: Subject: Re: [Sframe] Partial decodability and IDUs X-BeenThere: sframe@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Nov 2020 16:01:12 -0000 --=-bfgKan+LDXfZqRQSXF7f Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi, If one look at this from the RTP payload level what becomes of potential us= e and in some case necessary I think are the following. The sender will put one or more IDU into independent SFRAMEs and provide th= e meta data to it that can include the following: - Media Stream + Encoder instance - Scalability layer and depdendency information - Decoding order - Associated Capture/Presentation time - timestamp for any consumption related time line (if not redundant) - If it represent a switch point (IDR, Intra etc) - Will produce output (Video marker bit type, i.e. this IDU(s) will result= in decoded output) (likely for which layers) - Reliability level (How necessary the data is, parameter sets is necessar= y for any decoding, versus a non-referenced slice).=20 The one or more IDUs is a trade-off if between overhead and what benefits t= here are to encoded different IDUs in indepdendent SFRAMES. For example encoding H.26x parameterset NALUs independently depend on how one like to use them, = or if they should be teamed with their related "slice" NALUS, in a STAP style aggregation of NALUs inside the SFRAME.=20 I think the point is to look at this what would be needed to enable the functionality in the RTP payload format and for the SFUs to be able to corr= ectly select what SFRAMEs to forward to a particular receiver.=20 If some implementation makes is easy for themselves and aggregate more in o= ne SFRAME that should be possible. It is a trade-off to a large degree between functionality, cost of doing repair of missing parts, and complexity to implement.=20 There are some basic level which should be very easy to accomplish and neve= r should be gone below becauase then one is truly throwing away the basic functionalities of RTP.=20 I still think one needs to write a bit of analysis of the different media encoders functionality and what are shared concepts, what have special quir= ks that needs to be thought about. We also I think need to find the terminolgy= for what the different aspects like I listed above really should be called and = how they are mapped to different encoders, in audio, video, tactile, etc.=20 I will note that we didn't make much headway in the RTP streams debate befo= re we actually created the taxonomy of that. People where talking past each other= s.=20 I will also note that a very interesting aspect of the reliability puzzle f= or these codecs is what you can determine about a missing RTP packet based on = that you know which SSRC and sequence number that is missing. So if you have a scalable codec and transmit all the packets on a single SSRC, then a missin= g sequence number (detected through the gap) will not tell the receiver if it needs the packet urgently or not (such a difference between a base layer or= the highest enchancement layer). In some applications that might not matter in otheres it may. So in the later cases you might want to distribute the laye= rs on different SSRCs so that one can immediately determine how important the information is. I think that possibility will exist if this RTP payload for= mat is done correctly. Cheers Magnus Westerlund --=-bfgKan+LDXfZqRQSXF7f Content-Type: application/x-pkcs7-signature; name="smime.p7s" Content-Disposition: attachment; filename="smime.p7s" Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgEFADCABgkqhkiG9w0BBwEAAKCCEtww ggYHMIID76ADAgECAhALRm3NcHtuMGWutmt5cXntMA0GCSqGSIb3DQEBCwUAMEcxCzAJBgNVBAYT AlNFMREwDwYDVQQKDAhFcmljc3NvbjElMCMGA1UEAwwcRXJpY3Nzb24gTkwgSW5kaXZpZHVhbCBD QSB2MzAeFw0xNzEyMTUwNzIyMjNaFw0yMDEyMTUwNzIyMjJaMHAxETAPBgNVBAoMCEVyaWNzc29u MRowGAYDVQQDDBFNYWdudXMgV2VzdGVybHVuZDEtMCsGCSqGSIb3DQEJARYebWFnbnVzLndlc3Rl cmx1bmRAZXJpY3Nzb24uY29tMRAwDgYDVQQFEwdlcmFtc3dkMIIBIjANBgkqhkiG9w0BAQEFAAOC AQ8AMIIBCgKCAQEAsKlHZvB3TsmLEDtPSiFFKAh73S2wApt+laqg5eXTqonqnzT9ykEGL2dx9mBT +2WZiIKxo4w2sisVl3EEYTqXTkctpur7cN29gLC8F3tJHGI2sUVpO9AwpVrN+UuHEVetHt7hdxW9 uYd0LJJ8TP6/wGkIfAFaZxlZUn79O2eHElfih1iVIiTZXLcEe1rBJtzhUNRHWgOm2vQlDJ4sCpig GFq5w+XSRviEQMkQZRvw1CQmb35QS/C/T36ogzIRHDuAdkoSaiUOY/S2dLp4HkwvOOg+tADpaHkr bdmdnjKGrYSnJigmxw14pJugxL/Vb2EeVcgpAfVVst7Lm4POPRI8+wIDAQABo4IBxDCCAcAwSAYD VR0fBEEwPzA9oDugOYY3aHR0cDovL2NybC50cnVzdC50ZWxpYS5jb20vZXJpY3Nzb25ubGluZGl2 aWR1YWxjYXYzLmNybDCBggYIKwYBBQUHAQEEdjB0MCgGCCsGAQUFBzABhhxodHRwOi8vb2NzcDIu dHJ1c3QudGVsaWEuY29tMEgGCCsGAQUFBzAChjxodHRwOi8vY2EudHJ1c3QudGVsaWFzb25lcmEu Y29tL2VyaWNzc29ubmxpbmRpdmlkdWFsY2F2My5jZXIwKQYDVR0RBCIwIIEebWFnbnVzLndlc3Rl cmx1bmRAZXJpY3Nzb24uY29tMFUGA1UdIAROMEwwSgYMKwYBBAGCDwIDAQESMDowOAYIKwYBBQUH AgEWLGh0dHBzOi8vcmVwb3NpdG9yeS50cnVzdC50ZWxpYXNvbmVyYS5jb20vQ1BTMB0GA1UdJQQW MBQGCCsGAQUFBwMEBggrBgEFBQcDAjAdBgNVHQ4EFgQU5ivuhpU51W4UhBDBWwf8XsU837YwHwYD VR0jBBgwFoAUHHsZnpecdqwgPdjc45Fq49stplMwDgYDVR0PAQH/BAQDAgWgMA0GCSqGSIb3DQEB CwUAA4ICAQBn0FKukg00UN/c/ESpxSIaYTrsd8liHHMu5rLpOBNOacpGNBMGNgUDDt4QihhoQR3c vhYXCrAM59NTvw0HNlgqHZoEeVY7YnJJYnJXDCLUfkK5Dn28E3QrzykkF6giUOXDyF9mhWYbSAkJ yx0Yj0Xc8en3wYNyoFYEqjlKtZrdV0pcgFzEeXVLS8DWrzSy7+KfUtDOEiM6H3zO3nsq++KBmsOi SKkWn4oYERZg5KElEAHis9av+3KIaEPnOAt8QRWRpFfGZ4d89F16qFvElup5n7l864FqxnC2friD o4hLQY6ENaOaYIihXhbl2UYxAGDk89aJm/S5pYyq7wzm+KK3IcUl60rmc8SJlt6QXKw0wXEOE1Mu bauYKMsad2s8jD+rEkXp+agTRl+sezWaRxHBpxuUKDd6MhwDig3SZi1qP7D/Ds4V+JLIjjUJc25l 9tvMGC9+lqI0P+vMI3Zyrou0NNfb55uLQaq18O+7BZ8Kv7jvFdxYyUgbxQ0SPEiyhylcLHAmJeLC QaiZHmCREBkCLKSf0O4lE2TrVzdOD38wjzuQ27U3UddVCD9EQ3tF7o6EVhpxJJUlB6xe/2UWwy4Z la71dKLUhakdVrN5abzxqFWvzOAT9nBa2HzYVBtpbcu6KGh72YJ+M79fa9iIkcQCgUnw3gIAeWd/ /n4YbY2QhDCCBgcwggPvoAMCAQICEAtGbc1we24wZa62a3lxee0wDQYJKoZIhvcNAQELBQAwRzEL MAkGA1UEBhMCU0UxETAPBgNVBAoMCEVyaWNzc29uMSUwIwYDVQQDDBxFcmljc3NvbiBOTCBJbmRp dmlkdWFsIENBIHYzMB4XDTE3MTIxNTA3MjIyM1oXDTIwMTIxNTA3MjIyMlowcDERMA8GA1UECgwI RXJpY3Nzb24xGjAYBgNVBAMMEU1hZ251cyBXZXN0ZXJsdW5kMS0wKwYJKoZIhvcNAQkBFh5tYWdu dXMud2VzdGVybHVuZEBlcmljc3Nvbi5jb20xEDAOBgNVBAUTB2VyYW1zd2QwggEiMA0GCSqGSIb3 DQEBAQUAA4IBDwAwggEKAoIBAQCwqUdm8HdOyYsQO09KIUUoCHvdLbACm36VqqDl5dOqieqfNP3K QQYvZ3H2YFP7ZZmIgrGjjDayKxWXcQRhOpdORy2m6vtw3b2AsLwXe0kcYjaxRWk70DClWs35S4cR V60e3uF3Fb25h3QsknxM/r/AaQh8AVpnGVlSfv07Z4cSV+KHWJUiJNlctwR7WsEm3OFQ1EdaA6ba 9CUMniwKmKAYWrnD5dJG+IRAyRBlG/DUJCZvflBL8L9PfqiDMhEcO4B2ShJqJQ5j9LZ0ungeTC84 6D60AOloeStt2Z2eMoathKcmKCbHDXikm6DEv9VvYR5VyCkB9VWy3subg849Ejz7AgMBAAGjggHE MIIBwDBIBgNVHR8EQTA/MD2gO6A5hjdodHRwOi8vY3JsLnRydXN0LnRlbGlhLmNvbS9lcmljc3Nv bm5saW5kaXZpZHVhbGNhdjMuY3JsMIGCBggrBgEFBQcBAQR2MHQwKAYIKwYBBQUHMAGGHGh0dHA6 Ly9vY3NwMi50cnVzdC50ZWxpYS5jb20wSAYIKwYBBQUHMAKGPGh0dHA6Ly9jYS50cnVzdC50ZWxp YXNvbmVyYS5jb20vZXJpY3Nzb25ubGluZGl2aWR1YWxjYXYzLmNlcjApBgNVHREEIjAggR5tYWdu dXMud2VzdGVybHVuZEBlcmljc3Nvbi5jb20wVQYDVR0gBE4wTDBKBgwrBgEEAYIPAgMBARIwOjA4 BggrBgEFBQcCARYsaHR0cHM6Ly9yZXBvc2l0b3J5LnRydXN0LnRlbGlhc29uZXJhLmNvbS9DUFMw HQYDVR0lBBYwFAYIKwYBBQUHAwQGCCsGAQUFBwMCMB0GA1UdDgQWBBTmK+6GlTnVbhSEEMFbB/xe xTzftjAfBgNVHSMEGDAWgBQcexmel5x2rCA92NzjkWrj2y2mUzAOBgNVHQ8BAf8EBAMCBaAwDQYJ KoZIhvcNAQELBQADggIBAGfQUq6SDTRQ39z8RKnFIhphOux3yWIccy7msuk4E05pykY0EwY2BQMO 3hCKGGhBHdy+FhcKsAzn01O/DQc2WCodmgR5VjtickliclcMItR+QrkOfbwTdCvPKSQXqCJQ5cPI X2aFZhtICQnLHRiPRdzx6ffBg3KgVgSqOUq1mt1XSlyAXMR5dUtLwNavNLLv4p9S0M4SIzoffM7e eyr74oGaw6JIqRafihgRFmDkoSUQAeKz1q/7cohoQ+c4C3xBFZGkV8Znh3z0XXqoW8SW6nmfuXzr gWrGcLZ+uIOjiEtBjoQ1o5pgiKFeFuXZRjEAYOTz1omb9LmljKrvDOb4orchxSXrSuZzxImW3pBc rDTBcQ4TUy5tq5goyxp3azyMP6sSRen5qBNGX6x7NZpHEcGnG5QoN3oyHAOKDdJmLWo/sP8OzhX4 ksiONQlzbmX228wYL36WojQ/68wjdnKui7Q019vnm4tBqrXw77sFnwq/uO8V3FjJSBvFDRI8SLKH KVwscCYl4sJBqJkeYJEQGQIspJ/Q7iUTZOtXN04PfzCPO5DbtTdR11UIP0RDe0XujoRWGnEklSUH rF7/ZRbDLhmVrvV0otSFqR1Ws3lpvPGoVa/M4BP2cFrYfNhUG2lty7ooaHvZgn4zv19r2IiRxAKB SfDeAgB5Z3/+fhhtjZCEMIIGwjCCBKqgAwIBAgIQU7h+g+GcmSiTsJtJHOy46zANBgkqhkiG9w0B AQsFADA3MRQwEgYDVQQKDAtUZWxpYVNvbmVyYTEfMB0GA1UEAwwWVGVsaWFTb25lcmEgUm9vdCBD QSB2MTAeFw0xNTEwMjcxMjE2NDZaFw0yNTEwMjcxMjE2NDZaMEcxCzAJBgNVBAYTAlNFMREwDwYD VQQKDAhFcmljc3NvbjElMCMGA1UEAwwcRXJpY3Nzb24gTkwgSW5kaXZpZHVhbCBDQSB2MzCCAiIw DQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBAOzy3wAAuFDyp7vYVLfGk/fjwao71MNGNLSzzl5D tjQtMtl2ZLPZyX6ViqzTN9JOb7uZ6KxuGSpReQvt8XOh7iIhkKH9W5hRpbjTsJmUMJd6zifhOpNK 6iSU3q44+FjsQL1lVtcguUuFG6aZN0N3GFVbgt6jRrASF8t/3wy9bHPAIfMyPybpg6Y2PH5/1Nwk TepoDSmK69LGV+lV2IK6U9OWayZXZFIFIDCoGyFlhFxAEgN+qZ2+Rqg/0TM0oCHvKO2ELSGmAdnJ kwizR42ji/Y9SYTSuG75mzSe6OfCGWM8Db/xvy/20aLEPXNu1PvOgzY63WZ6cmkWnjMlVJ90pWC2 haqDm3Yf8TRdjUvAl7Pz1bTuexwShzIGakL7MkCYrEqHMRaojI/VStloQgW76E76zQ2byw5QxrhO UbisBSKRzlTlOZQgYFFAbG6ViF8DOpJh/ygtQwuTLUM5r15G7eynQV1AMTNCWcX+HUvgArUw6RfW 9L58uA68GjktFTV8s9RlDsUqsNcLqeXaV28S2WMday0YGaq/bloS8AD7KuumUKH+Ri9IGO9mJvP0 5tvDHjKpLvv80c3WLJnJU/aznYHYEt2+jjKHOTqdGTxL/zMdpRSQFSuu+KM8NoYrkU1VJqKga+QL sgqKghMp99gu1P1e6KsqseWHdXORrMbjqkBXAgMBAAGjggG4MIIBtDCBigYIKwYBBQUHAQEEfjB8 MC0GCCsGAQUFBzABhiFodHRwOi8vb2NzcC50cnVzdC50ZWxpYXNvbmVyYS5jb20wSwYIKwYBBQUH MAKGP2h0dHA6Ly9yZXBvc2l0b3J5LnRydXN0LnRlbGlhc29uZXJhLmNvbS90ZWxpYXNvbmVyYXJv b3RjYXYxLmNlcjASBgNVHRMBAf8ECDAGAQH/AgEAMFUGA1UdIAROMEwwSgYMKwYBBAGCDwIDAQEC MDowOAYIKwYBBQUHAgEWLGh0dHBzOi8vcmVwb3NpdG9yeS50cnVzdC50ZWxpYXNvbmVyYS5jb20v Q1BTMEsGA1UdHwREMEIwQKA+oDyGOmh0dHA6Ly9jcmwtMy50cnVzdC50ZWxpYXNvbmVyYS5jb20v dGVsaWFzb25lcmFyb290Y2F2MS5jcmwwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMA4G A1UdDwEB/wQEAwIBBjAdBgNVHQ4EFgQUHHsZnpecdqwgPdjc45Fq49stplMwHwYDVR0jBBgwFoAU 8I9ZOACz9Y+algzV6/p7qhfoExIwDQYJKoZIhvcNAQELBQADggIBAFBYa/HVjDu0LqtXQ8iMp8PL Fpqchf41ksQY6R1AsoZbaBUu0NQlAQ9GzlC1pmI5s0cJnuaZI0xV6TiWS3/R2p9UgW61XD9CTIUb AL31mY3BdJf3P46gzKgQEca/DlFjq9GVmuPS4q90BLNgvgoxoHubc3C6s0OaY1sbnay5EhnvrAE4 Q511FlxmJPLnRmQGpieeXa3cPegFfY1kJDKyyFRypF1RuRLXcdMIgKEy5NX1bS3M9dQ4mgmUmVT2 d33UiKSEYQ6s/B+LFaaz4LywXSv2o3W4kbHoQs86IWst821ww0wxsCpEfClIvF7fBw2QkbG/1Pwu zAuLVStEhDzkAqOrMGctKyNEaBsyAn7Eq2eCa8QDXnkmagp9QPsNFs/oqnXj9j1cVtH9a4OPzhtg 0pd7gd0NzU/5QxibXqbYvouQgihGXHQDmaL4ruN7C4arMUqRo82YnREsKL7h3j/jtmzcMLc9Q07F 04QQd/iSR1Y5pIi6PdNBiE2/4uyAXS6KOIGZrPbNQUNrZtwiQpqQNl8AUzgegfPwrYFlFocpaF3d 1m5r+2VKKqiRQVfYPGYeZnWfkcz06JoAhc/9mjbHXSP9hvWYzeLRuoZqHGUdjOX9DIQb926OneV7 C5WMIjSY8ORkamG/HKqngmjypL3gSc6oG/E6B+1i6Ds5j0Qpj5aQMYICzTCCAskCAQEwWzBHMQsw CQYDVQQGEwJTRTERMA8GA1UECgwIRXJpY3Nzb24xJTAjBgNVBAMMHEVyaWNzc29uIE5MIEluZGl2 aWR1YWwgQ0EgdjMCEAtGbc1we24wZa62a3lxee0wDQYJYIZIAWUDBAIBBQCgggFDMBgGCSqGSIb3 DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTIwMTEyMDE2MDA1M1owLwYJKoZIhvcN AQkEMSIEIP9bwGZUwLboSXMX7qjdg/ynnMlATyJugshY7cSFgsE8MGoGCSsGAQQBgjcQBDFdMFsw RzELMAkGA1UEBhMCU0UxETAPBgNVBAoMCEVyaWNzc29uMSUwIwYDVQQDDBxFcmljc3NvbiBOTCBJ bmRpdmlkdWFsIENBIHYzAhALRm3NcHtuMGWutmt5cXntMGwGCyqGSIb3DQEJEAILMV2gWzBHMQsw CQYDVQQGEwJTRTERMA8GA1UECgwIRXJpY3Nzb24xJTAjBgNVBAMMHEVyaWNzc29uIE5MIEluZGl2 aWR1YWwgQ0EgdjMCEAtGbc1we24wZa62a3lxee0wDQYJKoZIhvcNAQEBBQAEggEAjuDC5a4FKtWd 9DU5U1roDW3FljoTTV3rxitcpvws0j38lzbixEBLuO/9N3bp6wMGncqKtYXFTaubJEjRqi/RKyRc oFixnRZdrXShOps3sMhUfNp0sDzZ9hENwNS3b8AIfN8KT9yu1lUHl3J7VDQw46OC4LNz56xLj3Ho U51JFdmYYS+CVn1MXC/baHuIurBGRso1AdTjC1iq1PI06V4mCYUpbvZrLSc6NYEO9FBjMbMcTS4o T0t8AB7JxKcW3zD8CTrn5U/+12haHpTnifmsYStsvL7zXmXFrhNYvUNnTffKJXAH/AOlBf61bCfE zXuVp93D0pRftRUBs55sLagrnQAAAAAAAA== --=-bfgKan+LDXfZqRQSXF7f-- From nobody Fri Nov 20 08:23:49 2020 Return-Path: X-Original-To: sframe@ietfa.amsl.com Delivered-To: sframe@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 43B423A0B5D for ; Fri, 20 Nov 2020 08:23:47 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.898 X-Spam-Level: X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=wire-com.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5CIhx-4GoNdZ for ; Fri, 20 Nov 2020 08:23:45 -0800 (PST) Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com [IPv6:2a00:1450:4864:20::531]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D8C1E3A0B5A for ; Fri, 20 Nov 2020 08:23:44 -0800 (PST) Received: by mail-ed1-x531.google.com with SMTP id q16so10098998edv.10 for ; Fri, 20 Nov 2020 08:23:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wire-com.20150623.gappssmtp.com; s=20150623; h=from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=ytuyvQGrFX74LwkQk0tis5PscocxwncrZHllxTEVwfk=; b=v68RvJ+5ajVQDmmFFsVuxy6OIGmaeNhTXw22xZpjsDDTmU6eUjw2eF0u89Y6TfOugk cGXS7qFNMiB22PJeR1Rc2qg9YpWmQDHyoVTdU30EVNfvFDi9S98VxVI4lWVjhugU6voH htv2KbPc4TF088rha/uh8ZdATLlZC7mrlM2HyY60JJiyws0H9BjSRj6VeeiOJjZn/w6l xiMclZe0/fz+RTd9+KtwPXyf3rn0CA3CFIHRqjzylAC1/fURL2BlGLlMOAt6ity6KuhK zRW86ILn6I3DCgbe6DZElwQ5XybtzLmYoQOPnk9aFtJsZ2PB/WifGYW2jSDvRPzqLCkp 8hPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=ytuyvQGrFX74LwkQk0tis5PscocxwncrZHllxTEVwfk=; b=FYzGDfwgGDmclQcVSVap5iR+1IrnAmMLvoClMKBMkqy5jk8Dcc28WXDUnl4LtbIART yWG/u0lyQcqhW7hc7ADShE5WtdGLoQfdOPfxojVWqfEO/5uZbTFJuOAPWafpWo+iapNn onvMTh2PSvCW8pcQ5yXY7AAHel5CBSCUGARfhilx8zdbGHH6kbi9YCTiHtWTB9ZOLB0W ClC1rBRBMr1hJC6ugFRAgT8JAm532G6VGFsoANI2RQgDJDAY7VvdQiaWUiSb9XrzF+ci 1GuXE+CsDbiK+7uYQ+m1InCBcJS2Hve2VucAaHUDqW/YroJjiWjX6Fkj5JFGQK4Y2oYX rTgw== X-Gm-Message-State: AOAM531tn+gwVIf+YH3fDqYb5QL4lK+SwP6doNXQfmWc6KQg53bkNTOt 9l56mZ0gf1cD3/KMcUregFrz+w== X-Google-Smtp-Source: ABdhPJyae7c2YF+ZW+YvW6Rsa8Ce4zaFxHJ0v7Xqo/XM+ADngHF/biMd17gniVL459lfmRB090QXYQ== X-Received: by 2002:aa7:dc05:: with SMTP id b5mr2506481edu.47.1605889422925; Fri, 20 Nov 2020 08:23:42 -0800 (PST) Received: from rmbp.fritz.box ([134.3.30.253]) by smtp.gmail.com with ESMTPSA id a26sm1299733edt.74.2020.11.20.08.23.41 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 20 Nov 2020 08:23:42 -0800 (PST) From: Raphael Robert Message-Id: Content-Type: multipart/alternative; boundary="Apple-Mail=_43C6DFC5-C62A-48F9-BD17-0FCB1BA71C27" Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.20.0.2.21\)) Date: Fri, 20 Nov 2020 17:23:40 +0100 In-Reply-To: Cc: sframe@ietf.org To: Richard Barnes References: X-Mailer: Apple Mail (2.3654.20.0.2.21) Archived-At: Subject: Re: [Sframe] To tree or not to tree? X-BeenThere: sframe@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Nov 2020 16:23:47 -0000 --Apple-Mail=_43C6DFC5-C62A-48F9-BD17-0FCB1BA71C27 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Thanks for the detailed write-up! I agree that option 1 is very problematic for a number of reasons and I = don=E2=80=99t see a good reason to pursue it. I think you hit the nail on the head when you asked whether we want = into-epoch forward secrecy or not. In uses cases where call participants comes together ad-hoc as an = ephemeral group, there is certainly little need for it. In other = scenarios, where a fixed set of participants have reoccurring calls it = might well be that there won=E2=80=99t be an epoch change for a = considerable time (because no one joins or leaves the group). For these = use cases option 2 & 3 come into play. Option 2 solves the problem, but the solution comes at the price of = implementation complexity. Option 3 could be much simpler, namely it would be enough to use a = unique session ID to derive a base key that is unique to the session: sframe_epoch_secret =3D MLS-Exporter("SFrame 10 MLS", "", AEAD.Nk) session_secret =3D HKDF-Expand(sframe_epoch_secret, session_id, AEAD.Nk) sender_base_key[index] =3D HKDF-Expand(session_secret, = encode_big_endian(index, 8), AEAD.Nk) This is easy to implement and efficient but a little more error-prone = since forward secrecy will only be achieved if session_id is unique = between sessions. Re-using the same value will not yield forward = secrecy. Raphael > On 20 Nov 2020, at 16:17, Richard Barnes wrote: >=20 > Hi all, >=20 > One point that was raised during the SFrame meeting at IETF 109 was = whether the MLS-SFrame integration should re-use the MLS "secret tree" = structure. =20 >=20 > For those who might not be deep in MLS, the secret tree provides = forward secrecy for MLS messages sent within an epoch, without the need = to generate base keys for all participants [1]. In other words, if you = have a group where (a) only a few participants are speaking and (b) they = ratchet their key after each message, then the secret tree assures that = compromise of any group member with current state won't leak the keys = for messages that have already been sent/received. >=20 > Let's consider three possible integrations: >=20 > 1. SFrame uses the same secret tree as MLS (export at the leaves) > 2. SFrame exports a single secret, then makes its own secret tree > 3. SFrame exports a single secret, then uses a simpler scheme (like = the current one) >=20 > I would posit that (1) is not workable. That requires a tight = coupling between the MLS and SFrame stacks, which will often not be = tractable, e.g., in situations where the media logic is in a separate = thread or process from the MLS logic. >=20 > (2) only adds value over (3) if SFrame senders ratchet their keys. = Otherwise, there's no forward secrecy boundary; criterion (b) above = doesn't apply. The current MLS-SFrame document has no provision for = ratcheting within an epoch. We could do it, but it would require more = bits of header to send a "generation" as MLS does, to indicate how many = times you've ratcheted. It also seems like situations where you have = mostly silent participants are more rare in real-time cases than in = messaging cases. >=20 > So my preference would still be for (3), largely because intra-epoch = forward secrecy seems like a pretty secondary consideration here. If = intra-epoch forward secrecy is a problem people want to solve, then we = should do ratcheting, and we should do the secret tree. >=20 > --Richard >=20 > [1] = https://github.com/mlswg/mls-protocol/blob/master/draft-ietf-mls-protocol.= md#secret-tree-secret-tree = --=20 > Sframe mailing list > Sframe@ietf.org > https://www.ietf.org/mailman/listinfo/sframe --Apple-Mail=_43C6DFC5-C62A-48F9-BD17-0FCB1BA71C27 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Thanks for the detailed write-up!

I agree that option 1 is very = problematic for a number of reasons and I don=E2=80=99t see a good = reason to pursue it.
I think you hit the nail on = the head when you asked whether we want into-epoch forward secrecy or = not.

In uses = cases where call participants comes together ad-hoc as an ephemeral = group, there is certainly little need for it. In other scenarios, where = a fixed set of participants have reoccurring calls it might well be that = there won=E2=80=99t be an epoch change for a considerable time (because = no one joins or leaves the group). For these use cases option 2 & 3 = come into play.
Option 2 solves the problem, but = the solution comes at the price of implementation complexity.
Option 3 could be much simpler, namely it would be enough to = use a unique session ID to derive a base key that is unique to the = session:

sframe_epoch_secret =3D MLS-Exporter("SFrame 10 MLS", "", = AEAD.Nk)
session_secret =3D = HKDF-Expand(sframe_epoch_secret, session_id, AEAD.Nk)
sender_base_key[index] =3D = HKDF-Expand(session_secret, encode_big_endian(index, 8), = AEAD.Nk)

This = is easy to implement and efficient but a little more error-prone since = forward secrecy will only be achieved if session_id is unique between = sessions. Re-using the same value will not yield forward = secrecy.

Raphael

On 20 Nov 2020, at 16:17, Richard Barnes = <rlb@ipv.sx> = wrote:

Hi all,

One point that was raised during the SFrame meeting at IETF = 109 was whether the MLS-SFrame integration should re-use the MLS "secret = tree" structure. 

For those who might not be deep in MLS, = the secret tree provides forward secrecy for MLS messages sent within an = epoch, without the need to generate base keys for all participants = [1].  In other words, if you have a group where (a) only a few = participants are speaking and (b) they ratchet their key after each = message, then the secret tree assures that compromise of any group = member with current state won't leak the keys for messages that have = already been sent/received.

Let's consider three possible integrations:

1. SFrame uses the same = secret tree as MLS (export at the leaves)
2. SFrame exports a single secret, then makes its own secret = tree
3. SFrame exports a single = secret, then uses a simpler scheme (like the current one)

I = would posit that (1) is not workable.  That requires a tight = coupling between the MLS and SFrame stacks, which will often not be = tractable, e.g., in situations where the media logic is in a separate = thread or process from the MLS logic.

(2) only adds value over (3) if SFrame = senders ratchet their keys.  Otherwise, there's no forward secrecy = boundary; criterion (b) above doesn't apply.  The current = MLS-SFrame document has no provision for ratcheting within an = epoch.  We could do it, but it would require more bits of header to = send a "generation" as MLS does, to indicate how many times you've = ratcheted.  It also seems like situations where you have mostly = silent participants are more rare in real-time cases than in messaging = cases.

So my preference would still be for (3), largely because = intra-epoch forward secrecy seems like a pretty secondary consideration = here.  If intra-epoch forward secrecy is a problem people want to = solve, then we should do ratcheting, and we should do the secret = tree.

--Richard

[1] https://github.com/mlswg/mls-protocol/blob/master/draft-ietf-ml= s-protocol.md#secret-tree-secret-tree
--
Sframe mailing list
Sframe@ietf.org
https://www.ietf.org/mailman/listinfo/sframe

= --Apple-Mail=_43C6DFC5-C62A-48F9-BD17-0FCB1BA71C27-- From nobody Sun Nov 22 14:48:25 2020 Return-Path: X-Original-To: sframe@ietfa.amsl.com Delivered-To: sframe@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E334F3A0EB5 for ; Sun, 22 Nov 2020 14:48:23 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.001 X-Spam-Level: X-Spam-Status: No, score=0.001 tagged_above=-999 required=5 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, NICE_REPLY_A=-0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=cosmosoftware-io.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7h6R8unxE9yo for ; Sun, 22 Nov 2020 14:48:22 -0800 (PST) Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 823123A0EB4 for ; Sun, 22 Nov 2020 14:48:22 -0800 (PST) Received: by mail-wr1-x430.google.com with SMTP id o15so16831289wru.6 for ; Sun, 22 Nov 2020 14:48:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cosmosoftware-io.20150623.gappssmtp.com; s=20150623; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding:content-language; bh=5hy6CksF0eS/3iCmLVZ8y917Z+D5KCQ9Citn3q3GNjg=; b=wiMyJ8vLb2frGWcEYDVwtGjB40MunR3P1BpN8ML5cb/kG11MYbwoc78AqHE0pz9/Ki JES7O8aDUlzCN1M1syIgGZhHdz6pbUHU0KqwvInAnxUQArn7IFO3W8HpewXiOp3KEchN G8kGGMSocXNZKZOMWVRT3cK+an68lC5RrST+WFdd5pDsv8cGp0YKTPdsGdQInXAw3jlV pkSi7bHkx+Q+g/RX49x+mJ1GMVD8h1aJ1CtSGpHj04nZvQwwVxxHEJLzVytT2lw0MJBR ven9VfsXaMG4/qV6uRjZRXRQQ+IlbvD6SvQT3v+74TZgnWrgmHw911NdMPdM9MoIXVuZ ov7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=5hy6CksF0eS/3iCmLVZ8y917Z+D5KCQ9Citn3q3GNjg=; b=F6Is6OgsF9dMofWHmqUOLIomSmdQj/Ks9VPeSVRuzz5e7zhmsJAHzICV3E2qlJ9dKY fkmn0sIAfI3JVxqK1L1SjfAfE0IstKK3heqtkweAcxe3AXnSneqQFj+BYA9roe7RbTxF S6MRybqZ3eaioGD1NcknB7Zp8vtftquuga0ULoL/zcvK0es8rD2o/UscCgtUnhVs4mfG KAssZtC/375TbCJSJSBgWILzRJO9KTRA7udQcKwA66DzqhdH8+UnJ1Np5acO37mZ1UB6 +ZnD58W4xfcVMUWlSXT2YuoYghESsReZas49BwyF/yMM6Li09KFp3lON4BE0BkSu3uiA PUyg== X-Gm-Message-State: AOAM531NRxoa/jD9U1M1+gwRXtWv1rQ8MCWauZltlSkC1rIlJRlko84d Wda0JElXCcHdPrxTzXM4iwRHmU4FbMH7Og== X-Google-Smtp-Source: ABdhPJxH6AfJnZEc4m77mQpTy0aVHKB0zEGKZvQ0sWpRwZ0JLXdbz+eU5QllZ4dNepYGe93IR8Q36Q== X-Received: by 2002:adf:f48c:: with SMTP id l12mr4626067wro.280.1606085300235; Sun, 22 Nov 2020 14:48:20 -0800 (PST) Received: from [192.168.0.11] (79.108.125.160.dyn.user.ono.com. [79.108.125.160]) by smtp.gmail.com with ESMTPSA id f20sm12487701wmc.26.2020.11.22.14.48.19 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 22 Nov 2020 14:48:19 -0800 (PST) To: Magnus Westerlund , "bernard.aboba@gmail.com" Cc: "rlb@ipv.sx" , "sframe@ietf.org" References: <7eadfc25-d326-3a02-f88f-bfdc02b9a42c@cosmosoftware.io> <2752b6f0-ec61-6a19-e17c-0347c451a17c@cosmosoftware.io> From: Sergio Garcia Murillo Message-ID: Date: Sun, 22 Nov 2020 23:48:18 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US Archived-At: Subject: Re: [Sframe] Partial decodability and IDUs X-BeenThere: sframe@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Nov 2020 22:48:24 -0000 On 20/11/2020 17:01, Magnus Westerlund wrote: > Hi, > > If one look at this from the RTP payload level what becomes of potential use and > in some case necessary I think are the following. > > The sender will put one or more IDU into independent SFRAMEs and provide the > meta data to it that can include the following: > - Media Stream + Encoder instance > - Scalability layer and depdendency information > - Decoding order > - Associated Capture/Presentation time > - timestamp for any consumption related time line (if not redundant) > - If it represent a switch point (IDR, Intra etc) > - Will produce output (Video marker bit type, i.e. this IDU(s) will result in > decoded output) (likely for which layers) > - Reliability level (How necessary the data is, parameter sets is necessary for > any decoding, versus a non-referenced slice). > > The one or more IDUs is a trade-off if between overhead and what benefits there > are to encoded different IDUs in indepdendent SFRAMES. For example encoding > H.26x parameterset NALUs independently depend on how one like to use them, or if > they should be teamed with their related "slice" NALUS, in a STAP style > aggregation of NALUs inside the SFRAME. > > I think the point is to look at this what would be needed to enable the > functionality in the RTP payload format and for the SFUs to be able to correctly > select what SFRAMEs to forward to a particular receiver. From my point of view the that the partial decodability is not something that the SFU cares about. The only RTCP recovery I am aware of that could apply for IDUs is the slice loss indication which requires macroblock information, which is a way lower information layer than the one SFUs works with (it may not even be available if payload is encrypted). Moreover, the SFU will not rely the SLIs received from the receiving endpoint as it has no way of tracking SLIs requests from endpoints and map them to already sent ones in order to avoiding overflowing the encoder with SLIs requests. So I am afraid that the theoretical benefit of partial decoding would be reduced a lot in the typical usage of SFrame (i.e. no p2p use case). Also, before adding support from it, I would like to get real implementation feedback as it is currently not implemented in (lib)webrtc and I have not heard anynone asking for it ever. > If some implementation makes is easy for themselves and aggregate more in one > SFRAME that should be possible. It is a trade-off to a large degree between > functionality, cost of doing repair of missing parts, and complexity to > implement. And also complexity to specify. > There are some basic level which should be very easy to accomplish and never > should be gone below becauase then one is truly throwing away the basic > functionalities of RTP. > > I still think one needs to write a bit of analysis of the different media > encoders functionality and what are shared concepts, what have special quirks > that needs to be thought about. We also I think need to find the terminolgy for > what the different aspects like I listed above really should be called and how > they are mapped to different encoders, in audio, video, tactile, etc. > > I will note that we didn't make much headway in the RTP streams debate before we > actually created the taxonomy of that. People where talking past each others. Are you proposing to do something like rfc7656 but for video codecs? I think that would be great, but I also understand that it would be an different draft than the packetization or description header, right? > I will also note that a very interesting aspect of the reliability puzzle for > these codecs is what you can determine about a missing RTP packet based on that > you know which SSRC and sequence number that is missing. So if you have a > scalable codec and transmit all the packets on a single SSRC, then a missing > sequence number (detected through the gap) will not tell the receiver if it > needs the packet urgently or not (such a difference between a base layer or the > highest enchancement layer). In some applications that might not matter in > otheres it may. So in the later cases you might want to distribute the layers on > different SSRCs so that one can immediately determine how important the > information is. I think that possibility will exist if this RTP payload format > is done correctly. While I think this information is very valuable (in fact, it is a must have from the SFU perspective), I think it belongs to the SFU information header extension part and not the packetization part (although they could be implemented together). Best regards Sergio From nobody Sun Nov 22 15:10:58 2020 Return-Path: X-Original-To: sframe@ietfa.amsl.com Delivered-To: sframe@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1D71C3A0F47 for ; Sun, 22 Nov 2020 15:10:57 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.198 X-Spam-Level: X-Spam-Status: No, score=-0.198 tagged_above=-999 required=5 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nAt4rBEF39zI for ; Sun, 22 Nov 2020 15:10:56 -0800 (PST) Received: from mail-pl1-x629.google.com (mail-pl1-x629.google.com [IPv6:2607:f8b0:4864:20::629]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E8BC93A0F46 for ; Sun, 22 Nov 2020 15:10:55 -0800 (PST) Received: by mail-pl1-x629.google.com with SMTP id l11so7934098plt.1 for ; Sun, 22 Nov 2020 15:10:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:content-transfer-encoding:mime-version:subject:date:message-id :references:cc:in-reply-to:to; bh=USIaPKUfjOM7rj/NV+H4RK3NePTxwUpHwxHMZ+U0sA4=; b=uEfX4LII0r6uef4EPWcBCAvAw4COpUXIPY0x0dbaPHEHMtgb+YLl+2XByBaIlye65B HXJIVBSw2+QzRP0Ti5uRIJS56ItS8xgQ1ThaF+R5vnD6Av/EmyuZ6RucBF2YhDe8jQX9 VViGXIy28yuR/1ekbk32YllAOy2sKicPRrQ8FvUHKmIvp4BPBN2Cx3FVCbzh99M7g4Mu HGdZihN+BUUwLMnT3Hti9g+P9L10j2SBoS185ouIAp+rolnH6Ziy/hMZNVCgyIhXBrUl Hgv/5MEJ7pOT3tQ1iWD3tc794q9CPElRFo40GEncX8D8qnqF5xAzKhAPp5yX412B+6AZ sV6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:content-transfer-encoding:mime-version :subject:date:message-id:references:cc:in-reply-to:to; bh=USIaPKUfjOM7rj/NV+H4RK3NePTxwUpHwxHMZ+U0sA4=; b=BljzrDv5WpQfPadV0sxrmktrbmJYyV+4u2Y5wvplTmtcX1TtJWl5328KqJAR3y5DvS 7vn1Yz+FLDCHFsL2O7R5YYOTE3g8VDNHSA4QHpIy/peQfu6lJEP3o34PwasfqTV5QUPs 1jahtDt7QzmPI7EzfA/Mjd1wbAgNIcDsKmhgnjtFTMcnoDiPZ1mt1ldFDXZ9SQG68hBm D3r+FSWmJwD4ZUO7rUZQZx+AU8NPDK7t9isUmA5V1Jv7wuwHGkigmUfwmnr8als8PnRS oZMakDXBcjN+JsyHZOhYDzNIbJyGNdtbJnqYZIMtQve9KXmOAH/GOu5mhGKCEBHM98a7 Rqjg== X-Gm-Message-State: AOAM530kWqu/7ZbtdY+zpE01QLkPT7jA0gzrVujoqZdZiLxhwTkHyRex ya+mMaf4J23g11VvvSHBFTJ5sEXzfx1OAw== X-Google-Smtp-Source: ABdhPJzqmLzKm3KkBWU8FMntJZC7pMFpHyHaSi/XZDpItC3mAAmeyX2Eu69NwqTkDnB735XsYrsnJw== X-Received: by 2002:a17:902:e787:b029:d9:f88d:c32d with SMTP id cp7-20020a170902e787b02900d9f88dc32dmr5858639plb.79.1606086654818; Sun, 22 Nov 2020 15:10:54 -0800 (PST) Received: from [192.168.1.121] (c-71-227-236-207.hsd1.wa.comcast.net. [71.227.236.207]) by smtp.gmail.com with ESMTPSA id u24sm9948777pfm.51.2020.11.22.15.10.53 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 22 Nov 2020 15:10:54 -0800 (PST) From: Bernard Aboba X-Google-Original-From: Bernard Aboba Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (1.0) Date: Sun, 22 Nov 2020 15:10:52 -0800 Message-Id: References: Cc: Magnus Westerlund , rlb@ipv.sx, sframe@ietf.org In-Reply-To: To: Sergio Garcia Murillo X-Mailer: iPhone Mail (18B92) Archived-At: Subject: Re: [Sframe] Partial decodability and IDUs X-BeenThere: sframe@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Nov 2020 23:10:57 -0000 On Nov 22, 2020, at 2:48 PM, Sergio Garcia Murillo wrote: >=20 > The only RTCP recovery I am aware of that could apply for IDUs is the slic= e loss indication which requires macroblock information, which is a way lowe= r information layer than the one SFUs works with (it may not even be availab= le if payload is encrypted). >=20 > Moreover, the SFU will not rely the SLIs received from the receiving endpo= int as it has no way of tracking SLIs requests from endpoints and map them t= o already sent ones in order to avoiding overflowing the encoder with SLIs r= equests. >=20 > Also, before adding support from it, I would like to get real implementati= on feedback as it is currently not implemented in (lib)webrtc and I have not= heard anynone asking for it ever. [BA] Although RPSI and SLI Sections are included in several codec documents,= I have not seen them implemented. We do not support SLI or RPSI in our SFU.= The reasoning is that with RTX and FEC and other messages supported in WebR= TC (NAK, PLI, FIR), then SLI and RPSI do not provide incremental value.=20 >> So in the later cases you might want to distribute the layers on differen= t SSRCs so that one can immediately determine how important the >> information is. I think that possibility will exist if this RTP payload f= ormat is done correctly. [BA] We do support MRST with H.264/SVC, and found some advantages in doing s= o (not having to rewrite sequence numbers in an SFU). However MRST is not su= pported in VP8, VP9, AV1 and most H.264/SVC implementations, and the perform= ance cost of translation outweighed the forwarding performance advantages.= From nobody Mon Nov 23 05:45:04 2020 Return-Path: X-Original-To: sframe@ietfa.amsl.com Delivered-To: sframe@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 48AEF3A0B1C for ; Mon, 23 Nov 2020 05:45:02 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0 X-Spam-Level: X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, NICE_REPLY_A=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=wickr-com.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xQJHyOunjpYk for ; Mon, 23 Nov 2020 05:45:00 -0800 (PST) Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9864B3A0B1F for ; Mon, 23 Nov 2020 05:45:00 -0800 (PST) Received: by mail-wr1-x429.google.com with SMTP id s8so18609371wrw.10 for ; Mon, 23 Nov 2020 05:45:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wickr-com.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-language:content-transfer-encoding; bh=3VD31X4Xxj5g45yo3SoyeF6457Cj3fgW6wELlpTpp54=; b=QCOA4SewKhRZ4a9NSMxAUtI0yKYxyHTv46g87gEeczo4y4lePa+CA+DyqNorE52Hho EEtmoX+QVZoM+pkykBzgM6+71vpGpEiatymuAddPCG0bgVrusBKBKfYdFw46ppB2lhFl Yi/izE9kpKnRgqSzR+6U2XT8gZm6/FgF42+dR3GDSP9W/OpytaoFy2tcDIISuFIyPXPM bVVlu/vkRnMJobwT2NO2wH55jS8bdOaVqQyJ5KacoVPWOv5jG1BC30FX3uRRWlCq6uBn mpVq0DhuaruNGMhBaylS6lifxXnAtB3gLaMEeQT8f8ZJpWPn7yohF62rWzIvF13P801j jIwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=3VD31X4Xxj5g45yo3SoyeF6457Cj3fgW6wELlpTpp54=; b=WX/ZoCis5YqssMlO5aSAIVzQHQzlIsYRcCTODV0NeD6zs/bThgsIkeE11eZGpkmXOb EHEJaVxFCfSwsTyey8SInFJcOz+bL0Jle/65lgpo9uWuqdyrRt8NFU1MGMcSvJuiDF78 27CqbQV+pTHWC/oTTrDbElDtR4uKDF/n2gPPubgQBUyxq1txSBTxZI8bbb2JQoDObZLp xORRdUXs79k1wZ6O1NPsQ1DQz0vcRS2qX6xAWbPhGtDvllU/zZcxjycibBQ11DoK95Fu vBPVEaEbTxeZxkpYs6yPEnNpU3VIq6N6+pnGD/48Y5Sb/D0ZZFo7qM/DPOCMOjfkh4lk gZfg== X-Gm-Message-State: AOAM530+WQST7nld7RaEeeFRaqxRZtdA5GTKAkCCY0thw+9qiYRA8/gi 3a7BPvkgC4im3OnbU2x/hOiiJtH8C1lnzA== X-Google-Smtp-Source: ABdhPJz4gP+qQwrVexymDLBe/LT2IFV5TBcuaByN/ifN6zJjyVJxoczddfovGY3rrPIeVP2yQAwsxQ== X-Received: by 2002:adf:f3d1:: with SMTP id g17mr7844642wrp.201.1606139098843; Mon, 23 Nov 2020 05:44:58 -0800 (PST) Received: from [192.168.1.137] (84-114-27-5.cable.dynamic.surfer.at. [84.114.27.5]) by smtp.gmail.com with ESMTPSA id z6sm17224741wmi.1.2020.11.23.05.44.57 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 23 Nov 2020 05:44:58 -0800 (PST) To: sframe@ietf.org, Raphael Robert , Richard References: From: Joel Alwen Message-ID: Date: Mon, 23 Nov 2020 14:44:56 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Archived-At: Subject: Re: [Sframe] To tree or not to tree? X-BeenThere: sframe@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Nov 2020 13:45:02 -0000 To start with, I think forward secrecy (FS) within epochs can be valuable for SFrame. (E.g. for long or repeating convo's with the same set of participants.) Further responses are inline... > Raphael Robert Fri, 20 November 2020 16:23 UTC In other > scenarios, where a fixed set of participants have reoccurring calls it might > well be that there won’t be an epoch change for a considerable time (because > no one joins or leaves the group). > ... > Option 3 could be much simpler, namely it would be enough to use a unique > session ID to derive a base key that is unique to the session: > > sframe_epoch_secret = MLS-Exporter("SFrame 10 MLS", "", AEAD.Nk) > session_secret = HKDF-Expand(sframe_epoch_secret, session_id, AEAD.Nk) > sender_base_key[index] = HKDF-Expand(session_secret, > encode_big_endian(index, 8), AEAD.Nk) > > This is easy to implement and efficient but a little more error-prone since > forward secrecy will only be achieved if session_id is unique between > sessions. Re-using the same value will not yield forward secrecy. I might not get what your saying, but if I did then I don't think this gives any FS, even between sessions, as long as they use the same value for sframe_epoch_secret regardless of the choices of session_id. (And if they dont use the same sframe_epoch_secret value then I don't see how the value of session_id matters for FS.) Suppose sframe_epoch_secret is used to derive sender_base_key[i]. After session i is done participants still need to be able to reproduce sframe_epoch_secret if they want derive sender_base_key[i+1] for session i+1. In that case they could just as well recompute sender_base_key[i] again too. So no there's no FS for session i. Alternatively they could pre-compute all sender_key_base values for potential future epochs but that seems inelegant to me. Bellow I describe my preferred solution to precomputing and storing all potentially useful sender_key_base values. >> On 20 Nov 2020, at 16:17, Richard Barnes wrote: >> 1. SFrame uses the same secret tree as MLS (export at the leaves) >> 2. SFrame exports a single secret, then makes its own secret tree >> 3. SFrame exports a single secret, then uses a simpler scheme (like the current one) ATM I'm leaning towards option 2 or 3 (though not with a "simpler" schedule, just different one more aligned with SFrame use cases). When it comes to an SFrame epoch (i.e. a time period during which no one joins/leaves the session) I think there are two constraints that informing which FS symmetric key schedules are acceptable: a. Parties shouldn't need to explicitly coordinate who will use which key/nonce pairs during the epoch. b. Ciphertexts in the media streams may be dropped and/or delivered out of order. The tree-based symmetric key derivation hierarchy in MLS was explicitly designed to accommodate these constraints and while being relatively efficient so, to me, it looks like a good solution for SFrame as well. Thus option 2 doesnt sound bad to me. Having said that, Raphael highlights the case where epochs may be broken up into sessions (e.g. when the same set of parties on an old call want to resume their convo in a new call). If that's an important enough scenario to cater too then the following Option 3 solution might be better. Namely, use one binary tree per session and string the roots of the trees together in a symmetric ratchet (i.e. a chain). This would minimize the state participants need to maintain between sessions (effectively a single secret) while still giving us FS between (and within) sessions despite them being part of the same epoch. For the sake of concreteness: chain_secret[0] = MLS-Exporter("SFrame 10 MLS", "", AEAD.Nk) chain_secret[i] = HKDF-Expand(chain_secret[i-1], "chain", AEAD.Nk) session_secret[i] = HKDF-Expand(chain_secret[i], "sesion"+session_id, AEAD.Nk) and session_secret[i] is the secret at the root of the i-th sessions tree-based symmetric key schedule. The j-th leaf of the tree initiates a symmetric ratchet that defines the key/nonce pairs used to encrypt the j-th SFrame stream in the session. So packet number k of SFrame stream j in session i is encrypted using the key/nonce pair at position k in symmetric ratchet rooted at the leaf j of the i-th tree; namely the one rooted at secret session_secret[i]. Yet between session i and i+1 participants only need to store chain_secret[i+1]. - Joël