Subject: [avtext] Frame marking & VP9 SVC

From: Sergio Garcia Murillo <[email-masked]>
Date: Tue, 28 Mar 2017 15:54:24 +0200

Hi,

I have been implementing the framemarking for PERC and I have some 
doubts regarding the VP9 LID mapping.

First, is that according to 
https://tools.ietf.org/html/draft-ietf-payload-vp9-03 there are no 
quality layers on VP9 SVC they are considered as case of spatial layers:

    VP9 supports quality layers as spatial layers without any resolution
    changes; hereinafter, the term "spatial layer" is used to represent
    both spatial and quality layers.

So, shouldn't the LID reference to the spatial layer ID only  and omit 
the quality layer id completely? Also, the spatial layer id is 3 bits on 
that draft.

Also, in order to be able to implement VP9 SVC layer selection, the SFU 
needs the information of the P and U bits of the VP9 header description:

    P: Inter-picture predicted layer frame.  When set to zero, the layer
       frame does not utilize inter-picture prediction.  In this case,
       up-switching to current spatial layer's frame is possible from
       directly lower spatial layer frame.  P SHOULD also be set to zero
       when encoding a layer synchronization frame in response to an LRR
       [I-D.ietf-avtext-lrr 
<https://tools.ietf.org/html/draft-ietf-payload-vp9-03#ref-I-D.ietf-avtext-lrr>] message (seeSection 5.4 
<https://tools.ietf.org/html/draft-ietf-payload-vp9-03#section-5.4>).  When P is set to
       zero, the T bit (described below) MUST also be set to 0 (if
       present).

    U: Switching up point.  If this bit is set to 1 for the current
       frame with temporal layer ID equal to T, then "switch up" to a
       higher frame rate is possible as subsequent higher temporal
       layer frames will not depend on any frame before the current
       frame (in coding time) with temporal layer ID greater than T.

Without that info, the SFU won't be able to upscale, so it would be 
required to add that information on the VP9 LID.
Including that changes, it would be something like this:

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |  ID=2 |  L=2  |S|E|I|D|B| TID |0|0|0|P|U|SID  |    TL0PICIDX  |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Best regards
Sergio

---

From: "Mo Zanaty (mzanaty)" <[email-masked]>
Date: Tue, 28 Mar 2017 16:53:39 +0000

Hi Sergio,

Good catch. The VP9 payload format has changed since the Frame Marking LID extensions were defined. The VP9 authors expect it may change further. Based on discussion during today's AVT session, we will remove VP9 from the Frame Marking draft (which will help avoid MIS-REF later), and move that information to the VP9 payload format so it can evolve without diverging.

The discussion on U and P bits was more nuanced. The main point is Frame Marking should help identify when layer refresh has occurred and when layer up switching is possible. We will add some text on this for simple, fixed scalability structures, and note that complex, non-fixed structures require payload-specific inspection and logic that no extra bits will completely eliminate.

Cheers,
Mo

From: avtext <[email-masked]<mailto:[email-masked]>> on behalf of Sergio Murillo <[email-masked]<mailto:[email-masked]>>
Date: Tuesday, March 28, 2017 at 9:54 AM
To: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: [avtext] Frame marking & VP9 SVC

Hi,

I have been implementing the framemarking for PERC and I have some doubts regarding the VP9 LID mapping.

First, is that according to https://tools.ietf.org/html/draft-ietf-payload-vp9-03 there are no quality layers on VP9 SVC they are considered as case of spatial layers:

So, shouldn't the LID reference to the spatial layer ID only  and omit the quality layer id completely? Also, the spatial layer id is 3 bits on that draft.

Also, in order to be able to implement VP9 SVC layer selection, the SFU needs the information of the P and U bits of the VP9 header description:

      [I-D.ietf-avtext-lrr<https://tools.ietf.org/html/draft-ietf-payload-vp9-03#ref-I-D.ietf-avtext-lrr>] message (see Section 5.4<https://tools.ietf.org/html/draft-ietf-payload-vp9-03#section-5.4>).  When P is set to

Without that info, the SFU won't be able to upscale, so it would be required to add that information on the VP9 LID.
Including that changes, it would be something like this:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |  ID=2 |  L=2  |S|E|I|D|B| TID |0|0|0|P|U| SID |    TL0PICIDX  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Best regards
Sergio

---

From: Sergio Garcia Murillo <[email-masked]>
Date: Tue, 28 Mar 2017 19:02:48 +0200

Hi Mo,

Thanks for checking it out.

Just one comment, please take into account that in PERC context the 
payload is encrypted, so the SFU will not have access to the media 
payload and will only be able to rely on the frame marking extensions to 
perform the selective forwarding.

Best regards
Sergio

On 28/03/2017 18:53, Mo Zanaty (mzanaty) wrote:
> Hi Sergio,
>
> Good catch. The VP9 payload format has changed since the Frame Marking 
> LID extensions were defined. The VP9 authors expect it may change 
> further. Based on discussion during today's AVT session, we will 
> remove VP9 from the Frame Marking draft (which will help avoid MIS-REF 
> later), and move that information to the VP9 payload format so it can 
> evolve without diverging.
>
> The discussion on U and P bits was more nuanced. The main point is 
> Frame Marking should help identify when layer refresh has occurred and 
> when layer up switching is possible. We will add some text on this for 
> simple, fixed scalability structures, and note that complex, non-fixed 
> structures require payload-specific inspection and logic that no extra 
> bits will completely eliminate.
>
> Cheers,
> Mo
>
> From: avtext <[email-masked] 
> <mailto:[email-masked]>> on behalf of Sergio Murillo 
> <[email-masked] <mailto:[email-masked]>>
> To: "[email-masked] <mailto:[email-masked]>" <[email-masked] 
> <mailto:[email-masked]>>
> the quality layer id completely? Also, the spatial layer id is 3 bits 
> on that draft.
>
> Also, in order to be able to implement VP9 SVC layer selection, the 
> SFU needs the information of the P and U bits of the VP9 header 
> description:
>