Subject: [avtext] Some comments on draft-berger-avtext-framemarking

From: Bernard Aboba <[email-masked]>
Date: Mon, 23 Mar 2015 12:17:26 -0700

The basic idea of this draft seems sound to me, and most of  the fields proposed for the 'Mandatory Extension' (Section 2.1) also make sense to me (with the possible exception of the "B" bit, since for temporal scalability this should be derivable from the TID). 
 
However, I did want to comment on the Optional Extensions (Section 2.2) section, where additional fields such as the PictureID and H265-LayerId are discussed. 
 
In particular, I would argue for the following additional information within Optional Extensions: 
 
Additional Layer Identifiers beyond the TID
 
Layer identifiers enable the SFU to determine the temporal,
      spatial or quality layer of a given packet, which is used in
      forwarding/drop decisions.  Within H.264/SVC this information is
      present in the DID(3 bits), QID (4 bits) and TID (3 bits) fields.
      Within VP8, it is present in the TID field (2 bits).  Within
      H.265, it is present in the LayerID (6 bits) and TID (3 bits)
      fields.

   TL0PICIDX

      This field enables an SFU to determine whether a received frame is
      dependent on a base layer frame which the SFU has not previously
      received in its entirety.  This is important information since it
      may represent the first indication that all or part of a base
      layer frame needs to be recovered before the arrival of subsequent
      base layer frames that depend on it.  Within H.264/SVC the
      IDRPICID and TL0PICIDX fields are present if the Y bit is set.
      Within VP8, the TL0PICIDX field is present if the Y bit is set.

However, I would argue against the PictureID field, for the following reasons: 

While the PictureID field is utilized within the RPSI feedback
      message, as well as being supported within the VP8 and VP9 codecs,
      its use is not essential to an SFU.  RPSI feedback messages are rarely supported within
      existing SVC and simulcast implementations, since better loss
      recovery mechanisms are available.  As a result, an SFU does not
      need to originate an RPSI message which would require the
      PictureID.    Where it is necessary for the SFU to determine which
      picture a given packet belongs to, RTP header fields such as the
      timestamp and sequence number can be used instead.

Similarly, I don't believe that Macroblock identifiers should be considered (the draft doesn't propose this, but sometimes beating a not-yet-dead horse is useful): 
 
While macroblock identifiers are universally supported within
      video codecs, they are not used by SFUs today. SLI feedback messages (which
      require a starting macroblock) are rarely supported within
      existing SFUs, since better loss recovery mechanisms are
      available.  Therefore a codec-independent SFU should never need to
      source an SLI message.

---

From: "Mo Zanaty (mzanaty)" <[email-masked]>
Date: Tue, 24 Mar 2015 22:05:15 +0000

Hi Bernard,

The "B” bit is for base layer sync, like the “Y” bit in VP8 (but unlike the “Y” bit in H.264 SVC PACSI). It indicates this packet only depends on the base layer. This would only be derivable from TID if TID=0. If TID>0, there is no way to derive this, it must be signaled explicitly.

The codec-specific extensions are admittedly incomplete. At a minimum, we need to define H.264 (SVC), H.265 (v2/SHVC), VP8, and VP9. The key question to answer for those codecs is what info is critical for RTP switch operations. Info that is only useful for endpoints should not be necessary in the header extension since it can be extracted from the payload.

I agree that for H.264 (SVC), this should include DID and QID; and for H.265, LayerID. I also agree PictureID may be unnecessary. I’m not sure TL0PICIDX is necessary, but agree we should explore it.

Regards,
Mo

On 3/23/15, 3:17 PM, Bernard Aboba <[email-masked]<mailto:[email-masked]>> wrote:

Layer identifiers enable the SFU to determine the temporal, spatial or quality layer of a given packet, which is used in forwarding/drop decisions. Within H.264/SVC this information is present in the DID(3 bits), QID (4 bits) and TID (3 bits) fields. Within VP8, it is present in the TID field (2 bits). Within H.265, it is present in the LayerID (6 bits) and TID (3 bits) fields.

TL0PICIDX

This field enables an SFU to determine whether a received frame is dependent on a base layer frame which the SFU has not previously received in its entirety. This is important information since it may represent the first indication that all or part of a base layer frame needs to be recovered before the arrival of subsequent base layer frames that depend on it. Within H.264/SVC the IDRPICID and TL0PICIDX fields are present if the Y bit is set. Within VP8, the TL0PICIDX field is present if the Y bit is set.

While the PictureID field is utilized within the RPSI feedback message, as well as being supported within the VP8 and VP9 codecs, its use is not essential to an SFU.  RPSI feedback messages are rarely supported within existing SVC and simulcast implementations, since better loss recovery mechanisms are available. As a result, an SFU does not need to originate an RPSI message which would require the PictureID. Where it is necessary for the SFU to determine which picture a given packet belongs to, RTP header fields such as the timestamp and sequence number can be used instead.

While macroblock identifiers are universally supported within video codecs, they are not used by SFUs today. SLI feedback messages (which require a starting macroblock) are rarely supported within existing SFUs, since better loss recovery mechanisms are available. Therefore a codec-independent SFU should never need to source an SLI message.