Subject: Re: [payload] Comments on draft-ietf-payload-rtp-h265-02

From: "Wang, Ye-Kui" <[email-masked]>
Date: Mon, 3 Mar 2014 19:07:06 +0000

Comments 17-22 are on PACI, which is Stephan's baby, and I will thus let Stephan to address them.

My responses to comments 23 upwards are interleaved below (again, for convenience, I only included comments 23 upwards).

BR, YK

23. Section 5:
Otherwise (sprop-
      depack-buf-nalus  is  equal  to  0  for  an  RTP  stream),  the
      transmission order of NAL units carried in the RTP stream MUST be
      the same as the NAL unit decoding order.

This fails to describe how you determines the NAL unit decoding order. I assume in RTP sequence order and for each payload in the order the NALUs are included?

[YK]: No, the NAL unit decoding order is described by the de-packetization process in Section 6, which applies in all cases, regardless of sprop-depack-buf-nalus being 0 or greater than 0. Yes, when sprop-depack-buf-nalus is equal to 0, the order would be as you assumed above. I don't see anything wrong or missing in this regard. However, if you think more clarification is needed anywhere, please suggest and let us know where.

24. Section 6:

When MST is in use, NAL units of all RTP
   streams are stored in the same de-packetization buffer.

I think it needs to be clarified that this is all RTP streams from the same encoder instance and media source. Not all RTP streams from multiple encoder instances and media sources.

[YK]: Yes. But on other hand, in my understanding, we always refer to the context of one "encoder instance and media source" in RTP payload format for a particular codec, right? Is it OK that we just make this generic clarification?

25. Section 7.1:
   The receiver MUST ignore any unspecified parameter.

First, I don't know if this is the right place to included this rule.
Secondly, should it not be "unknown" parameters for the one processing the media format parameters?

[YK]: OK, I will change the wording to "unrecognized". However, to me this is a good place to say this - we did it the same way and in the same place in RFCs 6184/6190. However, if you know a better place, please let us know.

26. Section 7.1:

profile-space, profile-id:
            Otherwise (the RTP stream is a dependent RTP stream), the
            following applies, with j being the value of the sub-layer-
            id parameter:

            o profile_space = sub_layer_profile_space[j]
            o profile_id = sub_layer_profile_idc[j]

The following text raises some concerns.
A) These are defined as being stream specific values. It is unclear how one deals with these if one only want to configure the general capability within a RTP payload type.

[YK]: Note that these are "elementary stream" specific values, not "RTP stream" specific values. For example, if there are 3 temporal sub-layers, with TemporalId equal to 0, 1, and 2, respectively, and each is carried in one RTP stream, then the values of the RTP stream corresponding to the temporal sub-layers with TemporalId equal to 2 are the property of the bitstream or elementary stream consisting of all the three temporal sub-layers. And since this RTP stream is not a dependent RTP stream (because no other RTP stream depends on it), the general capability is assigned.

B) It is unclear if one may or are required to create different RTP payload types for the different scalability layers. From my perspective it is crucial that one can avoid a requirement to create layer specific payload types as this can result in total consumption of the available payload type space in an RTP session.

Thus, I see a need to clarify if and when one uses the above assignment.
And how the counter part in any signalling exchange is supposed to interpret these cases.

[YK]: To me, when a spec does not disallow the use of different RTP payload types, which is the case with the current spec, then the use is allowed. Do you see any reason to impose an restriction on use of payload types for the different RTP streams in a multi-stream transmission? If not, I think we can just add a note to say that there is no restriction imposed herein for the use of payload types. 

The same concerns relate to the tier-flag, level-id:

            Otherwise (the RTP stream is a dependent RTP stream), the
            following applies, with j being the value of the sub-layer-
            id parameter:

            o tier-flag = sub_layer_tier_flag[j]
            o level-id = sub_layer_level_idc[j]

27. Section 7.1:
       profile-compatibility-indicator:

Isn't this a sprop parameter?

[YK]: It is the same as profile, tier, and level. And you can see in later text, it is part of the media format configuration.

Also it is not clear what "j" is:

            If the RTP stream is not a dependent RTP stream, the
            following applies with j = 0..31:

            o The 32 flags = general_profile_compatibility_flag[j]

Am I correct that j in this case are the integer identifying a particular HEVC profile?

[YK]: That is true. This is clear from the semantics of the flag as defined in the HEVC spec. We can try to add a bit more clarification on this.

28. Section 7.1:
profile-compatibility-indicator:

It is not clear to me that this parameter has long term applicability.
If one includes this, is that a promise until signaling changes to not provide a stream that breaks the provided value? I wonder how one deals with the use cases like AD splicing. One stream may ensure this, is it the role of the signalling to enforce that what one says applies, or will be renegotiated in signalling?

[YK]: Again, the nature and the usage of this parameter is the same as profile, tier, and level. So, yes, it would be a promise when provided, in either SDP offer/answer or declarative usage.

29.

      sub-layer-id:

         This parameter MAY be used to indicate the highest allowed
         value of TID in the stream.  When not present, the value of
         sub-layer-id is inferred to be equal to 6.

This parameter is written in a generic form, however the O/A Section indicates this to declare a property of the stream to be sent. But not explicit discussion of this parameter is provided. So what is the meaning of this parameter?

A) I promise that the streams I send does not use more than X sub-layers

B) I require the peer sending me a stream to not use more than X sub-layers.

[YK]: It is A. Would it be clear enough in your view if we add that this parameter shall not be used for any other purpose than indicating a property of the stream (which to me is sort of clear from its current semantics)?

30. Section 7.1:
sprop-vps, sprop-sps and sprop-pps:

I think one unclarity and one that existed in H.264 also, is the precedence between out-of-band and in-band parameter sets. And when they start to apply. The later is especially important in cases when you actually transmit a different set in a new signalling exchange then what has previously been used.

Can some clarification on this be included?

[YK]: OK, we can add a clarification to state that, out-of-band parameter sets can be put at the beginning of the bitstream output to the decoder, while in-band parameter sets can be sent to the decoder in the same way as the receiver outputs other types of NAL units from the de-packetization process.

31. Section 7.1:
   Many of the parameters. I notices that quite a lot of the parameters are not explicitly defining the value range allowed for the parameters.
This is especially true for integer style values. See the many max- parameters.

[YK]: Right. We have been doing this all the time, including RFCs 6184/6190. Are you saying that from now that the value range of each parameter needs to be clearly specified in RTP payload formats?

32. Section 7.1:
  max-dpb: Formation error

[YK]: Sorry, I don't know what is error by comparing to the semantics to that of other parameters. Could you please clarify?

33. Section 7.1:

cap-parameter = tier-flag / level-id / max-lsr
                            / max-lps / max-br

These ABNF elements are not defined. I think they need to be.

[YK]: This came from you guys. Can you please provide the missing definition?

34. Section 7.2.1:

Source specific parameters:

   o  The OPTIONAL parameters "sprop-vps", "sprop-sps", and "sprop-
      pps", when present, MUST be included in the "a=fmtp" line of SDP
      or conveyed using the "fmtp" source attribute as specified in
      section 6.3 of [RFC5576].  For a particular media format (i.e.
      RTP payload type), "sprop-vps" "sprop-sps", or "sprop-pps" MUST
      NOT be both included in the "a=fmtp" line of SDP and conveyed
      using the "fmtp" source attribute.

This text exemplifies that there are important differencies between RTP session level configuration and individual source level configuration. I think some more general discussion of the potential constraints and implications of the two methods needs to be considered. Especially when you are prevented from signalling both session and stream level.

Stream level requires on to know that this stream will be sent and its configuration. And when you also can't provide a session level set that is default fall back if no stream specific is provided then you get to choose between. These parameter constraints applies to all streams, or no pre-provided parameters can be provided at all, and you need per stream signalling.

[YK]: OK, we will try to add some usage description along the line mentioned above, probably with some help from the original proponent of this source-specific feature.

35. Section 7.2.2:

          Informative note:  The above parameters apply for any stream
          sent by a declaring entity with the same configuration; i.e.
          they are dependent on their source.  Rather than being bound
          to the payload type, the values may have to be applied to
          another payload type when being sent, as they apply for the
          configuration.

I find this confusing. I don't know if it is a typo and should say "independent of their source", or if it is the definition of source here that is wrong.

It might be that the paragraph before the above:
   o  The parameters sprop-depack-buf-nalus and sprop-depack-buf-bytes
      describe the properties of the RTP stream that the offerer or the
      answerer is sending for the media format configuration.  This
      differs from the normal usage of the Offer/Answer parameters:
      normally such parameters declare the properties of the stream
      that the offerer or the answerer is able to receive.  When
      dealing with HEVC, the offerer assumes that the answerer will be
      able to receive media encoded using the configuration being
      offered.

actually needs to separate between end point specific configurations that apply to the whole RTP session through the payload type or if is on specific media encoding basis and thus needs to be connected to the
SSRC(s) that carries the particular media encoding.

[YK]: It seems to be a good catch. The same wording was inherited from RFC 6184, and appeared in RFC 3984 as well, for which you were a co-author. Could you authors of RFC 3984 (e.g. Stephan, Miska, Magnus) try to help figure out what was the actual intent when this wording was firstly introduced into RFC 3984?

36. Section 7.2.2:

      min_spatial_segmentation_idc  equal  to  or  larger  than  dec-
      parallel-cap.spatial-seg-idc of the capability point.  A stream
      that is sent based on choosing a capability point with parallel
      tool    type    't'    from    dec-parallel-cap    MUST    have
      entropy_coding_sync_enabled_flag     equal     to     0     and
      min_spatial_segmentation_idc  equal  to  or  larger  than  dec-
      parallel-cap.spatial-seg-idc of the capability point.

It looks like someone managed to turn this text into straight left and right columns, this is not allowed in IETF, we use ragged rights.

[YK]: I guess you meant to align text only to the left margin, not to both left and right margins? If that is the case, we actually did this in all the places in the Word based template for preparing of the draft. Do you see any issues in other places as well? Can you please let us know some specific line for which the style should be changed?

37. Section 7.2.2:

   o  An offerer has to include the size of the de-packetization
      buffer,  sprop-depack-buf-bytes,  and  sprop-depack-buf-nalus,  in
      the  offer  for  an  interleaved  HEVC  stream  or  for  the  MST
      transmission mode.  To enable the offerer and answerer to inform
      each  other  about  their  capabilities  for  de-packetization
      buffering in receiving streams, both parties are RECOMMENDED to
      include depack-buf-cap.  For interleaved streams or in MST, it is
      also RECOMMENDED to consider offering multiple payload types with
      different buffering requirements when the capabilities of the
      receiver are unknown.

This paragraph contains two mentions of "interleaved", the only two in the whole specification. I assume a copy and paste from RFC 6184 that wasn't converted properly.

[YK]: No they are intentionally left here, as interleaved packetization is allowed. Maybe we can add one sentence or two to make this more explicit.

38. Section 7.2.2.:

 However, when out-of-band transport of parameter
      sets  is  used,  parameter  sets  MAY  still  be  additionally
      transported   in-band   unless   explicitly   disallowed   by   an
      application.

To me it doesn't appear that this rule is O/A specific. It should apply also to declarative usage. I propose that this is moved to the parameter definition itself.

[YK]: OK. Section 7.2.4 (Parameter set considerations) should be a good place to hold this.

39. Section 7.2.2.:

       o In MST, the answerer MUST be prepared to use the parameter
          sets included in sprop-vps, sprop-sps, and sprop-pps of all
          RTP streams that a particular RTP stream depends on, when
          present (either included in the "a=fmtp" line of SDP or
          conveyed using the "fmtp" source attribute), for decoding the
          incoming NAL unit stream.

I become uncertain how this really works. Especially in the context of recv-sub-layer-id. So the offerer includes the parameter sets for all the layers it intended to send. Then the answerer sub-sets that selection using recv-sub-layer-id. Is the intention to say that the answerer needs to try to apply the parameter sets that matches the selected layers.

[YK]: When MST is in use, parameter sets may be split and transmitted in different RTP streams. When a reciver chooses to receive a set of the RTP streams, the receiver must be prepared to use parameter sets carried in all RTP streams of the set, not just parameter sets carried in the highest RTP stream. We can try to make this clearer.

40. Section 7.2.2:
       o If the level to use in the answerer-to-offerer direction is
          equal to the default level in the answer, the offerer MUST be
          prepared to use the parameter sets included in sprop-vps,
          sprop-sps, and sprop-pps (either included in the "a=fmtp"
          line of SDP or conveyed using the "fmtp" source attribute)
          for decoding the incoming NAL unit stream.  Otherwise, the
          offerer  MUST  ignore  sprop-vps,  sprop-sps,  and  sprop-pps
          (either included in the "a=fmtp" line of SDP or conveyed
          using the "fmtp" source attribute) and the answerer MUST
          transmit parameter sets in-band.

Sorry, I missing something here. In the Answerer to Offerer direction, there are no reason for the answerer to not provide parameter sets that matches the offerer's declared receiving capability. Thus, why discuss the "default" level?

[YK]: Good catch. The answerer should just provide parameter sets that matches the offerer's declared receiving capability if out-of-band parameter sets transmission is used. Will change this in the next version.

41. Section 7.2.2:
       o In MST, the offerer MUST be prepared to use the parameter
          sets included in sprop-vps, sprop-sps, and sprop-pps of all
          RTP streams that a particular RTP stream depends on, when
          present (either included in the "a=fmtp" line of SDP or
          conveyed using the "fmtp" source attribute), for decoding the
          incoming NAL unit stream.

I am uncertain if this is intended to mean something else than, if provided attempt to use them.

[YK]: This is a repeat of comment #39. See my reply to comment #39 above.

42. Section 7.2.2:
      Parameter sets associated with one source MUST
      only be used to decode NAL units conveyed in RTP packets from the
      same source.

The question is what is meant with "same source" in the above. Is it SSRC, Media Source or Media Encoding, or even Endpoint.

[YK]: I think this means the same media source or the same media encoding instance. Will change in the next version.

43. Section 7.2.2:

                                          sendonly --+
            answer: recvonly, recv-sub-layer-id --+  |
              recvonly w/o recv-sub-layer-id --+  |  |
      answer: sendrecv, recv-sub-layer-id --+  |  |  |
        sendrecv w/o recv-sub-layer-id --+  |  |  |  |
                                         |  |  |  |  |
      profile-space                      C  X  C  X  P
      profile-id                         C  X  C  X  P
      tier-flag                          C  X  C  X  P
      level-id                           C  X  C  X  P

      C: configuration for sending and receiving streams

I don't get this usage or definition of C for these parameters.

So these parameters are receiver capabilities, and they needs to be symmetrically used in the signalling, however the actually used values in the transmission direction from an O/A agent only needs to be within the receivers provided parameters. Thus, they are not a "configuration"
for the sending direction.

[YK]: These are configuration parameters, meaning indicating both what is to be sent and what is to be received for sendrecv and must be used symmetrically, and only the latter for recvonly. 

Also I really don't get the X's in the answer: recvonly, recv-sub-layer-id, and answer: sendrecv, recv-sub-layer-id. The answering agent is going to receive stream in both these cases, thus it needs to declare the payload type, and these parameters MUST be included (unless defaulted). So why is they marked with X?

[YK]: When recv-sub-layer-id is included in an SDP answer, it means that the answer chooses one of the multiple operation points corresponding to the offered or declared sub-layers in the sprop-vps, and in this case there is no need to repeat the information of profile, level etc. in the SDP, thus there are marked with X (MUST NOT be present).

44. Section 7.2.2:

                                          sendonly --+
            answer: recvonly, recv-sub-layer-id --+  |
              recvonly w/o recv-sub-layer-id --+  |  |
      answer: sendrecv, recv-sub-layer-id --+  |  |  |
        sendrecv w/o recv-sub-layer-id --+  |  |  |  |
                                         |  |  |  |  |
      interop-constraints                C  X  C  X  P
      profile-compatibility-indicator    C  X  C  X  P

I don't understand how these parameters really are used for receiver capability declarations, so I don't understand why the Cs are C and not P.

[YK]: As explained earlier, these are also parts of the media type configuration, same as profile, tier, and level, and their usage are the same as those too.

---

From: Magnus Westerlund <[email-masked]>
Date: Mon, 17 Mar 2014 15:22:31 +0100

Hi,

Removing the issues where I am happy with your response and have no
further to add.

On 2014-03-01 00:31, Wang, Ye-Kui wrote:
> -----Original Message----- From: payload
> [mailto:[email-masked]] On Behalf Of Magnus Westerlund 
> Sent: Friday, February 28, 2014 7:47 AM To: [email-masked];
> [email-masked] Subject: [payload]
> Comments on draft-ietf-payload-rtp-h265-02
> 
> Hi,
> 
> I have reviewed the H.265 RTP payload format and have a some
> comments.
> 
> 5. Section 1.2: o Transmission of HEVC NAL units of the same
> bitstream within a single RTP stream (note that RTP stream is used
> equivalently as RTP flow in this memo) or multiple RTP streams
> 
> I would propose to make clear that "multiple RTP streams" is within
> one or more RTP sessions.
> 
> [YK]: The motivation is to allow for the multiple RTP streams to be
> within either one or more RTP sessions. I hope changing "or multiple
> RTP streams" to "or multiple RTP streams within one or more RTP
> sessions" would be good to clarify this.
> 

Yes, that would be a relevant clarification.

> 
> 11. Section 4.1: Sequence number (SN): 16 bits
> 
> Set and used in accordance with RFC 3550.
> 
> To my understanding when using single stream transmission the
> Decoding order is recovered soley from RTP sequence numbers and order
> of the NALUs within each RTP payload. This usage should be mentioned
> here as it affects the impact if anyone would inject other payloads
> in the same RTP packet stream.
> 
> [YK]: The current draft also supports interleaved transmission,
> wherein decoding order can be different from RTP sequence number
> order. Decoder order number (DON) values are used for
> de-packetization.

That is really not clear. My reading of the draft, really indicated that
you removed all the interleaving support that was present in H.264's
payload format and that DONs really was to support scalability.

If it is intended to support Interleaving, then the draft needs to be
explicit about this support and discuss the limitations on that
mechanism, including how the use of the signalling attribute to
negotiate buffer capability.

> 
> 12. Section 4.1: I am missing the SSRC usage for this RTP payload. In
> SST that is the classical identification of a particular encoding of
> a media source. But for MST the SSRC usage is different as one needs
> to know which SSRC in which RTP sessions that are associated. I think
> this dependency needs to be noted.
> 
> [YK]: OK. I'd appreciate if you could provide some concrete language
> for this.

Ok, a draft for you to word smith.

synchronization source (SSRC): 32-bits

   Used to identify the source of the RTP packets. In SST mode a single
SSRC will be used for all the parts of a single encoding. In MST mode,
each SSRC is used for a packet stream containing a sub-set of encoding
layers for a single encoding. Thus, a receive is required to correctly
associate the set of SSRCs that included parts of the same encoding. See
Section X.Y for further discussion of this association.

What I considered but haven't included is if more about the dependency
tree needs to be expressed here, or if that is sufficient to be covered
elsewhere?

> 
> 15. Section 4.7: If sprop-depack-buf-nalus is greater than 0, the
> DOND field MUST be present in an aggregation unit that is not the
> first aggregation unit in an AP, and the variable DON for the
> aggregated NAL unit is derived as equal to the DON of the preceding
> aggregated NAL unit in the same AP plus the value of the DOND field
> plus 1 modulo 65536.
> 
> I really wonder over the "+1" usage in the DOND field. That prevents
> two NALUs in an AP to have same DON. Which at least Section 4.5 hints
> to as being valid. If that is not valid, then Section 4.5 is in
> error. Thus, this definition can cause issues and I don't see any
> issues with defining the field without the plus. That limits the
> difference to increment with a maximum of 255 instead of 256.
> 
> [YK]: I don't see there is an error. Section 4.5 allows two NALUs in
> an AP to have the same DON, but it does not require them to have
> different DONs. To me, both options would work, and it is just a
> matter of design taste. To my knowledge there are already some
> implementations of the payload format, thus I'd prefer to keep the
> "plus 1", though I don't have a strong opinion herein.

I am fine with this, as long as people are okay with that they will not
be able to assign the same DON value to two consecutive NALUs within a
single packet.

Cheers

Magnus Westerlund

----------------------------------------------------------------------
Services, Media and Network features, Ericsson Research EAB/TXM
----------------------------------------------------------------------
Ericsson AB                 | Phone  +46 10 7148287
Färögatan 6                 | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden | mailto: [email-masked]
----------------------------------------------------------------------

---

From: Magnus Westerlund <[email-masked]>
Date: Mon, 17 Mar 2014 16:54:05 +0100

Hi,

Here are my response, I have removed all issues where I am happy with
the response.

On 2014-03-03 20:07, Wang, Ye-Kui wrote:
> My responses to comments 23 upwards are interleaved below (again, for
> convenience, I only included comments 23 upwards).
> 
> BR, YK
> 
> 23. Section 5: Otherwise (sprop- depack-buf-nalus  is  equal  to  0
> for  an  RTP  stream),  the transmission order of NAL units carried
> in the RTP stream MUST be the same as the NAL unit decoding order.
> 
> This fails to describe how you determines the NAL unit decoding
> order. I assume in RTP sequence order and for each payload in the
> order the NALUs are included?
> 
> [YK]: No, the NAL unit decoding order is described by the
> de-packetization process in Section 6, which applies in all cases,
> regardless of sprop-depack-buf-nalus being 0 or greater than 0. Yes,
> when sprop-depack-buf-nalus is equal to 0, the order would be as you
> assumed above. I don't see anything wrong or missing in this regard.
> However, if you think more clarification is needed anywhere, please
> suggest and let us know where.

What might not be clear in Section 5, is what is counted as transmission
order within an aggregation packet. That is also not given for
sprop-depack-buf-nalus=0 in Section 4.7.

But, you might be correct that the actual text that I am missing is the
one that belongs to Section 6, on how to derive decoding order from
received packets when sprop-depack-buf-nalus=0.

> 
> 24. Section 6:
> 
> When MST is in use, NAL units of all RTP streams are stored in the
> same de-packetization buffer.
> 
> I think it needs to be clarified that this is all RTP streams from
> the same encoder instance and media source. Not all RTP streams from
> multiple encoder instances and media sources.
> 
> [YK]: Yes. But on other hand, in my understanding, we always refer to
> the context of one "encoder instance and media source" in RTP payload
> format for a particular codec, right? Is it OK that we just make this
> generic clarification?

"for a particular codec" can be misinterpreted as meaning some other
format, like H.264. Thus, I would prefer if you try to be more explicit
so that it isn't misunderstood. Multi source and multi-stream (within a
single media type) hasn't been that common.

> 26. Section 7.1:
> 
> profile-space, profile-id: Otherwise (the RTP stream is a dependent
> RTP stream), the following applies, with j being the value of the
> sub-layer- id parameter:
> 
> o profile_space = sub_layer_profile_space[j] o profile_id =
> sub_layer_profile_idc[j]
> 
> The following text raises some concerns. A) These are defined as
> being stream specific values. It is unclear how one deals with these
> if one only want to configure the general capability within a RTP
> payload type.
> 
> [YK]: Note that these are "elementary stream" specific values, not
> "RTP stream" specific values. For example, if there are 3 temporal
> sub-layers, with TemporalId equal to 0, 1, and 2, respectively, and
> each is carried in one RTP stream, then the values of the RTP stream
> corresponding to the temporal sub-layers with TemporalId equal to 2
> are the property of the bitstream or elementary stream consisting of
> all the three temporal sub-layers. And since this RTP stream is not a
> dependent RTP stream (because no other RTP stream depends on it), the
> general capability is assigned.

Well, I think you need to be explicit about what "stream" you mean in
the text throughout the whole payload format.

> 
> B) It is unclear if one may or are required to create different RTP
> payload types for the different scalability layers. From my
> perspective it is crucial that one can avoid a requirement to create
> layer specific payload types as this can result in total consumption
> of the available payload type space in an RTP session.
> 
> Thus, I see a need to clarify if and when one uses the above
> assignment. And how the counter part in any signalling exchange is
> supposed to interpret these cases.
> 
> [YK]: To me, when a spec does not disallow the use of different RTP
> payload types, which is the case with the current spec, then the use
> is allowed. Do you see any reason to impose an restriction on use of
> payload types for the different RTP streams in a multi-stream
> transmission? If not, I think we can just add a note to say that
> there is no restriction imposed herein for the use of payload types.

No, I don't see a reason to impose a restriction. But, if something is
intended to be possible it is better to be explicit so that the lazy
implementor doesn't miss that it is required to be capable of handling
such a configuration.

> 
> 27. Section 7.1: profile-compatibility-indicator:
> 
> [YK]: It is the same as profile, tier, and level. And you can see in
> later text, it is part of the media format configuration.
> 
> If the RTP stream is not a dependent RTP stream, the following
> applies with j = 0..31:
> 
> [YK]: That is true. This is clear from the semantics of the flag as
> defined in the HEVC spec. We can try to add a bit more clarification
> on this.

Thanks, a pointer is likely all that is necessary, but as someone who
only read a few very specific parts of the HEVC spec, I don't know these
are defined in the HEVC specification.

> 
> 28. Section 7.1: profile-compatibility-indicator:
> 
> It is not clear to me that this parameter has long term
> applicability. If one includes this, is that a promise until
> signaling changes to not provide a stream that breaks the provided
> value? I wonder how one deals with the use cases like AD splicing.
> One stream may ensure this, is it the role of the signalling to
> enforce that what one says applies, or will be renegotiated in
> signalling?
> 
> [YK]: Again, the nature and the usage of this parameter is the same
> as profile, tier, and level. So, yes, it would be a promise when
> provided, in either SDP offer/answer or declarative usage.

Good, can you make this clearer somehow?

> 
> 29.
> 
> sub-layer-id:
> 
> This parameter MAY be used to indicate the highest allowed value of
> TID in the stream.  When not present, the value of sub-layer-id is
> inferred to be equal to 6.
> 
> This parameter is written in a generic form, however the O/A Section
> indicates this to declare a property of the stream to be sent. But
> not explicit discussion of this parameter is provided. So what is the
> meaning of this parameter?
> 
> [YK]: It is A. Would it be clear enough in your view if we add that
> this parameter shall not be used for any other purpose than
> indicating a property of the stream (which to me is sort of clear
> from its current semantics)?

If it is only A, then I think it should have a sprop prefix.

> 
> 30. Section 7.1: sprop-vps, sprop-sps and sprop-pps:
> 
> I think one unclarity and one that existed in H.264 also, is the
> precedence between out-of-band and in-band parameter sets. And when
> they start to apply. The later is especially important in cases when
> you actually transmit a different set in a new signalling exchange
> then what has previously been used.
> 
> [YK]: OK, we can add a clarification to state that, out-of-band
> parameter sets can be put at the beginning of the bitstream output to
> the decoder, while in-band parameter sets can be sent to the decoder
> in the same way as the receiver outputs other types of NAL units from
> the de-packetization process.

I am uncertain that what you try to say here. I interpret it as:

As soon as you get the out-of-band parameters sets they can be added to
the front of the queue of NALUS that are given to the decoder. Is that
what you intended?

> 
> 31. Section 7.1: Many of the parameters. I notices that quite a lot
> of the parameters are not explicitly defining the value range allowed
> for the parameters. This is especially true for integer style values.
> See the many max- parameters.
> 
> [YK]: Right. We have been doing this all the time, including RFCs
> 6184/6190. Are you saying that from now that the value range of each
> parameter needs to be clearly specified in RTP payload formats?

>From my perspective, being explicit about what values you expect reduces
the risk for error or becoming subject to an attack as you can parse
according to the spec, and if the input is malformed or malicious you
can give an error with a clear reason. If you are not explicit, you
don't know what to expect here. Thus, I think explicit ranges or at
least data types, i.e. what data and format to parse the value as.

> 
> 32. Section 7.1: max-dpb: Formation error
> 
> [YK]: Sorry, I don't know what is error by comparing to the semantics
> to that of other parameters. Could you please clarify?

Sorry, poorly worded. The lines on page 55 overflows the 72 column
boundaries.

> 
> 33. Section 7.1:
> 
> cap-parameter = tier-flag / level-id / max-lsr / max-lps / max-br
> 

Yes, I know. Here is an proposal.

tier-flag = "tier-flag" EQ ("0" / "1")
level-id  = "level-id" EQ 1*3DIGIT  ; (0-255)
max-lsr   = "max-lsr" EQ  1*20DIGIT ; (0-18,446,744,073,709,551,615)
max-lps   = "max-lps" EQ 1*10DIGIT  ; (0-4,294,967,295)
max-br    = "max-br"  EQ 1*20DIGIT  ; (0-18,446,744,073,709,551,615)
EQ = "="
DIGIT = <RFC 5234>

This is based on my understanding that tier-flag really only is a binary
flag. level-id is an unsigned 8-bit value. max-lsr and max-br clearly
has the potential for being larger than 32-bit integer, thus I used a
64-bit integer. I think 4G of pixel resolution is likely sufficient.

Any errors in these assumptions?

> 
> 35. Section 7.2.2:
> 
> Informative note:  The above parameters apply for any stream sent by
> a declaring entity with the same configuration; i.e. they are
> dependent on their source.  Rather than being bound to the payload
> type, the values may have to be applied to another payload type when
> being sent, as they apply for the configuration.
> 
> I find this confusing. I don't know if it is a typo and should say
> "independent of their source", or if it is the definition of source
> here that is wrong.
> 
> It might be that the paragraph before the above: o  The parameters
> sprop-depack-buf-nalus and sprop-depack-buf-bytes describe the
> properties of the RTP stream that the offerer or the answerer is
> sending for the media format configuration.  This differs from the
> normal usage of the Offer/Answer parameters: normally such parameters
> declare the properties of the stream that the offerer or the answerer
> is able to receive.  When dealing with HEVC, the offerer assumes that
> the answerer will be able to receive media encoded using the
> configuration being offered.
> 
> actually needs to separate between end point specific configurations
> that apply to the whole RTP session through the payload type or if is
> on specific media encoding basis and thus needs to be connected to
> the SSRC(s) that carries the particular media encoding.
> 
> [YK]: It seems to be a good catch. The same wording was inherited
> from RFC 6184, and appeared in RFC 3984 as well, for which you were a
> co-author. Could you authors of RFC 3984 (e.g. Stephan, Miska,
> Magnus) try to help figure out what was the actual intent when this
> wording was firstly introduced into RFC 3984?

What, you are holding me responsible for what I wrote 10 years ago? ;-)

Considering this is SDP O/A, and looking at it again. I think it refer
to that an SDP agent sending a Offer or Answer with
sprop-depack-buf-nalus and sprop-depack-buf-bytes included, are
responsible to ensure that any HEVC encoding and its RTP packetization
conforms to the declared parameters. This applies to each encoding
independently, even if multiple encodings are transmitted.

Thus, I think this text would be clearer if one adds "endpoint" after
source.

> parallel-cap.spatial-seg-idc of the capability point.  A stream that
> is sent based on choosing a capability point with parallel tool
> type    't'    from    dec-parallel-cap    MUST    have 
> It looks like someone managed to turn this text into straight left
> and right columns, this is not allowed in IETF, we use ragged
> rights.
> 
> [YK]: I guess you meant to align text only to the left margin, not to
> both left and right margins? If that is the case, we actually did
> this in all the places in the Word based template for preparing of
> the draft. Do you see any issues in other places as well? Can you
> please let us know some specific line for which the style should be
> changed?

Correct, I have seen this in some few places.

Lines:
33
58, for three lines
262
285
299, for 3 lines
376
etc.

3275, for 5-lines

Search for ".   " and you will find a large number of these issues.

> 
> 37. Section 7.2.2:
> 
> o  An offerer has to include the size of the de-packetization buffer,
> sprop-depack-buf-bytes,  and  sprop-depack-buf-nalus,  in the  offer
> for  an  interleaved  HEVC  stream  or  for  the  MST transmission
> mode.  To enable the offerer and answerer to inform each  other
> about  their  capabilities  for  de-packetization buffering in
> receiving streams, both parties are RECOMMENDED to include
> depack-buf-cap.  For interleaved streams or in MST, it is also
> RECOMMENDED to consider offering multiple payload types with 
> This paragraph contains two mentions of "interleaved", the only two
> in the whole specification. I assume a copy and paste from RFC 6184
> that wasn't converted properly.
> 
> [YK]: No they are intentionally left here, as interleaved
> packetization is allowed. Maybe we can add one sentence or two to
> make this more explicit.

As said earlier, if this is a supported feature of the payload format,
then you need to discuss it explicitly.

> 43. Section 7.2.2:
> 
> sendonly --+ answer: recvonly, recv-sub-layer-id --+  | recvonly w/o
> recv-sub-layer-id --+  |  | answer: sendrecv, recv-sub-layer-id --+
> |  |  | sendrecv w/o recv-sub-layer-id --+  |  |  |  | |  |  |  |  | 
> profile-space                      C  X  C  X  P profile-id
> C  X  C  X  P tier-flag                          C  X  C  X  P 
> level-id                           C  X  C  X  P
> 
> So these parameters are receiver capabilities, and they needs to be
> symmetrically used in the signalling, however the actually used
> values in the transmission direction from an O/A agent only needs to
> be within the receivers provided parameters. Thus, they are not a
> "configuration" for the sending direction.
> 
> [YK]: These are configuration parameters, meaning indicating both
> what is to be sent and what is to be received for sendrecv and must
> be used symmetrically, and only the latter for recvonly.

This is at a least wrong for the level-id which a peer agent in an O/A
may downgrade. The rest I agree the payload format itself requires to be
kept in answer, or the corresponding PT dropped.

> 
> Also I really don't get the X's in the answer: recvonly,
> recv-sub-layer-id, and answer: sendrecv, recv-sub-layer-id. The
> answering agent is going to receive stream in both these cases, thus
> it needs to declare the payload type, and these parameters MUST be
> included (unless defaulted). So why is they marked with X?
> 
> [YK]: When recv-sub-layer-id is included in an SDP answer, it means
> that the answer chooses one of the multiple operation points
> corresponding to the offered or declared sub-layers in the sprop-vps,
> and in this case there is no need to repeat the information of
> profile, level etc. in the SDP, thus there are marked with X (MUST
> NOT be present).

This appears very strange to me. The Offerer includes a VPS that
contains options for what it can send. The answering party, i.e. which
will receive this stream needs in accordance to SDP O/A rules declare
which PTs it can receive. This PT must be corresponding to the one that
the Offerer proposed to send and in which the VPS was included.

For me this implies the following:

1. The parameters indicating the basic configuraiton of the payload
type, i.e profile, level ,tier etc. MUST be present in the answer with
the same values as in the offer it responds to. Otherwise the PT
matching rule fails.

2. In addition the PT to receive on MUST be declared as existing to
ensure that any middleboxes doesn't block it. Thus, explicit indication
of what will go in it is really recommended, especially considering i
all the other cases that information will be explicit here. Thus, I
think the X's are plain wrong. Or I am still misunderstanding something
here.

> 
> 44. Section 7.2.2:
> 
> sendonly --+ answer: recvonly, recv-sub-layer-id --+  | recvonly w/o
> recv-sub-layer-id --+  |  | answer: sendrecv, recv-sub-layer-id --+
> |  |  | sendrecv w/o recv-sub-layer-id --+  |  |  |  | |  |  |  |  | 
> I don't understand how these parameters really are used for receiver
> capability declarations, so I don't understand why the Cs are C and
> not P.
> 
> [YK]: As explained earlier, these are also parts of the media type
> configuration, same as profile, tier, and level, and their usage are
> the same as those too.

First of all these parameters are not included as the ones that have to
be symmetrically provided. I also think they could be SDP agent
specifically given. But, that can be discussed. But, according to the
current text, I fail to see that these needs to be handled as the
profile-id, level-id, etc.

The only constraint I find for these two flags applies under multicast.

Either they need text, or these C's need to be changed.

Cheers

Magnus Westerlund