Subject: [payload] DON in H.265 RTP draft

From: "Mo Zanaty (mzanaty)" <[email-masked]>
Date: Fri, 6 Mar 2015 03:08:40 +0000

Hi,
Based on implementation experience with DON in draft-ietf-payload-rtp-h265-07, here are some suggested changes in the draft to simplify implementations and improve robustness.

DON (Decoding Order Number)

DON is only necessary when using interleaved transmission (for error resilience) or reordered frames (for coding efficiency), if willing to incur the additional latency and complexity of these techniques. These techniques are not used in many (probably most) systems, so DON should not be required in those cases.

However, the current draft requires DON when tx-mode is MRST or MRMT (or sprop-max-don-diff>0). This causes unnecessary complexity, RTP packet overhead, and potential for RTP packet parsing failure due to conditional DON fields whose presence depends on subtle SDP rules. If implementations interpret these subtle SDP rules differently or improperly, RTP packet parsing completely fails, which is undesirably fragile.

The suggested change is to revert to the original design in the individual 00 draft (and RFC 6190), which uses separate explicit payload structures for RTP packets without DON (FU-A, STAP-A) and with DON (FU-B, STAP-B). Receivers must still support DON, but senders only use it when necessary, i.e. interleaved transmission or reordered frames. This has several benefits.

a. RTP packet parsing is more robust since RTP packets are self-describing with no dependency on subtle SDP rules.
b. RTP packet overhead is reduced in cases where DON is unnecessary, including many/most MRST and MRMT cases.
c. Aligns better with H.264 so those implementations can more easily add H.265.

The only drawback is this requires 2 extra NAL unit types (51, 52), but that is acceptable since the space is doubled in H.265 (64 vs. 32 in H.264).

If there is agreement on this suggested change, I will propose specific text changes. Although conceptually simple, it does impact a non-trivial amount of text, mostly repeated.

Regards,
Mo

---

From: "Wang, Ye-Kui" <[email-masked]>
Date: Fri, 6 Mar 2015 06:04:17 +0000

Hi Mo,

See my comments inline below.

BR, YK

From: payload [mailto:[email-masked]] On Behalf Of Mo Zanaty (mzanaty)
Sent: Thursday, March 05, 2015 7:09 PM
To: [email-masked]
Cc: [email-masked]
Subject: [payload] DON in H.265 RTP draft

Hi,

[YK] No, DON is needed when either interleaved transmission or multi-stream transmission (MRST/MRMT), otherwise DON is not needed. Reordered pictures for improve coding efficiency is not a reason for DON.

[YK] Could you please clarify what is the unnecessary complexity and RTP packet overhead? Also, when tx-mode is present, the value is clear, and when not present, the default value is specified. This rule is simple and clear. Could you please provide an example use case where a problem can occur?

[YK] If you do that, then you would also need to specify different packetization modes, and which payload structures are allowed in which packetization mode, and you would also need to specify different modes of de-packetization. Basically, the complexity would be similarly to the sum of RFC 6184 and RFC 6190. That makes it significantly more complicated than what we have now. Note that we intentionally avoided all these to simply the design and this was welcomed by the group.

[YK] I thus certainly disagree to go back. Note that in the individual 00 draft a lot of details were not included yet. Feel free to try to do the text changes, and if you make it working and complete you would agree that things would become much more complicated than what we have now.

Regards,
Mo

---

From: "Mo Zanaty (mzanaty)" <[email-masked]>
Date: Tue, 10 Mar 2015 07:13:40 +0000

Hi YK, see Mo: inline.
Regards,
Mo

On 3/6/15, 1:04 AM, Wang, Ye-Kui <[email-masked]<mailto:[email-masked]>> wrote:

Hi Mo,

BR, YK

From: payload [mailto:[email-masked]] On Behalf Of Mo Zanaty (mzanaty)
Sent: Thursday, March 05, 2015 7:09 PM
To: [email-masked]<mailto:[email-masked]>
Cc: [email-masked]<mailto:[email-masked]>
Subject: [payload] DON in H.265 RTP draft

Hi,

Mo: DON is not needed for MRST/MRMT unless reordered frames are used. See mst-mode=NI-T in H.264 SVC RFC 6190. The same applies to MRST/MRMT in H.265. H.264 did not force the complexity of DON for the very common cases of no reordered frames, so H.265 should likewise not force such complexity.

Mo: DON is unnecessary complexity and packet overhead. Existing RTP header fields already provide decode order without introducing an artificial, redundant DON. It was introduced in H.264 RFC 3984 for the interleaved mode, which was never widely deployed. It was retained in H.264 SVC RFC 6190 for the interleaved mode and MST modes with reordered frames. It should be retained in H.265 for the same (rare) cases, but not forced in unnecessary (and very common) cases. It is also fragile for RTP packet layout to depend on subtle SDP rules. As stated above, a severe problem can occur if implementations have different views of whether DON should be present or not, due to different SDP implementations or even transient renegotiation. RTP packet parsing completely fails because one side thinks payloads start at offset N while the other thinks N+2 due to DON conditionally present based on SDP rules.

Mo: I very much welcomed the simpler design of no different packetization modes negotiated in SDP. Thanks for that. I do not suggest we change that. I suggest what I stated above. “Receivers must still support DON, but senders only use it when necessary.”

Mo: Ok, I will do the text changes and report back.

Regards,
Mo

---

From: Yago Sanchez <[email-masked]>
Date: Tue, 10 Mar 2015 10:15:22 +0100

Hi Mo, 

The reason for forcing to use DON for MRST and MRMT is actually to reduce the implementation complexity. We could have followed the design in RFC 6190 by having four transmission modes NI-T, NI-TC, NI-C and I-C but it was agreed from the beginning that such a thing should be avoided. 

You are right when you say that the same principle of NI-T could be used without requiring the presence of the DON field, but this would lead for sure to the same result we had in RFC 6190, four modes. NI-TC was found very useful back then to solve issues of time stamp based (NI-T) decoding order recovery under loss events.

Note additionally that the NI-T case requires the dependent RTP (enhancement layes) streams to contain data for all sampling instances that exist in their dependee RTP streams (base layer or lower layers), e.g. by using the empty NAL units described in RFC 6190, which also incur some overhead. Only if the decoding order is equal to the presentation order (that is what I interpret from your comment about frame reordering) this requirement would not be necessary. This is something we would need to signal additionally, adding dependency on subtle SDP rules similar to the ones you mention.

In any case, I think that your proposal now introduces more problems than benefits. The design we chose at the beginning was concretely to avoid all this potential problems that would require specific solutions making the design much more complicated only to save a couple of bytes required to add the DON field. Therefore, I am here with Ye-Kui and would prefer to keep the design as it is now.

Best regards,
Yago

On 10 Mar 2015, at 08:13, Mo Zanaty (mzanaty) <[email-masked]> wrote:

> Hi YK, see Mo: inline.
> Regards,
> Mo
> 
> On 3/6/15, 1:04 AM, Wang, Ye-Kui <[email-masked]> wrote:
> 
> Hi Mo,
>  
> _______________________________________________
> payload mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/payload

---

From: "Mo Zanaty (mzanaty)" <[email-masked]>
Date: Tue, 10 Mar 2015 17:37:42 +0000

Hi Yago,

I fully agree with reducing complexity by eliminating SDP negotiation of different packetization modes for different transmission modes. I applaud the authors for those efforts, and strongly support retaining them. My suggestion is further complexity reduction, but perhaps I need to provide more details to convey this. I will look into the specific text changes as YK suggested. If the result is simpler, I will share it. If not, I will abandon it, and share that as well.

Regards,
Mo

On 3/10/15, 5:15 AM, Yago Sanchez <[email-masked]<mailto:[email-masked]>> wrote:

Hi Mo,

Best regards,
Yago

On 10 Mar 2015, at 08:13, Mo Zanaty (mzanaty) <[email-masked]<mailto:[email-masked]>> wrote:

Hi YK, see Mo: inline.
Regards,
Mo

Hi Mo,

BR, YK

Hi,

Regards,
Mo

_______________________________________________
payload mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/payload