Subject: [payload] new draft - RTP Payload for TTML Timed Text

From: James Sandford <[email-masked]>
Date: Thu, 17 Jan 2019 17:39:58 +0000

Dear IETF Payload WG,

I have uploaded a draft specifying an RTP payload format for TTML Timed Text https://datatracker.ietf.org/doc/draft-sandford-payload-rtp-ttml/

"""
This memo describes a Real-time Transport Protocol (RTP) payload format for TTML, an XML based timed text format for live and file based workflows from W3C.  This payload format is specifically targeted at live workflows using TTML.
"""

"""
TTML (Timed Text Markup Language) is a media type for describing timed text such as closed captions (also known as subtitles) in television workflows or broadcasts as XML.  This document specifies how TTML should be mapped into an RTP stream in live workflows including, but not restricted to, those described in the television broadcast oriented EBU-TT Part 3 specification.
"""

The television broadcast industry is currently progressing towards a future where broadcast facilities are based on IP networks as opposed to the specialist cabling and equipment currently used. One aspect of this work is the SMPTE2110 family of specifications which specifies, among other things, the use of RTP for audio, video, and other media. Work has previously been done by Thomas Edwards to provide an RTP profile for carrying ancillary data bit streams with maximum interoperability with traditional SDI (Serial Digital Interface) systems. These bit streams may contain closed captions, timing information, triggering signals for equipment etc. That work has provided an easy way to transition current data streams to IP in broadcast facilities. But these data streams still use legacy formats bound by the restrictions of SDI. This new draft specifies a payload to carry a modern timed text format natively over RTP within broadcast facilities.

I would be very grateful if you could review and support this document and I look forward to working with the group.

Regards,
James

==========
James Sandford
R&D Engineer

BBC Research and Development
5th Floor
Dock House
MediaCityUK
Salford
M50 2LH

Tel: 030304 (09549)
Web: http://www.bbc.co.uk/rd

----------------------------

http://www.bbc.co.uk
This e-mail (and any attachments) is confidential and may contain personal views which are not the views of the BBC unless specifically stated.
If you have received it in error, please delete it from your system.
Do not use, copy or disclose the information in any way nor act in reliance on it and notify the sender immediately.
Please note that the BBC monitors e-mails sent or received.
Further communication will signify your consent to this.

---------------------

---

From: Kjetil Oftedal <[email-masked]>
Date: Thu, 17 Jan 2019 19:53:45 +0100

On 17/01/2019, James Sandford <[email-masked]> wrote:
> I have uploaded a draft specifying an RTP payload format for TTML Timed Text
> https://datatracker.ietf.org/doc/draft-sandford-payload-rtp-ttml/
>

Hi,

Can you please elaborate a bit on the RTP Timestamping requirements
outlined here?
As far as I can see TTML already specifies the timing of the text in
relation to some epoch.
Will the processing of the RTP packet require the receiver/sender to
rewrite the timing data in TTML-information in relation to the RTP
timestamp? Or written another way: Is the RTP Timestamp considered the
epoch of the TTML document?

Best regards,
Kjetil Oftedal

---

From: Thomas Edwards <[email-masked]>
Date: Thu, 17 Jan 2019 19:12:08 +0000

I'm not sure what the author was thinking, but I'd think that in many cases, a TTML RTP flow would be often associated with another RTP media flow (such as RFC 4175/SMPTE ST 2110-20, AES67, etc.) and hopefully the RTP timestamp time would line up...

In RFC 8331 (RTP for SMPTE Ancillary Data), it was suggested they were tightly locked together:
"When an ANC data RTP stream is to be associated with an RTP
         video stream, the RTP timestamp rates SHOULD be the same to
         ensure that ANC data packets can be associated with the
         appropriate frame or field.  Otherwise, a 90 kHz rate SHOULD be
         used."

But that might be less appropriate for TTML which is less directly associated with video fields/frames.

-Thomas

On 1/17/19, 10:53 AM, "payload on behalf of Kjetil Oftedal" <[email-masked] on behalf of [email-masked]> wrote:

    On 17/01/2019, James Sandford <[email-masked]> wrote:
    > https://urldefense.proofpoint.com/v2/url?u=https-3A__datatracker.ietf.org_doc_draft-2Dsandford-2Dpayload-2Drtp-2Dttml_&d=DwICAg&c=uw6TLu4hwhHdiGJOgwcWD4AjKQx6zvFcGEsbfiY9-EI&r=lekNOOM5noV61zrPH3rwPyhtNnLLWoLEHgd0quQxly8&m=lcOWqoAOZng6eG5ibGyA-k5wqdyjNwxcOsa251X-eLg&s=eGgyZQYCqKg2ArgO0aFCaNFoX8fZTyL7cOP55goWDY4&e=
    >
    
    Hi,

    Best regards,
    Kjetil Oftedal
    
    _______________________________________________
    payload mailing list
    [email-masked]
    https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_payload&d=DwICAg&c=uw6TLu4hwhHdiGJOgwcWD4AjKQx6zvFcGEsbfiY9-EI&r=lekNOOM5noV61zrPH3rwPyhtNnLLWoLEHgd0quQxly8&m=lcOWqoAOZng6eG5ibGyA-k5wqdyjNwxcOsa251X-eLg&s=tiBjx73Dw6-x1Yj7MlpvB_bNw67I5UVkduJQHa8xwDA&e=

---

From: James Sandford <[email-masked]>
Date: Fri, 18 Jan 2019 09:40:34 +0000

Hello,
Thank you for your replies.

Kjetil. TTML and its variants (e.g. EBU-TT) do define a number of possible timing methods where absolute times are specified within the document. This has a number of disadvantages in the streaming use case. *There is not necessarily a guarantee that the reference clock used within the document is identical to that in the RTP header. The document could have been originally generated on a different system and a naive implementation could persist the original timestamps/reference clock when packaging the document into an RTP stream.
*The timestamps in the document may end up contradicting that in the RTP header. The replication of data, i.e. timestamps in the RTP header and within the document, feels like poor design that leaves more room for bugs in implementations.
*Unrestricted, timestamps in a document may specify changes in the past. They shouldn't, but there would be nothing to stop them.
*A convention for converting each timing method within the document to each possible reference clock at the RTP level would be required to correctly align closed-captions/subtitles in a TTML stream with other media such as Video. This could potentially increase the complexity of specifications and implementations.
*Where absolute timings are used in the streamed documents, replaying a stream may require re-writing of the timestamps within the payload against the new playout time
*A broadcast center may implement devices such as buffers for aligning, for example, external contributions from different points around the world with vastly differing latencies. Allowing timing to be specified within the document would require TTML specific knowledge to correctly align multiple streams. As the draft currently stands, this alignment/buffering could be done at the RTP level.

Some of these potential issues could of course be solved by senders/receivers "doing the sensible thing". Others cannot. And it would be preferable to reduce the potential for bugs/poor implementations as much as possible. This draft aims to reduce the risk of all of these points by moving the timing to the RTP level while still allowing a level of internal timing that allows, for example, animation of text. We would also like to discourage the implementation of senders which send, for example, an entire program's worth of subtitles in a single document. This would obviously be poor use of RTP.

As the draft is currently written, it will require the sender/reader to have knowledge of the RTP time to correctly calculate the display time of subtitles. Though this would be a requirement wherever TTML is combined with RTP video/audio anyway. Section 4.2.1 specifies a TTML profile that restricts the time base within the TTML document to Media time and defines it as 0-based against the RTP time for each payload. So you are correct, Kjetil, an RTP Timestamp is considered the epoch of its respective TTML document.

I hope this helps.

Regards,
James

==========
James Sandford
R&D Engineer

Tel: 030304 (09549)

________________________________________
From: Thomas Edwards [[email-masked]]
Sent: 17 January 2019 19:12
To: Kjetil Oftedal; James Sandford
Cc: [email-masked]
Subject: Re: [payload] new draft - RTP Payload for TTML Timed Text

-Thomas

    Hi,

    Best regards,
    Kjetil Oftedal

-----------------------------
http://www.bbc.co.uk
This e-mail (and any attachments) is confidential and
may contain personal views which are not the views of the BBC unless specifically stated.
If you have received it in
error, please delete it from your system.
Do not use, copy or disclose the
information in any way nor act in reliance on it and notify the sender
immediately.
Please note that the BBC monitors e-mails
sent or received.
Further communication will signify your consent to
this.
-----------------------------

---

From: "Roni Even (A)" <[email-masked]>
Date: Sun, 20 Jan 2019 08:09:36 +0000

Hi,
Thanks for submitting the document. I did a quick review and the table of content has all the required sections
I see that there is interest in this work from the mailing list discussion and new RTP payload is in the charter of the WG so we can most probably progress the document.
Some comments as WG chair

The IANA section should include the registration template. I looked at the reference to the ttml document (https://www.w3.org/TR/ttml-profile-registry/ ) and noticed the ttml+xml registration template, this should be in this document, I also expect some more content in section 5 and 6
As for the security consideration, the length field is per packet but I did not see any concerns about the document size, is it unbound which can cause denial of service attack by sending large documents. Please look at RFC8088 (https://tools.ietf.org/html/rfc8088 ) to see how to write the important specification sections in section 7.

As an individual, I saw some discussion about the clock rate so assume it will be addressed. I noticed that the default rate is 1000 yet the example in 7.2.1 has 90000 so I was also looking for some guidelines about how to select the clock rate.

Roni Even
Payload WG co-chair

From: payload [mailto:[email-masked]] On Behalf Of James Sandford
Sent: Thursday, January 17, 2019 7:40 PM
To: [email-masked]
Subject: [payload] new draft - RTP Payload for TTML Timed Text

Dear IETF Payload WG,

"""

"""

Regards,
James

==========
James Sandford
R&D Engineer

Tel: 030304 (09549)

http://www.bbc.co.uk

---------------------

---

From: James Sandford <[email-masked]>
Date: Thu, 24 Jan 2019 15:49:46 +0000

Hello,
I've uploaded a new version that hopefully addresses these concerns. https://datatracker.ietf.org/doc/draft-sandford-payload-rtp-ttml/01/

Regards,
James

==========
James Sandford
R&D Engineer

Tel: 030304 (09549)
Web: http://www.bbc.co.uk/rd
________________________________
From: Roni Even (A) [[email-masked]]
Sent: 20 January 2019 08:09
To: James Sandford; [email-masked]
Subject: RE: new draft - RTP Payload for TTML Timed Text

Hi,

Roni Even
Payload WG co-chair

Dear IETF Payload WG,

"""

"""

Regards,
James

==========
James Sandford
R&D Engineer

Tel: 030304 (09549)

http://www.bbc.co.uk

---------------------

http://www.bbc.co.uk

---------------------