Subject: Re: [avtext] draft-ietf-avtext-rtp-grouping-taxonomy-00

From: Bernard Aboba <[email-masked]>
Date: Tue, 11 Feb 2014 16:19:34 -0800

Alex said: "One possible solution for the SST/MST redefinition issue is to use the terms: single-port transmission and single-SSRC transmission, as well as the corresponding multi- variants. Acronyms: Sp / Ss - Mp / Ms. 

SST per RFC 6190 is single-port, single SSRC.

MST per RFC 6190 is multi-port, multi-SSRC.

What is being referred to as Single Stream Transmission here, is single-port covering both single-SSRC and multi-SSRC cases."[BA] Agree with Alex about what the definitions were in RFC 6190.  Also agree that it would be best to use new abbreviations if we are going to define new terms. The subject is confusing enough that we don't need to add any more :)Section 3.2.1 says: "   Scalable Video Coding [RFC6190] has a mode of operation where Encoded Streams and Dependent Streams from the SVC Media Encoder is grouped together in a single Source Packet Stream using the SVC RTP Payload format."This isn't right, because Single Session Transport in RFC 6190 could refer to either a single stream or multiple.  So I think Alex is correct. Section 3.3.2 says: "   Multi-stream transmission (MST) is a mechanism by which different portions of a layered encoding of a Source Stream are sent using separate Packet Streams (sometimes in separate RTP sessions)."So here what the document is calling "Multi-stream transmission" is referring to multiple-SSRC, either with a single-port or multiple ports.

---

From: Bo Burman <[email-masked]>
Date: Wed, 12 Feb 2014 18:03:45 +0000

Bernard, I get confused what you think is specified in 6190.

On one hand you agree with Alex what the definitions were in 6190, assumedly referring to the text "SST per RFC 6190 is single-port, single SSRC".

On the other hand, it seems you object to exactly that ("Encoded Streams and Dependent Streams from the SVC Media Encoder is grouped together in a single Source Packet Stream") in your next comment: "This isn't right, because Single Session Transport in RFC 6190 could refer to either a single stream or multiple.  So I think Alex is correct".

Could you please clarify?

In 6190 section 5.1 Packetization Rules for Single-Session Transmission, it says "When a set of layers including one or more SVC enhancement layers is transmitted in an RTP session, the set SHOULD be carried in one RTP stream that SHOULD be encapsulated according to this memo". The definition of "RTP packet stream" in section 3.1.2 of 6190, is "A sequence of RTP packets with increasing sequence numbers (except for wrap-around), identical payload type and identical SSRC (Synchronization Source), carried in one RTP session. Within the scope of this memo, one RTP packet stream is utilized to transport one or more layers".

So, to me it seems that multiple RTP streams are in fact not disallowed by SST, but 6190 is entirely silent on how such multiple streams would be mapped to multiple SSRC. It in fact seems to be agnostic to the use of SSRC and (assumedly) relies entirely on information in the SVC bitstream. Correct? In that case, can any number of SSRC be used to describe the SVC stream (I know it SHOULD only be one) from a single Media Source?

Furthermore, in another mail in this thread, I noted Magnus says there is a specific problem in SST to handle multiple Media Sources (in taxonomy terms, like multiple cameras) and to know which SSRC belongs to which Media Source. I tend to agree, given the above.

In any case, I think it will be unwise to re-define anything, we should only clarify. I hope to do that by including text that describe SST and MST in taxonomy terms, but I still have difficulties to understand exactly how to describe them.

Cheers,

Bo

From: avtext [mailto:[email-masked]] On Behalf Of Bernard Aboba
Sent: den 12 februari 2014 01:20
To: [email-masked]
Subject: Re: [avtext] draft-ietf-avtext-rtp-grouping-taxonomy-00

Alex said:

"One possible solution for the SST/MST redefinition issue is to use the terms:

single-port transmission and single-SSRC transmission, as well as the corresponding multi- variants. Acronyms: Sp / Ss - Mp / Ms.

What is being referred to as Single Stream Transmission here,

is single-port covering both single-SSRC and multi-SSRC cases."

[BA] Agree with Alex about what the definitions were in RFC 6190.  Also agree that it would be best to use new abbreviations if we are going to define new terms. The subject is confusing enough that we don't need to add any more :)

Section 3.2.1 says:

"   Scalable Video Coding [RFC6190<http://tools.ietf.org/html/rfc6190>] has a mode of operation where Encoded Streams and Dependent Streams from the SVC Media Encoder is grouped together in a single Source Packet Stream using the SVC RTP Payload format."

This isn't right, because Single Session Transport in RFC 6190 could refer to either a single stream or multiple.  So I think Alex is correct.

Section 3.3.2 says:

"   Multi-stream transmission (MST) is a mechanism by which different portions of a layered encoding of a Source Stream are sent using separate Packet Streams (sometimes in separate RTP sessions)."

So here what the document is calling "Multi-stream transmission" is referring to multiple-SSRC, either with a single-port or multiple ports.

---

From: Bernard Aboba <[email-masked]>
Date: Wed, 12 Feb 2014 10:51:20 -0800

Bo said: 

"So, to me it seems that multiple RTP streams are in fact not disallowed by SST"[BA] Correct.  There are some implementations that send multiple RTP streams within a single session. [Bo Burman]"6190 is entirely silent on how such multiple streams would be mapped to multiple SSRC. It in fact seems to be agnostic to the use of SSRC and (assumedly) relies entirely on information in the SVC bitstream. Correct? "[BA] For multiple streams (whether sent on separate RTP sessions or the same session), it is necessary to recover the decoding orderacross the multiple streams, as described in RFC 6190 Section 4.11.  This leads to multiple source transport requiring distinct packetizationfrom that used in single source transport, as described in Section 5.2.  IMHO, this was a major mistake, which has lead to interoperabilityissues with H.264/SVC transport.  That issue is being addressed in the HEVC over RTP payload specification where there is no longer distinctpacketization modes for SST and MST and DON support is mandatory. With respect to signaling, RFC 6190 provides examples of single session transport (7.3.1, 7.3.2) and multi-session transport (7.3.3, 7.3.4), but does not provideexamples of how to signal multiple sources within the same session.   With BUNDLE, RFC 5761 and RFC 5583 dependency groups, I believe this isnow possible.   However, since the topic is somewhat "bleeding edge" it will probably not end up in the HEVC over RTP spec, eventhough that will cover temporal scalability. [Bo]In that case, can any number of SSRC be used to describe the SVC stream (I know it SHOULD only be one) from a single Media Source?[BA] Yes, this has been implemented, and has been shown to have some advantages.  For example, when dropping a layer, sequence number rewriting can potentially be avoided.  Also, FEC can be applied to some layers and not others. 
 [Bo] Furthermore, in another mail in this thread, I noted Magnus says there is a specific problem in SST to handle multiple Media Sources (in taxonomy terms, like multiple cameras) and to know which SSRC belongs to which Media Source. I tend to agree, given the above.
 [BA] There is a need for a receiver to understand the purpose of each incoming stream.  The ways in which this can be done are underdiscussion.  As an example, App-Id/Receive-App-Id may be one way to handle this. [Bo] In any case, I think it will be unwise to re-define anything, we should only clarify. [BA] I agree. [Bo] I hope to do that by including text that describe SST and MST in taxonomy terms, but I still have difficulties to understand exactly how to describe them.
 [BA] To be clear, my objection was not to defining new terms Single Stream Transport and Multiple Stream Transport ,but to redefining the meaning of the SST and MSTabbreviations used in RFC 6190.

---

From: Bo Burman <[email-masked]>
Date: Thu, 13 Feb 2014 23:26:44 +0000

I would then suggest to simply clarify that, as 6190 define them, both SST and MST can use one or more Encoded Packet Streams (SSRC) per Media Source.

So, to elaborate, there is a SST-SingleStream with all scalable layers of a Media Source sent in a single Packet Stream (with a single SSRC). SST-MultiStream uses a single RTP Session where the scalable layers are spread across multiple different Packet Streams. In MST-SS, the scalable layers use multiple RTP Sessions, each carrying a single Packet Stream. Finally, the scalable layers in MST-MS use multiple Packet Streams in each of the multiple RTP Sessions.

Then there is the complication that scalable layers signaled as MST (in SDP) that are also BUNDLE'd are in fact combined into a single RTP Session, effectively becoming equivalent to SST-MS.

Comments?

From: Bernard Aboba [mailto:[email-masked]]
Sent: den 12 februari 2014 19:51
To: Bo Burman; [email-masked]
Subject: RE: [avtext] draft-ietf-avtext-rtp-grouping-taxonomy-00

Bo said:

"So, to me it seems that multiple RTP streams are in fact not disallowed by SST"

[BA] Correct.  There are some implementations that send multiple RTP streams within a single session.

[Bo Burman]

"6190 is entirely silent on how such multiple streams would be mapped to multiple SSRC.

It in fact seems to be agnostic to the use of SSRC and (assumedly) relies entirely on information in the SVC bitstream. Correct? "

[BA] For multiple streams (whether sent on separate RTP sessions or the same session), it is necessary to recover the decoding order

across the multiple streams, as described in RFC 6190 Section 4.11.  This leads to multiple source transport requiring distinct packetization

from that used in single source transport, as described in Section 5.2.  IMHO, this was a major mistake, which has lead to interoperability

issues with H.264/SVC transport.  That issue is being addressed in the HEVC over RTP payload specification where there is no longer distinct

packetization modes for SST and MST and DON support is mandatory.

With respect to signaling, RFC 6190 provides examples of single session transport (7.3.1, 7.3.2) and multi-session transport (7.3.3, 7.3.4), but does not provide

examples of how to signal multiple sources within the same session.   With BUNDLE, RFC 5761 and RFC 5583 dependency groups, I believe this is

now possible.   However, since the topic is somewhat "bleeding edge" it will probably not end up in the HEVC over RTP spec, even

though that will cover temporal scalability.

[Bo]

In that case, can any number of SSRC be used to describe the SVC stream (I know it SHOULD only be one) from a single Media Source?

[BA] Yes, this has been implemented, and has been shown to have some advantages.  For example, when dropping a layer,

sequence number rewriting can potentially be avoided.  Also, FEC can be applied to some layers and not others.

 [BA] There is a need for a receiver to understand the purpose of each incoming stream.  The ways in which this can be done are under

discussion.  As an example, App-Id/Receive-App-Id may be one way to handle this.

[Bo] In any case, I think it will be unwise to re-define anything, we should only clarify.

[BA] I agree.

[Bo] I hope to do that by including text that describe SST and MST in taxonomy terms,

but I still have difficulties to understand exactly how to describe them.

 [BA] To be clear, my objection was not to defining new terms Single Stream Transport and Multiple Stream Transport ,but to redefining the meaning of the SST and MST

abbreviations used in RFC 6190.

---

From: Suhas Nandakumar <[email-masked]>
Date: Thu, 13 Feb 2014 15:41:30 -0800

Hello Bo,

  I like this way of enumerating  all the possibilities . Few comments
inline.

Cheers
Suhas

On Thu, Feb 13, 2014 at 3:26 PM, Bo Burman <[email-masked]> wrote:

>  I would then suggest to simply clarify that, as 6190 define them, both
> SST and MST can use one or more Encoded Packet Streams (SSRC) per Media
> Source.
>
>
>
> So, to elaborate, there is a SST-SingleStream with all scalable layers of
> a Media Source sent in a single Packet Stream (with a single SSRC).
> SST-MultiStream uses a single RTP Session where the scalable layers are
> spread across multiple different Packet Streams. In MST-SS, the scalable
> layers use multiple RTP Sessions, each carrying a single Packet Stream.
> Finally, the scalable layers in MST-MS use multiple Packet Streams in each
> of the multiple RTP Sessions.
>

[Suhas] Can we make a 2X2 table and add the terms in the matching cells.
Something on the lines below
                                              Single RTP Session
    Multiple RTP Sessions
  Single Packet Stream           * SST-SingleStream *
*MST-SS*
  Multiple Packet Streams       * SST-MultiStream                    MST-MS*

I am thinking that this table will go along with the above definition and
might aid in the clarity.

>
> Then there is the complication that scalable layers signaled as MST (in
> SDP) that are also BUNDLE'd are in fact combined into a single RTP Session,
> effectively becoming equivalent to SST-MS.
>
>
>

I think we need to make the above categorization independent of the
underlying transport. but i do agree with your overall direction.

>
> *From:* Bernard Aboba [mailto:[email-masked]]
> *Sent:* den 12 februari 2014 19:51
> *To:* Bo Burman; [email-masked]
> *Subject:* RE: [avtext] draft-ietf-avtext-rtp-grouping-taxonomy-00
>
>
>
> Bo said:
>
> _______________________________________________
> avtext mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/avtext
>
>

---

From: Jonathan Lennox <[email-masked]>
Date: Mon, 24 Feb 2014 19:17:23 +0000

(As an individual)

I think the important distinction, both from an implementation and a specification perspective, is between single-stream and multi-stream transmission.  Multi-stream cases are where you have issues of separation and reassembly, stream association, and the like — where all the complexity arises.

Whether those multiple streams are sent in multiple sessions or not seems like a secondary issue.  It’s obviously important as an deployment choice, and has some signaling implications, but I don’t think it makes a lot of difference for either implementation or media-level specification.

On another point: in your table, I don’t understand MST-SS.  How can a single stream be sent in multiple sessions?  Based on the latest version of the taxonomy draft, it seems like you’re interpreting this as meaning multiple streams in multiple sessions, where the streams have the same SSRC.  That may be worth calling out as a special case (since it’s the mode that RFC 3550 defines for MST) but I think that calling that “single stream” is not a good idea.  They still have separate sequence number spaces, probably timestamp spaces, feedback, etc.

On Feb 13, 2014, at 6:41 PM, Suhas Nandakumar <[email-masked]<mailto:[email-masked]>> wrote:

Hello Bo,

  I like this way of enumerating  all the possibilities . Few comments inline.

Cheers
Suhas

On Thu, Feb 13, 2014 at 3:26 PM, Bo Burman <[email-masked]<mailto:[email-masked]>> wrote:
I would then suggest to simply clarify that, as 6190 define them, both SST and MST can use one or more Encoded Packet Streams (SSRC) per Media Source.

[Suhas] Can we make a 2X2 table and add the terms in the matching cells. Something on the lines below
                                              Single RTP Session               Multiple RTP Sessions
  Single Packet Stream            SST-SingleStream                  MST-SS
  Multiple Packet Streams        SST-MultiStream                    MST-MS

I am thinking that this table will go along with the above definition and might aid in the clarity.

Then there is the complication that scalable layers signaled as MST (in SDP) that are also BUNDLE’d are in fact combined into a single RTP Session, effectively becoming equivalent to SST-MS.

I think we need to make the above categorization independent of the underlying transport. but i do agree with your overall direction.

Comments?

From: Bernard Aboba [mailto:[email-masked]<mailto:[email-masked]>]
Sent: den 12 februari 2014 19:51
To: Bo Burman; [email-masked]<mailto:[email-masked]>
Subject: RE: [avtext] draft-ietf-avtext-rtp-grouping-taxonomy-00

Bo said:

[Bo Burman]

[Bo]

[BA] I agree.

_______________________________________________
avtext mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/avtext

_______________________________________________
avtext mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/avtext

---

From: Bo Burman <[email-masked]>
Date: Tue, 25 Feb 2014 08:22:26 +0000

Inline

From: Jonathan Lennox [mailto:[email-masked]]
Sent: den 24 februari 2014 20:17
To: Suhas Nandakumar
Cc: Bo Burman; [email-masked]
Subject: Re: [avtext] draft-ietf-avtext-rtp-grouping-taxonomy-00

(As an individual)

I think the important distinction, both from an implementation and a specification perspective, is between single-stream and multi-stream transmission.  Multi-stream cases are where you have issues of separation and reassembly, stream association, and the like - where all the complexity arises.
[BoB] Agree

Whether those multiple streams are sent in multiple sessions or not seems like a secondary issue.  It's obviously important as an deployment choice, and has some signaling implications, but I don't think it makes a lot of difference for either implementation or media-level specification.
[BoB] Again, agree. One of the major points I wanted to make was that "SST" not necessarily means that only a single stream is used.

On another point: in your table, I don't understand MST-SS.  How can a single stream be sent in multiple sessions?  Based on the latest version of the taxonomy draft, it seems like you're interpreting this as meaning multiple streams in multiple sessions, where the streams have the same SSRC.  That may be worth calling out as a special case (since it's the mode that RFC 3550 defines for MST) but I think that calling that "single stream" is not a good idea.  They still have separate sequence number spaces, probably timestamp spaces, feedback, etc.
[BoB] That was not my intent with MST-SS; it is multiple streams in total, yes, but only a single stream, single SSRC, per RTP Session. MST-MS would use multiple streams, multiple SSRC, in each of the multiple RTP Sessions. The distinction between SS and MS is thus use of multiple SSRC or not in an RTP Session.

Hello Bo,

Cheers
Suhas

Comments?

Bo said:

[Bo Burman]

[Bo]

[BA] I agree.

---

From: Jonathan Lennox <[email-masked]>
Date: Tue, 25 Feb 2014 15:35:21 +0000

On Feb 25, 2014, at 3:22 AM, Bo Burman <[email-masked]<mailto:[email-masked]>> wrote:

Inline

Similarly

Okay, I see the distinction you’re making; but I think the terminology you’ve chosen is confusing.

The top-level distinction, I think, needs to be between single-stream and multi-stream transmission.  Then there are distinctions between various ways of transmitting the streams that make up multi-stream transmission.

Personally, I would retroactively redefine the acronyms SST and MST as “single-stream transmission” and “multi-stream transmission”, despite 6190.  At least in my personal recollection, when 6190 was being written, no one thought there was a use case for multi-stream transmission in a single session — and indeed, there was some desire to discourage it from being used — so single-stream and single-session mode were thought of as basically synonymous.

Or if you think this would be confusing, we could instead come up with alternate terms for the two modes.  But I think that the grid where you imply that single/multiple stream and single/multiple session are orthogonal choices is a bad idea.

---

From: Bo Burman <[email-masked]>
Date: Wed, 26 Feb 2014 08:50:04 +0000

So, just to confirm, your point is that "SST" in 6190 terms is really what I chose to call "SST-SS", right? And, you suggest that any use of multiple RTP streams (Packet Streams) should be defined as using "MST", irrespective of how they are transported? In effect, re-defining 6190 "Single/Multiple Session Transmission" to "Single/Multiple Stream Transmission" (as, I noted, is already done in the HEVC payload format draft).

How do you suggest to handle the fact that you have been able, and still will be able, to signal "SST" for an SDP media block in 6190 and use multiple streams in that media block? Is that not a confusion that we should try to remove/clarify, and is not the Taxonomy an appropriate place for this? If you have a suitable remedy, please suggest text that we can use.

/Bo

From: Jonathan Lennox [mailto:[email-masked]]
Sent: den 25 februari 2014 16:35
To: Bo Burman
Cc: Suhas Nandakumar; [email-masked]
Subject: Re: [avtext] draft-ietf-avtext-rtp-grouping-taxonomy-00

Inline

Similarly

Okay, I see the distinction you're making; but I think the terminology you've chosen is confusing.

Personally, I would retroactively redefine the acronyms SST and MST as "single-stream transmission" and "multi-stream transmission", despite 6190.  At least in my personal recollection, when 6190 was being written, no one thought there was a use case for multi-stream transmission in a single session - and indeed, there was some desire to discourage it from being used - so single-stream and single-session mode were thought of as basically synonymous.

---

From: Jonathan Lennox <[email-masked]>
Date: Wed, 26 Feb 2014 16:28:52 +0000

On Feb 26, 2014, at 3:50 AM, Bo Burman <[email-masked]<mailto:[email-masked]>> wrote:

So, just to confirm, your point is that “SST” in 6190 terms is really what I chose to call “SST-SS”, right? And, you suggest that any use of multiple RTP streams (Packet Streams) should be defined as using “MST”, irrespective of how they are transported? In effect, re-defining 6190 “Single/Multiple Session Transmission” to “Single/Multiple Stream Transmission” (as, I noted, is already done in the HEVC payload format draft).

Right, that’s what I’m thinking.

How do you suggest to handle the fact that you have been able, and still will be able, to signal “SST” for an SDP media block in 6190 and use multiple streams in that media block? Is that not a confusion that we should try to remove/clarify, and is not the Taxonomy an appropriate place for this? If you have a suitable remedy, please suggest text that we can use.

Do you mean multiple streams with SST encodings of different sources?  That’s unremarkable, I think.  Or do you think that SST would actually let you use multiple packet streams for different layers of the same source?

/Bo

From: Jonathan Lennox [mailto:[email-masked]]
Sent: den 25 februari 2014 16:35
To: Bo Burman
Cc: Suhas Nandakumar; [email-masked]<mailto:[email-masked]>
Subject: Re: [avtext] draft-ietf-avtext-rtp-grouping-taxonomy-00

Inline

Similarly

---

From: Bo Burman <[email-masked]>
Date: Thu, 27 Feb 2014 14:37:04 +0000

From: Jonathan Lennox [mailto:[email-masked]]
Sent: den 26 februari 2014 17:29

Right, that's what I'm thinking.
[BoB] OK, so what I then believe is needed in the Taxonomy is a clarification that although SST and MST definitions in 6190 contain ...Session..., it assumes a single Packet Stream per Media Source in that "Session". This also means that a re-definition of SST and MST reading as ...Stream... would be equivalent with the original intent and more in line with the rest of the Taxonomy.

Do you mean multiple streams with SST encodings of different sources?  That's unremarkable, I think.  Or do you think that SST would actually let you use multiple packet streams for different layers of the same source?
[BoB] Multiple SST Media Sources in a single RTP Session is straightforward and all possible issues should already be covered by draft-ietf-avtcore-rtp-multi-stream. I don't know what I was thinking of, because SST has no issue. Sorry.

On the other hand, there is a statement on MST in 1.2.2 of 6190 that says "In MST, each RTP session is associated with one RTP stream, which may carry one or more layers". I guess that is no longer correct, especially in the context of BUNDLE where several media blocks can contribute to a single RTP session. To reflect that, I think we should clarify in the Taxonomy that multiple 6190 Packet Streams from a single Media Source can be present in a single RTP Session, despite what was originally intended and what is said in 6190. Agree?

Related to that, can you confirm that an 6190 SDP media block will still only result in a single RTP Packet Stream per Media Source? Or, do you see a need to relax that for MST and clarify that there may be multiple Packet Streams per Media Source even in the case of a single SDP media block using 6190 MST? As I see it, that would be desirable to align with the new definition of SST and MST.

/Bo

Inline

Similarly

---

From: Jonathan Lennox <[email-masked]>
Date: Thu, 27 Feb 2014 23:28:02 +0000

On Feb 27, 2014, at 9:37 AM, Bo Burman <[email-masked]<mailto:[email-masked]>> wrote:

Right, that’s what I’m thinking.
[BoB] OK, so what I then believe is needed in the Taxonomy is a clarification that although SST and MST definitions in 6190 contain …Session…, it assumes a single Packet Stream per Media Source in that “Session”. This also means that a re-definition of SST and MST reading as …Stream… would be equivalent with the original intent and more in line with the rest of the Taxonomy.

That seems correct to me.

Do you mean multiple streams with SST encodings of different sources?  That’s unremarkable, I think.  Or do you think that SST would actually let you use multiple packet streams for different layers of the same source?
[BoB] Multiple SST Media Sources in a single RTP Session is straightforward and all possible issues should already be covered by draft-ietf-avtcore-rtp-multi-stream. I don’t know what I was thinking of, because SST has no issue. Sorry.

Right, good.

 On the other hand, there is a statement on MST in 1.2.2 of 6190 that says “In MST, each RTP session is associated with one RTP stream, which may carry one or more layers”. I guess that is no longer correct, especially in the context of BUNDLE where several media blocks can contribute to a single RTP session. To reflect that, I think we should clarify in the Taxonomy that multiple 6190 Packet Streams from a single Media Source can be present in a single RTP Session, despite what was originally intended and what is said in 6190. Agree?

Yes.

That would probably be my understanding of it, but I think this is an area where SDP semantics are still pretty unclear.

That said, I think SDP issues are probably out of the scope of Taxonomy.

Inline

Similarly