Subject: RTSP Bakeoff issues

From: [email-masked] (Ron Frederick)
Date: Tue, 08 Aug 2000 17:38:59 -0700

On July 24th and 25th, there was an RTSP Interoperability "bakeoff" hosted
here at Entera in Fremont, CA. We had a total of about 27 attendees from
7 organizations: Apple, Cisco, Entera, Microsoft, Real, Sun, and the
University of Technology in Darmstadt. The goals were to test interoperability
among the various RTSP clients, proxies, and servers and to provide feedback
to the IETF about the RTSP spec to help it progress forward in the standards
process. I gave a presentation to the MMUSIC working group at the IETF in
Pittsburgh last week on some of the issues we discovered. There wasn't time
for a full discussion on all of them at the meeting, though, so I've written
up a summary below hoping to continue the discussion here... The issues below
were produced jointly by all the participants in the bakeoff, but they haven't
specifically seen this summary yet. Credit goes to the group as a whole, but
blame for any mistakes you find goes to me. :)

Comments on any and all of the issues below would be greatly appreciated. I
tried to list specific ideas that came up either during the bakeoff or during
the discussions at the MMUSIC session last week, but further work is required
on most of these points, particularly in figuring out how to make changes with
the least possible disruption to existing implementations...

1. When it is known, passing a real URL instead of "*" on an OPTIONS command
that's sent by a client can be useful when proxies are involved, as it lets
the proxy pass the options request through to the origin server and get the
real list of features supported by that server rather than replying with a
list of what _it_ supports. A suggestion was made that this should be pointed
out and explicitly encouraged in the spec.

2a. A question came up about whether servers need to support a SETUP on a URL
without a preceding DESCRIBE on a given control connection. This can come up
if the DESCRIBE was done earlier on some other connection or if the SDP
description was retrieved by some other out-of-band means. My personal
opinion is that servers should be strongly encouraged to always support
this and _not_ tie any state to a given underlying control connection. There
might be other reasons why the URLs returned in a DESCRIBE might become
stale and no longer work after some period of time, but that could happen
even when the DESCRIBE and SETUP happen on the same connection. In such a
case, a new DESCRIBE can be sent to get the updated information, but all of
this should be independent of which control connections are used. Other
opinions?

2b. Another question came up about whether servers were required to allow a
SETUP on only a subset of tracks in an aggregate object. I couldn't find
anything in the spec which discussed this, but I'd like to recommend that
this at least be a "SHOULD".

2c. Finally, there were questions about whether it was legal to allow SETUPs
for tracks from multiple different (possibly aggregate) objects in a single
session, such that they could be controlled together using a single PLAY
command. This seems like very useful functionality to me, but it raises
questions about what URL should be used in the PLAY command in that case and
there are some issues that come up between this and the proposals recently
discussed here in the RTSP extensions draft draft-sheedy-mmusic-rtsp-ext-01.
The proposal at the bakeoff was to allow a "*" in place of the URL in PLAY
when a Session was specified, with the meaning being to play all the tracks
that were SETUP in that session. However, if you want to support reuse of
a single session for multiple different sets of streams as was described in
the extensions draft, something more complicated is going to be required.
Further work is needed to reconcile all of these cases.

3. Queued PLAY as described in the spec was not yet implemented by many of
the participants in the bakeoff. While it seems like it could be a very
useful feature, there were some serious complications in the current design
that make it difficult to implement. In particular, it's currently very
difficult to return a reasonable set of values in the "RTP-Info" header of
the PLAY reply, as it would require knowing exactly what the sequence number
and timestamp will be when the server is done playing all of the previously
queued data, and this information may be difficult or impossible to compute
without actually reading and parsing all of that data. It might be more
reasonable to move this information from the PLAY reply to some other
independent message sent from the server to the client right before a PLAY
spurt starts at the server. For the first PLAY in any given session, this
message would be sent immediately after the PLAY reply, but for queued
segments it would be delayed until that segment was about to begin playing.
There was also an independent request for a "stream done" indication when
playback finished, and it seems like this same message could serve that
purpose, providing the sequence number and timestamp just past the end of
the packets which were transmitted when playback finishes.

4. It would be useful to clarify in the spec that the CSeq field should be
kept unique within a single control connection between the client and the
server. There was some confusion about whether it might instead be somehow
tied to a Session.

5a. A suggestion was made that URLs in the RTP-Info header be quoted in some
way, to avoid confusion in parsing. At the MMUSIC session, a recommendation
was made to use SIP-style quoting for this where the URL is placed in angle
brackets. This sounds like a reasonable solution, but some thought is needed
about how to make the transition here, as existing clients are unlikely to
correctly handle stripping off the angle brackets.

5b. The relationship (or lack thereof) between "seq" and "rtptime" in the
RTP-Info header could use more clarification. The spec does already say that
they don't necessarily correspond to the same time, but it might be useful to
add a detailed example that shows why this is the case and explains how each
is computed. On a related note, there was a suggestion that it might be
useful to have another field in this header that specified the timestamp of
the first real packet which will be sent (which _would_ correspond to the
seq field) so that clients would know what kind of delay to expect before
data starts arriving on that track.

6a. There are some situations where it would be useful to get some kind of
session state between a client and server even before any media streams are
requested. An obvious thing to use for this would be the Session header, but
right now a session can only be created when a SETUP is done. A suggestion
was made to add some kind of new command or field in something like the
OPTIONS command which could request the server assign and return a Session
without needing to do a SETUP.

6b. All of the examples currently found in the spec use numeric session IDs.
While the BNF does clearly show the session IDs can contain other characters,
it might be good to change the examples to show that to reinforce the fact
that they should be treated as opaque strings.

6c. Currently, the spec does not require the server to return a Session header
in the SETUP reply. However, this makes it difficult to do some kinds of
aggregate control (such as the example earlier about tracks from different
objects played together). To make the client's job easier, should we consider
requiring session IDs to be returned?

6d. More explanation about timeouts would be useful. In particular, the spec
speaks about session timeouts, but it says very little about timeouts on
control connections and how exactly the teardown of control connections
affects the teardown of sessions. In particular, section 12.37 talks about
RTSP command activity being needed to reset the timeout, but Section A
later mentions other wellness information such as RTCP reports being good
enough for that, suggesting that it should be possible to start playback
of a session and then tear down the control connection without fear of
the session timing out as long as RTCP reports are sent. Not all servers
implement this, however. A clearer specification of exactly what servers
must support here would be useful.

7a. Currently, the description of the Transport header says that "multicast"
is the default for _all_ transports. However, for some such as RTP/AVP/TCP,
multicast doesn't make sense. Should it be changed to make "unicast" the
default for some transports like TCP?

7b. Most existing implementations of RTP/AVP/TCP are actually of the
"interleaved" variety. However, clients don't have a good way of explicitly
asking for "interleaved", since the current BNF requires the "interleaved"
attribute to contain the channel numbers used (and those are generally
picked by the server). Should "RTP/AVP/TCP" always imply "interleaved" or
should the spec be changed to allow "interleaved" with no arguments as a
legal option so clients can request it? Is there any interest in supporting
the non-interleaved TCP transport described in the RTP/AVP Profile? If not,
should we recommend that be removed from the profile, or changed in some way
to be closer to the RTSP interleaved one?

7c. The "port", "client_port", and "server_port" options currently allow
either a single port number or "port1-port2" to specify a range. However,
the spec isn't really clear about what is meant by each of these cases. In
the case of RTP, does specifying a single port imply that RTCP will be on
that port + 1? If a range of ports is specified where the second port number
_isn't_ the first port + 1, what does that mean?

7d. A suggestion was made that proxies be reminded to explicitly add a
"source" field to the Transport header they pass back to clients, to make
sure client RTCP reports are sent to the right place. Otherwise, they could
end up getting sent to the origin server. Also, we may want to encourage
servers to echo back full transport information, including client-specified
fields like "client_port", to make it easier for proxies.

8. Right now, section B of the spec suggests setting the RTP marker bit in
an audio stream to mark a talk spurt boundary on the first packet of any new
PLAY request. However, in practice, this can be difficult to implement as
many servers don't know exactly what kind of data they are sending (and thus
whether it is audio or not). Also, there's no equivalent to this which works
for video. This same information can can gotten from the sequence number
given in the RTP-Info header, however, and perhaps that's all we need to
provide.

9a. There was some confusion about the proper way to compute the base URL
to use for expanding "a=control:" relative URLs in SDP descriptions returned
by DESCRIBE. When a Content-Base header is not present but a Content-Location
header is, should that be used as-is, or should its final component after the
last '/' be stripped off the way it is for HTTP?

9b. Is it legal for the session-level "a=control:" in the SDP description to
be a relative URL?

9c. If a session-level "a=control:" _is_ specified, should that be used as
the base URL for all the media-level "a=control:" relative URLs, or should
they still be interpreted as being relative to the original base?

9d. There are a few different ad-hoc schemes right now for specifying
alternate tracks in an SDP description, so that clients and servers can
do a limited form of content negotiation. It would be nice to standardize
on a single way to do this. At the second MMUSIC session, there was a fair
amount of support in favor of adding some kind of flow-id attribute, allowing
multiple alternatives to all list the same flow ID as a way for clients to
know they should only play back one of those alternatives at any given time
rather than all of them. It would be good to try and flesh out this idea
further.

9e. A suggestion was made that the "cliprect" field in SDP for video tracks
might be useful to include even in cases where video is being played back at
its natural size, allowing a client to prepare an appropriately sized window
without having to wait for data to start arriving.

10. A question came up about whether RTSP proxies should be expected to
preserve port number mappings between multiple SETUP requests in a given
session. In other words, if a client does multiple SETUPs with a given
client_port, should a proxy be expected to pass on that request to the
origin server with a common client_port (though possibly different
particular port numbers)? Similarly, if a server replies to multiple requests
with the same server_port pair, should the proxy use one server_port pair
for its connection back to the client?

11. Questions came up about when servers needed to handle multiple header
lines with the same name. At the MMUSIC session, a comment was made that
this is another place we might be able to use what SIP did. It explicitly
mentions that headers whose values are comma separated lists must allow the
items to appear on multiple separate headers lines and that the order must
be preserved. Copying that text into the RTSP spec might be useful. It would
also be useful to clarify exactly what other line continuation conventions
must be supported, if any.
--
Ron Frederick
[email-masked]

---

From: Anup Rao <[email-masked]>
Date: Tue, 08 Aug 2000 18:36:04 -0700

Ron,

Thanks for the excellent summary.  One comment below:

> given in the RTP-Info header, however, and perhaps that's all we need to

I'd like to make a case for the way it currently is ie. set the marker bit in the
audio stream to mark the first packet for any new PLAY request.

It allows for the logical seperation of the controlling(RTSP) entity and the
playback entity.  It is quite possible to not receive the  PLAY response (with the
RTP-Info header) before the first packet, hence resulting in some confusion. In
such a case, one might point out that  the RTP-Info header is no use. However  I
think the RTP-Info is advisory  information(in that one can get by without),  but
the marker bit is  more than that. I think that will perhaps become more evident
for implementations that do indeed keep contiguous timestamps across PLAYs as
specified in the same section.

This "logical seperation" is even more important if the RTP stream is directed at
a third party (ie the controller and the playback agent are on different hosts)
which does not have the RTP-Info header.
In comparison, teaching the server to know whether its audio nor not (in essence a
boolean flag either derived from the file extension, file header, or simply put in
as seperate meta-information with the file) seems to be simple.

Thanks
Anup.

---

From: [email-masked] (Ron Frederick)
Date: Fri, 11 Aug 2000 17:59:06 -0700

Hi Anup...

In message <[email-masked]> you write:
>> [setting the marker bit in a server at the start of PLAY]
> 
> I'd like to make a case for the way it currently is ie. set the marker bit
> in the audio stream to mark the first packet for any new PLAY request.
> 
> playback entity.  It is quite possible to not receive the  PLAY response
> (with the RTP-Info header) before the first packet, hence resulting in some
> confusion. In such a case, one might point out that the RTP-Info header is
> no use. However I think the RTP-Info is advisory  information (in that one
> can get by without), but the marker bit is more than that. I think that
> will perhaps become more evident for implementations that do indeed keep
> contiguous timestamps across PLAYs as specified in the same section.
> 
Actually, the intent of the marker bit in RTP was also to only be advisory
information, at least in the particular case being discussed here. The
marker bit for audio is meant to signal a logical point to adjust the size
of the playout buffer by adding or removing silence. It should not be
needed to achieve correct decoding of the audio data. It should also not
really matter whether or not the bit is set at the very beginning of the
first PLAY request sent, or any play request sent after a substantial pause,
as the receiver buffer would be empty at that point. It only becomes an
issue on something like queued PLAY of multiple requests or a seek during
an already playing stream. In those cases, the only "bad" behavior if the
marker bit is not set is that the receiver doesn't get a chance to readjust
their buffer, instead letting the audio data play with a gap defined by the
origin server's timestamps.

> This "logical seperation" is even more important if the RTP stream is
> directed at a third party (ie the controller and the playback agent are on
> different hosts) which does not have the RTP-Info header. In comparison,
> teaching the server to know whether its audio nor not (in essence a
> boolean flag either derived from the file extension, file header, or simply
> put in as seperate meta-information with the file) seems to be simple.
> 
I do understand the logical separation argument, and in fact I've designed
systems in the past where the control was on one piece of hardware and the
RTP data processing was on another. However, in those cases there generally
was still some other internal control channel opened between the two systems
and you generally already needed to be passing messages between them when
streams are started up. Passing the RTP-Info at this time is straightforward,
if you really need it. In many cases, you can probably also get by without
it, though, if you had to.

As for changing the server to know about whether a particular track is
audio, the reality is that there are a large number of file formats out
there which aren't going to be easy to change at this point, and many of
them are containers for arbitrary types of content with no indication of
whether it is audio or not. While it would be possible to have some
additional external information of this sort, that's not very practical.
The server would have to have some kind of process where it scanned new
content to figure out what all the tracks were and then prompted the user
about each should be treated as audio. I just don't see this be worthwhile
for the small amount of gain we're achieving here. We also run the risk of
an audio payload format coming along which uses the marker bit in some
other way (which a payload is free to do) -- do we really want a server
changing it in cases where it doesn't know the details of the audio
payload type?
--
Ron Frederick
[email-masked]

---

From: Sean Sheedy <[email-masked]>
Date: Thu, 07 Sep 2000 10:04:30 -0700

Thanks, Ron, for the excellent summary.

Some belated thoughts on a few of the issues raised are interspersed
below.

> this should be independent of which control connections are used. Other

A SETUP without a preceding DESCRIBE should be allowed.  I agree that
tying state  to a specific control connection via DESCRIBE is a bad
idea. It simply doesn't work for non-persistent control connections such
as UDP.

It may be worthwhile to allow client to combine the features of SETUP
and DESCRIBE in the same request.  For example, if a new header is
included in a SETUP request (e.g., "PrepareAsset"), the server could
return the DESCRIBE data in SETUP response body.  Opinions on this?

> this at least be a "SHOULD".

Not all servers or media types can support this.  E.g., dynamically
breaking apart an already combined MPEG-2 presentation into its
elementary streams is awkward at best.

> Further work is needed to reconcile all of these cases.

What if the server returns a dynamically created aggregate URL in the
SETUP response?  That way the client has an unambiguous name to use to
specify the entire collection.  For example:

C->S  SETUP stream1
S->C  OK Session=1234 URI=myaggregate1
C->S  SETUP stream2 Session=1234
S->C  OK Session=1234 URI=myaggregate1
C->S  PLAY Session=1234 URI=myaggregate1
S->C  OK

> the packets which were transmitted when playback finishes.

We (nCUBE) have implemented queued PLAY.  MPEG-2 doesn't have the time
stamp problems, etc., on transitions, fortunately.  Even so, we've found
callbacks on transitions useful for clients, who typically want to track
the presentation state.

> requiring session IDs to be returned?

Yes.  This was already my interpretation, and the interpretation of the
client writers I've worked with.  Without requiring a session ID to be
returned on SETUP, servers would have major problems properly handling
non-persistent control connections.

> must support here would be useful.

I agree that this needs to be clarified.  Clients need to know exactly
what is required of them to maintain their sessions with a minimum of
overhead.  This is particularly important for delivery mechanisms (such
as MPEG-2) that don't contain a built in backchannel from the client to
the server.

I'd like to propose two complementary liveness detection mechanisms
which are independent of the content delivery mechanism: explicit and
implicit.  Explicit detection uses a reference to a particular session
to update the liveness indicator for that session.  Implicit detection
uses activity on a control connection to update the liveness indicator
for all sessions associated with that control connection.

Explicit liveness detection uses the PLAY and GET_PARAMETER methods as
mentioned in the RFC.  Any request received from a client which
explicitly references a session resets the session time-out counter. 
This mechanism is most useful in environments where each client controls
a small number of sessions, particularly one which is not using
persistent control connections.

Implicit liveness detection resets the time-out counter for all sessions
associated with a client connection any time a syntactically valid RTSP
request is received on the connection.  For this purpose, a stand alone
CRLF could count as a syntactically valid RTSP request, although it
would not result in any response from the server.  Note that this
mechanism is not meaningful when using connectionless protocols
such as UDP for sending requests.

Implicit liveness detection provides a very low overhead, scalable
liveness mechanism in environments which use persistent connections.  In
particular, RTSP clients which need to control many sessions
simultaneously benefit from this strategy.

Note that this proposal doesn't address which type of liveness detection
must be supported by a server, or how a client can determine which type
a server allows.

> default for some transports like TCP?

Yes.  Defaults should never be nonsensical.

> _isn't_ the first port + 1, what does that mean?

The ports should always be specified explicitly.  Assuming that an
unspecified port means "add 1" is not valid for non-RTP transports. 
Allowing a generic list of ports rather than just a range would be good. 
I'm not sure of the right syntax for this, though.  It can't be a comma
separated list since commas are already used for lists of transports.

Sean

-- 
Sean Sheedy                                         [email-masked]

---

From: [email-masked] (Ron Frederick)
Date: Thu, 07 Sep 2000 12:30:37 -0700

Hi Sean...

> Some belated thoughts on a few of the issues raised are interspersed
Thanks -- my replies are below. I'd love to see other opinions here as well.
Many of the things I put into the summary had no clear answers yet, but it'd
be nice to get some more discussion around them so they can be addressed
somehow in future versions of the spec.

> idea. It simply doesn't work for non-persistent control connections such
Agreed.

> return the DESCRIBE data in SETUP response body.  Opinions on this?
In the general case, this could be problematic, as the URL sent on a DESCRIBE
is often different than the one(s) sent on a SETUP. You actually need the
information in the DESCRIBE reply to decide which SETUP message(s) you need
to send. I was thinking more of the case where the DESCRIBE was done
previously or where you got the description by other means when I first
raised this issue, and in those cases you wouldn't need the description to
be sent again on the SETUP. If you do need it but you don't need it before
sending the SETUP, you can always have your client pipeline the DESCRIBE and
SETUP commands in a single message, and then process both replies. That
should be about as quick from a network standpoint as one combined request,
and wouldn't require any protocol changes.

> elementary streams is awkward at best.
An already-combined MPEG-2 presentation isn't likely to show up as an
aggregate object of the kind described here, though. The only time this is
an issue is when the SDP description lists multiple tracks that need to each
be SETUP separately. In such a case, the server is clearly capable of
splitting up the data, as it is sending it to two different destinations.
The only question is whether a client should have the ability to ask it to
send one of those two streams and not the other (or more generally some
subset of all the streams).

> C->S  PLAY Session=1234 URI=myaggregate1
In a case like this, I'm still trying to understand what value the URI
provides at all in the "PLAY", when you already have the Session field. In
your example, you could do just as well leaving out the URI in the PLAY and
just using a "*", along with Session: 1234.

The one place where I could see it potentially being useful would be if you
wanted to set up several different pieces of content in the same session
with the intent of playing alternate versions of them rather than playing
all of them simultaneously. In that case, the URI in the PLAY could let you
specify which subset of the items to play. This sounds complicated to
implement from a server standpoint, though, and I'm not sure we'd want to
mandate something like that in the spec.

> callbacks on transitions useful for clients, who typically want to track
How do you know what sequence number to return in the RTP-Info in the case of
queued play, when you haven't finished playing the previous piece(s)? Also,
when you say that MPEG-2 doesn't have the timestamp problem, is that just
because all of your movies have a known length up front and a common RTP
clock rate, or is there something more to it than that?

> non-persistent control connections.
Agreed. Does anyone else think this would be an unreasonable change?

> [clarification of session timeouts]
> 
I can see value in each of these approaches, but I'm not sure we'd want to
complicate things by allowing servers to support only one or the other.
There can also be some questions about what it means for a session to be
"associated with a client connection". If you have multiple connections
open from a client that are sending commands about a particular session,
which one is the session associated with? Also, if the client drops a
control connection and opens a new one but hasn't sent any commands about
that session yet, how would the server know it to associate that session
with the new connection?

The explicit approach is definitely simplest. In addition, though, I think
we definitely want to allow for RTCP-based liveness, so that clients don't
necessarily need to keep the control connection open at all during a
long playing stream. I'm not really sure we need to worry much about a
single control connection having so many sessions that an implicit type
of approach is required, but if that is an issue, I think we need to very
clearly define how streams are associated with a given connection...

> Yes.  Defaults should never be nonsensical.
The problem here is that you need semantic information to realize that the
default is nonsensical in this case. If a server is doing some kind of
simple pattern matching on transports, it needs a special case for each
type of transport now to know whether "unicast" or "multicast" is implied
when neither is specified... This might be worth doing, but there definitely
is a cost to it and some serious backward compatibility issues to work out.

> separated list since commas are already used for lists of transports.
I would also vote in favor of always specifying both ports. Again, though,
we need to decide how to deal with backward compatibility.
--
Ron Frederick
[email-masked]

---

From: Jonathan Sergent <[email-masked]>
Date: Thu, 07 Sep 2000 13:43:22 -0700

/// [email-masked] (Ron Frederick):
 ] In the general case, this could be problematic, as the URL sent on a DESCRIBE
 ] is often different than the one(s) sent on a SETUP. You actually need the
 ] information in the DESCRIBE reply to decide which SETUP message(s) you need
 ] to send. 

I was going to say the same thing.

[...]
 ] The only time this is
 ] an issue is when the SDP description lists multiple tracks that need to each
 ] be SETUP separately. In such a case, the server is clearly capable of
 ] splitting up the data, as it is sending it to two different destinations.
 ] The only question is whether a client should have the ability to ask it to
 ] send one of those two streams and not the other (or more generally some
 ] subset of all the streams).

I agree, but I wonder how to handle this situation:

DESCRIBE $A
	(session description includes stream URLs $b, $c, and $d)
SETUP $b
SETUP $c
PLAY $A
	($b and $c start playing)
SETUP $d

All SETUPs and PLAYs are on the same RTSP session.  Does stream d start
playing immediately, in sync with the rest of the presentation?  Does
the server send 455?  The RFC is unclear.  Banning it is probably
unnecessary, but it's unclear whether you have to send another PLAY in
order to get it to start streaming (I would say "yes").  This is
potentially useful in situations where the client knows that one stream
does not begin for several minutes and so it can avoid the extra
resource utilization in the mean time.

[...]
 ] The one place where I could see it potentially being useful would be if you
 ] wanted to set up several different pieces of content in the same session
 ] with the intent of playing alternate versions of them rather than playing
 ] all of them simultaneously. In that case, the URI in the PLAY could let you
 ] specify which subset of the items to play. This sounds complicated to
 ] implement from a server standpoint, though, and I'm not sure we'd want to
 ] mandate something like that in the spec.

I think it's ambiguous, too, given this text in the spec, under PAUSE:
   If the request URL names a stream, only
   playback and recording of that stream is halted. For example, for
   audio, this is equivalent to muting.

If the server allows PLAY of only part of what has been SETUP (as you
describe above), then how does it distinguish an enqueued PLAY request
for only one stream from a request to restart PLAY of that stream after
it has been paused?  For example:

DESCRIBE $A
SETUP $b
SETUP $c
SETUP $d
PLAY $A\nRange: npt=0-60
PAUSE $b\nRange: npt=10
[30 seconds pass]
PLAY $b

Does $b restart in sync with $A, or does it start playing from start to
finish once $A has completed?  If there is a Range header, it is
clearly an enqueued request, since there is no way to say "start
playing in sync with the rest of the presentation" in Range... there
probably should be.

Also, if the client PAUSEs only one stream out of the presentation,
then it PAUSEs the entire presentation, then it PLAYs the entire
presentation, does the previously muted stream stay muted?

[...]
 ] The explicit approach is definitely simplest. In addition, though, I think
 ] we definitely want to allow for RTCP-based liveness, so that clients don't
 ] necessarily need to keep the control connection open at all during a
 ] long playing stream. I'm not really sure we need to worry much about a
 ] single control connection having so many sessions that an implicit type
 ] of approach is required, but if that is an issue, I think we need to very
 ] clearly define how streams are associated with a given connection...

I agree.

Another possibility would be to send a "ping" GET_PARAMETER request but
with more than one session ID listed, to ping all of the sessions in
one request.

 ] The problem here is that you need semantic information to realize that the
 ] default is nonsensical in this case. If a server is doing some kind of
 ] simple pattern matching on transports, it needs a special case for each
 ] type of transport now to know whether "unicast" or "multicast" is implied
 ] when neither is specified... This might be worth doing, but there definitely
 ] is a cost to it and some serious backward compatibility issues to work out.

How about getting rid of the default and requiring one or the other?

-- 
Jonathan Sergent / [email-masked]

---

From: Dirk Pranke <[email-masked]>
Date: Thu, 07 Sep 2000 21:16:15 -0700

Hi Ron,

Here are my comments, based on Oracle's RTSP implementation in the
Oracle Video Server (OVS), which is quite similar to nCUBE's ... I apologize 
for the length of the post, but I think it's necessary to preserve context.

> > idea. It simply doesn't work for non-persistent control connections such

Me too.

> and wouldn't require any protocol changes.

Clearly the general case is problematic, but it should be allowed for some
cases; we have situations where we can send the SETUP w/o needing a prior
DESCRIBE, but we still need other information that could be contained in the
description. There are two potential problems trying to work around this
with DESCRIBE:

 (1) a general problem with DESCRIBE is that its stateless whereas SETUP is
     stateful, and the DESCRIBE and SETUPs are not temporally aligned. For
     example, OVS has assets whose descriptions change in real time, and it
     is important to freeze the description that was in place at SETUP time.
     For this reason, DESCRIBE was useless to us and our initial implementations
     didn't even support it (we do now).

 (2) Unless I'm missing something, authorization concerns may prohibit a stateless 
     DESCRIBE from being pipelined with a SETUP, or more generally, _all pipelining_ 
     (a rolling nonce in the Authorization-Info: can cause a pipeline stall).

> subset of all the streams).

Yes, and I would think it should be.

> mandate something like that in the spec.

Yup. It seems like if you try to mix aggregate and individual operations, you're
in for a world of pain.

> clock rate, or is there something more to it than that?

Not all the world uses RTP :) If you're streaming raw MPEG-2 Transport over IP
or other types of networks, then a decent MPEG-2 client will recover from
timestamp discontinuities, and doesn't need RTP-Info sequencing information
(although it doesn't hurt, either).

> Agreed. Does anyone else think this would be an unreasonable change?

I think a returned Session: should be mandatory.

> clearly define how streams are associated with a given connection...

I certainly think the spec should clearly describe how to explicit liveness
over the control connection, and I would not be offended if implementations
needed to support that for a minimum level of interoperability (perhaps as
a negotiable option). There are certainly implementations out there that
can also do RTCP-based liveness, and that seems valid to me as well, so I
would not mind if that was negotiated as well. I think the spec's current
vagueness is not acceptable, as it has clearly _already_ caused interoperability
problems.

I myself am not fond of the implicit approach, because it does have many of the
problems you describe, and to me it is aesthetically unappealling. 

The case Sean was describing where you would have a client controlling many
streams is where you have an RTSP client that is acting as a bridge for other
protocols (e.g., MPEG DSM-CC, or legacy streaming protocols). Oracle's server
introduced the concept of a "bridge" into RTSP, and has SETUP_BRIDGE and 
TEARDOWN_BRIDGE and other mechanisms to indicate aggregation and control. I
think another way to accomplish the same thing would be to simple set up sub-streams
on a single session.

One possible complication for all of this: I don't recall the spec being clear
as to whether or not you _could_ send commands for multiple streams over a single
transport connection. I believe we thought that that wasn't allowed, e.g. a transport
connection had at most one session associated with it. If a connection can have
multiple sessions, then the overhead can also be reduced.

> is a cost to it and some serious backward compatibility issues to work out.

I would get rid of the defaults.

> we need to decide how to deal with backward compatibility.

Yup.

-- Dirk

Dirk Pranke
Oracle Interactive Television Division

---

From: Henning Schulzrinne <[email-masked]>
Date: Fri, 08 Sep 2000 13:34:04 -0400

Dirk Pranke wrote:

> > somehow in future versions of the spec.

I will be trying to edit in changes. Thus, it would be helpful if you
could point out also where in the spec you suggest changes.

> > > idea. It simply doesn't work for non-persistent control connections such

I'm trying to find places in the spec that require DESCRIBE before
SETUP.

> Yes, and I would think it should be.

Added in SDP section:

A client is \textbf{not} required to issues {\SETUP} requests for all
streams within an aggregate object. Servers {\SHOULD} allow the client   
to ask for only a subset of the streams.  

> (although it doesn't hurt, either).

And even good RTP implementations will work just fine with timestamp
and/or sequence number discontinuities. These can occur for any number
of reasons.

> > Agreed. Does anyone else think this would be an unreasonable change?

Tentatively changed the 'all but SETUP, OPTIONS' entry to just OPTIONS.

> I think a returned Session: should be mandatory.

Text in Session section changed to:

The session
identifier is chosen by the media server (see
Section~\ref{sec:session-id}) and {\MUST} be returned in the {\SETUP} 
response.  Once a client receives a \header{Session} identifier, it
{\MUST} return it for any request related to that session.

> vagueness is not acceptable, as it has clearly _already_ caused interoperability

I assume we're talking here about server detecting client liveness.

One approach is to use the SIP session timer mechanism, in combination
with Require/Supported. That way, the server can decide whether it can
either live with clients that don't support it or not. I don't think it
would hurt to have a mechanism that's similar to the SIP one, as it's
already documented.

http://www.cs.columbia.edu/~hgs/sip/drafts/draft-ietf-sip-session-timer-02.txt

> multiple sessions, then the overhead can also be reduced.

I don't recall that restriction being the case. Would there be any
reason to disallow this, given that the server can easily distinguish
the requests? This would be in line with HTTP and SIP usage, too.

> I would get rid of the defaults.

Is there a problem with existing clients that omit the parameter, based
on the default assumption? Otherwise, I've removed the sentence and
added a word about having to insert one or other other (unicast or
multicast).

> > we need to decide how to deal with backward compatibility.

The +1 rule only applies to RTCP and is meaningless elsewhere.

Currently, 'port' is only described specifically for RTP, where a pair
(range) is required.

Thus, I don't see the problem except that maybe 'port' needs to be
defined for non-RTP sessions.
-- 
Henning Schulzrinne   http://www.cs.columbia.edu/~hgs

---

From: [email-masked] (Ron Frederick)
Date: Fri, 08 Sep 2000 15:52:24 -0700

> resource utilization in the mean time.
This is a good question, and I'm not sure the spec clearly defines the
behavior right now. I would say the server is always free to send a 455,
but if it doesn't it shouldn't play $d unless/until you send another PLAY
command. Sending a PLAY without a Range should probably be good enough,
though, and would basically cause $d to pick up in the middle at exactly
the same play point as the other currently active streams.

> playing in sync with the rest of the presentation" in Range... there
With no Range header, I'd say it definitely would start up immediately in
sync with the other streams currently playing. With a Range header, I guess
the only thing that would make sense would be a queued play, as there's no
current support for a single session having multiple streams playing at
different points simultaneously.

> presentation, does the previously muted stream stay muted?
I'd say a PAUSE of the whole presentation resets all state about individual
streams. If you PLAY the aggregate URL, it plays all of the streams.

> with more than one session ID listed, to ping all of the sessions in
Yes - Session could be one of the headers that allows either a comma-separated
list or multiple header instances, at least in this case.
--
Ron Frederick
[email-masked]

---

From: [email-masked] (Ron Frederick)
Date: Fri, 08 Sep 2000 16:39:36 -0700

> could point out also where in the spec you suggest changes.
Thanks, Henning -- I'll try to go back through my list of issues and the spec
and see if there are specific suggestions I can make.

> I'm trying to find places in the spec that require DESCRIBE before
I don't think anything in the spec currently requires this, and indeed many
of the implementations didn't require it. However, the question was raised
about whether it was mandatory for servers to support SETUP without a
preceding DESCRIBE. If we believe it should be (and I do), it would be good
to add that to the spec.

> to ask for only a subset of the streams.  
Sounds good.

>> Not all the world uses RTP :) If you're streaming raw MPEG-2 Transport over
>> IP or other types of networks, then a decent MPEG-2 client will recover from
Both of these statements are evading the point of my question. According to
Appendix B of the spec, a server should maintain continuous sequence numbers
and timestamps when playing multiple segments back to back. How exactly can
it do that and still return a timely answer (with proper RTP-Info) to a
queued PLAY, though, when it doesn't know yet how many packets will be in
the currently playing segment? How would _you_ suggest a server implement
this for the RTP transport?

The only reasonable thing I can think of here is what I put in the original
summary, where a different message is sent from the server to a client when
a segment finishing playing, indicating the sequence number and timestamp of
the next segment (which may or may not already be queued up). This could also
serve as a "stream done" notification, which others have asked for.

> Tentatively changed the 'all but SETUP, OPTIONS' entry to just OPTIONS.
Sounds good.

> would hurt to have a mechanism that's similar to the SIP one, as it's
What would you use instead of re-INVITE here? I don't think we want to be
resending the SETUP. It might be ok to re-send the PLAY, but that only
works when the stream is actively playing something. We need this mechanism
to work even when the session is paused. It might be possible to use
SET_PARAMETER, but this header would be in the main command header, not the
content, so it isn't all that good a fit either.

To be honest, I'm not sure we need as complicated a mechanism as this for
RTSP. The current mechanism of using PLAY/GET_PARAMETER "pings" plus RTCP
(when available) is probably good enough. The main issue I think that needed
clarification was the distinction between RTSP session and transport level

> the requests? This would be in line with HTTP and SIP usage, too.
I've always assumed that there was no correlation at all between transport
connections and RTSP sessions. This meant that you were free to use a single
connection for as many sessions as you liked and that you were free to send
commands for a single session over several different transport connections
at once if you liked. This is stated pretty well in section 1.1, I think.
However, I'm not sure it is remembered when people read other sections of
the document. :)

I'm not sure exactly where it would be best to put it, but I think it would
be good to emphasize somewhere that the mechanism for timing out sessions is
independent of the transport-level control connections, and vice-versa. The
latter is particularly important -- a session should _not_ be cleaned up just
because a client drops their control connection.
> added a word about having to insert one or other other (unicast or
Yes - compatibility with existing clients and servers was my main concern
here. Some clients seemed to put "RTP/AVP/TCP;unicast" and others didn't.
Similarly, some servers _required_ "unicast" to be present for it to work
and others didn't. Moving forward, I think the cleanest thing is probably
to always require "unicast" or "multicast" to be specified explicitly, but
there will still be some time where we'll have both clients & servers out
there which may not specify it.

> defined for non-RTP sessions.
The current grammar doesn't actually require it to be a pair/range. It allows
it to be a single port number. It also doesn't currently specify the "+1"
default in that case.
--
Ron Frederick
[email-masked]

---

From: [email-masked] (Ron Frederick)
Date: Fri, 08 Sep 2000 16:53:02 -0700

> for the length of the post, but I think it's necessary to preserve context.
Thanks, Dirk - see my follow-ups below...

>      didn't even support it (we do now).
I think part of the confusion here is that there are really two different
types of information being discussed... There's the general (typically
fairly static) information about an asset that you need to know before you
can even issue a set of SETUP messages for it and there's some additional
information which you might want to know after issuing the SETUP that is
specific to the particular state that was established there.

In RTSP, there's currently very little state in the second category. About
the only thing is the Transport information, which is returned in the SETUP
reply. For a server like what you are describing, I think it is perfectly
reasonable to have other headers in the SETUP reply and/or actually use the
content body of the SETUP reply to return this information. If all the
client needs to know is the URL, it might be perfectly reasonable for it
to issue a SETUP without a DESCRIBE in a system like this.

>      (a rolling nonce in the Authorization-Info: can cause a pipeline stall).
Yes, that's a good point. That's a limitation of the authentication scheme,
though. It would be possible to design one with rolling nonces that could
still allow several parallel commands to be issued, as long as each had its
own independent chain of nonces and a way of saying which chain the next
command was following on from.

I think I covered the rest in my reply to Henning...
--
Ron Frederick
[email-masked]

---

From: Dirk Pranke <[email-masked]>
Date: Fri, 08 Sep 2000 16:59:30 -0700

Ron Frederick wrote:

 ...

> this for the RTP transport?

I was simply trying to point out that you *can* implement queued play without
needing to worry about RTP-Info, since you it's irrelevant if you're not
using RTP. I agree with you that you do have the problem if you're using RTP.

-- Dirk

Dirk Pranke

---

From: Dirk Pranke <[email-masked]>
Date: Fri, 08 Sep 2000 17:03:14 -0700

Ron Frederick wrote:

  ...

> to issue a SETUP without a DESCRIBE in a system like this.

Yup, and the way we did this is to optionally indicate that we wanted the full 
description as the content-body of the SETUP.

> command was following on from.

Yes, this would be possible but somewhat weaker, and also more complicated to 
implement since you'd need to track the valid nonces in a more complicated manner.

-- Dirk

Dirk Pranke

---

From: Jonathan Sergent <[email-masked]>
Date: Fri, 08 Sep 2000 17:19:28 -0700

/// [email-masked] (Ron Frederick):
 ] With no Range header, I'd say it definitely would start up immediately in
 ] sync with the other streams currently playing. With a Range header, I guess
 ] the only thing that would make sense would be a queued play, as there's no
 ] current support for a single session having multiple streams playing at
 ] different points simultaneously.

I misread the section about PLAY with no range.  Sorry for the confusion.

I think that there is a race when the client sends a PLAY request
without a Range header, very close to the when the server is scheduled
to stop playing the current range.  The server automatically returns to
the Ready state when the range ends.  The incoming request will be
interpreted differently depending on whether it arrives before or after
the server's state change.

I think there is a similar race for PAUSE near the end of the range.
PAUSE should probably be a no-op in the Ready state.  Having the server
return an error seems wrong (but maybe it's not, and the client just
needs to be written to deal with this and hide it from the end-user).

---

From: Sean Sheedy <[email-masked]>
Date: Tue, 12 Sep 2000 09:44:21 -0700

> mandate something like that in the spec.

I'm looking for a way to combine this use of "*" with that proposed in
my draft for use when changing URI's on PLAY.  How about the following,
which is admittedly a bit rough around the edges:

A "*" for the URI always matches the session's active URI.  The active
URI is maintained as part of the session state, and changes as the
session changes state.  At a high level, the active URI is one of: 
   - The currently playing URI, if in the playing state.
   - The previously played URI, if in the ready state and a previous
     play occurred.
   - The collection of all SETUP URI's, if in the ready state and no
     previous play occurred.

We can explicitly specify the changes to the active URI by adding to the
state diagram from the RFC:

 state       message received   next state   active URI becomes
 Init        SETUP              Ready        request
             TEARDOWN           Init         0
 Ready       PLAY               Playing      request
             SETUP              Ready        active + request
             TEARDOWN           Init         0
             RECORD             Recording    request
 Playing     PLAY               Playing      request
             PAUSE              Ready        active (no change)
             TEARDOWN           Init         0
             SETUP              Playing      ???
 Recording   RECORD             Recording    request
             PAUSE              Ready        active (no change)
             TEARDOWN           Init         0
             SETUP              Recording    ???

Definitions:
  active  = active URI
  request = URI in request

It's not clear to me what the proper behavior is when a SETUP is
received in the playing or recording states.

A "*" URI which matches a null URI is an error.  In other words, a
"SETUP * ..." from init state is illegal.

> clock rate, or is there something more to it than that?

As Dirk has pointed out, this isn't an issue when delivering MPEG-2
without RTP encapsulation (our usual environment).

> clearly define how streams are associated with a given connection...

After reading the rest of the discussion about this issue, I've come
to agree that explicit references to sessions and RTCP-based
liveness are the mechanisms appropriate for the spec.  I think the
following still needs to be added:

 - A well-defined mechanism for client and server to negotiate
   an appropriate liveness strategy.  SIP or a subset thereof
   seems like the appropriate approach.

 - The option for clients to use multiple session identifiers
   on GET_PARAMETER liveness messages.  This would minimize the
   overhead for clients controlling multiple sessions (e.g.,
   the bridge servers in Dirk's previous message).

Sean

(503) 531-6427