Subject: Presentations & MUTE in RTSP

From: Eric Fleischman <[email-masked]>
Date: Mon, 21 Apr 1997 09:16:56 -0700

I think that there is widespread agreement in the RTSP community that
MUTE should be optional. I certainly haven't heard anybody requesting
that it even be a recommended feature. Thus, I don't think that this is
at issue here. Rather, my push-back is concerning where it is
appropriate for requests such as MUTE to be used.

The nature of my concern is about the concept of Presentation in RTSP
(see definition in section 1.3). Many/most/all people of this list come
from an MBONE background and naturally think in terms of segregated
multimedia streams such as what commonly appears on the MBONE today.
That is great. However, the majority of existing multimedia content was
created by the CD-ROM community. Vendors of streaming engines (such as
ourselves) are very interested in being able to stream *both* types of
media content (i.e., CD-ROM and MBONE).

While both types of content (CD-ROM, MBONE) could theoretically be
treated as RTSP PRESENTATIONS, the CD-ROM content is explicitly designed
to be *exclusively* treated in that manner. Most/much of this content is
interleaved. This content was designed to be treated as a unit and the
"splitting out" of various media streams from the unit at the server-end
takes quite a bit of overhead. The serving out of this CD-ROM content
(when it is treated as a unit as it was designed) has minimal overhead,
thus permitting servers of such "bundled data" to be able to
simultaneously satisfy huge number of users. 

[An aside: Streaming Servers need to support huge number of users if
they support "on-demand" uses. However, they also need to support large
number of users for a single "live" event because multicast is not yet
universally deployed. Thus, when one announces a multicasted event,
streaming servers also need to support N number of unicast sessions for
the same event for those users who are unable to receive multicast.
Thus, server performance is the single most critical element when
designing streaming products.]

Thus, my point is that the implication of MUTE varies tremendously
vis-a-vis whether the data is originally segregated streams (as in
MBONE) or whether it is originally a "unit" (as in CD-ROM technology). 

Because the handling of segregated streams is significantly more
computationally intensive at the server than handling CD-ROM content, I
can well imagine streaming products saying something like "anybody who
uses segregated streams obvously doesn't care all that much about server
performance. Therefore, let them have MUTE for that type of data since
the overhead of MUTE is minimal compared to the "big whack" in
performance associated with segregating the streams to begin with.
However, RTSP Presentations (i.e., CD-ROM data) will not support MUTE
because that would introduce large overheads into the performance."

Thus, in this hypothetical situation, a single product may support MUTE
for one data approach but will not support MUTE for another data
approach. I would like to community to be aware of this so that we could
hopefully tailor MUTE (and other alternatives like it) appropriately.

Please note that this is not a matter of data communications transports
in that both CD-ROM and MBONE data may well be carried via RTP. In the
former case, the whole CD-ROM bundle will be carried over a single RTP
session. The RTP payload type will be establed due to the information
contained in the DESCRIBE method (most probably via the second example
in section 9.2) and a dynamic payload type will then be assigned to
carry that "bundle".

> -----Original Message-----
> From:	Gray Cope [SMTP:[email-masked]]
> Sent:	Friday, April 18, 1997 11:06 PM
> To:	Eric Fleischman
> Cc:	[email-masked]
> Subject:	Re: MUTE in RTSP
> 
> Eric,
> 
> I think I understand your concern regarding some content creators
> not wanting their "multimedia bundle" altered by muting individual
> streams within their presentation.
> 
> So if the RTSP specification did not _require_ the MUTE request to be
> implemented would that address your concerns?
> 
> Ultimately the content creator or distributor can exercise control
> over how their presentation is delivered.  That is they can specify
> that their content not be delivered by a service that implements
> the MUTE request.
> 
> Gray
> 
>

---

From: Karl Auerbach <[email-masked]>
Date: Mon, 21 Apr 1997 12:03:30 -0700 (PDT)

> MUTE should be optional.

I haven't seen a concensus call, but, as I have said in e-mail to this
list, I'm against optional features in general.  Optional parts tend to
cause interoperperability problems, are undertested, and tend to make
users unhappy.  

Can you imagine how you would feel if the caller could tell your telephone
that the "mute" button on your phone should be inoperative for a
particular call? 
 
As far as my contribution to concensus goes, it is my feeling that if
there is a presentation running and the presentation is composed of
distinct streams[*], then the person in control of that presentation
should have the non-optional ability to tell the server(s) which of the
streams should be advanced with the flow of presentation time but the
data should not be transmitted onto the network.

[*] There's actually a somewhat fuzzy line here -- some "encodings",
eg. mpeg, have both audio and video mixed in a way that a server could
separate them, but often at considerable processing cost.  I would say
that a server need not be required to "mute" one of these streams
independent of the other.  The fuzzy line is where an "encoding" ends
and a file format begins.  I realize that from the point of view of
the person at the receiving end that this should make no difference
and contradicts my example, above, about the on-again-off-again action
of the mute button on someone's telephone.

> .... However, the majority of existing multimedia content was
> created by the CD-ROM community. 

I disagree.  The vast bulk of multimedia content is created daily by
people communicating with one another.  Add to that the professionally
generated broadcast TV and CDROMs become nearly invisibile as a tiny,
almost insignificant, speck.

		--karl--

---

From: Henning Schulzrinne <[email-masked]>
Date: Mon, 21 Apr 1997 19:16:13 -0400

In the case of 'bundled' (CD ROM) presentations shipped as a single RTP
stream, that stream is also a single RTSP stream (single RTSP URL). It
wouldn't really make any sense to pause/mute/play anything but the whole
show. If you allow anything else, you are back to having to unbundle and
reassemble the CD ROM presentation. Thus, MUTE for the whole show works
just fine (whether it's generally useful in that case is a different
issue).

We should also note that several bundled and unbundled streams may be
merged into a 'virtual presentation' by the receiver. In that case,
MUTEing may be useful, even if you mute the whole bundled stream. The
server of the bundled stream doesn't know about this and doesn't care -
this is strictly done by the receiver.

We have a mechanism (OPTIONS) where a client can easily detect the
availability of MUTE, etc., so you don't have to wait for a 5xx error
message. However, if MUTE is not generally supported in servers, smart
clients would want to fall back to PAUSE/PLAY, so that MUTE doesn't buy
much in terms of simplifying clients.

Suggestion: we can specify it as optional, since there will likely be
some servers that want to implement it and they might as well all do it
in the same way.

---

From: Stephen Casner <[email-masked]>
Date: Tue, 22 Apr 1997 16:09:24 -0700 ()

Henning,

> just fine (whether it's generally useful in that case is a different

This whole debate was moot because the bundled stream Eric wants to
send is a single RTSP stream so it is not even possible to express a
desire to MUTE only part of it.  So, the presence of the MUTE function
shouldn't cause any burden for Eric's server.

> this is strictly done by the receiver.

Right, so there is still value in having MUTE for the bundled stream
as a whole.

> much in terms of simplifying clients.

If you make MUTE optional so that clients have to be prepared to use
PAUSE/PLAY instead, then you might as well not have it.  Clients would
just implement PAUSE/PLAY and ignore MUTE unless the result was
significantly better in some way.

As an aside, for the next debate I suggest that we focus on the real
technical issues underlying the usage scenarios and avoid red herrings
such as the question of author control over a presentation.  Even in a
cinema, one can cover his eyes or plug his ears to mute one of the
individual "media".  Unless, of course, he is in Malcolm McDowell's
seat in "Clockwork Orange".
							-- Steve

---

From: Sampo Syreeni <[email-masked]>
Date: Wed, 23 Apr 1997 16:15:00 +0300 (EET DST)

>I think that there is widespread agreement in the RTSP community that

Or that it should not be (at all).

>designing streaming products.]

Agreed. Furthermore, if one uses segregated data streams with multicast
sessions, one is bound to have some problems if all the synchronization
burden is pushed to the client end. The clients should largely be blind,
as far as the application viewer is concerned, as to whether the streams
come from multicast or unicast sources. This makes heavy client side
buffering more difficult.

>hopefully tailor MUTE (and other alternatives like it) appropriately.

Such unnecessary polymorphism in the server environment should not, in my
oppinion, be condoned.

Sampo Syreeni (Decoy/dAWN), student, <[email-masked]>

---

From: Sampo Syreeni <[email-masked]>
Date: Wed, 23 Apr 1997 16:44:35 +0300 (EET DST)

>that the "mute" button on your phone should be inoperative for a

This isn't the problem here. In the context of RTSP, MUTE is just an
optimization. Nobody tells you that you cannot turn display of some single
stream off and still have it coming in from the network.

>almost insignificant, speck.

Multimedia as in more than direct AV broadcast. That means true
presentations and controllable content in this context.

---

From: Sampo Syreeni <[email-masked]>
Date: Wed, 23 Apr 1997 16:52:35 +0300 (EET DST)

On Mon, 21 Apr 1997, Henning Schulzrinne wrote:

>show.

PAUSE no. MUTE yes. You wouldn't want to pause the soundtrack to a movie
only to discover that it is out of sync when you start it. But you sure as
hell would like to mute it if you got tired of hearing the ear piercing
whistle they used...

>If you allow anything else, you are back to having to unbundle and

Let's assume you're deaf. Would there not be sense in MUTEing the
soundtrack of a movie if a text was available?

---

From: Stephen Casner <[email-masked]>
Date: Thu, 24 Apr 1997 12:25:19 -0700 ()

On Wed, 23 Apr 1997, Sampo Syreeni wrote:

> >It's not clear to me what the server serving a single QuickTime/ASF
> >movie would do when told to PAUSE the audio. It appears it would have to
> >do the same disassembly or just offer QuickTime movies as a single
> >entity.

Nothing.  There is no way to express that PAUSE request because RTSP
knows only about the single combined stream.

> Furthermore, if one uses segregated data streams with multicast
> burden is pushed to the client end.

Not true.  The burden is at the client end no matter how the data gets
there, because the client end is where the presentation occurs.

> The clients should largely be blind,
> come from multicast or unicast sources.

True, but the synchronization effort is the same for multicast or unicast.

> >Whether the media streams arrive at the receiver unbundled or not, because
> >the time-to-render characteristics of each kind of rendering hardware
> >(audio/video/etc), the receiver must often unbundle them and do
> >considerable amounts of inter-stream time biasing in order to get get them
> >to be rendered at the same time.
> 
> But such buffering is easier to do because you do not have to take into
> account the delay jitter of the network, ...

Not true.  To achieve continuous playback, you need to provide
sufficient buffering with either transport method.  The minimum
buffering delay is achieved by putting all the buffering together in
one place, accommodating all delay sources, and dynamically adapting
that to the minimum value.

> There are also bound
> to be more synchronization problems over a heterogeneous cloud of routers
> and networks when operating with segregated data streams than with
> interlaced ones.

Not really, unless some streams are given priority over others.  But,
in fact one may want to do that when achieving low latency on one
stream is more important than presenting the streams in sync.

> soundtrack of a movie if a text was available?

This is indeed one of the motivations for sending the audio and video
streams separately.
							-- Steve

---

From: Sampo Syreeni <[email-masked]>
Date: Mon, 28 Apr 1997 15:16:29 +0300 (EET DST)

>knows only about the single combined stream.

Assuming that we allow composite media streams which is (I gather) what
this discussion has been all about. If one allows such interlaced,
composite streams as full QuickTime/ASF, no problems arise in this regard.
However, it's not clear whether one should endorse multiple independent
stream formats like this. Another way to do the same thing would be to
disallow composite streams and deliver all data as interdependent
monomedium streams (audio, video, text etc.). This approach enhances
orthogonality (one would probably use only one or two different stream
types for a single type of media), but requires disassembly and
independent control of the constituent streams at both ends of the
delivery channel. 

I'm strongly for allowing composite streams myself. The overhead of
the disassembly stage required in the other approach can be too heavy.

>there, because the client end is where the presentation occurs.

But when you have to account for differing delays in the delivery of the
different streams, you need considerably more buffer space, possibly
messier algorithms, and you have to worry about the integrity of the
separate streams instead of just taking care of the single incoming
pipe.

Now, when you multicast, you cannot use transport throttleback for
congestion control as there are multiple recipients, also, if there are
synchronization problems (widely differing latencies along the different
delivery paths or something like that), you cannot adapt to them
end-to-end. In this case a single stream with tighly interlocked
(interlaced) substreams would seem more stable. Comments?

>True, but the synchronization effort is the same for multicast or unicast.

Yep, but it would be very desirable to use tranport layer flow control to
regulate the data flow when doing unicast streams. How do you do this
when you multicast?

>sufficient buffering with either transport method.

But the amount of sufficient is much more with network latencies counted
in.

>The minimum

Which should be easier with interlaced streams. Especially if you cannot
use flow control.

>Not really, unless some streams are given priority over others.

No? With QoSR present, you probably have to reserve separate flows for the
different RT streams. So the routes will probably be different. This, in
case, considerably increases the amount of buffering needed. Add to that
the cost fo buffering video and you're going to have a hard, hard time
implementing a working system.

>This is indeed one of the motivations for sending the audio and video

Indeed. And I don't say that you should limit the protocol to interlaced
datastreams. On the contrary, I agree that separate streams should be
endorsed. But I also think that interlaced streams have some attributes
that make them (e.g.) useful for multicasting and low-resource
environments. So they shouldn't be banned.