Subject: RE: MUTE in RTSP

From: Eric Fleischman <[email-masked]>
Date: Fri, 18 Apr 1997 16:44:39 -0700

Streaming a presentation (i.e., multiple multimedia streams treated as a
single entity) is the most efficient streaming approach possible for a
server. 

Should one then treat one media type differently from another then one
is no longer treating the multiple streams as a unit. Whenever you do
this you increase server overhead tremendously. 

The amount of "tremendously" depends on the implementation. If you have
an approach especially tailored for maximum server efficiency such as
ours, the overhead of doing this is immense.

> -----Original Message-----
> From:	David Oran [SMTP:[email-masked]]
> Sent:	Friday, April 18, 1997 11:05 AM
> To:	Eric Fleischman; [email-masked]
> Subject:	Re: MUTE in RTSP
> 
> At 09:41 AM 4/18/97 -0700, Eric Fleischman wrote:
> [snip]
> >In the first instance, all media streams are treated as a unit and
> are
> >transparently manipulated together. In most/many cases,
> "presentations"
> >will be the product of content creators who have designed these
> streams
> >"off line" to function as that unit to create the intended multimedia
> >experience. A MUTE, in this context, would introduce an orthogonal
> >concept of providing independent control for one of those streams in
> the
> >"multimedia bundle". This is hostile to the basic underlying
> philosophy
> >of what a "presentation" actually is. MUTE is totally inappropriate
> >within a "presentation" since it wars with the "core underlying
> >philosophy" of what a "presentation" actually is.
> >
> Huh? I don't follow this at all. Let's take a presentation like a
> carefully
> ported version of John Waters' famous film Pink Flamingos, which has
> video,
> audio, and smell-o-rama. Now, I'm watching this in my kitchen and I
> say,
> sheesh, I like the movie, but this smell-o-rama stuff really stinks!
> So, I
> hit the "mute-the-smell" button on my player. Now, of course you can
> do
> this as a purely local function, but I see no reason why you shouldn't
> be
> able to tell the server that the smell is muted so the server doesn't
> have
> to transmit all the synthesis instructions to the local olfactory
> molecule
> replicator.
> 
> Is John Waters going to come down to my house a berate me because I
> did
> something "hostile to the basic undelying philosophy of what a
> presentation
> is"?...well maybe John Waters *would* do that - never mind :-).
> 
> >In the second instance, RTSP provides for the independent control of
> >multiple media streams which the user (via the intermediate
> >registration/announcement facilities) has orchestrated into a single
> >logical session. RTSP has already provided the user with the ability
> to
> >do a MUTE within this context by permitting him/her to PAUSE the
> audio
> >stream and at a subsequent time to PLAY with Offset that same stream.
> A
> >Mute in this instance, therefore, is merely a "simplification" to the
> >PAUSE...PLAY with Offset sequence. I have no objection to a MUTE if
> it
> >is solely targeted to this (non-presentation) environment since it
> >automates what the "Offset" should be in the "Play with Offset" part
> of
> >the sequence. More importantly, a MUTE in this context is fully
> >consistent with the "underlying philosophies" of such logical
> sessions. 
> >
> You just did an "undef" of MUTE by saying it's just PAUSE followed by
> PLAY.
> That's OK with me, except then you'll invoke the "presentation
> principle"
> above to say I aint' oughta do dat.
> 
> If we can just treat MUTE as a friendly server optimization that a
> client
> can send over to save the server some work, what's all the
> broughhahaha about?
> 
> What am I missing here?
> 
> 
> -----------------
> David R. Oran
> Cisco Systems		Direct: 408-527-0567
> 7 Ladyslipper Lane	Home Office: 508-264-2048,  Home: 508-263-2705
> Acton, MA 01720	EMail: [email-masked]

---

From: Henning Schulzrinne <[email-masked]>
Date: Sat, 19 Apr 1997 10:20:54 -0400

> single entity) is the most efficient streaming approach possible for a

Just to avoid confusion, may I suggest that we distinguish two cases of
multimedia, from a delivery standpoint:

1) 'multiple-media, single-stream': data is read from a single file
(say, QuickTime, ASF, whatever) [this part is obviously invisible to the
outside world] and [importantly] delivered in some interleaved stream
format on a single network 'connection', that is, the same UDP port and
multicast address and SSRC (if RTP) for all media.

2) logical grouping (if you like, virtual presentation): the client
knows that audio and video belong together, but they get controlled by
separate RTSP URLs and the server doesn't know or care. Audio and video
are delivered separately and synchronized at the receiver. Audio and
video may originate at the same physical server, but not necessarily.

Note that the issue of MUTEing is independent of whether the server
knows about the relationship between the streams or not.

We had the discussion as to whether interleaving (= single RTP/network
stream with multiple media) is better or not a few times, without much
progress; I believe RTSP should be agnostic on this front.

> ours, the overhead of doing this is immense.

Serving data seems to consist roughly of three parts:

1) get the data from permanent storage or a cache fronting for it

2) schedule delivery at appropriate time intervals to maintain
approximate timing and adhere to network flow contracts, if applicable;
increase the logical clock (normal play time) of the stream
corresponding to progression of time within the stream

3) put the stuff out on the network.

MUTEing, in a sense, just skips (1) and (3), but keeps increasing the
clock as in (2). Thus, it seems hard to believe that this would consume
more resources than doing 'real' delivery. This is convenient, but not
necessary, if the client is streaming several media and synchronizing
them. As has been pointed out, the client can PAUSE and PLAY (at the new
time point) the stream to achieve the same effect. Since it's generally
a good idea to have the client do work rather than the server, this may
be an appropriate mechanism.

If you have a multimedia stream that you need to physically disassemble
(say, recreate a new QuickTime file) to disable a particular media, this
is clearly more work than just shoveling the whole thing out the door.
In some cases (plenty of bandwidth), a server may decide that it's too
busy to bother and let the client turn down the volume if the client
doesn't want to listen to the audio. 

It's not clear to me what the server serving a single QuickTime/ASF
movie would do when told to PAUSE the audio. It appears it would have to
do the same disassembly or just offer QuickTime movies as a single
entity.

The whole area of group RTSP streams (presentations) is still a bit
fuzzy and requires more discussion. I think we should settle that issue
first. MUTE is something we can always offer later, if needed.

---

From: Karl Auerbach <[email-masked]>
Date: Sat, 19 Apr 1997 12:15:56 -0700 (PDT)

> single entity) is the most efficient streaming approach possible for a

I do not believe that assertion.

In fact, when a composition of streams can be sent by distinct, but time
synchronized computers (or processors) that bundling would tend to add
artificial limitations.

As has been said many times on this list: Real implementations have
demonstrated that the true burden of achieving synchronized rendering
among multiple streams falls on the receiver.  Whether the streams are
emitted in tight or loose synchrony at the source(s) is somewhat
irrelevant. 

		--karl--

---

From: Sampo Syreeni <[email-masked]>
Date: Wed, 23 Apr 1997 15:37:38 +0300 (EET DST)

>progress; I believe RTSP should be agnostic on this front.

Agreed.

>more resources than doing 'real' delivery.

Assuming you have a server architecture that is optimized for sending data
from readily interleaved files with (possible) precalculated data to gain
performance, an additional (and possibly superfluous) state ni the
optimized playback core /can/ do damage. I guess this is what is meant by
'highly optimized for speed'. And in a case like this, the implicit
interleaving may mean that the only phase that you really /can/ skip is
phase 3. In a well optimized situation in which the outgoing network
resources allow sending without contention and/or competition, phase 3
isn't going to be a problem. So, in cases like this, you don't necessarily
gain anything (except for a decrease in network workload; with resource
reservation, not necessarily even that) but you may end up paying a
penalty in the engine core.

>This is convenient, but not

Indeed it is. The /only/ reason I can come up with to using a separate
MUTE is a very synchronization problem (you cannot be sure that no glitch
with respect to the presentation clock is introduced by the pause).

>do the same disassembly or just offer QuickTime movies as a single

This is precisely the kind of situation I alluded to above. Readily
interleaved data may preclude the usage of MUTE anyway. And to gain
performance, one often uses interleaving/preprocessing to reduce head
latencies and other similar bottlenecks...

>first. MUTE is something we can always offer later, if needed.

Agreed.

Sampo Syreeni (Decoy/dAWN), student, <[email-masked]>

---

From: Sampo Syreeni <[email-masked]>
Date: Wed, 23 Apr 1997 15:50:58 +0300 (EET DST)

>synchronized computers (or processors) that bundling would tend to add

Yes. /When./ But in the case of realtime streams, at least, that is not
going to help. And on the server side, interlaced streams do have a strong
appeal because of performance reasons. So they should be possible, at
least. But of course, in most cases it is attractive to be able to
distribute the workload of streaming a presentation, especially when the
number of streams grows to tens or more. In this case even the network may
limit the usability of interlaced tranmission techniques.

>emitted in tight or loose synchrony at the source(s) is somewhat

True. If the receivers have sufficient resources available to accomplish
such synchronization. But in cases where realtime, high definition video,
a LAN environment and desktop computers are used, interlaced data streams
are /much/ better; it is hard to implement sufficient queues on PCs and if
the underlying network has tolerable jitter (many LANs and, indeed, ATM
clouds do), interlacing can ease up the requirements imposed on the reader
considerably.