Subject: RE: [SIP] DTMF and Universal Input

From: "Fairlie-Cuninghame, Robert" <[email-masked]>
Date: Fri, 11 Aug 2000 03:04:57 -0700

Hi Skip,

I think you have made some very important points regarding some of the
hurdles the "forked (keystrokes) streams" approach must overcome before it
can be implemented/adopted in a widespread manner. 

I agree that actual transport mechanism (Reliable/redundant RTP or RTP over
TCP or other) is a secondary concern as long as the simplicity, bandwidth
and processing power is comparable.
I also agree that a large change conceptual change needs to be made so that
keystrokes are treated as a _separate_ stream (like video or data) rather
than just an audio-vocoder optimization/extension conveying the same
information.

The largest hurdle IMO is broaching the whole problem of how to referr to a
particular stream in an SDP session and requesting that it be forked? (and
later that a particular copy can be terminated).

What limit do you set on the number of times the stream can be forked? This
number will definitely need to be greater than two streams if there multiple
application servers in the signalling path, 3, 4, 5, where do you stop ?
[This would be a very powerful DoS tool if abused, eg, tell an unsecured
client to send a 100 streams of audio to the target server.] 

How do you determine if a UA supports stream forking? 

IMHO, as Skip says,  these are really issues that need to be solved by SIP
(or atleast by SIP's use of SDP) and needs to become a core requirement of
UA's to make the system useful in a widespread manner.

RTP splitters (located in the network) for the keystroke stream were
proposed as an alternative to the forking of streams, however this approach
could easily cause unacceptable arrival delays between the audio & keystroke
RTP packets at the remote UA (or the complete loss of one stream). 

Regards,

Robert.

-- My opinions are my own. I tried selling them once but everybody
	seems to already have one. -- 

> -----Original Message-----
> From: Skip Cave [mailto:[email-masked]]
> Sent: Friday, August 11, 2000 4:48 PM
> To: SIP Mail List
> Subject: [SIP] DTMF and Universal Input
> 
> 
> Henning Schulzrinne said:
> >Besides the philosophical arguments, can somebody please 
> succinctly and
> 
> >technically characterize why RTP transport of digits does not work or
> is
> >not sufficient?
> http://lists.bell-labs.com/pipermail/sip/2000q3/002087.html
> 
> Skip says:
> I will try to answer Henning and summarize my issues with RFC 2833.
> 
> Probably the most important issue is the requirement for a user agent
> server in the network to redirect and replicate keystroke 
> sessions just
> as it would redirect and replicate audio or video streams.
> 
> Lets take this scenario:
> 
> 1) A UA client (Term1) INVITES a UA server (Term2) and establishes an
> audio session, a keystroke session, and a video session between Term1
> and Term2 using SDP. Term1 is the transmitter of the media 
> (keystrokes,
> audio, video) in all three sessions. Term2 is the receiver of 
> all media
> in all sessions.
> 
> 2) Term2 wishes to redirect Term1's audio media stream to a 
> third party,
> a UA server called Term3. Term2 RE-INVITES Term1's audio session to go
> to Term3's port. However, the keystroke and video sessions 
> transmitting
> from Term1 to Term2 are NOT to be disturbed. We only want to redirect
> Term1's audio session to Term3. All other sessions are to be 
> left alone.
> If audio and keystrokes are in the same SDP session, how do 
> we do this?
> Keystrokes need to have their own session in this scenario. RFC 2833
> puts both media types in one session.
> 
> 3) After we finish redirecting Term1's audio to Term3, we 
> decide that a
> fourth party needs to be involved (Term4). We want both Term1's audio
> and Term1's keystrokes to be sent to Term4, but we DON'T want 
> to disturb
> the keystrokes and video going from 1-2, or the audio going from 1-3.
> Term2 re-invites Term1 again, asking Term1 to replicate his keystroke
> session and send a copy of the keystroke session to Term4. 
> Term1 is now
> essentially multi-unicasting his keystroke session to both Term2 and
> Term4. Similarly, in the re-invite we ask Term1 to multi-unicast his
> audio to Term2 and Term4. If audio and keystrokes are in the same SDP
> session, how do we do this? Keystrokes need to have their own 
> session in
> this scenario.
> 
> I can go on in this vein, but you get the idea. Why do we require
> keystrokes to be in the same session as the audio? Why did we pick the
> audio session instead of the video session to carry keystrokes? Better
> yet, why not give keystrokes their own session permanently. This makes
> the above senarios much simpler, and doesn't affect the PSTN-PSTN
> transport problem, other than you need two SDP sessions (tones and
> audio)for that particular requirement. Simply modify RFC 2833 
> to require
> two sessions - one for keystrokes and signaling, and one for voice
> audio. Now you can treat them as the separate entities they 
> really are,
> and not a hybrid construct that is specific to the PSTN. I 
> haven't heard
> any reasons why this isn't a better solution.
> 
> Henning Schulzrinne said:
> >As a side note, I'm still not sure how useful this 
> audio/DTMF split is
> >in practice, given the wide use of voice recognition.
> http://lists.bell-labs.com/pipermail/sip/2000q3/001933.html
> 
> As I have tried to point out in my previous posts, not every 
> application
> vendor will want to use voice recognition on every application. there
> are many applications where speech recognition is not appropriate, and
> could even be detrimental. There are many applications that send
> keystrokes in one direction and voice in the other (IVR), and these
> types of applications are becoming MORE popular, not less. Most
> important, virtually all of the applications my company is 
> implementing
> need keystrokes from a caller to go one place, and audio from a caller
> to go to another (or many other places).
> 
> Henning Schulzrinne said:
> >not all end systems will implement anything beyond basic 2833, in
> >practice you'll end up with trivial 'RTP splitters' that route DTMF
> >packets to one server and audio to another.
> http://lists.bell-labs.com/pipermail/sip/2000q3/001933.html
> 
> Actually, that is my key point. in order to enable 
> applications like the
> one described above, the split between keystrokes and tones 
> NEEDS TO BE
> IN THE BASE RFC 2833 DOCUMENT. Otherwise you need splitters, or other
> nefarious devices to implement most applications. We don't need
> splitters for sending audio two places. Why would we require 
> a splitter
> to send audio one place and keystrokes another? Just reinvite 
> a separate
> keystroke session.
> 
> Henning Schulzrinne said:
> >Finally (separate issue), I just don't see that there is a practical
> >difference between DTMF and 'user input'. User input is just a subset
> of
> >DTMF (minus loudness). Hopefully, both will disappear over time, as
> user
> >devices have richer input capability. We don't want to redo
> >telnet/rlogin or X11 in SIP.
> http://lists.bell-labs.com/pipermail/sip/2000q3/001933.html
> 
> I agree, at least about the issue of DTMF and user input being similar
> (assuming you mean keystrokes). I don't agree about the keys
> dissapearing over time (See my post about addressing)
> 
> Skip said:
> >Some devices will use speech recognition for addressing. 
> Some will not.
> 
> >Those that can't or won't accept the costs of speech reco will still
> have
> >to have some method of indicating an address at a minimum. I will be
> >willing to bet that that method will include at a minimum, a 
> set of 10
> >numeric keys and an ENTER and DELETE (or equivalent) button. The
> keyboard
> >may have a thousand other keys, but it WILL have numerics.
> http://lists.bell-labs.com/pipermail/sip/2000q3/002083.html
> 
> So perhaps DTMF and user keystroke input can be considered similar
> enough that we could use a single protocol to transport both types of
> things. Fair enough. We can default things like level or frequency or
> time stamps when the transmitting device is a packet terminal that
> doesn't care about those things. As long as keystrokes are in 
> a separate
> session from the audio, I can deal with these issues.
> 
> So that brings us to my second issue. Even if keystrokes have 
> their own
> session, they still need to be delivered by a reliable 
> protocol. Henning
> seems to feel that RTP with reliability extensions (RFC 2198) 
> will fill
> this bill.
> 
> Henning Schulzrinne said:
> >RTP with redundancy is at least as reliable as SIP transport
> http://lists.bell-labs.com/pipermail/sip/2000q3/002087.html
> 
> RFC 2198 seems to be targeted specifically to reliably 
> delivering audio
> by transmitting redundant audio data. I was unable to find any mention
> of how single-datum  events such as a non-audio keystroke from a SIP
> terminal could be made reliable using RFC 2198. (I can think of some
> ways, but none of them are specified in the document). I 
> expect that, if
> we are to use RFC 2198, we will have to extend it to deal 
> with non-audio
> keystrokes. I'm not saying it can't be done, just that we haven't said
> how to do it yet. By the way, Colin Perkins, a participant in this
> thread, was an author of the 2198 RFC for audio redundancy. 
> Colin, if I
> am putting words in your mouth, speak up.
> 
> There are several existing mechanisms for reliable delivery of
> single-datum events. TCP, SCTP, and telnet come to mind. Would it be
> easier to just select an existing reliable protocol for keystroke
> transport, or develop RFC 2198 to the point it could be used for
> single-datum non-audio events like a keystroke. Admittedly, 
> it would be
> nice to not require another protocol stack in devices that already
> implement RTP, but what about things that aren't phones? Should we
> require that any SIP device with keys support reliable RTP? Even if it
> doesn't have a microphone or speaker? Is reliable RTP simpler or more
> complex than the other reliable delivery mechanisms?
> 
> Why should we bias a protocol such as RFC 2833 towards devices with a
> microphone and a keyboard? Why not bias it towards devices with video
> camera and a keboard? Why not devices with just a keyboard?
> 
> Henning Schulzrinne said:
> >every Ethernet phone will need to support RFC 2833 (and it's pretty
> >trivial to implement, given that RTP/RTCP is needed anyway). 
> Instead of
> 
> >adding another mechanism, it would seem better to have the generic
> >ability to get end systems to send to multiple copies of 
> audio streams,
> 
> >including subsets. This is more generally useful for 
> multi-party calls.
> 
> http://lists.bell-labs.com/pipermail/sip/2000q3/001933.html
> 
> I'm not convinced that reliable RFC2833 is all that trivial for
> non-audio keystrokes, since that hasn't been defined yet. I'm also not
> convinced that reliable RFC2833 is vastly simpler or more complex that
> other reliable protocols. I agree that a single protocol is 
> nice, and if
> reliable RTP isn't vastly more complex than a different reliable
> protocol, we could use it.
> 
> I DEFINITELY agree with Henning that all SIP terminals should be
> required to have the capability to send multiple copies of audio
> streams. This should be in the base SIP (or is it SDP) specification.
> This functionality is a MUST for a large set applications (virtually
> every application my company builds). Multi-party calls, operator
> intercept, voice logging, all of these applications would be much more
> difficult and costly to implement, if session replication 
> capability is
> not present in EVERY SIP terminal (and gateway). All SIP terminals
> should also be required to support requests for multiple copies of a
> video session. They should all be required to support requests for
> multiple copies of a keystroke session. We should consider this as one
> of the highest priorities for updates to the basic SIP doc. 
> What is the
> process for getting this requirement into the base document?
> 
> Don't know what this has to do with splitting keystroke sesions from
> audio, though.
> 
> Actually, I don't find this requirement for all devices (including
> audio-less devices) to use reliable RTP particularly objectionable. As
> long as reliable RTP delivery of keystrokes can be shown to 
> be close to
> as efficient in bandwidth and processing power as any of the other
> transport protocol candidates, a reliable version of RFC 2833 
> would be a
> viable approach to use. As long as the keystrokes have their own
> session, and reliable RTP isn't too much of a hog compaired 
> to the other
> choices, let's use it. Of course we still need to define it (reliable
> RTP for non-audio, single-datum events) first.
> 
> Comments?
> 
> Skip Cave
> Sr. Principal Engineer
> InterVoice-Brite Inc.
> 
> 
> 
> _______________________________________________
> SIP mailing list
> [email-masked]
> http://lists.bell-labs.com/mailman/listinfo/sip
> 

_______________________________________________
SIP mailing list
[email-masked]
http://lists.bell-labs.com/mailman/listinfo/sip

---

From: "Skip Cave" <[email-masked]>
Date: Wed, 16 Aug 2000 11:05:46 -0500

Skip Cave said:
>> Simply modify RFC 2833 to require two sessions 
>> - one for keystrokes and signaling, and one for voice audio. 
>> I haven't heard any reasons why this (separate sessions)isn't 
>> a better solution.

Robert Fairlie-Cuninghame said:
> I also agree that a large change conceptual change needs
> to be made so that keystrokes are treated as a 
> _separate_ stream (like video or data) rather
> than just an audio-vocoder optimization/extension 
> conveying the same information.
Archive: http://lists.bell-labs.com/pipermail/sip/2000q3/002106.html

Henning Schulzrinne said:
> (Combining keystrokes & audio in a single session) Has to do with ease of 
> synchronization at the tone reconstruction point.
> You want to allow both:
> - embedded in audio stream for recreating audio + tones with tight
> synchronization;
> - just events, for entities that don't care about audio.
> The above mechanism (RFC 2833) supports this.
Archive: http://lists.bell-labs.com/pipermail/sip/2000q3/002124.html

Henning seems to be saying that, by putting DTMF keystrokes in the same session with audio, is is in some sense "easier" for the receiver to synchronize the key entry with the compressed audio stream. Also he seems to imply that you can synchronize the two media types more "tightly" if they are in the same stream.

Of course, there are lots of applications that must synchronize multiple media streams. A video and audio application comes to mind. Or video and keystrokes. If what Henning says is true, then all media that needs to be tightly synchronized (audio, video, keystrokes, force-feedback controls, etc.....) should be placed in a single RTP session. 

Henning, are you saying that if you need to tightly synchronize video and audio, both media types should be placed in the same session? Or video & keystrokes? Doesn't SIP allow synchronization between two separate sessions just as easily as if they were in the same session? Surely, if you use the same timestamps for a DTMF stream and an audio stream, you can exactly re-synchronize them at the receiver end, regardless if they are in the same session or not?

RFC 1890, Henning's last word on Video & Audio streaming, specifically states:
"The payload types currently defined in this profile carry either audio or video, but not both". Therefore, audio & video are defined as separate media types with separate sessions. You should still should be able to synchronize audio & video even if they are in separate sessions.

For another option, Colin Perkins writes:
> I suggest you read RFC 2793 which describes how to use 
> RFC 2198 redundancy as a `reliable' transport for non-audio keystrokes. 
http://lists.bell-labs.com/pipermail/sip/2000q3/002113.html

RFC 2793 - "RTP Payload for Text Conversation" is an RFC which I had somehow overlooked (thank you, Colin) which proposes a method to use RTP to transport T.140 "text conversation" sessions. It includes the assumption that the T.140 session will be a separate session from other media (good). However, it doesn't provide a way to indicate keystroke duration (bad). RFC 2793 proposes the same optional redundancy mechanism (RFC 2198) as Henning promotes to provide "reliable" transport in RCF 2833. 

RFC 2793 does not implement a traditional acknowledge-each-transmission kind of reliability. Instead, it uses RFC 2198 (like 2833) which proposes sending each datum more than once, thus raising the tolerance somewhat to lost packets. I doesn't eliminate errors due to lost packets, it just lowers the probability of occurance. I'm not sure if this is the best reliability mechanism for keystroke transport, but it is certainly better than doing nothing. Reliabile transport should not be an option for keystrokes, it should be a requirement.

An important point here is that we have at least two different ways to exchange keystroke data between two SIP devices RFC 2833 (DTMF part) and RFC 2793. They each use different mechanisms to send the keystrokes, with different encodings. 

The two keystroke transport mechanisms even use different key encodings. RFC 2833 uses the following encoding for keys:

Event  encoding (decimal)
_________________________
0--9                0--9
*                     10
#                     11
A--D              12--15

RFC 2833 doesn't define encodings for any other keys. It assumes that all devices will only have a maximum of 16 keys. RFC also lumps lots of other non-keystroke-related stuff into the same category as keystrokes - call progress tones, hook flash, fax tones, etc.

RFC 2793 proposes ISO/IEC 10646-1, level 3 (Unicode) encoding to define a much larger set of keys, but the 16 keys defined in RFC 2833 keys are included. 

0x0030         DIGIT ZERO
0x0031         DIGIT ONE
0x0032         DIGIT TWO
0x0033         DIGIT THREE
0x0034         DIGIT FOUR
0x0035         DIGIT FIVE
0x0036         DIGIT SIX
0x0037         DIGIT SEVEN
0x0038         DIGIT EIGHT
0x0039         DIGIT NINE
0x0041         LATIN CAPITAL LETTER A
0x0042         LATIN CAPITAL LETTER B
0x0043         LATIN CAPITAL LETTER C
0x0044         LATIN CAPITAL LETTER D
0x002A         ASTERISK
0x0023         NUMBER SIGN

As you can see, the two protocols have two vastly different ways to encode the same thing - keystrokes.

If I can summarize:
RFC 2833 was designed to solve a very narrow problem set - the transport of all mechanically-generated PSTN tones and other PSTN events across a packet network, when the PSTN audio has been compressed by a non-tone-friendly compression scheme. RFC 2833 defines a large set of other events besides DTMF in the event transport - call progress signaling, flash, fax tones, etc, to aid in the transport of PSTN functionality across a packet network.

Henning has shown how this mechanism can be extended to cover other application requirements (i.e. keystroke transport) using SDP media control mechanisms (m=), albeit it is not a perfect fit. 

RFC 2793 was designed to solve a different problem set - the transport of durationless keystrokes. This RFC uses ISO/IEC 10646-1, level 3 to define a much larger set of keys, but the keys on a DTMF phone are also included in the set.

Both RFCs propose an optional usage of RFC 2198 for psuedo-reliable transport. I am not convinced that this mechanism is preferrable to more traditional reliable protocols, but it may be adequate. However, for keystrokes a reliability mechanism should be required, not optional.

So why do we have two ways to transport keystrokes? 

In my company's applications we plan to receive SIP INVITES from PSTN/SIP Gateways, Softswitches, SIP phones, SIP mobile devices, and all manner of SIP terminals. We will INVITE sessions with all of these same types of devices. 

ALL of the key input from these different sources should use the same protocol. The key input that we send to all of these devices should use the same protocol. We don't want to have to use different protocols to send keystrokes to a SIP gateway, a SIP wireless device, a Softswitch, or a PC. We don't want to have to know ahead of time what type of device we are calling to determine what kind of session to INVITE to get keystrokes. All keystrokes, whether it be DTMF or 101-key PC keyboard should use the same protocol. 

It should be fairly easily to combine the PSTN-specific characteristics of RFC 2833 (duration, level, etc), with the separate session and large defined keyset of RFC 2793. By making level and frequency parameters optional, we can deal with both kinds of transmission devices. We should consider whether the redundancy reliability mechanism (RFC 2198) provides adequate reliability for keystroke transport.

If PSTN transport vendors want to stick fax tones and call progress in with the compressed audio in a gateway-to-gateway RTP session, that's fine. just don't put keystrokes in there. The're a different animal. 

Skip Cave
Sr. Principal Engineer
InterVoice-Brite Inc.