Subject: [Speechsc] Re: Speechsc digest, Vol 1 #153 - 2 msgs

From: "John Potemri" <[email-masked]>
Date: Thu, 6 Nov 2003 15:06:11 -0500

Brian,

I did wonder when someone would raise "record" on the reflector. I know
many people are talking about it.  I would caution to push "record" into
the recognize package (a.k.a. resource type). Someone may want to offer
record capabilities via MRCP and not have recognition.

I would prefer to see record defined as a separate resource. MRCP does
allow multiple resource types on the same control channel and share the
same source stream, so you could record while recognizing, too. This keeps
the two state machines and protocol interface clean.

I don't want to open a can of words, but I suppose similar arguments for
more granular resource types could be made around:
   - "speak" supporting both TTS and audio - what if I only supported
   audio? At least in this case you wouldn't have both resources "active".
   - "recognize" resources that only do DTMF grammars.

But then again, these are "types" within the "resource" and one could argue
that you support the functional interface specified by the resource
definition, but may not support certain types. This could have been
resolved out-of-band via some resource management framework. But record in
recognition is different. I understand that this may not be a speech
vendors preferred way to look at things.

This does beg the question of MRCP as the protocol for media engine ("media
server") functions. You'll note that while not defined, the spec does
suggest fax as a type, too. Perhaps someone has already had this dialog. I
have excluded addessing conferencing at this point because I know who will
jump all over me.

-John

P.S. And while we comment against MRCPv2, is it out-of-scope to mention the
effects relative to MRCP(v1) for which people might make parallel
suggestions?

John Potemri
NMS Communications
100 Crossing Blvd
Framingham, MA  01702
+1-508-271-1369
[email-masked]

|---------+---------------------------->
|         |           speechsc-request@|
|         |           ietf.org         |
|         |           Sent by:         |
|         |           speechsc-admin@ie|
|         |           tf.org           |
|         |                            |
|         |                            |
|         |           11/06/2003 12:00 |
|         |           PM               |
|         |           Please respond to|
|         |           speechsc         |
|         |                            |
|---------+---------------------------->
  >--------------------------------------------------------------------------------------------------------------|
  |                                                                                                              |
  |       To:       [email-masked]                                                                            |
  |       cc:                                                                                                    |
  |       Subject:  Speechsc digest, Vol 1 #153 - 2 msgs                                                         |
  >--------------------------------------------------------------------------------------------------------------|

Send Speechsc mailing list submissions to
             [email-masked]

To subscribe or unsubscribe via the World Wide Web, visit
             https://www1.ietf.org/mailman/listinfo/speechsc
or, via email, send a message with subject or body 'help' to
             [email-masked]

You can reach the person managing the list at
             [email-masked]

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Speechsc digest..."

Today's Topics:

   1. Question on Proxy, and media redirect. (Alex Lee)
   2. RE: Question on Proxy, and media redirect. (Eberman, Brian)

----- Message from Alex Lee <[email-masked]> on Wed, 05 Nov 2003
17:18:40 -0500 -----
                                                  
      To: [email-masked]                       
                                                  
 Subject: [Speechsc] Question on Proxy, and media 
          redirect.                               

Hi,

    We have looked at the way the sessions are set up with MRCPv2, and
find that it isn't well-defined for some proxy applications.  The
situation we're thinking about is that there's a proxy fronting multiple
ASR servers at the back end, where each ASR server may have different
capabilities (e.g. Server 1 supports English, Server 2 supports
French).  Sometimes, it may be desirable to use different servers for
different recognition sessions on the same call.

    When it is desired that a different server is to be used for
different recognition sessions, there are two possibilities for how this
may be done.  First, if the client is very smart, then it can be aware
of many different servers it has access to, and setup independent
sessions to these servers based on what it needs.  The other possibility
is to build a client that isn't very smart, but it will access an MRCP
proxy that is capable of routing its recognition requests, based on the
parameters of each recognition session.  For efficient implementation of
the Proxy, the Proxy should not proxy the audio data, but rather it
should only proxy the SIP and MRCP control messages.

    In order to do that, however, the spec itself must allow for some
kind of media re-routing (similar to a SIP re-invite) initiated from the
server when a "RECOGNIZE" method is called from the MRCP client.  Right
now, MRCPv2 assumes that the same media channel will be used within one
session.

    One way to work around this is to allow for a SIP INVITE to be sent
from the Server to the Client in the middle of a "recognize" method, to
change some properties of the media channel.  In the use case with the
proxy switching back-end servers, the client would initiate a session
with the proxy.  Then, based on the parameters of the MRCP RECOGNIZE
command (e.g. which grammars to use for recognition, which languages to
use, etc.), the proxy would choose the back-end server to use for the
recognition.  After the real back-end server has been chosen, the proxy
would go back to the client to modify the media properties, so that the
client can send the audio directly to the real server.

    The following diagram illustrates the interaction of such a
situation.  The single arrows are messages sent via the SIP channels,
while the double arrows are messages sent via the control channels.

Client                           Proxy                           Server
   |                               |                               |
   | -------- SIP INVITE --------> |                               |
   |                               |                               |
   | <------- SIP INVITE --------- |                               |
   |     Port# for both control    |                               |
   |            and media          |                               |
   |                               |                               |
   | ---------- SIP ACK ---------> |                               |
   |                               |                               |
   |                               |                               |
   | ======= TCP connect() ======> |                               |
   |                               |                               |
   | ==== MRCP DEFINE-GRAMMAR ===> |                               |
   |                               |                               |
   | <=== MRCP DEFINE-GRAMMAR ==== |                               |
   |                               |                               |
   | ====== MRCP SET-PARAM ======> |                               |
   |                               |                               |
   | <===== MRCP SET-PARAM ======= |                               |
   |                               |                               |
   | <=== More MRCP SET-PARAM ===> |                               |
   |       and DEFINE-GRAMMAR      |                               |
   |                               |                               |
   | ==== MRCP RECOGNIZE (1) ====> |                               |
   |                               |                               |
   |                               | -------- SIP INVITE --------> |
   |                               |                               |
   |                               | <------- SIP INVITE --------- |
   |                               |     Port# for both control    |
   |                               |            and media          |
   |                               |                               |
   |                               | ---------- SIP ACK ---------> |
   |                               |                               |
   | <------- SIP INVITE --------- |                               |
   |      New Port# of media       |                               |
   |                               |                               |
   | -------- SIP INVITE --------> |                               |
   |                               |                               |
   | <--------- SIP ACK ---------- |                               |
   |                               | ======= TCP connect() ======> |
   |                               |                               |
   |                               | <==== All MRCP SET-PARAM ===> |
   |                               |       and DEFINE-GRAMMAR      |
   |                               |        sent by client         |
   |                               |                               |
   |                               | ====== MRCP RECOGNIZE ======> |
   |                               |                               |
   |                               | <===== MRCP RECOGNIZE ======= |   |
|                               |
   | <==== MRCP RECOGNIZE (2) ==== |                               |
   |                               |                               |
   |                               | <=== MRCP START-OF-SPEECH === |   |
|                               |
   | <=== MRCP START-OF-SPEECH === |                               |
   |                               |                               |
   |                               | < MRCP RECOGNITION-COMPLETE = |   |
|                               |
   | < MRCP RECOGNITION-COMPLETE = |                               |
   |                               |                               |
   |                               |                               |

    Is there any possibility to explicitly include this mechanism, or an
alternative, in the MRCPv2 spec that would allow for the usage of a
proxy where it needs change the back-end server media properties in the
middle of the session?  I have seen that "RE-DIRECT" has been mentioned
in the MRCPv2 spec in section 8.5.3, but there hasn't been any example,
explicit description, or rules on how this is to be done.  Any
extensions or clarification on this topic is much appreciated.  Thanks.

Alex...

----- Message from "Eberman, Brian" <[email-masked]> on Thu, 6
Nov 2003 08:19:29 -0500 -----
                                                     
      To: [email-masked]                          
                                                     
      cc: "Eberman, Brian"                           
          <[email-masked]>               
                                                     
 Subject: RE: [Speechsc] Question on Proxy, and      
          media redirect.                            

I really think Alex has brought up an important point here.  We need to be
able to cleanly separate the RTP/ audio streaming from the control stream
to
get a cleaner specification.  Proxy scaling should really be a key issues
for MRCPv2 and I thought it was in the base requirements. This seems like a
significant whole.

I've seen two other places in implementations of VoiceXML browsers that
need
to be clarified for v2:

1) There isn't any way to cleanly implement a pure recording function under
MRCP to implement the record function of VoiceXML without having the
VoiceXML implementation box routing the audio. I think we need to resolve
this issue.  VoiceXML recording can be terminated by speech recognition so
it is actually more like a recognition then a recording.

2) The SPEECH-MARKER event for TTS doesn't indicate the time of the event.
So if a system is distributing the RTP stream and the TTS stream and
potential mixing audio using the MARKER it doesn't know the time in the RTP
stream when the audio mark was hit.

The stream control semantics seem to need a little work here.
-Brian

-----Original Message-----
From: Alex Lee [mailto:[email-masked]]
Sent: Wednesday, November 05, 2003 5:19 PM
To: [email-masked]
Subject: [Speechsc] Question on Proxy, and media redirect.

Hi,

    We have looked at the way the sessions are set up with MRCPv2, and find
that it isn't well-defined for some proxy applications.  The situation
we're
thinking about is that there's a proxy fronting multiple ASR servers at the
back end, where each ASR server may have different capabilities (e.g.
Server
1 supports English, Server 2 supports French).  Sometimes, it may be
desirable to use different servers for different recognition sessions on
the
same call.

    When it is desired that a different server is to be used for different
recognition sessions, there are two possibilities for how this may be done.
First, if the client is very smart, then it can be aware of many different
servers it has access to, and setup independent sessions to these servers
based on what it needs.  The other possibility is to build a client that
isn't very smart, but it will access an MRCP proxy that is capable of
routing its recognition requests, based on the parameters of each
recognition session.  For efficient implementation of the Proxy, the Proxy
should not proxy the audio data, but rather it should only proxy the SIP
and
MRCP control messages.

    In order to do that, however, the spec itself must allow for some kind
of media re-routing (similar to a SIP re-invite) initiated from the server
when a "RECOGNIZE" method is called from the MRCP client.  Right now,
MRCPv2
assumes that the same media channel will be used within one session.

    One way to work around this is to allow for a SIP INVITE to be sent
from
the Server to the Client in the middle of a "recognize" method, to change
some properties of the media channel.  In the use case with the proxy
switching back-end servers, the client would initiate a session with the
proxy.  Then, based on the parameters of the MRCP RECOGNIZE command (e.g.
which grammars to use for recognition, which languages to use, etc.), the
proxy would choose the back-end server to use for the recognition.  After
the real back-end server has been chosen, the proxy would go back to the
client to modify the media properties, so that the client can send the
audio
directly to the real server.

    The following diagram illustrates the interaction of such a situation.
The single arrows are messages sent via the SIP channels, while the double
arrows are messages sent via the control channels.

Client                           Proxy                           Server
   |                               |                               |
   | -------- SIP INVITE --------> |                               |
   |                               |                               |
   | <------- SIP INVITE --------- |                               |
   |     Port# for both control    |                               |
   |            and media          |                               |
   |                               |                               |
   | ---------- SIP ACK ---------> |                               |
   |                               |                               |
   |                               |                               |
   | ======= TCP connect() ======> |                               |
   |                               |                               |
   | ==== MRCP DEFINE-GRAMMAR ===> |                               |
   |                               |                               |
   | <=== MRCP DEFINE-GRAMMAR ==== |                               |
   |                               |                               |
   | ====== MRCP SET-PARAM ======> |                               |
   |                               |                               |
   | <===== MRCP SET-PARAM ======= |                               |
   |                               |                               |
   | <=== More MRCP SET-PARAM ===> |                               |
   |       and DEFINE-GRAMMAR      |                               |
   |                               |                               |
   | ==== MRCP RECOGNIZE (1) ====> |                               |
   |                               |                               |
   |                               | -------- SIP INVITE --------> |
   |                               |                               |
   |                               | <------- SIP INVITE --------- |
   |                               |     Port# for both control    |
   |                               |            and media          |
   |                               |                               |
   |                               | ---------- SIP ACK ---------> |
   |                               |                               |
   | <------- SIP INVITE --------- |                               |
   |      New Port# of media       |                               |
   |                               |                               |
   | -------- SIP INVITE --------> |                               |
   |                               |                               |
   | <--------- SIP ACK ---------- |                               |
   |                               | ======= TCP connect() ======> |
   |                               |                               |
   |                               | <==== All MRCP SET-PARAM ===> |
   |                               |       and DEFINE-GRAMMAR      |
   |                               |        sent by client         |
   |                               |                               |
   |                               | ====== MRCP RECOGNIZE ======> |
   |                               |                               |
   |                               | <===== MRCP RECOGNIZE ======= |   |
|                               |
   | <==== MRCP RECOGNIZE (2) ==== |                               |
   |                               |                               |
   |                               | <=== MRCP START-OF-SPEECH === |   |
|                               |
   | <=== MRCP START-OF-SPEECH === |                               |
   |                               |                               |
   |                               | < MRCP RECOGNITION-COMPLETE = |   |
|                               |
   | < MRCP RECOGNITION-COMPLETE = |                               |
   |                               |                               |
   |                               |                               |

    Is there any possibility to explicitly include this mechanism, or an
alternative, in the MRCPv2 spec that would allow for the usage of a proxy
where it needs change the back-end server media properties in the middle of
the session?  I have seen that "RE-DIRECT" has been mentioned in the MRCPv2
spec in section 8.5.3, but there hasn't been any example, explicit
description, or rules on how this is to be done.  Any extensions or
clarification on this topic is much appreciated.  Thanks.

Alex...

_______________________________________________
Speechsc mailing list
[email-masked] https://www1.ietf.org/mailman/listinfo/speechsc

_______________________________________________
Speechsc mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/speechsc

_______________________________________________
Speechsc mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/speechsc

---

From: Sarvi Shanmugham <[email-masked]>
Date: Thu, 06 Nov 2003 13:35:59 -0800

Hi John,
       I agree with all your comments. These are things that I was 
planning to propose too,  as additions to the specification, but haven't 
been able to get to it due to some other priority work items.
       Like you, I am also interested in seeing the Record as a separate 
resource for purposes very similar to the ones stated by you and Brian.
       But I still see a need for recording capabilities associated with 
the recognition resources to be able to record and manage utterances. I 
still have no problem looking at the Recording engine as a separately 
controlable standalone resource, as long as for simplicity sake, we 
allow the Recognize operation to trigger the start and stop of the 
recording engine if needed.
      Another place I suspect this Recording or the Record utterances 
operation is going to be usefull, is for Speaker verification. This 
requires the user to record/buffer user utterances spoken  separately or 
during a recognize operation and then apply the recorded/buffered audio 
to the Speaker Verification Engine.

    I also agree with john that there is need to be able to play only 
audio from a synthesizer.
The I was hoping we could achieve this, is by using the smae messages 
with different limitations based on the specific resource profiles.
       For Example:  with the Synthesizer we could have 3 different 
profiles. Audio-Player, poor-mans TTS(concatenating audio bits), 
advanced TTS.  The major difference  between the 3 is  the synthesizer 
data sent in the SPEAK method. 
        1.  Audio player may only support , the SSML <audio>, and the 
<mark> tags.
         2. Poor man TTS can support a few more SSML tags such as 
<say-as> , <phoneme>,  <voice> <xml:lang> etc.
         3. The Advanced TTS could support  everything else.

I agree this same approach applies to Recognizers that are capable of 
DTMF only or both DTMF and ASR.

Sarvi

John Potemri wrote:

>   |                               | <===== MRCP RECOGNIZE ======= ||                               |
>   |                               | <=== MRCP START-OF-SPEECH === ||                               |
>   |                               | < MRCP RECOGNITION-COMPLETE = ||                               |
>   |                               | <===== MRCP RECOGNIZE ======= ||                               |
>   |                               | <=== MRCP START-OF-SPEECH === ||                               |
>   |                               | < MRCP RECOGNITION-COMPLETE = ||                               |