Subject: Re: [Speechsc] MRCPv2 comments

From: Sarvi Shanmugham <[email-masked]>
Date: Mon, 10 Nov 2003 19:53:06 -0800

Some of the features that you are requesting are not available in the 
current spec because the current specification is leveraging the 
existing MRCPv1 specification. That said,
I do plan to add some of the functionality you have requested. Look for 
individual reposnses before.

[email-masked] wrote:

>Hi,
>
>These are some things that I noticed on a quick inspection of MRCP v2.0  spec.
>
>Fetching, Storing and deleting TTS content
>-------------------------------------------------------------
>
>1. When the TTS receives an inline content, it has a content-id associated with it. If it has a cache-control field, then that would determine how long this content is kept on the server.  However in case of a scenario where there MIGHT be a need to keep TTS content for an entire page browse (so no cache control is activated) and then if the user navigates to a new page, the client would decide to delete that particular TTS content, it cannot do so. Basically, there is no DELETE_CONTENT directive that can take the "content-id" as a parameter. If there are other ways to do this, these are not so clear. 
>
With TTS there is no explicit content create or delete.  There is inline 
content, that is stored on the server for the life of the SPEAK request. 
And there is URI referenced content(either specified as  a URI, or 
specified as a URI from inside an inline content), which may be held in 
a local cache on the media server. The client uses the Cache-Directive 
fields to tell the media server, what caching directive values to use in 
the media servers document cache for thsi session or Request..

None of these cache directives applies for the inline content itself, 
but only applies for content that is URI referenced and hence those that 
the media server would have to go and fetch and possibly cache.

>
>
>2. For TTS content, the spec doesnt specify if content-id is needed for both inline content and URI's. It specifies for ASR that content-id is needed only for inline recognition grammars and not for external ones.
>
currently the Content-ID is used in recognizers because you could 
load/complile grammars with a DEFINE-GRAMMARS command  and then use it 
later in a RECOGNIZE command. Here we need the content-id to relate the 2.

There doesn't seem to be a similar need in TTS. Or is this question 
related to your requirement above??

>
>3. The spec does not specify which all TTS methods can have the content as a payload. I would assume from the spec that only the SPEAK request would have the content as a payload. However, the spec defines message headers like "audio fetch hint" and "fetch hint" that is used with SET-PARAMS, GET-PARAMS and SPEAK. They have values of "prefetch" indicating that the content may be prefetched and "safe" that says the content need to be downloaded only when needed. If (according to my understanding) there is prefetch enabled with a SET-PARAM request, and the content can be specified ONLY through a SPEAK request, I dont see a prefetch capability here. There should be some method like LOAD-CONTENT that can take either an inline content or a URI to fetch (and the content addressed through a content-id). 
>
The fetch hints are to be used when the server has to go fetch dat from 
outside. This may be thec ase if the SSML contains either <audio> tags 
pointing ot external audio files or may be other SSML content. The fetch 
hints are to be used in these cases to see if the SPEAK request should 
speaking when all the contents are have been loaded or if it can start 
speaking as more content is being fetched in the background(streaming 
mode). Audio-fetch-hint is specifically for the <audio> tag and hence 
supports a streaming as one of its options.  The regular fetch hint 
applies for XML documents that the Synthesizor or the Recognizer may 
have to fetch from outside before speaking or recognizing. The 
difference betweens eding thme in a SET-PARAM method as opposed to a 
RECOGNIZE or SPEAK request is that the values apply for session or for 
that particular request.

>
>Queueing
>-------------
>
>MRCPv2 does not specify using a QUEUE method. If there are preloaded content (for TTS) on the server that could be referenced using a content-id, the client should be able to choose multiple content through the id's and create a queue to be played back. It can be argued that multiple SPEAK requests can be sent and they naturally form a queue to be played back. However, consider the scenario where you have sent multiple SPEAK and the TTS is playing them back. Then the client decides to PAUSE this and play another TTS content. With the current spec, if you pause this, the next SPEAK will be appended to this queue rather than spoken immediately. If a STOP gets sent, the whole queue gets cleared. 
>
>If a QUEUE is implemented, then this queue can be paused (through a PAUSE_QUEUE or something) and a new TTS can be played. The TTS would send back a PLAY_COMPLETE after which the client can resume playback of the queue.  
>
I'd like to gauge how important the need is to be able to pause a SPEAK 
request, play something else and then  resume a previously paused play 
request.
If there is s strong need for it, My suggestion to address this is using 
the existing mechanisms instead of adding a separate QUEUE 
implementation and operation.

I suspect we could address your problem, by fixing the current 
SPEAK/PAUSE/RESUME mechanism and use the Request-Identifier and the 
Request state (ACTIVE/PENDING/COMPLETE) to provide a fix. I will get 
back to you with a clearer proposal on this.

>
>Audio recorder
>---------------------
>
>MRCPv2 does not specify the use of an audio recorder with RECORD, and FETCH capability.
>There is provision in the ASR methods for recording only those waveforms that were used for recognition with fetch capability.
>
This has been addressed on a recent thread. The summary is that that I 
agree there is a need for a separate RECORD resource, I will add a 
propose mechanism in the next revision of the specification.

>
>ASR
>-------
>
>1. MRCPv2 does not have provision to delete pre-loaded grammars if the client decides they are no longer needed.
>
The current specification doesn't support server level or global 
preloaded grammars at all. This is one of the requirements per the MRCP 
requirements document and I will be adding support for it as well.

>
>2. No INTERMEDIATE_RECOGNITION_RESULT send event from the recognizer and provision for setting this with the message fields (eg SET-PARAMS)
>
Can you elaborate on the need for this. Are you refering to recognition 
in stages(recgonizer, NL etc.) or are you refering to parital 
recognition. If so could you provide an example of its usage.

Thx,
Sarvi

>
>best regards,
>
>Sailesh.
>
>Sailesh Kumar Sathish      GSM +358 50 4835679
>Nokia Research Center      Desk +358 3 272 5679
>P.O.Box 100, Tampere      [email-masked]
>Finland -33721
>
>
>
>_______________________________________________
>Speechsc mailing list
>[email-masked]
>https://www1.ietf.org/mailman/listinfo/speechsc
>
>  
>

_______________________________________________
Speechsc mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/speechsc

---

From: [email-masked]
Date: Tue, 25 Nov 2003 15:25:12 +0200

Hi Mr Sarvi,

Thank you for the reply. Please find my comments below.

>the media server would have to go and fetch and possibly cache.

Exactly. Wouldnt there be cases when the TTS inline content would have to be stored beyond the SPEAK request timeline? As a crude example, say for instance the ASR couldnt recognize the speech data and the TTS would send back a text saying "I didnt understand, please say again". This text might have to be repeated multiple times within a page (or throughout the session). The text could be inline and can have a content-id associated with it where subsequent SPEAK request can invoke through the content-id. Also there could be other application specific TTS text that the author could write that will give feedback to the user based on ASR response metadata. The author can set multiple TTS text (each text with own id) at the server and the client can later invoke a particular TTS text based on an interaction condition (or based on metadata from the recognition result). 

>recognition. If so could you provide an example of its usage.

I am referring to partial results when they become available. For example, here is an interaction for a car sales application

user: get me the best quotes for minivans in Manhattan area
system: what models are you interested in? 
user: Dodge, Ford, Mitsubishi, Audi, Volkswagen, BMW
system: finding quotes for minivan
.. Dodge
.. Ford
..Mitsubishi
..Audi
..Vokswagen
..BMW

Please wait while the system searches for best quotes

In the interaction above, the ASR grammar would be written in such a way as to allow the user to make multiple selections (even multiple selections in mixed initiative). It would be desirable then to give feedback to the user when each model gets recognized. This result would be sent back to the client when a match in an active grammar is found. This could be coupled with a Speech_Timeout_Value when the recognizer would decide that the user has done all the selections.

Thank you.

Regards,

Sailesh.