Subject: RE: [Speechsc] MRCPv2 Comments

From: [email-masked]
Date: Tue, 2 Dec 2003 09:11:33 +0200

Hi Jeff,

Thanks for the reply. I guess you misunderstood the example a bit.

I was trying to put a use case where an INTERMEDIATE_RECO_RESULT primitive from the media server (Speech Recognizer) would come in handy. This primitive is sent by the Speech Recognizer when an intermediate result is obtained and could be sent to the client. In this case, the Speech Recognizer would not wait for the entire length till the whole result (of what the user said) becomes ready. The system here is the media server that is reponsible for interpreting what the user has said. 

The user would say something like

U: show me quotes for Ford, Dodge, Mitsubishi, Audi, Volkswagen

On the browser side, there would be a variable "minivan" that can take in multiple values. Once the whole set of values are filled for this variable, the client will then process this (and send to a server for getting the quotes).

When the user says the above dialog, the media server should not wait for the entire list of cars to be recognized before sending back the result. It can use an INTERMEDIATE_RECO_RESULT primitive to send back each result so that when each word gets recognized, it could be displayed back at the user terminal.

Let me put it again (MS is the media server, in this case the Speech Recognizer, CB is the client side browser (can be GUI or Voice Browser, U is the user)

U: Show me the quotes for Ford, Dodge, Mitsubishi, Audi, Volkswagen

MS: INTERMEDIATE_RECO_RESULT Ford

CB: You selected Ford (Assuming this is GUI)

MS:  INTERMEDIATE_RECO_RESULT Dodge

CB: You selected Ford, Dodge

MS: INTERMEDIATE_RECO_RESULT Mitsubishi

CB: You selected Ford, Dodge, Mitsubishi

MS: INTERMEDIATE_RECO_RESULT Audi

CB: You selected Ford, Dodge, Mitsubishi, Audi

MS: INTERMEDIATE_RECO_RESULT Volkswagen

CB: You selected Ford, Dodge, Mitsubishi, Audi, Volkswagen

MS: RECOGNITION_COMPLETE (after timeout)

CB: Please confirm your selections

User confirms the selections and the client will process the variable values by sending to content server that would return the quotes. This of course is a short list and there could be cases where the interaction and list could be more exhaustive and complex. Its good to add support for this. Hope I am a bit more clear :)

best regards,

Sailesh.

> -----Original Message-----
> From: ext Jeff Kusnitz [mailto:[email-masked]]
> Sent: 02 December, 2003 02:31
> To: Sathish Sailesh (NRC/Tampere)
> Cc: Hariharan Ramalingam (NRC/Tampere); [email-masked]
> Subject: Re: [Speechsc] MRCPv2 Comments
> 
> 
> I agree with your first example - not necessarily for TTS 
> itself, as I'd 
> almost expect an SSML engine to be able to cache frequently 
> seen text. But 
> it also would be useful to precache some audio files - large ones in 
> particular would be useful to have cached in advance (yes, I 
> realize once 
> they're fetched they can stay cached, but to get them cached, 
> one has to 
> render an SSML document currently).
> 
> In the second example, I'm not sure I follow - you'd like the 
> dialog to go 
> something like:
> 
> 
>         system: finding quotes for minivan
>         user: .. Dodge
>         system: got dodge
>         user: .. Ford
>         system: got ford
>         user: ..Mitsubishi
>         system: got mitsubishi
>         user: ..Audi
>         system: got audi
>         user: ..Vokswagen
>         system: got Vokswagen
>         user: ..BMW
>         system: got BMW
> 
>         [some app-specified delay elapses]
> 
>         system: Please wait while the system searches for best quotes
> 
> And then, the system will search for the list of cars?  If 
> so, this seems 
> like something that could/should be implemented by the client or the 
> application rather than the server.  The VoiceXML, in fact, 
> would probably 
> be something like a single field with a grammar, and in the 
> field's filled 
> handler, it appends the reco results to an array of results 
> queues a "got 
> xxx" prompt, and clears the field (which causes the FIA to 
> reselect it), 
> and then in the noinput handler, which happens after the timeout, the 
> searching takes place.
> 
> Or did I misunderstand what you're trying to do?
> 
> Thanks,
> Jeff
> 
> 
> 
> 
> 
> [email-masked]
> Sent by: [email-masked]
> 11/25/2003 05:25 AM
>  
>         To:     <[email-masked]>
>         cc:     <[email-masked]>
>         Subject:        Re: [Speechsc] MRCPv2 Comments
> 
> 
> Hi Mr Sarvi,
> 
> Thank you for the reply. Please find my comments below.
> 
> >With TTS there is no explicit content create or delete.  
> There is inline 
> content, that is stored on the server for the life of the 
> SPEAK request. 
> And there is URI referenced content(either specified as  a URI, or 
> specified as a URI from inside an inline content), which may 
> be held in 
> a local cache on the media server. The client uses the 
> Cache-Directive 
> fields to tell the media server, what caching directive 
> values to use in 
> the media servers document cache for thsi session or Request..
> 
> None of these cache directives applies for the inline content itself, 
> but only applies for content that is URI referenced and hence 
> those that 
> >the media server would have to go and fetch and possibly cache.
> 
> 
> Exactly. Wouldnt there be cases when the TTS inline content 
> would have to 
> be stored beyond the SPEAK request timeline? As a crude 
> example, say for 
> instance the ASR couldnt recognize the speech data and the 
> TTS would send 
> back a text saying "I didnt understand, please say again". 
> This text might 
> have to be repeated multiple times within a page (or throughout the 
> session). The text could be inline and can have a content-id 
> associated 
> with it where subsequent SPEAK request can invoke through the 
> content-id. 
> Also there could be other application specific TTS text that 
> the author 
> could write that will give feedback to the user based on ASR response 
> metadata. The author can set multiple TTS text (each text 
> with own id) at 
> the server and the client can later invoke a particular TTS 
> text based on 
> an interaction condition (or based on metadata from the recognition 
> result). 
> 
> >2. No INTERMEDIATE_RECOGNITION_RESULT send event from the 
> recognizer and 
> provision for setting this with the message fields (eg SET-PARAMS)
> >
> Can you elaborate on the need for this. Are you refering to 
> recognition 
> in stages(recgonizer, NL etc.) or are you refering to parital 
> >recognition. If so could you provide an example of its usage.
> 
> I am referring to partial results when they become available. 
> For example, 
> here is an interaction for a car sales application
> 
> user: get me the best quotes for minivans in Manhattan area
> system: what models are you interested in? 
> user: Dodge, Ford, Mitsubishi, Audi, Volkswagen, BMW
> system: finding quotes for minivan
> .. Dodge
> .. Ford
> ..Mitsubishi
> ..Audi
> ..Vokswagen
> ..BMW
> 
> Please wait while the system searches for best quotes
> 
> In the interaction above, the ASR grammar would be written in 
> such a way 
> as to allow the user to make multiple selections (even 
> multiple selections 
> in mixed initiative). It would be desirable then to give 
> feedback to the 
> user when each model gets recognized. This result would be 
> sent back to 
> the client when a match in an active grammar is found. This could be 
> coupled with a Speech_Timeout_Value when the recognizer would 
> decide that 
> the user has done all the selections.
> 
> Thank you.
> 
> Regards,
> 
> Sailesh.
>  
> Sailesh Kumar Sathish      GSM +358 50 4835679
> Nokia Research Center      Desk +358 3 272 5679
> P.O.Box 100, Tampere      [email-masked]
> Finland -33721
> 
> 
> _______________________________________________
> Speechsc mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/speechsc
> 
> 
> 

_______________________________________________
Speechsc mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/speechsc

---

From: Jeff Kusnitz <[email-masked]>
Date: Tue, 2 Dec 2003 14:04:06 -0800

Thank you for the clarification - the updated example makes much more 
sense than my interpretation of the initial one :-)

Jeff

---

From: Jeff Kusnitz <[email-masked]>
Date: Tue, 2 Dec 2003 14:20:32 -0800

I've got a couple of questions about bargein as it relates to the MRCP v2 
spec.

1. Currently, the recognition server has no idea about whether bargein is 
enabled, disabled, etc.  When speech start is detected, it's supposed to 
fire off a message to the client, and, if the tts resource is on the same 
server, fire a message off there as well, where the tts server will decide 
whether or not to interrupt the prompt.  Would it not make sense to have a 
boolean "barge-in" field added to the recognition header to tell it 
whether it should bother reporting bargein to the client/tts resource or 
not?

2. VoiceXML 2.0 supports two different types of bargein - speech, which 
means "speech was detected", and "hotword", which means that something 
from an active DTMF or speech grammar was recognized.  MRCP seems to only 
have a single type of bargein - "speech".  Lack of support for "hotword" 
makes it difficult to implement a VoiceXML 2.0 browser.  I'd like to see a 
"barge-in-type" field added to the recognition header which allowed for 
"speech" or "hotword"

Thanks,
Jeff