Subject: RE: [Speechsc] Managing big grammars

From: "Shanmugham, Saravanan" <[email-masked]>
Date: Tue, 7 Jun 2005 12:09:22 -0700

Sorry for the late reply. The way I think this should be addressed is
to have support for compiled grammars as URI. All Large or Huge grammars
are going to be URI referenced and never embedded in a DEFINE grammar. 
This means that the URI could point to compiled grammar file.

Another way is to use the chache capability. When you access a grammar
through HTTP is to cache it. In general instead of caching of the XML
grammar you could decide to cache the compiled version instead. This
would mean that you might do an initial DEFINE-GRAMMAR to force a
compile of the grammar and future references to that grammar URI would
hit the cache and hence the compiled version.

Sarvi

     -----Original Message-----
     From: [email-masked] 
     [mailto:[email-masked]] On Behalf Of Corby Anderson
     Sent: Thursday, May 26, 2005 2:11 PM
     To: [email-masked]
     Subject: [Speechsc] Managing big grammars
     
     What are folks' thoughts on how we could manage big 
     grammars with MRCP?
     
     There are two usage cases that I'm interested in: a Large 
     Grammar that takes several seconds to compile, and a Jumbo 
     Grammar that takes many many minutes to compile.  A Large 
     Grammar might contain several hundred or thousand entries. 
      A Jumbo Grammar might contain hundreds of thousands of entries.
     
     For a Jumbo Grammar, there's no way that a user can wait 
     for a grammar to be uploaded and compiled via the 
     DEFINE-GRAMMAR or RECOGNIZE methods.  A grammar that large 
     needs to be pre-compiled so that the DEFINE-GRAMMAR or 
     RECOGNIZE method can simply load the grammar binary.
     
     For a Large Grammar, it might be okay to make one user 
     wait a few seconds for the grammar to compile -- but it 
     would be a big win if that compiled grammar were stored 
     with a known name so that /subsequent/ uses of the same 
     grammar don't have to pay the same compilation penalty.
     
     Our current speech recognition vendor's api provides 
     methods for checking for the existence of a dynamic 
     grammar, and for compiling a new grammar (named as the 
     client specifies). The api also provides a powerful and 
     flexible way to manage these stored grammars.  Is there 
     some MRCP-friendly way for dealing with these types of 
     grammars?  If not, then is one being considered?
     
     Corby Anderson
     Tellme Networks, Inc.

     _______________________________________________
     Speechsc mailing list
     [email-masked]
     https://www1.ietf.org/mailman/listinfo/speechsc

_______________________________________________
Speechsc mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/speechsc

---

From: "Reifenrath, Klaus" <[email-masked]>
Date: Wed, 8 Jun 2005 09:01:09 -0400

I am confused. Where does NLSM or EMMA fit in here? Both standards define
the recognition result format.
Caching solves the problem only partly, because cached grammars are not
permanent and cannot be shared between different servers. Hence only a
COMPILE-GRAMMAR method solves the use case of Corby.

Klaus 

-----Original Message-----
From: David R. Oran [mailto:[email-masked]] 
Sent: Mittwoch, 8. Juni 2005 14:41
To: Reifenrath, Klaus
Cc: 'Shanmugham, Saravanan'; Corby Anderson; [email-masked]
Subject: Re: [Speechsc] Managing big grammars

On Jun 8, 2005, at 8:28 AM, Reifenrath, Klaus wrote:

> I agree.
>
> BUT unfortunately there is no MRCPv2 interface to just compile a 
> grammar. So grammar compilation needs to be done with vendor specific 
> tools or APIs.
> We could add a new method (allowed only in idle state):
> COMPILE-GRAMMAR
> that requires the header field
> Compiled-Grammar-URI
> that specifies the location of the compiled grammar (like 
> Personal-Grammar-URI).
> Is it too late to add this simple optional method?
>
I don't see why we need this given we have DEFINE_GRAMMAR. If we simply
allow DEFINE_GRAMMAR to take compiled grammars as well as XML grammars as
Sarvi suggested, isn't this adequate?

I also think Sarvi's caching idea is a good thing for implementations to do,
and while it doesn't affect the protocol it might be worth pointing out this
optimization in the spec.

By the way, for NLSML we could have a hack which allowed a compiled version
of the grammar to be embedded in the XML, thus avoiding the need for
alternate URI forms, questions about wire encoding, and having to invent yet
another vendor-extension mechanism since I assume compiled grammars are
implementation-specific.

Might be worth suggesting something like this for EMMA too...

Dave.
> Klaus
> From: Shanmugham, Saravanan [mailto:[email-masked]]
> Sent: Dienstag, 7. Juni 2005 21:09
> To: Corby Anderson; [email-masked]
> Subject: RE: [Speechsc] Managing big grammars
>
> to have support for compiled grammars as URI. All Large or Huge 
> grammars are going to be URI referenced and never embedded in a DEFINE 
> grammar.
> grammar you could decide to cache the compiled version instead.
> This would
> mean that you might do an initial DEFINE-GRAMMAR to force a compile of 
> the grammar and future references to that grammar URI would hit the 
> cache and hence the compiled version.
>
> Sarvi
>

---

From: "Shanmugham, Saravanan" <[email-masked]>
Date: Wed, 8 Jun 2005 09:41:03 -0700

-----Original Message-----
     From: Reifenrath, Klaus [mailto:[email-masked]] 
     Sent: Wednesday, June 08, 2005 6:01 AM
     To: 'David R. Oran'; Corby Anderson
     Cc: Shanmugham, Saravanan; [email-masked]
     Subject: RE: [Speechsc] Managing big grammars
     
     I am confused. Where does NLSM or EMMA fit in here? Both 
     standards define the recognition result format.
     Caching solves the problem only partly, because cached 
Sarvi>> The advantage is that it is almost self optimizing. And how
often you want a grammar to be stored in cache is going to be based on
the Caching parameters U use for that grammar URI
     grammars are not permanent and cannot be shared between 
     different servers. Hence only a COMPILE-GRAMMAR method 
Sarvi>> I am not sure how a COMPILE-GRAMMAR method is going to address
sharing grammars across MRCP servers.

Sarvi
     solves the use case of Corby.
     
     Klaus 
     
     -----Original Message-----
     From: David R. Oran [mailto:[email-masked]]
     Sent: Mittwoch, 8. Juni 2005 14:41
     To: Reifenrath, Klaus
     Cc: 'Shanmugham, Saravanan'; Corby Anderson; [email-masked]
     Subject: Re: [Speechsc] Managing big grammars

     > Is it too late to add this simple optional method?
     I don't see why we need this given we have DEFINE_GRAMMAR. 
     If we simply
     allow DEFINE_GRAMMAR to take compiled grammars as well as 
     XML grammars as
     Sarvi suggested, isn't this adequate?
     
     I also think Sarvi's caching idea is a good thing for 
     implementations to do,
     and while it doesn't affect the protocol it might be worth 
     pointing out this
     optimization in the spec.
     
     By the way, for NLSML we could have a hack which allowed a 
     compiled version
     of the grammar to be embedded in the XML, thus avoiding 
     the need for
     alternate URI forms, questions about wire encoding, and 
     having to invent yet
     another vendor-extension mechanism since I assume compiled 
     grammars are
     implementation-specific.

     Dave.

---

From: "Reifenrath, Klaus" <[email-masked]>
Date: Thu, 9 Jun 2005 05:02:38 -0400

I do not think that a standard for compiled grammar is required, because the
client does not need to process the compiled grammar. It only passes the URI
to the engine. In networks with MRCP servers from different vendors the
client (only) need to take care, that the grammar is compiled by an MRCP
server of the same vendor that will be used for recognition. 
Note: There exists no standard for voice enrolled grammars. Hence we have
the same situation (no standard, client need to choose same vendor for
enrolment and recognition) for enrolment.

Klaus

-----Original Message-----
From: Qiru Zhou [mailto:[email-masked]] 
Sent: Mittwoch, 8. Juni 2005 23:17
To: Corby Anderson
Cc: [email-masked]
Subject: Re: [Speechsc] Managing big grammars

I think the problem here is there is no standard for compiled grammar
format. This is a vendor implementation specific which is not inter-operable
across different vendors. I agree that pre-compiled large grammars can
improve system response and processing efficiency. But I don't know if this
should be the part of standard. Same for cache, it is an implementation
specific issue. Another practice on this issue is to pre load and archive
large grammar in recognizer (i.e., add ARCHIVE-GRAMMAR method).

-- Qiru

Corby Anderson wrote:
> One feature that I'd like a COMPILE-GRAMMAR method to provide is the 
> ability to specify common back-end storage (like Sarvi mentioned) so 
> that compiling a Large Grammar one time could make it available to 
> other MRCP servers via the Compiled-Grammar-URI that Klaus suggested.  
> It would be especially useful to us if the *client* could specify the 
> URI where the grammar should be stored.  This would permit clients to 
> use grammar names that are meaningful to the client.  (The example for 
> Personal-Grammar-URI seems like the grammar name is specified by the 
> client, but it's worth spelling out.)
> 
> To round out the utility of a feature like this, we'd need some way to 
> determine if a grammar had already been compiled.  Since we're talking 
> about URIs, the client could just do a HEAD on the grammar to see if 
> it exists -- but does the client always have access to back-end 
> resources in this way?  If not, then a QUERY-GRAMMAR method which 
> takes a Compiled-Grammar-URI header could provide this capability.
> 
> Corby Anderson
> Tellme Networks, Inc.
> 
> 
> Reifenrath, Klaus wrote:
> 
>> I agree.
>> From: Shanmugham, Saravanan [mailto:[email-masked]] Sent: Dienstag, 
>> 7. Juni 2005 21:09
grammar.
>> Another way is to use the chache capability. When you access a 
>> grammar through HTTP is to cache it. In general instead of caching of 
>> the XML grammar you could decide to cache the compiled version 
>> instead. This would mean that you might do an initial DEFINE-GRAMMAR 
>> to force a compile of the grammar and future references to that 
>> grammar URI would hit the cache and hence the compiled version.
>>
>> Sarvi
>>
>> Grammar that takes several seconds to compile, and a Jumbo     Grammar 
>> that takes many many minutes to compile.  A Large     Grammar might 
>> contain several hundred or thousand entries.      A Jumbo Grammar 
>> might contain hundreds of thousands of entries.
>> for a grammar to be uploaded and compiled via the     DEFINE-GRAMMAR 
>> or RECOGNIZE methods.  A grammar that large     needs to be 
>> pre-compiled so that the DEFINE-GRAMMAR or     RECOGNIZE method can 
>> simply load the grammar binary.
>> wait a few seconds for the grammar to compile -- but it     would be a 
>> big win if that compiled grammar were stored     with a known name so 
>> that /subsequent/ uses of the same     grammar don't have to pay the 
>> same compilation penalty.
>> methods for checking for the existence of a dynamic     grammar, and 
>> for compiling a new grammar (named as the     client specifies). The 
>> api also provides a powerful and     flexible way to manage these 
>> stored grammars.  Is there     some MRCP-friendly way for dealing with 
>> these types of     grammars?  If not, then is one being considered?
>>         Corby Anderson
>>     Tellme Networks, Inc.

---

From: "Shanmugham, Saravanan" <[email-masked]>
Date: Sat, 11 Jun 2005 22:29:15 -0700

I am not sure I get this. There is already support for this. 
There is a bunch of caching related headers that already defined, and
they are based on HTTP. 
So when the client does a DEFINE-GRAMMAR for large grammar, the ideal
way is to refer to it using a HTTP URI, so that the MRCP server is
expected to go download and possibly compile it. 
When you do this, the server is expected to maintain a cache of
downloaded documents. The caching behaviour for these documents is
controlled by the MRCP client by setting the appropriate cache control
parameters in the DEFINE-GRAMMAR request. 
This controls how long the MRCP server is expected to cache such
documents. Also, since these large grammars are accessed by the MRCP
server through a HTTP client, the webserver serving these large
documents can also specify the caching properties of the documents it
serves. The MRCP server and HTTP client and cache are supposed to honor
this.

Nthis should cover all of your large grammar requirements below.

Sarvi  

     -----Original Message-----
     From: [email-masked] 
     [mailto:[email-masked]] On Behalf Of Corby Anderson
     Sent: Saturday, June 11, 2005 1:41 PM
     To: [email-masked]
     Subject: Re: [Speechsc] Managing big grammars
     
     Thanks, all, for considering those features.  I appreciate 
     the time you spent discussing them -- especially since the 
     spec is so close to completion.
     
     So let's quickly revisit the original two usage cases I 
     was interested in (Large grammars that might be several 
     thousand lines long and Jumbo grammars that might be 
     hundreds of thousands of lines long).  It sounds clear 
     that the group's intent is for vendors to provide back-end 
     tools for compiling, storing, and referring to Jumbo 
     grammar binaries; so Sarvi's option #1 would only apply to 
     the Large grammars.  When the client wants to recognize 
     with a Large grammar, it would have to upload the grammar 
     source to the MRCP server (which may or may not cache it 
     in source and/or binary form -- depending on how/if the 
     vendor chooses to implement this optimization).  Would it 
     be appropriate to include a hint to the client that the 
     MRCP server has cached the grammar?  For example, if a 
     Large grammar is uploaded with DEFINE-GRAMMAR, a header in 
     the 200 response could indicate that this grammar is 
     cached -- a client could use this information to avoid 
     uploading that Large grammar again for future recognitions 
     with that grammar on that MRCP server.
     
     The downside, of course, is that it raises cache 
     management issues: how long do these things stay in cache, 
     how can a client determine if a previously-cached grammar 
     is still in the cache, etc.  All of these are 
     surmountable, but I don't have a feel for whether they 
     belong in the spec or not.
     
     What does the group think?  If a vendor chooses to 
     implement Large grammar caching, should the spec cover 
     supplying hints to the client so that the client doesn't 
     have to upload the same (cached) grammar for subsequent 
     recogntions?  Even though this would be a vendor-optional 
     feature, I think it would common enough that the spec 
     would benefit from having a common mechanism to indicate 
     whether a grammar is cached.
     
     Thanks,
     Corby Anderson
     Tellme Networks, Inc.
     
     Dave Burke wrote:
     
     > I agree. Grammar compilation, caching, and sharing is a 
     problem for 
     > the MRCP server. Exposing this in the protocol puts unnecessary 
     > complexity on the client. For example, the client needs 
     to download 
     > grammars, implement its own HTTP caching, checksum 
     algorithms (some 
     > way of not having to push huge data to the MRCP server 
     to query) etc.
     >  
     > So I vote for point 1 below and include in the text that an MRCP 
     > server could also, as an optimisation, implement a 
     centralised common 
     > cache shared across multiple MRCP servers.
     >  
     > Dave
     >
     >     ----- Original Message -----
     >     *From:* Shanmugham, Saravanan <mailto:[email-masked]>
     >     *To:* Corby Anderson <mailto:[email-masked]> ; 
     [email-masked]
     >     <mailto:[email-masked]>
     >     *Sent:* Saturday, June 11, 2005 12:09 AM
     >     *Subject:* RE: [Speechsc] Managing big grammars
     >
     >     I would say that a common cache amoung a farm of 
     MRCP servers to
     >     optimize compilations would be a nice server side 
     optimization and
     >     shouldn't affect the MRCP protocol.
     >      
     >     my 2c.
     >     Sarvi
     >
     >         
     -----------------------------------------------------------
     -------------
     >         *From:* [email-masked]
     >         <mailto:[email-masked]>
     >         [mailto:[email-masked]] *On Behalf Of 
     *Corby Anderson
     >         *Sent:* Friday, June 10, 2005 11:52 AM
     >         *To:* [email-masked] <mailto:[email-masked]>
     >         *Subject:* Re: [Speechsc] Managing big grammars
     >
     >         Option #1 seems like a step in the right 
     direction, but having
     >         multi-server re-use would make it even more 
     useful. Option #1
     >         as worded implies that /each /MRCP server would have to
     >         compile grammar "X" the first time it's used;  
     I'm suggesting
     >         that we try and make one MRCP server's 
     compilation efforts
     >         useful to other MRCP servers (as long as they're the same
     >         vendor).  The use case I have in mind is a "farm" of
     >         recognition servers -- and when we compile one 
     of these Large
     >         grammars (grammar "X"), it would be great to 
     store it in some
     >         common storage area so that other MRCP servers 
     could check for
     >         its existence and load the binary. 
     >         It isn't necessary to have a standard compiled 
     grammar format
     >         -- it's just necessary to have a standard way to 
     reference
     >         them.  In a server farm environment containing multiple
     >         vendors' MRCP servers, wouldn't the client know 
     (or couldn't
     >         it discover during session setup) which kind of 
     server it was
     >         connecting to and so adjust the query/compile uri
     >         accordingly?  I envision something along the lines of:
     >
     >             MRCP/2.0 82 QUERY-GRAMMAR 123456
     >             Compiled-Grammar-URI:
     >             
     > 
     http://grammar-resource/<insert_vendor_here>/query?name=<gr
     ammar_name>
     >
     >
     >             MRCP/2.0 507 COMPILE-GRAMMAR 123654
     >             Compiled-Grammar-URI:
     >             
     http://grammar-resource/<insert_vendor_here>/store?name=<gr
     ammar_name>
     >             Content-type: application/srgs+xml
     >             Content-Length: 176
     >
     >             <?xml version="1.0" (etc.)
     >
     >
     >         Where the "grammar-resource" would be simple and general
     >         enough to be implemented by either the client or 
     the vendor.
     >
     >         Corby Anderson
     >         Tellme Networks, Inc.
     >
     >         Shanmugham, Saravanan wrote:
     >
     >>I agree, the issue is that there is no standard for 
     compilation and 
     >>compiled grammar format.  I am not sure a 
     COMPILE-GRAMMAR method is 
     >>the solution to this.
     >>
     >>The aim of compilation is to accelarate customer 
     response when dealing 
     >>with large-grammars.
     >>A few ways to achieve the above, like I mentioned earlier are,
     >>   1. The MRCP server could implement an optimization for large 
     >>grammars that leverages the grammar caching mechanism and its 
     >>capabilities. That is when they cache a grammar URI, 
     they also store 
     >>the compiled version of the grammar. They could do this 
     in a vendor specific compiled format.
     >>Large grammars are usually not dynamic and hence the 
     caching mechanims 
     >>should work reasonably well for this.
     >>   2. Have the MRCP client point to 
     pre-compiled-grammars so that when 
     >>the MRCP server goes and fetches it the large-grammar is already 
     >>compiled. There is the problem of making sure the compiled
     >>grammar(vendor-specific) matches the specific 
     recognition engine used.
     >>This is difficult to manage in a multivendor deployment.
     >>   3. Compile and store the grammars on the MRCP server 
     ahead of time 
     >>during deployments. And then refer to these compiled 
     grammars on the 
     >>MRCP server. This would have its own limitations, but would be 
     >>workable in many cases/deployments.
     >>
     >>
     >>We could add the text for the first above bullet into the MRCPv2 
     >>specification if no one has any objects to it.
     >>
     >>Thanks,
     >>
     >>     [mailto:[email-masked]] On Behalf Of Qiru Zhou
     >>     Sent: Wednesday, June 08, 2005 2:17 PM
     >>     To: Corby Anderson
     >>     Cc: [email-masked]
     >>     I think the problem here is there is no standard for 
     >>     compiled grammar format. This is a vendor implementation 
     >>     specific which is not inter-operable across different 
     >>     vendors. I agree that pre-compiled large grammars can 
     >>     improve system response and processing efficiency. But I 
     >>     don't know if this should be the part of standard. Same 
     >>     for cache, it is an implementation specific issue. Another 
     >>     practice on this issue is to pre load and archive large 
     >>     grammar in recognizer (i.e., add ARCHIVE-GRAMMAR method).
     >>     
     >>     -- Qiru
     >>     
     >>     Corby Anderson wrote:
     >>     >> grammars are going to be URI referenced and never 
     >>     embedded in a DEFINE grammar.
     >>     DEFINE-GRAMMAR 
     >>         would be a 
     >>     grammar, and 
     >>     >>     https://www1.ietf.org/mailman/listinfo/speechsc
     -----------------------------------------------------------
     -------------

---

From: "Wyss, Felix" <[email-masked]>
Date: Tue, 14 Jun 2005 08:08:27 -0500

Using a stale grammar for calls arriving while the new version is
fetched and compiled would be acceptable, provided:

  1) There is a flag in the DEFINE-GRAMMAR request to control this
behavior.  I can think of three possible options:

    a. Ensure grammar is always up-to date according to the HTTP
server's caching headers ("accuracy over latency")

    b. Allow stale grammars to be used ("latency over accuracy"), but
block when grammar is not in the cache at all

    c. Like b, but fail immediately if grammar is not already in the
cache; that way, the application can fallback to something else instead
of spuriously blocking.

  2) DEFINE-GRAMMAR returns a header or result-code indicating whether a
stale grammar is being used 

  3) If a DEFINE-GRAMMAR causes a stale grammar to be used, that grammar
must be used for the whole duration of that call or until the next
DEFINE-GRAMMAR is issued for it (i.e. the grammar won't unpredictably
change between recognitions just because a newer version became ready in
the meantime).  

  4) All standard conforming MRCP servers MUST implement the caching
behavior this way (i.e. it is not optional or up to the vendor like in
RFC 2616).  

Of course, this still requires the application to ping the servers to
update their caches; otherwise some calls may get very stale grammars.  

Felix

________________________________

From: Dave Burke [mailto:[email-masked]] 
Sent: Tuesday, June 14, 2005 07:16
To: Shanmugham, Saravanan; Wyss, Felix; Corby Anderson;
[email-masked]
Subject: Re: [Speechsc] Managing big grammars

That optimisation would be nice alright and is not restricted by HTTP
(see note from RFC 2616 below). 

It seems like we have got some good optimisation suggestions (read:
informative) that could go into the spec to address large grammars. If
memory serves, they are:

    1. An MRCP server may cache grammars used in DEFINE-GRAMMAR or
RECOGNIZE to be used over multiple MRCP sessions

    2. An MRCP server may cache the compiled version of a grammar

    3. An MRCP server may cache grammars centrally to allow other
servers take advantage of the cache

    4. An MRCP server, in response to a DEFINE-GRAMMAR, may inform other
MRCP servers to reload the new grammar into memory

    5. An MRCP server may choose to return a stale response while a new
grammar is being downloaded, compiled, and cached

-- Dave

>From RFC 2616:

   In some cases, the operator of a cache MAY choose to configure it to
   return stale responses even when not requested by clients. This
   decision ought not be made lightly, but may be necessary for reasons
   of availability or performance, especially when the cache is poorly
   connected to the origin server. Whenever a cache returns a stale
   response, it MUST mark it as such (using a Warning header) enabling
   the client software to alert the user that there might be a potential
   problem.

---

From: "Reifenrath, Klaus" <[email-masked]>
Date: Tue, 14 Jun 2005 08:51:49 -0400

Is 4) a realistic scenario? It would require that the MRCP server know each
other, which results in a configuration nightmare. And what happens if one
of them is down just at that moment... 

Slight reformulation of 2):
An MRCP server may cache a compiled version of a grammar

Klaus

-----Original Message-----
From: David R. Oran [mailto:[email-masked]] 
Sent: Dienstag, 14. Juni 2005 14:31
To: Dave Burke
Cc: [email-masked]; Shanmugham, Saravanan; Wyss, Felix
Subject: Re: [Speechsc] Managing big grammars

I like Dave's formulation. If there are no objections, I'm going to craft
this into a sub-section on "Grammar Caching"  under the section on "Grammar
Data". On looking where to put this, I noticed that the description is under
9.5.1 "recognizer Grammar Data" which is under "Recognizer message body".
Some of that material isn't really about message body (since it talks about
ow indirect content is handled), so I'm going to re-jigger things a bit to
improve the flow and move the parts about indirect content to the new
earlier section.

Speak now if you don't like this, because I'm going to do this later
today...

On Jun 14, 2005, at 8:16 AM, Dave Burke wrote:

> informative) that could go into the spec to address large grammars.  
> If memory serves, they are:
>     4. An MRCP server, in response to a DEFINE-GRAMMAR, may inform 
> other MRCP servers to reload the new grammar into memory
>     5. An MRCP server may choose to return a stale response while a 
> new grammar is being downloaded, compiled, and cached
>
> -- Dave
>
> From RFC 2616:
>
> From: Shanmugham, Saravanan
> To: Dave Burke ; Wyss, Felix ; Corby Anderson ; [email-masked]
> Sent: Tuesday, June 14, 2005 12:04 PM
> cool. That makes sense.
> The way I though this problem would have been addressed, is the MRCP 
> caching system to be smart.
> That is everytime there is a request a grammar for a grammar URI.  
> the MRCP server would do a fetch to check if the document has changed. 
> If it has, then it would download and compile the grammar.
> While this happenning existing calls that were using the older 
> compiled grammar continue to use it. Once this is ready, the previous 
> version of the grammar is allowed to expire-out.
>
> Would this not address your scenario and avoid having to do this work 
> around you are suggesting?
>
> Sarvi
>
> Sent: Tuesday, June 14, 2005 3:16 AM
> It's not that the URL needs a specific date or time, just that it has 
> a unique name. This is simply so that you can ensure the new grammar 
> is downloaded, compiled, and loaded ahead of any calls.
> Once compiled and loaded, the application grammar URIs can be updated 
> to reference the new grammar. This way you avoid the danger that a 
> bunch of calls, which arrive at the same time as the DEFINE- GRAMMAR 
> is happening, experience delays.
>
> Dave
> From: Shanmugham, Saravanan
> To: Dave Burke ; Wyss, Felix ; Corby Anderson ; [email-masked]
> Sent: Tuesday, June 14, 2005 7:58 AM
> I dont understand why the RUL has to encode and date and time into it. 
> Lets say we are refering to large_grammar.grxml
>
> When a client does a DEFINE-GRAMMAR on the MRCP server with this URI. 
> The MRCP server is expected to go download this URI and compile it. 
> The caching behaviour of this URI and how long it lives in the cache 
> is dependent on what is returned by the HTTP GET operation that 
> brought the grammar into the MRCP server. In addition, the behaviour 
> is also governed by Cache -Control parameters that the client can 
> specify for the DEFINE-GRAMMAR that would take precedence over what 
> was returned by the HTTP GET operation. This means that the MRCP 
> client can force the MRCP server to ignore the cache parameters it got 
> from the grammar webserver.
>
> So, for the large_grammar.grxml, if the webserver returned a large 
> max-age value the grammar and its compiled version in the cache of the 
> MRCP server should not get timedout. For any future requests for that 
> grammar, MRCP server should go do fetch of the document, compare the 
> max-age of the fetched coument and make sure the webserver says that 
> the document in the cache is not out-of-date.
> As long as the grammar document in the cache is not out of date, you 
> don't need to compile it again and you are fine. If the propoerty of 
> the large_grammar.grxml on the webserver had changed then the MRCP 
> server would be expected to refresh the cache and compile the newly 
> downloaded grammar.
>
>
> Am I missing something here.
>
> Sarvi
>
> From: [email-masked] [mailto:[email-masked]]
> On Behalf Of Dave Burke
> Sent: Sunday, June 12, 2005 12:12 PM
> To: Wyss, Felix; Shanmugham, Saravanan; Corby Anderson; 
> [email-masked]
> There is no doubt that grammar compilation and loading must be hidden 
> from the caller experience. What is in doubt is whether the MRCP 
> protocol needs to expose such control.
>
> The grammar loading problem is not dissimilar to cache seeding in the 
> Web world. Taking your use-case, I would implement this in MRCP as a 
> two-step process:
>
> (1) Make the large grammar available through HTTP and date-stamp the 
> URI, e.g. http://example.com/large_grammar_12June2005.grxml.
> Set the HTTP response header Cache-Control: max-age to a value larger 
> than the grammar change period.
>
> (2) When a new grammar is required, open an MRCP channel to each of 
> the MRCP servers. Issue a DEFINE-GRAMMAR with the relevant URI e.g.
> http://example.com/large_grammar_13June2005.grxml. Update the 
> application to point at the new grammar.
>
> [For a mid-size grammar where loading latency is negligible, and 
> assuming a centrally located compiled grammar cache, step (2) might 
> only require a DEFINE-GRAMMAR to one MRCP server].
>
> You are correct that the current MRCPv2 draft scopes the DEFINE- 
> GRAMMAR to a channel and this is a concern. However, this is resolved 
> with Sarvi's suggested text change.
>
> Dave
> From: Wyss, Felix
> To: Dave Burke ; Shanmugham, Saravanan ; Corby Anderson ; 
> [email-masked]
> Sent: Sunday, June 12, 2005 1:41 AM
> I disagree.  Predictability is *very* critical for an application 
> using ASR.  It must be able to make sure that when a recognition 
> operation is started, there is no latency while the ASR server 
> compiles and/or loads a large precompiled grammar into memory.  The 
> DEFINE-GRAMMAR command is scoped to a channel, which means by the time 
> the application has a channel (i.e. a caller is in the system), it is 
> unacceptable to block for more than fraction of a
> second while the grammars are being compiled and/or loaded.   
> Otherwise, the first caller after a grammar is changed encounters 
> delays and has a poor user experience or even hangs up.  Somebody may 
> say: well, just make a "priming" call after modifying the grammar to 
> ensure the grammar is cached.  However, that is not very practical for 
> automatically generated grammars; but worse yet, in large systems 
> there are commonly several load-balanced ASR severs and it will be 
> rather hard to ensure each of them is touched at least once.
>
>
>
> The way we solve this on our platform is as follows: Large grammars 
> that may change frequently, possibly several times a day, such as 
> company directory grammars that are generated from a database, can be 
> registered as "preloaded grammars".  This means, our platform ensures 
> the grammar is compiled, distributed to all ASR servers, and loaded 
> into their caches after a new version of that grammar has been 
> generated.  Each preloaded grammar has a name associated, which is 
> referenced through a special URI.  The application logic only knows 
> that URI and not the actual URI/filename of the underlying grammar 
> data.  The subsystem managing the preloaded grammars ensures that the 
> preloaded grammar URI resolves to the previous version of the grammar 
> until the new grammar is loaded and ready on all ASR servers.  Thus, 
> all callers that call into the system while the grammar is being 
> generated, compiled, and loaded on the servers will get the previous 
> version.  This ensures that there is no delay for the first caller 
> after a change.  We can do this as we are using the native APIs 
> provided by the ASR engine vendors we support and layer a 
> sophisticated framework and API on top of it.
>
>
>
> It would be nice if MRCP provided some support for schemes like that, 
> without having to open dummy channels to register grammars and pray 
> the MRCP server doesn't evict them too early.  An MRCP client has to 
> be able to guarantee that when a call comes through
> the system, everything is ready and it won't experience any delay.   
> As it stands now the MRCP protocol does not appear to provide support 
> for this.
>
>
>
> Thanks,
>
> Felix
>
>
>
> From: [email-masked] [mailto:[email-masked]]
> On Behalf Of Dave Burke
> Sent: Saturday, June 11, 2005 10:42
> To: Shanmugham, Saravanan; Corby Anderson; [email-masked]
> I agree. Grammar compilation, caching, and sharing is a problem for  
> complexity on the client. For example, the client needs to download  
> grammars, implement its own HTTP caching, checksum algorithms (some  
> way of not having to push huge data to the MRCP server to query) etc.
>
>
>
> server could also, as an optimisation, implement a centralised  
> common cache shared across multiple MRCP servers.
>
>
>
> Dave
>
> From: Shanmugham, Saravanan
>
> To: Corby Anderson ; [email-masked]
>
> Sent: Saturday, June 11, 2005 12:09 AM
>
> I would say that a common cache amoung a farm of MRCP servers to  
> optimize compilations would be a nice server side optimization and

---

From: "Reifenrath, Klaus" <[email-masked]>
Date: Tue, 14 Jun 2005 08:51:49 -0400

Klaus

---

From: "Shanmugham, Saravanan" <[email-masked]>
Date: Tue, 14 Jun 2005 12:15:12 -0700

I think all this is saying is that a MRCPv2 vendor could desing their
system such they can deploy a cluster of them with a common compiled
cache and a internal mechanism to announce to each other of a change.
Those internal mechanisms are many and are outside the scope of this
specification.

SArvi

     -----Original Message-----
     From: Reifenrath, Klaus [mailto:[email-masked]] 
     Sent: Tuesday, June 14, 2005 5:52 AM
     To: 'David R. Oran'; Dave Burke
     Cc: [email-masked]; Shanmugham, Saravanan; Wyss, Felix
     Subject: RE: [Speechsc] Managing big grammars
     
     Is 4) a realistic scenario? It would require that the MRCP 
     server know each other, which results in a configuration 
     nightmare. And what happens if one of them is down just at 
     that moment... 

     Klaus
     
     -----Original Message-----
     From: David R. Oran [mailto:[email-masked]]
     Sent: Dienstag, 14. Juni 2005 14:31
     To: Dave Burke
     Cc: [email-masked]; Shanmugham, Saravanan; Wyss, Felix
     Subject: Re: [Speechsc] Managing big grammars
     
     I like Dave's formulation. If there are no objections, I'm 
     going to craft this into a sub-section on "Grammar 
     Caching"  under the section on "Grammar Data". On looking 
     where to put this, I noticed that the description is under
     9.5.1 "recognizer Grammar Data" which is under "Recognizer 
     message body".
     Some of that material isn't really about message body 
     (since it talks about ow indirect content is handled), so 
     I'm going to re-jigger things a bit to improve the flow 
     and move the parts about indirect content to the new 
     earlier section.
     
     Speak now if you don't like this, because I'm going to do 
     this later today...
     
     On Jun 14, 2005, at 8:16 AM, Dave Burke wrote:
     
     >    In some cases, the operator of a cache MAY choose to 
     configure it 
     > to
     [mailto:[email-masked]]
     > From: [email-masked] 
     [mailto:[email-masked]]
     problem for 
     to download 
     algorithms (some 
     to query) etc.
     centralised common 
     optimization and

---

From: "Wyss, Felix" <[email-masked]>
Date: Wed, 15 Jun 2005 17:13:53 -0500

[inline]

> [Dave B] . If you invoke DEFINE-GRAMMAR with some kind of
> header to indicate fail if the object is not in the cache, is the MRCP
> server to go fetch it anyway for future use? Should this mechanism
> fail if the grammar is an intermediate cache somewhere between the
> MRCP server and HTTP origin server?

[Felix I. Wyss] Looking at this from an application's perspective, I
think the criteria should be: A "cache probing" DEFINE-GRAMMAR should
fail if the server cannot guarantee that an immediately following
recognition with that grammar will not incur noticeable delays (e.g.
>0.5s).  In practice I presume that will probably mean that the server
has to have the compiled version of the grammar already cached.  

I would say the command does not cause the grammar to be fetched if it
is not in the cache.

Felix