Subject: RE: [Speechsc] Managing big grammars

From: "Bennett, Patrick" <[email-masked]>
Date: Tue, 14 Jun 2005 11:36:13 -0500

> An MRCP server may cache a compiled version of a grammar

What good is this?  The more MAYs you have in the specification, the
more it becomes unusuable by clients.  While server implementers
certainly love MAYs (or to some extent even the SHOULDs) in RFC's (it
lets them take shortcuts), implementors who actually have to USE the
protocol, despise them.  RFCs littered with 'may do x' are nothing more
than general guidelines, not the strict protocol and behavioral
specification which MRCP requires.

If needs to be to be usable by carrier grade and scale implementors.
This isn't possible when the MRCP servers MAY do something.

I guess I could just say, the MRCP spec MAY be unusable for anything
other a simple test system with trivial, *static* grammars with no more
than a few concurrent sessions.
 
> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Reifenrath, Klaus
> Sent: Tuesday, June 14, 2005 7:52 AM
> To: 'David R. Oran'; Dave Burke
> Cc: [email-masked]; Shanmugham, Saravanan; Wyss, Felix
> Subject: RE: [Speechsc] Managing big grammars
> 
> Is 4) a realistic scenario? It would require that the MRCP server know
> each
> other, which results in a configuration nightmare. And what happens if
one
> of them is down just at that moment...
> 
> Slight reformulation of 2):
> An MRCP server may cache a compiled version of a grammar
> 
> Klaus
> 
> -----Original Message-----
> From: David R. Oran [mailto:[email-masked]]
> Sent: Dienstag, 14. Juni 2005 14:31
> To: Dave Burke
> Cc: [email-masked]; Shanmugham, Saravanan; Wyss, Felix
> Subject: Re: [Speechsc] Managing big grammars
> 
> I like Dave's formulation. If there are no objections, I'm going to
craft
> this into a sub-section on "Grammar Caching"  under the section on
> "Grammar
> Data". On looking where to put this, I noticed that the description is
> under
> 9.5.1 "recognizer Grammar Data" which is under "Recognizer message
body".
> Some of that material isn't really about message body (since it talks
> about
> ow indirect content is handled), so I'm going to re-jigger things a
bit to
> improve the flow and move the parts about indirect content to the new
> earlier section.
> 
> Speak now if you don't like this, because I'm going to do this later
> today...
> 
> On Jun 14, 2005, at 8:16 AM, Dave Burke wrote:
> 
> > That optimisation would be nice alright and is not restricted by
HTTP
> > (see note from RFC 2616 below).
> >
> > It seems like we have got some good optimisation suggestions (read:
> > informative) that could go into the spec to address large grammars.
> > If memory serves, they are:
> >     1. An MRCP server may cache grammars used in DEFINE-GRAMMAR or
> > RECOGNIZE to be used over multiple MRCP sessions
> >     2. An MRCP server may cache the compiled version of a grammar
> >     3. An MRCP server may cache grammars centrally to allow other
> > servers take advantage of the cache
> >     4. An MRCP server, in response to a DEFINE-GRAMMAR, may inform
> > other MRCP servers to reload the new grammar into memory
> >     5. An MRCP server may choose to return a stale response while a
> > new grammar is being downloaded, compiled, and cached
> >
> > -- Dave
> >
> > From RFC 2616:
> >
> >    In some cases, the operator of a cache MAY choose to configure it
> > to
> >    return stale responses even when not requested by clients. This
> >    decision ought not be made lightly, but may be necessary for
> > reasons
> >    of availability or performance, especially when the cache is
poorly
> >    connected to the origin server. Whenever a cache returns a stale
> >    response, it MUST mark it as such (using a Warning header)
enabling
> >    the client software to alert the user that there might be a
> > potential
> >    problem.
> >
> >
> > ----- Original Message -----
> > From: Shanmugham, Saravanan
> > To: Dave Burke ; Wyss, Felix ; Corby Anderson ; [email-masked]
> > Sent: Tuesday, June 14, 2005 12:04 PM
> > Subject: RE: [Speechsc] Managing big grammars
> >
> > cool. That makes sense.
> > The way I though this problem would have been addressed, is the MRCP
> > caching system to be smart.
> > That is everytime there is a request a grammar for a grammar URI.
> > the MRCP server would do a fetch to check if the document has
changed.
> > If it has, then it would download and compile the grammar.
> > While this happenning existing calls that were using the older
> > compiled grammar continue to use it. Once this is ready, the
previous
> > version of the grammar is allowed to expire-out.
> >
> > Would this not address your scenario and avoid having to do this
work
> > around you are suggesting?
> >
> > Sarvi
> >
> > From: Dave Burke [mailto:[email-masked]]
> > Sent: Tuesday, June 14, 2005 3:16 AM
> > To: Shanmugham, Saravanan; Wyss, Felix; Corby Anderson;
> > [email-masked]
> > Subject: Re: [Speechsc] Managing big grammars
> >
> > It's not that the URL needs a specific date or time, just that it
has
> > a unique name. This is simply so that you can ensure the new grammar
> > is downloaded, compiled, and loaded ahead of any calls.
> > Once compiled and loaded, the application grammar URIs can be
updated
> > to reference the new grammar. This way you avoid the danger that a
> > bunch of calls, which arrive at the same time as the DEFINE- GRAMMAR
> > is happening, experience delays.
> >
> > Dave
> > ----- Original Message -----
> > From: Shanmugham, Saravanan
> > To: Dave Burke ; Wyss, Felix ; Corby Anderson ; [email-masked]
> > Sent: Tuesday, June 14, 2005 7:58 AM
> > Subject: RE: [Speechsc] Managing big grammars
> >
> > I dont understand why the RUL has to encode and date and time into
it.
> > Lets say we are refering to large_grammar.grxml
> >
> > When a client does a DEFINE-GRAMMAR on the MRCP server with this
URI.
> > The MRCP server is expected to go download this URI and compile it.
> > The caching behaviour of this URI and how long it lives in the cache
> > is dependent on what is returned by the HTTP GET operation that
> > brought the grammar into the MRCP server. In addition, the behaviour
> > is also governed by Cache -Control parameters that the client can
> > specify for the DEFINE-GRAMMAR that would take precedence over what
> > was returned by the HTTP GET operation. This means that the MRCP
> > client can force the MRCP server to ignore the cache parameters it
got
> > from the grammar webserver.
> >
> > So, for the large_grammar.grxml, if the webserver returned a large
> > max-age value the grammar and its compiled version in the cache of
the
> > MRCP server should not get timedout. For any future requests for
that
> > grammar, MRCP server should go do fetch of the document, compare the
> > max-age of the fetched coument and make sure the webserver says that
> > the document in the cache is not out-of-date.
> > As long as the grammar document in the cache is not out of date, you
> > don't need to compile it again and you are fine. If the propoerty of
> > the large_grammar.grxml on the webserver had changed then the MRCP
> > server would be expected to refresh the cache and compile the newly
> > downloaded grammar.
> >
> >
> > Am I missing something here.
> >
> > Sarvi
> >
> > From: [email-masked] [mailto:[email-masked]]
> > On Behalf Of Dave Burke
> > Sent: Sunday, June 12, 2005 12:12 PM
> > To: Wyss, Felix; Shanmugham, Saravanan; Corby Anderson;
> > [email-masked]
> > Subject: Re: [Speechsc] Managing big grammars
> >
> > There is no doubt that grammar compilation and loading must be
hidden
> > from the caller experience. What is in doubt is whether the MRCP
> > protocol needs to expose such control.
> >
> > The grammar loading problem is not dissimilar to cache seeding in
the
> > Web world. Taking your use-case, I would implement this in MRCP as a
> > two-step process:
> >
> > (1) Make the large grammar available through HTTP and date-stamp the
> > URI, e.g. http://example.com/large_grammar_12June2005.grxml.
> > Set the HTTP response header Cache-Control: max-age to a value
larger
> > than the grammar change period.
> >
> > (2) When a new grammar is required, open an MRCP channel to each of
> > the MRCP servers. Issue a DEFINE-GRAMMAR with the relevant URI e.g.
> > http://example.com/large_grammar_13June2005.grxml. Update the
> > application to point at the new grammar.
> >
> > [For a mid-size grammar where loading latency is negligible, and
> > assuming a centrally located compiled grammar cache, step (2) might
> > only require a DEFINE-GRAMMAR to one MRCP server].
> >
> > You are correct that the current MRCPv2 draft scopes the DEFINE-
> > GRAMMAR to a channel and this is a concern. However, this is
resolved
> > with Sarvi's suggested text change.
> >
> > Dave
> > ----- Original Message -----
> > From: Wyss, Felix
> > To: Dave Burke ; Shanmugham, Saravanan ; Corby Anderson ;
> > [email-masked]
> > Sent: Sunday, June 12, 2005 1:41 AM
> > Subject: RE: [Speechsc] Managing big grammars
> >
> > I disagree.  Predictability is *very* critical for an application
> > using ASR.  It must be able to make sure that when a recognition
> > operation is started, there is no latency while the ASR server
> > compiles and/or loads a large precompiled grammar into memory.  The
> > DEFINE-GRAMMAR command is scoped to a channel, which means by the
time
> > the application has a channel (i.e. a caller is in the system), it
is
> > unacceptable to block for more than fraction of a
> > second while the grammars are being compiled and/or loaded.
> > Otherwise, the first caller after a grammar is changed encounters
> > delays and has a poor user experience or even hangs up.  Somebody
may
> > say: well, just make a "priming" call after modifying the grammar to
> > ensure the grammar is cached.  However, that is not very practical
for
> > automatically generated grammars; but worse yet, in large systems
> > there are commonly several load-balanced ASR severs and it will be
> > rather hard to ensure each of them is touched at least once.
> >
> >
> >
> > The way we solve this on our platform is as follows: Large grammars
> > that may change frequently, possibly several times a day, such as
> > company directory grammars that are generated from a database, can
be
> > registered as "preloaded grammars".  This means, our platform
ensures
> > the grammar is compiled, distributed to all ASR servers, and loaded
> > into their caches after a new version of that grammar has been
> > generated.  Each preloaded grammar has a name associated, which is
> > referenced through a special URI.  The application logic only knows
> > that URI and not the actual URI/filename of the underlying grammar
> > data.  The subsystem managing the preloaded grammars ensures that
the
> > preloaded grammar URI resolves to the previous version of the
grammar
> > until the new grammar is loaded and ready on all ASR servers.  Thus,
> > all callers that call into the system while the grammar is being
> > generated, compiled, and loaded on the servers will get the previous
> > version.  This ensures that there is no delay for the first caller
> > after a change.  We can do this as we are using the native APIs
> > provided by the ASR engine vendors we support and layer a
> > sophisticated framework and API on top of it.
> >
> >
> >
> > It would be nice if MRCP provided some support for schemes like
that,
> > without having to open dummy channels to register grammars and pray
> > the MRCP server doesn't evict them too early.  An MRCP client has to
> > be able to guarantee that when a call comes through
> > the system, everything is ready and it won't experience any delay.
> > As it stands now the MRCP protocol does not appear to provide
support
> > for this.
> >
> >
> >
> > Thanks,
> >
> > Felix
> >
> >
> >
> > From: [email-masked] [mailto:[email-masked]]
> > On Behalf Of Dave Burke
> > Sent: Saturday, June 11, 2005 10:42
> > To: Shanmugham, Saravanan; Corby Anderson; [email-masked]
> > Subject: Re: [Speechsc] Managing big grammars
> >
> >
> >
> > I agree. Grammar compilation, caching, and sharing is a problem for
> > the MRCP server. Exposing this in the protocol puts unnecessary
> > complexity on the client. For example, the client needs to download
> > grammars, implement its own HTTP caching, checksum algorithms (some
> > way of not having to push huge data to the MRCP server to query)
etc.
> >
> >
> >
> > So I vote for point 1 below and include in the text that an MRCP
> > server could also, as an optimisation, implement a centralised
> > common cache shared across multiple MRCP servers.
> >
> >
> >
> > Dave
> >
> > ----- Original Message -----
> >
> > From: Shanmugham, Saravanan
> >
> > To: Corby Anderson ; [email-masked]
> >
> > Sent: Saturday, June 11, 2005 12:09 AM
> >
> > Subject: RE: [Speechsc] Managing big grammars
> >
> >
> >
> > I would say that a common cache amoung a farm of MRCP servers to
> > optimize compilations would be a nice server side optimization and
> > shouldn't affect the MRCP protocol.
> >
> >
> >
> > my 2c.
> >
> > Sarvi
> >
> > _______________________________________________
> > Speechsc mailing list
> > [email-masked]
> > https://www1.ietf.org/mailman/listinfo/speechsc
> >
> 
> _______________________________________________
> Speechsc mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/speechsc
> 
> _______________________________________________
> Speechsc mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/speechsc

_______________________________________________
Speechsc mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/speechsc

---

From: "David R. Oran" <[email-masked]>
Date: Tue, 14 Jun 2005 12:53:03 -0400

<chair hat>
<philosophy alert>

> specification which MRCP requires.
IETF protocol specifications are about simplicity, interoperability,  
robustness, and flexibility, and we generally weigh our  
specifications toward those goals over goals of vertically integrated  
solutions which operate within tight and predictable bounds. Nothing  
in the HTTP specification, for example, says anything about how  
dynamic web content is computed, or how servers implement content  
caching.

If you are a system-builder you need to ensure that the components  
you select (e.g. MRCPv2 servers) meet the performance requirements  
for your applications. The current MRCPv2 design, as elaborated on by  
Sarvi, allows a client to ascertain to a limited degree whether a  
server is going to perform adequately (through the cache control  
parameters), but not to force a server to use any particular means to  
do so. I think this is the right tradeoff. You're certainly entitled  
to disagree.

> This isn't possible when the MRCP servers MAY do something.
I agree in the case where the results are semantically different and  
it is impossible for the client to ascertain what the server does,  
but not in cases where the alternative behaviors are explicitly there  
for flexibility and to avoid over-specification.

> than a few concurrent sessions.
I think we have deployed experience with MRCPv1 (which says even less  
on the subject) that contradicts this observation.

</philosophy alert>
</Chair hat>