Subject: OCP Core version head_sid4 available

From: Alex Rousskov <[email-masked]>
Date: Thu, 10 Apr 2003 10:52:30 -0600 (MDT)

To the best of my knowledge, all new specific OCP comments that caused
no further objections/questions are now applied to the draft. The
"application message" definition and related text has been changed
(improved?) again. I think we are getting close to a stable definition
though.

If you have any additions or corrections for the [new] misspelled
Acknowledgments section, please let me know. The authors list is not
frozen either, of course.

The [major] change log is quoted below. The latest snapshot, including
XML sources for those doing hands-on modifications, is available at

	http://www.measurement-factory.com/tmp/opes/

Please continue to comment and work on the to-do list.

Thank you,

Alex.

-------------- change log ----------------

Appendix B. Change Log
<217>
<218>
   o  Changed document labels to reflect future "WG draft" status.
<219>
   o  Added Acknowledgments section.
<220>
   o  Added "Core" to the title since we expect application specific
      drafts to follow and because this document, even when complete,
      cannot specify a "working" protocol without application-specific
      parts. This change is still debatable.
<221>
   o  Added reference to required future application-specific specs in
      the Introduction.
<222>
   o  Moved all rant about irrelevance of application protocols proxied
      by an OPES processor to the "Application proxies and OCP scope"
      section. Removed "processor input" and "processor output" terms.
      No reason to define a new term when its only purpose is to
      document irrelevance?
<223>
   o  Moved "OCP message" definition to the terminology section.
<224>
   o  Clarified "application message" definition based on recent WG
      discussions and suggestions. There seems to be consensus that
      "application message" is whatever OPES processor and callout
      server define or agree on, but OCP needs some minimal structure
      (content + metadata)
<225>
   o  Synced data and metadata definitions with the new "application
      message" definition.
<226>
   o  Simplified "Overall Operation" section since it no longer need to
      talk about irrelevance of application protocols proxied by an OPES
      processor.
<227>
   o  Illustrated nesting/relationship of key OCP concepts (application
      message, OCP message, transaction, connection, transport
      connection, etc.). The figure needs more work.
<228>
   o  Listed all from-processor and from-server OCP messages in one
      place, with references to message definitions.
<229>
   o  Added "services" message parameter, assuming that more than one
      service may be requested/executed with one transaction.
<230>
   o  Gave callout server ability to report what services were actually
      applied (see "services" parameter definition).

---

From: "The Purple Streak, Hilarie Orman" <[email-masked]>
Date: Thu, 10 Apr 2003 16:09:32 -0600

Maybe I misunderstood the purpose of app-messages, but I don't see how
to use the app-message-* directives to allow more than one "response"
from the callout server.  The OPES processor does app-message-start
for am-id, X, then some data, then app-message-end am-id=X, and the
callout server will respond with app-message-start am-id=X and
sometime later app-message-end am-id=X.  At that point all state about
am-id=X on both machines must be destroyed.  How can the callout
server send another version of the data it received from the OPES
processor and relate it to am-id=X?

Hilarie

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 10 Apr 2003 17:06:23 -0600 (MDT)

On Thu, 10 Apr 2003, The Purple Streak, Hilarie Orman wrote:

> Maybe I misunderstood the purpose of app-messages, but I don't see
> how to use the app-message-* directives to allow more than one
> "response" from the callout server.

This is achieved by responding with several all-message-start
messages, all with distinct am-ids.

> The OPES processor does app-message-start
> for am-id, X, then some data, then app-message-end am-id=X,

yes

> and the callout server will respond with app-message-start am-id=X

no, am-id=y1

> and sometime later app-message-end am-id=X.

no, app-message-end xid am-id=y1

In addition to that, it can respond with more (at _any_ time after
receiving app-message-start am-id=x):

	app-message-start xid am-id=y2
	app-message-start xid am-id=y3
	...
	app-message-start xid am-id=y4
	...
	app-message-end xid am-id=y3
	app-message-end xid am-id=y2
	...
	app-message-end xid am-id=y4

or whatever (interleaved in any way). It is important that there is no
[documented] relation between original and adapted flows. Usually, the
original application message is adapted according to requested
services, but other things might happen as well (errors, service ID
expansion, or whatever).

> How can the callout server send another version of the data it
> received from the OPES processor and relate it to am-id=X?

It does not need to relate to am-id=x because that is what transaction
ID (xid) is for. By definition, OCP transactions encapsulate
processing of the original application message (am-id=x). Technically,
we do not even need am-id parameter for original data flow.

Does this clarify?

Also, look at the new "adapted flow" figure in the draft (and related
figures). It also shows how multiple adapted application messages
within one transaction are possible. Can the figure be improved to
make the above more apparent?

Thanks,

Alex.

---

From: "The Purple Streak, Hilarie Orman" <[email-masked]>
Date: Thu, 10 Apr 2003 17:17:46 -0600

Thanks for am-id clarification.  In general, the OPES processor won't
know how many am's might be sent by the callout server or what kinds
they might be.  This makes it difficult to know what to start carrying
on the proxied flow.  Should the OPES processor wait until
transaction-close?  That seems inefficient.  How do people envision
using this multiple return message feature?

Hilarie

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 10 Apr 2003 17:37:30 -0600 (MDT)

On Thu, 10 Apr 2003, The Purple Streak, Hilarie Orman wrote:

> Thanks for am-id clarification.  In general, the OPES processor
> won't know how many am's might be sent by the callout server or what
> kinds they might be.

If this knowledge is needed, it can be relayed via metadata or
explicitly added to the protocol as an "expect more" flag/parameter.
My expectation is that the possibility of multiple adapted application
messages is application-dependent. Those OPES processors that support
applications like SMTP would wait for the callout server to close the
transaction.

> This makes it difficult to know what to start carrying
> on the proxied flow.

Not sure what you mean. The number of adapted application messages
does not really affect proxied flow. Usually, OPES processor would
start post-processing and sending out every application message it
receives immediately after it gets app-message-start message (whether
it is the first such message or 10th). Multiple adapted application
messages are only allowed for protocols like SMTP where that is not a
problem, of course. Did I misunderstood your statement?

> Should the OPES processor wait until transaction-close?
> That seems inefficient.

Why should it? It got an adapted application message it can proceed
with. OCP supports very efficient pipeline where there is no waiting
as long as the application protocol supports such behavior.

See http://www.ietf.org/internet-drafts/draft-ietf-fax-esmtp-conneg-06.txt
for example. There were other SMTP-related examples posted on the
list.

Alex.

---

From: "The Purple Streak, Hilarie Orman" <[email-masked]>
Date: Thu, 10 Apr 2003 19:00:41 -0600

I'm thinking of image conversion as an example, but anything with the
HTTP "vary" header more generally.  If the requested OPES service is
something general, like "decrease image size", one could imagine the
service returning several responses, like "black and white", "300x400
depth 3", etc.  Or the service "translate to local language" might
return "french", "german", and "italian" in some places.  The OPES
processor will look at the headers on the returned am's, looking for
something that matches the preferences of the user who caused the
content to be converted.  Although more than one might be acceptable,
one might be best.  Without knowing whether or not it's going to be
part of the transaction, the OPES processor would have to wait until
transaction close to send the result.

Hilarie

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 10 Apr 2003 22:13:05 -0600 (MDT)

> transaction close to send the result.

That is fine/unavoidable, I think. There are examples where pipeline
model does not (cannot) work all the way through. Any example where
the OPES processor does post-processing, and that post-processing
needs to know all adapted response(s) before starting, will break
pipeline at the OPES processor. Note that

	(a) it does not break the protocol, just prevents
	    certain optimizations
	(b) I cannot think of any optimization that is
	    not supported by OCP but will work better that OCP
	    here
	(c) OPES processor may relay user preferences to
	    the callout server so that the serve can produce
	    a single best response (if the server is smart
	    enough to do that). This can be done via metadata,
	    of course.

Alex.

---

From: "The Purple Streak, Hilarie Orman" <[email-masked]>
Date: Thu, 10 Apr 2003 23:56:27 -0600

It's not that the OPES processor wants only one response, it's that it
needs to know, in advance, which reponses are coming.  In fact, it may
need to cause early termination of some responses in order to manage
cache space.

Hilarie

---

From: Alex Rousskov <[email-masked]>
Date: Fri, 11 Apr 2003 06:56:07 -0600 (MDT)

> need to cause early termination of some responses in order to manage

Same answer, I believe. You are constructing an example where it is
impossible to avoid overheads if you assume no example-specific
negotiation of some kind.

Note, however, that it is possible to trade what you call "cache
space" for reponse time and local bandwidth: assuming reproduceable
responses, the OPES processor can send the same application message
for adaptation twice; during the first attempt, it will terminate
every adapted response after noting its metadata;  during the second
attempt, it will terminate all responses but the one(s) it preselected
after the first attempt.

Alex.