Subject: fixing "application message" mess

From: Alex Rousskov <[email-masked]>
Date: Mon, 7 Apr 2003 12:22:29 -0600 (MDT)

I may have found a simple and even elegant solution to the current
mess with the "application message" definition and related issues. The
clue was already in the draft but was disguised by poor presentation.
Your comments and questions helped to factor it out.

I will be making specific changes to the draft, but I want to share
the outline with the list, to give early comments/objections a
chance to be reflected in the next draft release. Here is the outline.

    0. The core of the problem is in overloading of
       the term "application". The solution is in
       specifying exactly which "application" we are
       talking about and specifying the relationships
       between different "application" terms.

    1. The are three applications in OPES/OCP environment
       (see diagram at the bottom of this message):

        - "input application" is specified by the
          application protocol that OPES processor
          is using to communicate with the previous
          application hop ("processor input")

        - "output application" is specified by the
          application protocol that OPES processor
          is using to communicate with the next
          application hop ("processor output")

        - "OCP application" is specified by the
          application protocol that is negotiated
          between OPES processor and the callout
          server (they may support several protocols
          and will have to pick one to use).

       Usually, input application is the same as output
       application. Often, input application is the same as
       OCP application.

    2. OCP is concerned with OCP application messages only. OCP
       agents MUST obey OCP application requirements when
       exchanging OCP application messages (embedded in OCP
       messages). OCP places no restrictions on processor input or
       output and has no requirements concerning the relationship
       between OCP application messages and processor input or
       output. Such restrictions and requirements may exist for
       OPES processors, but are outside of OCP scope.

    3. Meta-data may be used by OCP agents to relay information
       about input and output applications, including information
       about connections and message boundaries.

In other words, let OPES processor worry about its application input
and output. OPES processor negotiates with the callout server what OCP
application they should use and then handles necessary conversions, if
any. Naturally, the processor should not negotiate OCP applications
that it cannot convert input application to or output application from.

OCP application could be HTTP. Or it could be something like
"transfer-encoded HTTP message body with content-type and
transfer-encoding passed via meta-data". Etc...

Comments?

Thanks,

Alex.

                previous application hop
                ------------------------
                        |
                (processor input)
                        |
                        V
                   +---------+                           +-------+
                   |  OPES   | <== (OCP application) ==> |callout|
                   |processor|                           |server |
                   +---------+                           +-------+
                        |
                (processor output)
                        |
                        V
                --------------------
                next application hop

      legend:
         ===    communications using OCP
         ---    communications using other application protocols

---

From: Markus Hofmann <[email-masked]>
Date: Mon, 07 Apr 2003 16:28:11 -0400

> transfer-encoding passed via meta-data". Etc...

Provides quite some flexibility, but complicates the negotiation 
process... Who would specify/standardize possible OCP applications to 
be used? E.g., who would specify something like "transfer-encoded HTTP 
message body with content-type and transfer-encoding passed via 
meta-data". How do we ensure interoperability? By requesting some 
mandatory OCP applications (e.g. HTTP)?

-Markus

---

From: Alex Rousskov <[email-masked]>
Date: Mon, 7 Apr 2003 15:39:41 -0600 (MDT)

On Mon, 7 Apr 2003, Markus Hofmann wrote:

> process... Who would specify/standardize possible OCP applications
> to be used? E.g., who would specify something like "transfer-encoded
> HTTP message body with content-type and transfer-encoding passed via

The above negotiations (automated or via configuration files) are
unavoidable anyway! The only way to eliminate negotiations is to
support all pre-defined input/output/OCP application protocols in
every implementation, honor their message boundaries, _and_ transfer
complete application messages.

If we want to make support for an application protocol optional (e.g.,
some callout servers may not support SMTP but may support HTTP), then
OCP agents must negotiate because the agents are no longer 100%
compatible "by default".

If we want to support partial message exchange (e.g., OPES processor
extracts HTTP/SMTP message body and forwards that body along with some
MIME headers as meta-information), then OCP agents must negotiate
because the callout server needs to know whether it is getting a
complete message or just the body.

And so on.

The proposed solution does not make negotiation more complex. In fact,
it actually clarifies negotiation purpose. OCP negotiation must result
in an agreement regarding OCP application protocol, which includes the
usual things like name, version, message framing, meta-data (headers),
etc. If you look at it, it all boils down to how to encode/interpret
"meta-data" and how to interpret "data" (what is it I am getting?),
something we have to negotiate anyway if we want to remain application
agnostic. We can probably agree on the same meta-data encoding/format
for all applications. Then,

	OCP application protocol == required meta-data fields +
	                            "data" meaning

OPES WG may then publish an "HTTP as an OCP application protocol"
draft where the above things will be specified for HTTP. Other working
groups and individuals may publish their own application protocol
bindings.

OCP implementations will have to support at least one binding and able
to negotiate it. OPES processors will have to support mapping of their
input/output protocols to/from OCP application protocol they support
(usually it will be a no-op mapping with exception of tracing). Again,
this is no different from what we were going to have; the proposed
terminology/approach is meant as a clarification, not an extension!

Does the above sound convincing enough? Or do you still think I am
opening a pandora box (that was closed before)?

Thanks,

Alex.

---

From: Markus Hofmann <[email-masked]>
Date: Mon, 07 Apr 2003 18:36:17 -0400

Alex Rousskov wrote:

> unavoidable anyway! 

A valid question still is whether such negotiations should be built 
into OCP or not. I'm not taking either side (yet :), just saying that 
pros and cons should be looked at before we start adding more and more 
to OCP itself.

For example, one might also imagine that the OCP application to be 
used is (implicitely) indicated by the service to be called (and its 
description). Meaning, if an OPES processor decides to do a callout in 
order to execute Service A, one might assume that the description of 
Service A also tells the OPES processor what OCP application to use 
(likewise for the callout server). Note, that there might be multiple 
'services' with different OCP application bindings defined for one and 
the same callout application (e.g. a virus scanner accepting files via 
  HTTP, SMTP, and plain).

> The proposed solution does not make negotiation more complex.

Some kind of negotiation or agreement is required, sure. The question 
is whether this mechanism should be build into OCP or if it would be 
beneficial to assume this to happen "out-of-band". Just asking for the 
pros and cons.

> groups and individuals may publish their own application protocol

Sounds reasonable.

> opening a pandora box (that was closed before)?

Not yet sure about the "pandora box" :)

It's fine if we're convinced that this should be built into OCP, and 
if we keep the mechanisms in OCP as simple as possible and stay within 
our scope.

-Markus

---

From: Alex Rousskov <[email-masked]>
Date: Mon, 7 Apr 2003 17:46:50 -0600 (MDT)

On Mon, 7 Apr 2003, Markus Hofmann wrote:

> into OCP or not. I'm not taking either side (yet :), just saying
> that pros and cons should be looked at before we start adding more
> and more to OCP itself.

Absolutely. That is why I explicitly said "or via configuration
files". The scope of OCP negotiations is yet to be determined. I hope
we can make that determination after Reinaldo Penno suggests the first
negotiation mechanisms.

> description). Meaning, if an OPES processor decides to do a callout
> in order to execute Service A, one might assume that the description
> of Service A also tells the OPES processor what OCP application to
> use (likewise for the callout server).

Yes, a very reasonable setup.

> Note, that there might be multiple 'services' with different OCP
> application bindings defined for one and the same callout
> application (e.g. a virus scanner accepting files via HTTP, SMTP,
> and plain).

Yes.

> if we keep the mechanisms in OCP as simple as possible and stay
> within our scope.

We are not convinced yet :-). Reinaldo made some good points why
auto-negation is needed, and I think we need to wait for his
specification so that we can discuss specific trade-offs.

Alex.