Subject: Shortcuts

From: "The Purple Streak, Hilarie Orman" <[email-masked]>
Date: Thu, 10 Apr 2003 17:20:54 -0600

I had a returned thought on the issue of whether or not data had to
complete the loop between the OPES processor and the callout server.
If the proxied protocol is a store-and-forward type, like SMTP, then
it seems that the callout server might, quite properly, send
transformed messages directly to an application endpoint (SMTP server)
without going back through the OPES processor.

If this is proper, should we document it.  If it is improper, can
we say why?

Hilarie

---

From: Markus Hofmann <[email-masked]>
Date: Thu, 10 Apr 2003 21:57:26 -0400

> without going back through the OPES processor.

If that is the case, why would you need a callout protocol in the 
first place? Why wouldn't "OPES processor" and "callout server" talk 
SMTP between each other? "OPES processor" and "callout server" would 
just be two mail servers talking SMTP...

-Markus

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 10 Apr 2003 22:02:31 -0600 (MDT)

> If this is proper, should we document it.  If it is improper, can

Excellent question!

If a callout server accepts application message and then just forwards
an adapted version elsewhere, it is not (should not be) a callout
server. It is an OPES processor! Thus, it is absolutely proper to do
that as long as the entity in question does not pretend to be a
callout server and obeys all processor requirements.

Here is what you get in that case:

 -- (SMTP) --> OPES         --> (SMTP) --> OPES        --> (SMTP)
               processor 1                 processor 2

Which is perfectly fine and is obviously beyond OCP scope (OCP is
not involved here).

If, for some unnatural reason, the same thing is implemented using
OCP, it may still be OPES-legal as long as both OPES agents know what
they are doing:

                                          callout server
 -- (SMTP) --> OPES         --> (OCP) --> _and_ OPES     --> (SMTP)
               processor 1                processor 2

Essentially, "adaptation service" here means "I took care of it,
forget it". This is OK as long as processors cooperate. Note that the
callout server would be required to respond with that "I took care of
it" application message to its OPES processor 1 (that application
message/response will probably have no payload though, just metadata).
No need to documented this convoluted case, IMO.

A somewhat similar but more realistic example with an "I took care of
it" response would be a "black hole" service (e.g., a SPAM filter):

 -- (SMTP) --> OPES         --> (OCP) --> callout
               processor 1                server

The latter might be worth documenting in an "SMTP adaptation using
OPES" draft.

HTH,

Alex.

---

From: "The Purple Streak, Hilarie Orman" <[email-masked]>
Date: Thu, 10 Apr 2003 23:51:38 -0600

As has been mentioned before, because friends of OPES would like to
deal with OPES and its callout protocol for all content
transformation tasks.  Just makes life easier.

Hilarie

---

From: Alex Rousskov <[email-masked]>
Date: Fri, 11 Apr 2003 07:02:23 -0600 (MDT)

> transformation tasks.  Just makes life easier.

You have to provide a better (more detailed) motivation than that. In
your specific example, sending the message to the next SMTP hop is
actually "easier" than handling OCP. Moreover, any OPES processor that
supports rule language would already have the functionality to send
application messages to their next hop as opposed to a callout server
-- it is a common case in real-world rule sets or ACLs, and we are not
adding anything "new" here.

Alex.

---

From: "The Purple Streak, Hilarie Orman" <[email-masked]>
Date: Fri, 11 Apr 2003 11:08:41 -0600

It's desirable to avoid the round-trip latency whenever possible.  It
seems eminently possible and sensible to do this for SMTP.  That lets
one have an architecture with a centralized content-driven dispatcher
(OPES processor) but without round-trip latency between the OPES
processor and the callout server.

Hilarie

---

From: Alex Rousskov <[email-masked]>
Date: Fri, 11 Apr 2003 11:25:48 -0600 (MDT)

> processor and the callout server.

There is no round-trip latency in a simple implementation of your
example: There is no callout server and OCP. There is no "bump on the
wire" to introduce that latency. There are just two sequential hops
(two OPES processors) on the wire, pipelining as fast as they can.

If you insist on using OCP and a callout server, then there is still
no extra round-trip latency as long as both OPES processor and the
callout server can handle "I took care of it, forget it" mode of
operation that I described earlier. No modifications to OCP core are
required to support that mode on per-application basis. That mode may
be useful to implement black holes, for example (but I am repeating
myself).

Do you think we should add explicit "I took care of it, forget it"
support to OCP Core? Even though some connection-oriented application
protocols like HTTP cannot "forget" a message on a protocol level?

Thanks,

Alex.

---

From: "The Purple Streak, Hilarie Orman" <[email-masked]>
Date: Fri, 11 Apr 2003 12:48:26 -0600

I don't think any changes are necessary for OCP.  Not even "I took
care of it".  The only question is whether or not we need to document
shortcuts somewhere - ideally it would have been in the architecture
document, I suppose.  There are at least two cases for shortcuts,
redirects and cut-throughs; are there more?  Can we get agreement on
wording about when a shortcut is OK and when not, where should we put
the wording?

I think shortcuts are generally very difficult for streaming
protocols, by the way.  I'd like to hear differently, if there are
streaming experts out there.

Hilarie

---

From: Alex Rousskov <[email-masked]>
Date: Fri, 11 Apr 2003 13:36:57 -0600 (MDT)

On Fri, 11 Apr 2003, The Purple Streak, Hilarie Orman wrote:

> care of it".  The only question is whether or not we need to
> document shortcuts somewhere - ideally it would have been in the
> architecture document, I suppose.  There are at least two cases for
> shortcuts, redirects and cut-throughs; are there more?  Can we get
> agreement on wording about when a shortcut is OK and when not, where
> should we put the wording?

What would be the purpose of documenting a shortcut if no protocol
changes are required to support it? The purpose would determine
whether documentation is needed and where it should be placed, IMO.

If you want to document shortcuts (on high level) to illustrate OPES
architecture power, the architecture draft or use cases draft is the
place. If you want to document (in detail) a nice trick that may come
in handy for some, a separate document would be more appropriate,
IMO. Other reasons?

OCP Core does not seem like the right place since OCP may not be
involved in some of these shortcuts and since they might be
application-specific.

Alex.

---

From: jfcm <[email-masked]>
Date: Sat, 12 Apr 2003 12:19:30 +0200

At 03:57 11/04/03, Markus Hofmann wrote:
>If that is the case, why would you need a callout protocol in the first 
>place? Why wouldn't "OPES processor" and "callout server" talk SMTP 
>between each other? "OPES processor" and "callout server" would just be 
>two mail servers talking SMTP...

Absolutely true. This is a daisy chaining model. This is not excluding any 
other model. This is not exlusing callout protocol additions in metadata.
jfc

---

From: jfcm <[email-masked]>
Date: Sat, 12 Apr 2003 12:27:20 +0200

At 20:48 11/04/03, The Purple Streak, Hilarie Orman wrote:
>There are at least two cases for shortcuts, redirects and cut-throughs; 
>are there more?

Stops.

Let imagine an anti-spam system based unpon origin verification (calls the 
origin to checj an ID key is consistent).

1. the key is positive: I can short-cut to the pop server.
2. the key is not answered because there is no checking service, I redirect 
to another filtering system
3. the key is negative I kill the mail.

jfc