Subject: AW: Start on OPES protocol work

From: "Martin Stecher" <[email-masked]>
Date: Sun, 16 Feb 2003 00:03:01 +0100

Hi everybody,

I like to follow Markus' suggestion to talk about fundamental questions
first and not to compare mentioned protocols in the very beginning or list
their pros and cons here.

So here are just some thoughts about six topics:

Markus' first question
   > what type of interactions need to be supported (e.g. pure
   > request/response, or request with multiple replies, or...)
together with a sentence from section 3.3 of the requirements draft
   > a request/response MAY consist of one or more messages

1.
I believe that a key feature of the protocol will be a "preview" feature
or better a "multiple preview/multiple decision point" feature. While
this is one request/response pair on an application message we can look
at it as several protocol messages and I think it is best to define this
feature that way: The Client sends first part of the application message
via one protocol message and receives one protocol message response
telling to send the rest of the data too. A second protocol message then
encapsulates the rest of the application message.

Defining it this way should guarantee a very strict and clear protocol
message syntax that avoids those complicated phrases like "after sending
this, the client has to wait until blah blah" and this should then have
a positive effect on protocol implementations that should avoid that
messages on an open persistent connection run out-of sync.

2.
The same question from Markus points me to another feature that got on
my wish list last year: Although I know that we want to concentrate on
HTTP, section 3.11 encourages us to define a protocol that could be used
for any application-layer protocol. And unlike most application
protocols that have one source and one destination, SMTP has one sender
but many recipients. An OPES protocol that is prepared to encapsulate
SMTP messages should be aware that a callout service may have different
results for different recipients.
Therefore I'd welcome if the OPES protocol is able (or can be easily
extended) to send multiple replies on one request.

Next question from Markus:   
   > what type of data is exchanged in requests and replies
Maybe not a direct response but this came to my mind here:

3.
I like to minimize the message overhead (see my performance
considerations in point 5). Together with what I said above in the first
section, I think that a message header and footer (if a footer is
necessary at all) should be as short as possible. Usually, protocols
write a protocol version into each request and response - but why? Isn't
it enough to do this once when a callout connection is established.? All
the messages within a persistent connection will be of the same version.
When we try to keep this in mind, every message could be designed to be
a minimum around the payload.

Messages should also follow a same general syntax to make parsing as
simple and fast as possible and to allow to skip easily message types
that are optional.

Third question:
   > what are the security requirements
I hope that somebody could elaborate on this one. I have to admit that I
do not see the full implications of every single sentence and
requirement of section 5 of the requirements draft in detail.
How strong must the encryption and authentication be? Do we need to
support multiple different encryption methods? Do we need the newest
technology here?
When looking at what HTTPS does for HTTP, would that kind of SSL
encryption be sufficient for the OPES protocol?

4.
Whatever the finding regarding these security questions will be, I very
much want to follow the "MAY" in section 5.2 of the requirements and
keep this optional. Knowing that a common deployment scenario will be a
private network between OPES processor and callout server, we should not
enforce unnecessary security if it costs more than only a very little of
resources.

About performance

5.
I believe that performance is absolut key! I experienced that customers
very much compare the hardware costs of different filtering solutions.
And we are talking with this protocol about deployments at central
gateways handling thousands of requests per second. If one protocol
implementation needs 20% more resources than another this could mean
that another three servers have to be deployed. Having the best
technology does not guarantee the success as we all know.

Not a topic so far: Ease-Of-Implementation

6.
Another key to success will be that the OPES procotol can be implemented
easily. If the protocol uses fancy new technology that the majority of
edge device developers and filter developers does not know yet and if it
is a bit too complicated, the protocol will not be accepted.
I think that it is much better to build on protocol parts from the
application-level protocols we want to encapsulate. Developers that will
likely deal with the OPES protocol should recognize parts of the
protocol and should feel familiar with some concepts. The OPES protocol
implementation will often be an add-on or a rewrite of existing
implementations; being able to re-use parts of the existing code will be
very helpfull.
That's why I propose to build on concepts like chunked transfer encoding
and to use SSL as encryption method if possible and so on. Therefore I
also do not tend to binary or highly structured approaches.

Facit (sorry I have to violate my introduction here and start to name
some protocols ;-)

I think about a very lightweight but still powerfull and flexible
protocol, building on components of HTTP/HTTPS.
Only knowing a little about SOAP and BEEP, I think that they are too
complex to define a direct base for the OPES protocol, although there
seem to be very interesting concepts which could be integrated.

I even think that the base message structure of the OPES protocol should
be easier than ICAP/1.0 (you may remember that I misliked the
Encapsulated header stuff from the very beginning) and now think that
even more could be much simpler.

In order to fulfill all requirements the protocol will get rich anyway,
so let's try to keep the base as simple as possible.

Regards
Martin

---

From: "The Purple Streak, Hilarie Orman" <[email-masked]>
Date: Sat, 15 Feb 2003 17:48:23 -0700

>  encapsulates the rest of the application message.

Yes, preview is important.

>  messages on an open persistent connection run out-of sync.

"out-of sync"?  I'm not sure what the problem is.  Could you elaborate,
because it's important that we all understand what the requirements mean.

>  extended) to send multiple replies on one request.

Hmmmm - I'm not sure that there's a clear need for multiple replies,
interesting as it is.  Is this example for OPES located between the
sender's mail agent and the SMTP server?  Or for OPES located between
an SMTP server and the users, working on a message sent to a local
alias that is a list?  Somehow the callout server (or the OPES box?)
knows the expansion of the alias?

It might be useful for the OPES processor to advise the callout server
to cache the data for multiple uses, though.  In order to be
application agnostic the two OPES entities must agree on an ID for the
content - perhaps borrowed from the application, but in an OPES
callout protocol field.

>  support multiple different encryption methods? Do we need the newest

One good method would be enough for encryption.  3DES or AES would be fine.
Something strong enough to keep bright high school students from writing
letters to heads of large companies and explaining to them that their
computer club broke the key for the days encryption using 25 computers :-)

>  seem to be very interesting concepts which could be integrated.

SOAP might be considered complicated, but BEEP seems to be an entirely
different and simpler kind of thing.  Possibly simpler than HTTP.

---

From: Robert Collins <[email-masked]>
Date: 16 Feb 2003 16:47:38 +1100

On Sun, 2003-02-16 at 10:03, Martin Stecher wrote:

> write a protocol version into each request and response - but why? 

IMO, because it reduces the state to be held by the endpoints of the
connection.

> Isn't

Agreed. IMO this has a greater impact on performance than a few bytes
for version identification.

The thing to remember is that any general purpose protocol will be
slower than a specially tuned protocol. We're building a general purpose
tool here - so reliability, ease of implementation and interoperability
are (IMO) greater priorities.

Rob