Subject: RE: OPES protocol, pre-draft

From: "Martin Stecher" <[email-masked]>
Date: Fri, 21 Feb 2003 18:13:13 +0100

> > ICAP/1.0 is already pipelined, I think. Messages are sent in chunked
> > transfer encoding. ICAP clients send chunk N before waiting for the
> > modified chunk N-1 to return.
> 
> You are right, of course. I should have mentioned that ICAP does use
> pipelining. IIRC, ICAP pipes start only after message headers and do
> not allow any non-data messages to be present in-between (which forces
> ICAP to use a chunk-extension hack to indicate end-of-data).
> 
> Another ICAP pipes disadvantage is that they cannot eliminate
> callout-to-processor data flow when the callout processor did not
> modify the data. The proposed protocol allows for this important
> common-case optimization using "data-asis" messages.

ICAP's 204 response is something like this, isn't it?
Similar to your proposal, the ICAP client needs to advertise that it
keeps a copy of the data (by adding the Allow: 204 ICAP header) and
the ICAP server can then respond with "ICAP/1.0 204 Not modified"
not only after the preview but after it has seen more or all of the
HTTP data.

I see that the OPES pipe proposal goes further, it is really a next
generation protocol.

> 
> Finally, ICAP pipes place ICAP connections in direct dependence on
> HTTP connections. If I have to process 5 HTTP messages _concurrently_,
> I have to open 5 ICAP connections. With the proposed approach, I can
> use 1, 2, 3, 4, or 5 OPES connections. This allows for possibly
> important performance optimizations on high-traffic servers.

Fully agree.

> 
> These limitations of ICAP model apparently come from it being an
> extension to HTTP (rather than being implemented on top of HTTP).
> OPES pipes are not an optimization add-on but a design feature.
> 
> It remains to be seen if the proposed protocol is so much better that
> it actually makes sense to phase out a working/deployed ICAP protocol.

Yes, let's first get further with a proposal for a best-we-can-think-of-
OPES-protocol.
You wrote in a message on Wednesday:
> I expect ICAP proponents defend ICAP on this list.

There are probably not too many stronger supporters of ICAP than I am.
And I will proudly defend ICAP. But this does not mean that we cannot
work on something new and better.

It took nearly two years from a stable protocol draft to an almost RFC
status and about 18 month before we could convince a significant number
of vendors to support ICAP in their products.

The differences between ICAP/0.9, ICAP/0.95 and ICAP/1.0 are very noticable.

So, I do not see any argument why we cannot create a next generation protocol
that becomes at least as successful as ICAP/1.0. Yes, it will take some time
and it has to be much better. But this is the fun part, isn't it?

I like to talk about ICAP TNG ("the next generation" for all non-Trekkies) as
a working title because I think that the protocol's success can be maximized
if it really looks like a successor of ICAP/1.0 and is maybe even named
ICAP/2.0. As far as I see the current proposal could still work out in this
direction (and also in something very different, I know).

This is of course not in the focus of this WG and I understand that some of
you do not like this direction. It is just my personal motivation (sorry for
bothering you with this ;-)

Martin

---

From: Alex Rousskov <[email-masked]>
Date: Fri, 21 Feb 2003 10:42:27 -0700 (MST)

On Fri, 21 Feb 2003, Martin Stecher wrote:

> ICAP's 204 response is something like this, isn't it? Similar to
> your proposal, the ICAP client needs to advertise that it keeps a
> copy of the data (by adding the Allow: 204 ICAP header) and the ICAP
> server can then respond with "ICAP/1.0 204 Not modified" not only
> after the preview but after it has seen more or all of the HTTP
> data.

Yes, 204 Not modified works similarly but can only be applied to
unmodified message tails (including entire messages) whether OPES
"asis" mechanism aplies to individual data chunks in the pipeline. 204
Not modified is a one-time all-or-nothing decision. Same idea (asis
feature was inspired by ICAP of course), different scope. Asis feature
can be used to reduce data transfer from callout server to processor
to the necessary minimum even if callout server modifies the last
chunk of the message.

> So, I do not see any argument why we cannot create a next generation
> protocol that becomes at least as successful as ICAP/1.0. Yes, it
> will take some time and it has to be much better. But this is the
> fun part, isn't it?

Exactly.

> I like to talk about ICAP TNG ("the next generation" for all
> non-Trekkies) as a working title because I think that the protocol's
> success can be maximized if it really looks like a successor of
> ICAP/1.0 and is maybe even named ICAP/2.0. As far as I see the
> current proposal could still work out in this direction (and also in
> something very different, I know).

I agree that calling OPES protocol ICAP may help OPES
acceptance/deployment for non-technical reasons. To save time and
flames, I suggest to avoid the naming issue until OPES protocol is
shaped and the consensus is, in fact, that it is better than ICAP/1.0.

Thank you,

Alex.