Subject: RE: OPES protocol, additional message needed?

From: "Martin Stecher" <[email-masked]>
Date: Fri, 21 Feb 2003 18:31:47 +0100

Hi Alex,

>[...]
> 
> Am I downplaying the importance of change-probability and progress
> fields? Do you expect implementations hack them in as OPES extensions
> if we do not define them? Are there corresponding ICAP extensions
> already?
>[...]

There is no message like this in ICAP today and I am not aware of
an extension for change-probability.
The result is that ICAP servers start to implement the donwload
progress indication / data trickling features and as you pointed out
this belongs in the OPES processor.

But the OPES processor needs to have some input to determine whether
and what latency fighting method to use.
The overall question is: If I (the OPES processor) start to forward
original application message data (by data trickling or LateClearance
content encoding), how likely is it that I will get into deep trouble
when the callout server sends modified data?
It can only know if the callout server has sent some information
about this likelihood.

And regarding progress: Let's assume there is a way to forward progress
indication to the consumer, then the OPES processor can better work with
a numeric value (e.g. "still 10 seconds to go" or "30% done") rather
then only "the callout server is still working on it".

I know that there was some work to get this feature into an ICAP
extension but it has not yet been implemented.

Regards
Martin

---

From: Alex Rousskov <[email-masked]>
Date: Fri, 21 Feb 2003 11:29:33 -0700 (MST)

Martin,

	I agree 50%: Content modification is a primary OPES function.
Probability of a modification can be expressed in simple terms.
Moreover it may affect how the application message is handled (e.g.,
encoding or dropping if things go wrong). We probably should support
this in the core protocol.

	Note that LateClearance has to be decided before any headers
are sent; it must be possible to attach the hint to the consumer-start
OPES message. I will try to make necessary modifications.

	As for progress, my concern is that it is impossible to have a
simple machine-friendly format that describes "progress". Simple
format cannot express both "still 10 seconds to go" and "30% done",
and there are many other variations that some applications may want.
Moreover, "still 10 seconds to go" and "30% done" can be very wrong if
they are sent to the client without adjustments (OPES may not know of
all other hops on the way).

	Neither HTTP nor SMTP intermediaries have any sane way of
reporting this kind of progress back to the client. I am afraid of
adding pages of complicated features to the protocol that are not
going to be used much. To me, it smells like a good candidate for a
protocol extension (we still need to built in mechanism for those) not
a core feature. An extension may still be very well documented and
widely supported, of course. Do you think that progress indication
must be a core OPES feature?

Thank you,

Alex.

On Fri, 21 Feb 2003, Martin Stecher wrote:

> There is no message like this in ICAP today and I am not aware of an
> extension for change-probability. The result is that ICAP servers
> start to implement the donwload progress indication / data trickling
> features and as you pointed out this belongs in the OPES processor.
>
> and what latency fighting method to use. The overall question is: If
> I (the OPES processor) start to forward original application message
> data (by data trickling or LateClearance content encoding), how
> likely is it that I will get into deep trouble when the callout
> server sends modified data? It can only know if the callout server
> has sent some information about this likelihood.
>
> And regarding progress: Let's assume there is a way to forward
> progress indication to the consumer, then the OPES processor can
> better work with a numeric value (e.g. "still 10 seconds to go" or
> "30% done") rather then only "the callout server is still working on
> it".
>