Subject: Re: OPES protocol, pre-draft

From: Markus Hofmann <[email-masked]>
Date: Sun, 16 Mar 2003 23:09:14 -0500

Please apologize my late reply, but this just came to mind when 
catching-up with all the emails after coming back from vacation and 
reading through them for a second time on the plane...

Martin Stecher wrote:
> Example 2:
> Callout server analyzes beginning of application message and detects
> that it is not interested in this app. message:
> 
> A. Traditional approach, ICAP/1.0 like with multiple preview
>     0. C: Start of app. message 1
>     1. C: Here is preview chunk 1
>     2.                                S: Need another preview chunk
>     3. C: Here is preview chunk 2
>     4.                                S: Not interested (204 resp.)
>     5. C: Start of app. message 2
> 
> B. Proposed OPES protocol
>     0. C: Start of app. message 1
>     1. C: Chunk 1 of data             S: Start of app. message 1
>     2. C: Chunk 2 of data            
>     3. C: Chunk 3 of data             S: Not interested
>     4. C: Chunk 4 of data             
>     5. C: Start of app. message 2
> 
> In this case B sent more data and data chunks 3 and 4 will be ignored
> by the callout server. 

This might be obvious, but please allow me to add the following for 
clarification only... Chunks 3 and 4 can only be ignored (i.e. 
discarded) by the callout server if the OPES processor has sent them 
with the [copied] flag set in the respective <data-have amid offset 
size [copied]> message(s). If the [copied] flag was not set, the 
callout server must not ignore chunk 3 and 4, but simply copy/send 
them back to the OPES processor. Obviously, this can be done without 
further processing (i.e. servicing), but they still need to be sent 
back, since the OPES processor might no longer have a copy of these 
chunks. Note, that this is true even if chunks 1 and 2 were sent with 
the [copied] flag set.

Thanks,
   Markus

---

From: Alex Rousskov <[email-masked]>
Date: Mon, 17 Mar 2003 05:49:23 -0700 (MST)

Markus,

	We probably need a "sticky" [copied] flag meaning "I am going
to have a copy of every byte from now on, until further notice". This
way the callout server can remove itself from the loop earlier.

Thanks,

Alex.

---

From: "Oskar Batuner" <[email-masked]>
Date: Mon, 17 Mar 2003 11:08:17 -0500

Let me propose a simplification of the protocol: remove 
[copied] flag and make this behavior compulsory.

Reasons:

1. The main goal of chunks is to reduce network traffic. OPES server
implementation that discards message chunk by chunk as soon as one 
is send to callout server does not look a good idea. Even if call 
starts before the complete message is received by the OPES processor 
it should be able to assemble the complete message anyway. In 
asynchronous system OPES processor can not rely on ability of 
the callout processor to keep pace with the speed of incoming traffic.

2.Reliability. If the original message is discarded by OPES processor 
before transaction with the callout processor is completed callout 
processor failure (including connection outage) forces OPES processor 
to fail on this message, while preserved copy provides multiple 
ways for recovery - default action on callout server unavailable, 
repeat request to a different callout server. And the message 
was already there, savings from premature discarding are very 
small.

3. catch 22: if message processing is simple and anticipated 
processing time is small - so are the savings from message discarding.
If anticipated processing time is significant - savings may be 
bigger but so are the risks related to failure and needs for 
reliable recovery. 

Same thing with the message size - for small messages savings are 
negligible, for large messages savings from ability to continue 
without returning message body (e.g. in virus checking) out weight 
savings from message discarding.

4. For streaming data policy may be different but still predefined: 
for live streaming data should me discarded (or alternative action 
taken) if delay is too long, for preload situation is the same as 
with large message.

All of this shows that benefits of controlled discarding are too small 
to justify complications in the protocol and system implementation.

My .10

Oskar

> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]]On Behalf Of Alex Rousskov
> Sent: Monday, March 17, 2003 7:49 AM
> To: [email-masked]
> Subject: Re: OPES protocol, pre-draft
> 
> 
> 
> Markus,
>

---

From: Markus Hofmann <[email-masked]>
Date: Mon, 17 Mar 2003 12:47:34 -0500

Alex,

> way the callout server can remove itself from the loop earlier.

The same can be achieved by simply setting the [copied] flag in any 
outgoing <data-have> message from now on, couldn't it?

This way we wouldn't need yet another flag, and also we wouldn't need 
a way to overwrite a "sticky" flag that was previously set (see your 
"until further notice" remark above").

Thanks,
   Markus

---

From: Markus Hofmann <[email-masked]>
Date: Mon, 17 Mar 2003 13:02:31 -0500

Oskar,

> 1. The main goal of chunks is to reduce network traffic. OPES server

You probably mean "OPES processor". Let's try to stick to the 
terminology we've in the existing documents, might help avoiding 
misunderstandings.

> it should be able to assemble the complete message anyway.

This might require the OPES processor to possibly buffer huge amounts 
of data, which might impact scalability of the approach.

Imagine a large number of users simultaneously downloading large files 
via HTTP, and the OPES processor does a callout for virus scanning for 
each of these files (in parallel). Since virus scanning involves long 
delays in processing, quite some data would accumulate at the OPES 
processor for temporary buffering...

> was already there, savings from premature discarding are very 

Absolutely right, and that's why our architecture and protocol should 
allow an OPES processor to buffer the data. Then it's up to the OPES 
processor to decide whether to do that or not.

> bigger but so are the risks related to failure and needs for 

Yup, see my comments above.

Also, for some transactions it might be OK to fail when the callout 
server fails. Example: If a user requests mandatory virus scanning for 
file download, it might be perfectly fine to reply with an error 
messge when the callout server running the virus scanning fails, 
rather then sending back a non-scanned version from the buffer of the 
OPES processor.

Thanks,
   Markus

---

From: jfcm <[email-masked]>
Date: Tue, 18 Mar 2003 11:16:50 +0100

Dear Markus and all,
please let not confuse everything in sticking to two terminolgies 
corresponding to twe different and quite opposed thinkings. This will help 
identfying these thinkings and to benefit from them both. Or we will wind 
up with something no one will be happy with.

At 19:02 17/03/03, Markus Hofmann wrote:
>You probably mean "OPES processor". Let's try to stick to the terminology 
>we've in the existing documents, might help avoiding misunderstandings.

Notwidthstanding the document I do not understand yet what is an OPES and 
what is an OPES processor - and it seems that I am not alone. But I fully 
understand what is a dispatcher and what is a server. I may be wrong, but 
for me a processor is a CPU, and a server an identified virtual system 
running on one or serveral CPU. I may be wrong, but I think in the case the 
server uses several cpus the callout protocol could/should be use to permit 
such cpu to interoperate?

Also I have difficulties in understanding if an OPES is the end-service as 
provided or the service as  provuided to the end-user. Meaning, if the 
dispatcher is part of the OPES or not. Meaning, if the dispatcher user side 
interfaces are parts of the OPES.

>>implementation that discards message chunk by chunk as soon as one is 
>>send to callout server does not look a good idea. Even if call starts 
>>before the complete message is received by the OPES processor it should 
>>be able to assemble the complete message anyway.
>
>This might require the OPES processor to possibly buffer huge amounts of 
>data, which might impact scalability of the approach.

For examplen that remark makes sense if the "OPES processor" is a single 
machine (otherwise "buffer" is to be defined). But an OPES service can be 
permformed on several machines (this is to be considered (?) or operated(?) 
by the callout protocol), either successively (the service processing calls 
on several machines) or in parallel (several mechines offer the same 
service and must coordinate).

>Imagine a large number of users simultaneously downloading large files via 
>HTTP, and the OPES processor

so, here you then mean dispatcher.

>does a callout for virus scanning for each of these files (in parallel). 
>Since virus scanning involves long delays in processing, quite some data 
>would accumulate at the OPES processor for temporary buffering...

This is true for any inputs (HTTP or not). This means that the callout 
protocol should not be dependent on the fact that the input is HTTP, but to 
support different levels of service that an HTTP flow may chose from, the 
same for an SMTP, or an FTP flow.

What I mean is that we have to be clear about the different layers, and not 
to patch through layer violations. Inputs come through a dispatcher. 
Dispatchers relate with other dispatchers (and servers? I personallythink 
that dispatchers should be server front-ends, much simpler) using the 
callout protocol.

Nothing in the callout protocol is to be directly related to what happens 
before the dispatcher. Alex,  shown well that towards the user (the user is 
to relate with the OPES through the application). This is also true the 
other way: the server is to relate with traffic flow through the dispatcher.

jfc

---

From: "Oskar Batuner" <[email-masked]>
Date: Tue, 18 Mar 2003 10:04:28 -0500

Markus,

I think we may have a different implementations in mind. I am 
looking at web proxy server (surrogate, I just do not like 
this word) extended by OPES capabilities. For this model 
storing all incoming data is not a problem - disks are 
large and cheap, and storing data is very natural behavior 
for the system build around cache engine.

Correct me if I'm wrong, but it looks like you have 
in mind something like layer 7 switch. Such devise may have 
better throughput but very limited storage capabilities. 
Main differentiator is ability to keep data on disk. Hybrid 
devices are also possible, e.g. solid state cache. More interesting 
hybrid is L7 based OPES processor combined with cache farm.

I suppose that buffering policy will depend mostly on the device 
type. OPES processor with disk will store all intermediate data 
and use it's caching capabilities to enhance overall performance. 
L7 switch based device will tend to be very conservative on 
storage use and may need to exploit protocol capabilities for 
copy control.

To support all these needs we may do several things:

1. Dynamic (per-message) control, like in current proposal.
2. Stateful protocol with storage policy negotiated at handshake.
3. Different level of protocol implementation with device capabilities 
announced (but not negotiated) at handshake. 

I thing protocol should support all 3 policies. This may significantly 
simplify implementation of cache-based OPES processors.

BTW, hybrid system with L7 switch based OPES processor and cache farm 
points to an interesting twist: why not to consider the attached web 
cache as a special case of callout server? This may be just another 
OPES scenario with OPES rules used to create and dynamically 
control caching policy. To support this scenario callout protocol 
should accommodate queries - transactions where message body is present 
only in response, and caching requests - message body only in request, 
and rules support persistence (if request was not in cache send 
storage request on response) and splitting (send content server 
response to end user without waiting for storage request to finish).

> [mailto:[email-masked]]On Behalf Of Markus Hofmann
> Sent: Monday, March 17, 2003 1:03 PM

Yes, but:

- virus scanning was used as an example of scenario where storage on 
OPES processor is most beneficial - virus scanner may give results 
after processing small part of the message eliminating most of the traffic;

- with the spiky nature of web tarffic you can not rely on scanner ability 
to keep pace with the traffic. 

> rather then sending back a non-scanned version from the buffer of the 

Yes, but stored message provides OPES processor with ability to recover 
after callout server failure (by sending the same request to another 
server) transparantly to other OPES flow participans.

Oskar

---

From: Alex Rousskov <[email-masked]>
Date: Tue, 18 Mar 2003 10:04:22 -0700 (MST)

On Mon, 17 Mar 2003, Oskar Batuner wrote:

> Let me propose a simplification of the protocol: remove [copied]
> flag and make this behavior compulsory.

After reading the follow-ups to this proposal, I would like to start
by emphasizing that Oskar is essentially proposing changing the
default behavior from "move" to "copy". I think it is clear by now
(correct me if you have doubts!), that some systems have to use "move"
and some systems have to use "copy", often for OPES-unrelated
application/location-specific reasons.

Thus, if we yank [copied] out, we have to add [moved] and vice versa.
This is not a simplification of a protocol, just a change of the
default settings.

I think using "move" default and [copied] flag is better because
"move" is a simpler, basic operation; OPES "copy" operation is,
essentially, "store" + "move". That is why, IMO, "move" should stay as
the default and [copied] should be used to overwrite that default.

A possible alternative is to have the default dependent on the
application protocol binding. I think many OCP defaults are best
determined by the application protocol, but I am worried that having
many application defaults would make the protocol difficult to analyze
without referring to a specific application binding. Opinions?

Thank you,

Alex.

P.S. I assume that an overhead of passing a [copied] or [moved] flag
     to the other side is negligible, regardless of what OCP encoding
     we select. Thus, the question of the default is of minor
     importance. We can even have "mode=copied|move" option and have
     it mandatory to avoid deciding on the defaults. Again, I vote for
     "move" mode as the default because "move" is a basic operation
     and "copy" is not.

---

From: Alex Rousskov <[email-masked]>
Date: Tue, 18 Mar 2003 10:30:37 -0700 (MST)

On Mon, 17 Mar 2003, Alex Rousskov wrote:

> to have a copy of every byte from now on, until further notice".
> This way the callout server can remove itself from the loop earlier.

The "until further notice" part is a stupid idea, of course. Since the
callout server cannot see messages still in the [TCP] pipe, it would
never not be possible for the server to remove itself from the loop --
the server cannot guarantee that one of the still-unprocessed
data-have messages was not sent with the "notice: now I am not
copying!"  flag.

Once declared, the copying commitment has to be persistent for the
life of the data stream. What we need to support loop cuts is a
[copying-all] flag/state.  Once that application message flag is sent,
all further data messages MUST have a [copied] flag.

Sorry for not thinking this through well enough on the first try.
Hopefully there are no bugs in the revised version above.

Alex.

---

From: Markus Hofmann <[email-masked]>
Date: Tue, 18 Mar 2003 12:32:08 -0500

Oskar,

> for the system build around cache engine.

Even in this case you cannot assume to always being able to buffer the 
entire object - even the largest caches run out of disk space at some 
point and need to do some sort of cache replacement and conserve disk 
space. Even more, in case of streaming caches you might end up doing 
prefix caching, i.e. not having stored the entire object, but only the 
prefix of an object.

The point is that we must not require a specific implementation, but 
that our protocol should support various forms of implementations. I 
agree that buffering capacity is of less importance in the model 
you've described above, but the protocol should also support other 
forms, and as such I still see value in allowing the OPES processor to 
not buffer the entire object (meaning there seems to be value to 
having the [copied] flag.

> hybrid is L7 based OPES processor combined with cache farm.

Yup, that's one possible implementation form our protocol should support.

> storage use and may need to exploit protocol capabilities for 

Yup, and that's why I believe the callout protocol should *not* 
require the OPES processor to always store the entire object.

> simplify implementation of cache-based OPES processors.

I agree that the protocol should support different forms of 
implementations, but I'm not yet convinced that policies need to be 
negotiated "at handshake". With the current proposal, the OPES 
processor can dynamically decide what to buffer and what not, 
indicating this to the callout server via the [copied] flag. This 
seems pretty flexible. Is there a specific scenario that cannot be 
solved with that approach?

-Markus

---

From: Markus Hofmann <[email-masked]>
Date: Tue, 18 Mar 2003 13:10:26 -0500

> without referring to a specific application binding. Opinions?

Please let's avoid having too many different application defaults and 
options - options always complicate the protocol and if there's no 
really strong, strong reason for them, let's try to avoid them. In 
this specific case, I'd assume it's perfectly fine to defne a general 
default.

Thanks,
   MArkus

---

From: Markus Hofmann <[email-masked]>
Date: Tue, 18 Mar 2003 13:26:15 -0500

jfcm wrote:

> Notwidthstanding the document I do not understand yet what is an OPES 
> and what is an OPES processor - and it seems that I am not alone. But I
> fully understand what is a dispatcher and what is a server.

The "data dispatcher" is part of the OPES processor (see Figure 1 and 
corresponding text in the architecture draft). You might think of the 
OPES processor as a physical box/network element, and about the data 
dispatcher being a specific software component on this box. Note that 
"OPES processor" does not refer to a "processor" in the sense of a 
processor chip, but rather to the entire "box".

A little bit simplified, you might think of having two different types 
of boxes in the OPES architecture - the "OPES processor" and the 
"callout server". The OPES processor includes a data dispatcher 
(that's simply a functional component), which decides what services 
have to be executed on what messages.

Hope that helps.

-Markus

---

From: "Oskar Batuner" <[email-masked]>
Date: Tue, 18 Mar 2003 14:52:28 -0500

Martin, looking at your and other people reaction I agree
that we should keep protocol flexible.

The main change that can give a serious advantage is a well defined
implementation level with fixed "copy always" policy. OPES processor based on
web cache is a very real possibility, and at handshake it may be just a
notification.

I agree that from the protocol point view the same effect may be achieved by
utilizing these generic framework - just always ask/notify about keeping the
copy. But the possibility to implement a device that does not support that
flexibility may very helpful, and such possibility requires notification at
handshake: one should know partner capabilities in advance. This approach may
also benefit L7 type devices - callout server will know about OPES processor
limitations and adopt accordingly.

Oskar

> Sent: Tuesday, March 18, 2003 12:32 PM

---

From: Alex Rousskov <[email-masked]>
Date: Tue, 18 Mar 2003 13:58:23 -0700 (MST)

On Tue, 18 Mar 2003, Oskar Batuner wrote:

> I agree that from the protocol point view the same effect may be
> achieved by utilizing these generic framework - just always
> ask/notify about keeping the copy. But the possibility to implement
> a device that does not support that flexibility may very helpful,
> and such possibility requires notification at handshake: one should
> know partner capabilities in advance.

Yes, may be very helpful. No, does not _require_. The current [copied]
scheme supports devices that never copy, that always copy, and that
copy depending on some conditions.

> This approach may also benefit L7 type devices - callout server will
> know about OPES processor limitations and adopt accordingly.

I cannot think of any good example where "I will always copy"
declaration can be made without a reference to a specific application
message or transaction. It seems almost impossible to be that certain
in a general environment. Do you think it is worth supporting? Note
that we will support that kind of declaration in an application
message context.

The opposite declaration ("I will never copy") is, of course,
applicable to many devices. Do you think it is worth supporting?

Finally, do you think we need _negotiation_ capabilities here? "I may
copy if you want to, do you?", "yes please copy" or "no, do not copy".

I would like to see a complete set of copying-related features that we
want to support in order to propose a specific protocol solution.

Thanks,

Alex.

---

From: jfcm <[email-masked]>
Date: Wed, 19 Mar 2003 00:11:07 +0100

On 19:26 18/03/03, Markus Hofmann said:
>>understand what is a dispatcher and what is a server.

Dear Markus,
I thank you for your response. It is my reading of the document. If the 
words probably render the thinking at a given time, I think they are 
inapropriate now. What they mean is that the OPES are not   located on the 
"OPES processor" but one of the "callout server". Also the OPES processor 
only supports the dispatcher and the i/o.

>corresponding text in the architecture draft). You might think of the OPES 
>processor as a physical box/network element, and about the data dispatcher 
>being a specific software component on this box. Note that "OPES 
>processor" does not refer to a "processor" in the sense of a processor 
>chip, but rather to the entire "box".

I do not see the dispatcher as a CPU, nor as a machine, but as a diagram 
corresponding to a set of filtering rules. With I/O the in being the input 
data, the O being the output to the user or to the servers or to other 
dispatchers. This can be implemented in different ways.

>A little bit simplified, you might think of having two different types of 
>boxes in the OPES architecture - the "OPES processor" and the "callout 
>server". The OPES processor includes a data dispatcher (that's simply a 
>functional component), which decides what services have to be executed on 
>what messages.

Taking advantage of this, you do understand.
- has the callout server a dispatcher as a front end (the callout protocol 
performing between disptchers) or not (the callout protocol performing 
beteween a dispatcher and a remote server). In the first case I understand 
how the sytem can be smart, in the later case I don't.
- what does the dispatcher do with the part of the traffic it does not send 
to the server and it received?

You see I do not feel at ease with the focus on the protocol before having 
a clear, terse and comon understanding of what it relates, under which 
constraints and with which mission.
jfc

---

From: Markus Hofmann <[email-masked]>
Date: Tue, 18 Mar 2003 20:33:03 -0500

jfcm wrote:

> - has the callout server a dispatcher as a front end (the callout 
> protocol performing between disptchers) or not (the callout protocol 
> performing beteween a dispatcher and a remote server). 

The callout protocol is used between an OPES processor (which includes 
a data dispatcher) and a callout server.

> - what does the dispatcher do with the part of the traffic it does not 
> send to the server and it received?

Messages not being vecored out to a callout server are forwarded to 
their respective destinations. For example, an HTTP request would be 
forwarded towards the content provider (i.e. Web server), an HTTP 
response would be forwarded towards the content consumer (i.e. Web client)

-Markus

---

From: jfcm <[email-masked]>
Date: Wed, 19 Mar 2003 04:01:25 +0100

On 02:33 19/03/03, Markus Hofmann said:
>The callout protocol is used between an OPES processor (which includes a 
>data dispatcher) and a callout server.

Dear Markus,
I do not ask what the present paper says (IMHO in a confusing way as 
something "which includes a data dispatcher" is not a precise definition). 
I defined a dispatcher as a box in a scheme and I question the way it 
should work. The same to discuss the callout server as a server called by 
the OPES processor is confusing since it does not defines the architecture 
of the server system. To discuss the protocol without defining precisely 
which functional block it relates seems premature? Looks like what we all 
do: starting with "int main (..." instead of buidling a diagram of what we 
want to develop.

>Messages not being vecored out to a callout server are forwarded to their 
>respective destinations. For example, an HTTP request would be forwarded 
>towards the content provider (i.e. Web server), an HTTP response would be 
>forwarded towards the content consumer (i.e. Web client)

Sorry for the Franglish. I meant "while it wait for the response" of the 
callout something for the following part. As a policy does it buffer the 
data, to wich extent, request hem to be resent?

jfc