Subject: OPES protocol, pre-draft

From: Alex Rousskov <[email-masked]>
Date: Tue, 18 Feb 2003 16:25:23 -0700 (MST)

To bootstrap creation of OPES protocol specs and to show one possible
way to answer the 5 design questions raised on this list, I would like
to propose the following protocol outline. If you think this is too
premature to talk about protocol specifics, please suggest what must
be done first.

I believe this protocol core can be implemented on top of HTTP or
BEEP.  I am less sure about SOAP; I am not a SOAP expert.

I think that the relaxed flow of the proposed protocol is a good idea,
given the combination of robustness requirements and requirements to
support a diverse set of application protocols. Mandatory ACKs and
explicit client-server flow do little good when you are managing
several semi-independent I/O buffers and have to handle, for example,
OPES server disappearance/overload and multiple recipients of a single
message. This relaxed flow and focus on data manipulation are the key
design elements that make the protocol simple yet efficient and robust
(I hope).

The protocol is optimized for the common case when most data is
forwarded "as is", unaffected by the OPES server. Removing this
optimization can simplify the protocol further.

--------------------

Contents:

	1. Terminology and environment
	2. OPES processor-server communication
	2.1 Transaction-independent messages
	2.2 Transaction-dependent messages
	2.2.1 Common transaction-dependent properties
	2.2.2 Major message types
	3. Examples
	4. Transport connections
	5. Synchronization and error handling

1. Terminology and environment

draft-ietf-opes-protocol-reqs-03.txt defines the following information
flow:

  data provider --(original application message)-->
  -- [ OPES magic ] -->
  --(produced application messages)--> data consumers

The original and "produced" (forwarded) messages together form an
application protocol transaction. Note that there may be more
than one produced application message resulting from a single
original message.

When application protocol transaction involves a request-response
sequence (e.g., HTTP), the above scheme remains the same. There are
just two related transactions now:

  provider1 --(client req)--> [ OPES ] --(proxy req)--> consumer1
  provider2 --(server resp)-> [ OPES ] --(proxy resp)-> consumer2

Usually, provider1 is consumer2, and consumer1 is provider2.

Multiple related transactions do not change the nature of the
OPES protocol. The only difference is that more information may
need to be passed from OPES processor to OPES server. For
example, processing of the response flow may need some knowledge
about the request flow.  Individual transactions may become
related.

There may be no server response. Depending on the application
protocol, there may be multiple server and/or proxy responses.

2. OPES processor-server communication

OPES processor and OPES server exchange messages. The exchange is
bidirectional. There is no clear client or server role.  There is
no clear request/response message category either.

There are two major kinds of messages: transaction-independent
and transaction-dependent.

2.1 Transaction-independent messages

Transaction-independent messages exchange information unrelated
to any specific application protocol transaction. They are used
to enquire about supported OPES protocol options, control remote
logging, exchange user preferences, check connectivity status,
etc. Transaction-independent messages are not documented here.

2.2 Transaction-dependent messages

Transaction-dependent messages are the core of the OPES protocol.
These messages alter, block, redirect, clone application messages
to be forwarded to data consumers.  Each transaction-dependent
message is associated with a single application transaction by
means of a unique transaction identifier (xid).

It may be important to keep in mind that transaction-dependent
messages manipulate the state of these four buffers/connections and
associated meta-information:
    - data producer (incoming) buffer at the OPES processor
    - data producer (incoming) buffer at the OPES server
    - data consumer (outgoing) buffer at the OPES server
    - data consumer (outgoing) buffer at the OPES processor

The design keeps buffers as allows to prevent buffer overflows
and discard buffered content when possible. Note that we rely on
transport protocol to be both reliable and to stop sending us
more data (eventually) if we stop reading it. TCP has both
properties.

2.2.1 Common transaction-dependent properties

Some transaction-dependent OPES messages share the following common
properties.

    xid -- Application transaction identifier (Xaction ID)

    source -- Information about the data provider (i.e., the
	source of the application message). For messages
	originated from the OPES processor, the source describes
	the original data provider.  For messages originated from
	the OPES server, the source describes what provider
	information should be presented to the data consumer;
	OPES server may need to change how the original
	information looks to the other application side.

    destinations -- One or more destinations. A single
	destination is information about the data consumer (i.e,
	the destination of the application message). For messages
	originated from the OPES processor, destination describes
	the consumer as intended by the producer. For messages
	originated from the OPES server, the destination is the
	data consumer that should be used by the OPES processor;
	OPES server may need to change the intended recipient.
	Depending on the application, OPES processor may need
	to check that all destinations have been covered by
	OPES server.

    data size -- Specific data size in octets OR a special token
	meaning "all" or "maximum". The all-token may only be
	used when requesting data, never when sending it.

    reason -- This should probably be a numeric status code with
	an optional information string. We will use just strings
	for now.

2.2.2 Major message types

    OPES processor may send the following transaction-dependent
messages to the OPES server. [ Note: The XML-like syntax does NOT
imply that the messages should be implemented using XML. Text or
binary encodings can be used; the encoding decision is out of
scope of this document. ]

    <xaction-start xid services ...>
	Informs OPES server about a new application
	transaction. This message should probably identify OPES
	service(s) requested for this transaction and other
	transaction-global info unrelated to data buffering,
	sources, or destinations.

    <producer-start xid bid source destinations>
	Informs OPES server about a new request from the data
	producer. Buffer IDentifier (bid) uniquely identifies the data
	buffer used for this request. Bid can probably be set to
	xid unless we expect to handle protocols that may merge
	requests before forwarding them.

    <data-have bid offset size [copied] >
	Sends [a portion of] application message from the data
	producer buffer to the OPES server. If "copied" flag
	is set, the OPES server may assume that the corresponding
	data is buffered at the processor and may refer to it
	using <data-as-is> messages described below.

    <data-pause bid>
	Notifies OPES server that there will be no more data
	for this transaction (coming from the OPES processor)
	UNLESS OPES server explicitly asks for it using
	<data-need-data> message described below

    <data-end bid reason>
	Notifies OPES server that there will be no more data
	for this transaction (coming from the OPES processor)

    <producer-end bid reason>
	Notifies OPES server that there will be no more messages
	for this bid (coming from the OPES processor)

    <xaction-end xid reason>
	Notifies OPES server that there will be no more messages
	for this transaction (coming from the OPES processor)

    OPES server may send the following transaction-dependent
messages to the OPES processor.

    <consumer-start xid bid source destinations />
	Informs OPES processor that OPES server may want to send
	data from source to destination(s). The Buffer IDentifier
	(bid) is unique for the (xid, source, destinations)
	triplet and identifies consumer buffer at the OPES
	server. There may be other buffers/messages (bid
	triplets) associated with the same transaction (xid). Xid
	comes from the corresponding xaction-start message send
	by the OPES processor.

    <data-have bid offset size>
	Tells OPES processor to send the attached data to the
	data consumer

    <data-asis bid offset size>
	Tells OPES processor to use processor's own copy of the
	specified data to send to the data consumer. This message
	can only specify data fragments previously marked with
	"copied" flag in a <data-have> message from OPES processor.

    <data-wont-need bid offset size>
	Tells OPES processor that the server will never send
	data-asis message for the specified data range. This
	message can only specify data fragments previously marked
	with the "copied" flag in a <data-have> message from OPES
	processor. This optional message may help OPES processor
	to free its resources.

    <data-need bid offset size>
	Tells OPES processor to send the specified data segment
	to the OPES server (probably in response to data-pause
	message from the OPES server).

    <data-pause bid>
	Notifies OPES processor that it should not send more data
	for this transaction until OPES server explicitly asks
	for it using data-need message described above

    <data-end bid reason>
	Tells OPES processor that there will be no more data
	for this bid (coming from the OPES server)

    <consumer-end bid reason>
	Notifies OPES server that there will be no more messages
	for this bid (coming from the OPES server)

    <xaction-end xid reason>
	Notifies OPES server that there will be no more messages
	for this transaction (coming from the OPES server)

Note: There needs to be a way for OPES server to tell OPES
processor to terminate (or short-circuit) the forwarding of a
message. This feature needs to be added to the protocol, but it
should not change the overall design.

3. Examples

Here is an example of (not) filtering an HTTP message based
on HTTP headers:

	processor: <xaction-start xid1 services ...>
	processor: <producer-start xid1 bid11 source destination>
	processor: <data-have bid11 offset=0 size=headers copied>
	processor: <data-pause bid11>

	server: <consumer-start xid1 bid12 source destination >
	server: <data-asis bid12 offset=0 size=all>
	server: <xaction-end xid1 "end-of-HTTP-message">

Note that xaction-end implies consumer-end implies data-end, and
there is no reason for OPES processor to send a xaction-end
message to server if the server already sent xaction-end message.
The lines above are grouped about possible network I/O
boundaries; thus, only two network data packets may be required
to process a message if the OPES server decides it does not care
based on the headers.

Here is an example of redirecting an HTTP request by changing its
destination info and corresponding HTTP headers:

	processor: <xaction-start xid2 services ...>
	processor: <producer-start xid2 bid21 source destination>
	processor: <data-have bid21 offset=0 size=headers copied>
	processor: <data-pause bid11>

	server: <consumer-start xid2 bid22 source other-destination >
	server: <data-have bid22 offset=0 size=new-headers>
	server: <xaction-end xid2 "end-of-HTTP-message">

Finally, here is an example of modifying the "middle" part of
HTTP message body. The OPES server switches the message encoding
to chunked, to avoid buffering data to figure out new Content-Length.

	processor: <xaction-start xid3 services ...>
	processor: <producer-start xid3 bid31 source destination>
	processor: <data-have bid31 offset=0 size=headers copied>
	processor: <data-pause bid11>

	server: <consumer-start xid3 bid32 source destination >
	server: <data-have bid32 offset=0 size=new-headers>
	server: <data-wont-need bid31 offset=0 size=headers>
	server: <data-need bid31 offset=headers size=all>

	processor: <data-have bid31 offset=headers size=chunk1 copied>

	server: <data-asis bid32 offset=headers size=chunk1>

	processor: <data-have bid31 offset=chunkOff1 size=chunk2 copied>

	/* send modified chunk, tell processor to ignore the original */
	server: <data-have bid32 offset=newheaders+chunk1 size=chunk2mod>
	server: <data-wont-need bid31 offset=chunkOff1 size=chunk2>

	processor: <data-have bid31 offset=chunkOff2 size=chunk3 copied>
	processor: <data-end bid31 "end-of-HTTP-message">

	server: <data-asis bid32 offset=chunkOff2 size=chunk3>
	server: <xaction-end xid3 "end-of-HTTP-message">

Note that once the flow starts, there are no explicit synchronization
points or waiting. The above message order is not the only one
possible: most messages from the processor are not synchronized with
most messages from the server.

4. Transport connections

Transport connections would depend on the transport protocol
(HTTP, BEEP, etc.). It is important to note that regardless of
the transport protocol chosen, it is possible to multiplex
messages from the OPES processor (or from the server) over
several persistent connections -- transaction-dependent messages
do not depend on "connection" properties except for the basic
requirement that ordered messages use the same connection, in the
right order.

Simple connection-specific messages can be introduced if we want
to support keep-alive checks or if we want to retry aborted
connections.

5. Synchronization and error handling.

The protocol has very few explicit dependencies between messages.
It is trivial to imagine a case where incorrect processor or
server implementation would result in deadlocks or other bad
states.  All sorts of deadlocks are resolved using timeouts. If
there is no progress with the transaction for an
admin-configurable time, the transaction is aborted. Aborting at
OPES server side is easy:

	server: <xaction-end xid3 "deadlock">

On the processor side, specific actions would depend on the
protocol and state. For example, if no response bytes have been
sent to an HTTP client yet, then an error response can be sent.

It would be also possible, in some states, to eliminate OPES
server from processing if it fails. Supporting this behavior
would require having a copy of entire application messages even
is OPES server tells us it does not need a copy. The exact
behavior must be admin-configurable.

---------------------------

This is just a start, of course. Many details are not specified
yet. For example, can OPES server request that a sub-fragment of
copied fragment is forwarded "as is"? What is the best way to
handle modifications of a non-chunked HTTP transfer that change
total message-length?

Also note that many command/option names should probably be
changed/polished. It is quite possible, for example, to make the
protocol look like the next generation of ICAP if we want to.

I believe the proposed protocol covers current major ICAP capabilities
and supports a few desired optimizations mentioned on the mailing
list.  Is there anything major that the protocol core does not support
and that needs to be supported? (besides security and authentication
features that are irrelevant at this point).

For example, should we add commands to control persistency of
application-protocol connections from the OPES server (I do not think
so)? Should we make it possible for OPES server to change the
application protocol (I do not think so)?

Thank you,

Alex.

-- 
                            | HTTP performance - Web Polygraph benchmark
www.measurement-factory.com | HTTP compliance+ - Co-Advisor test suite
                            | all of the above - PolyBox appliance

---

From: "The Purple Streak, Hilarie Orman" <[email-masked]>
Date: Tue, 18 Feb 2003 19:30:15 -0700

It's a good start.

The OPES processor should make the decisions about what to send to
the consumer.  This might just be a matter of terminology, but the
processor is in control of the source and destination and should not
send messages to new destinations based on OPES server demands.

The start message should identify the total length of the data, if it
is available.  This might stretch over several "bids", see below.

The first response from the server should identify the new total length,
if it is available.  If the length will change, but the new size is
unknown, the server should indicate this.

There is some confusion about "destination" - the OPES server should
never change the destination (i.e., the endpoint), so I don't see why
it is needed.  In the redirection example, it would be sufficient to
change the headers, and the purpose of "destination" is a mystery to
me.

The relationship between the application-layer framing, the bid and
offset, and the OPES framing is not clear from your examples.  The
application data may be transmitted to the OPES server a packet at a
time - this will mean a different bid on every data message, if you
literally mean that a bid is a buffer id.  Otherwise, it should have
some other name.

Also, the information about a bid should include its total length.

It should be possible to send the start and end messages on a separate
transport connection for handling errors or congestion.

The case discussed for multiple services, multiple responses, should
be included.  To support it, one needs multiple service lists and an
id for each response.

It should be possible to indicate that the transmitted data comes
from several places in the bid.  This allows the OPES processor to omit
huge cookies and other junk; the response, by including this information,
helps the process limit the state and parsing.

Hilarie

---

From: Alex Rousskov <[email-masked]>
Date: Tue, 18 Feb 2003 22:56:46 -0700 (MST)

> send messages to new destinations based on OPES server demands.

It looks like a design decision to me (i.e., it can be done both
ways). I think that you may want destination modification by OPES
servers if you want to satisfy Martin's requirement to be able to
produce multiple SMTP messages (to several small groups of recipients)
from one original SMTP message (to one large group of recipients). My
understanding is that it MAY be OPES server responsibility to "split"
the original destination address(es) into two groups. That is why I
let OPES server to modify original destination info.

Other destination-modification examples include request redirection
(within a CDN or at the surrogate). Source-modification examples
include anonymizing proxies.

If there is a consensus that OPES server cannot modify source and
destination info, then we can simplify the protocol a little bit.
Is there a consensus regarding this design decision? Perhaps Abbie's
poll will show...

> is available.  This might stretch over several "bids", see below.

Good point. This extra info about anticipated message length should
not hurt and may be used for resource pre-allocation purposes. It MUST
NOT be treated as normative/final, of course. I will modify the
messages accordingly.

> unknown, the server should indicate this.

Agreed, except we also need to support the case where the OPES server
does not know whether the length will change. For example, if the
server replaces "foo" with "blah" and "bleh" with "bar", it can tell
the final length (and whether it will change) only after seeing all
message content.

To make things more general and symmetric, I would make it possible
(but optional) to supply this non-normative length estimate with every
relevant message, in both directions.

> change the headers, and the purpose of "destination" is a mystery to

See above for motivation. The destination is needed because the OPES
processor needs to know where to connect to forward the request. It is
Bad Design to have OPES processor guess that information from
[possibly modified] message headers. This, again, assumes that we want
OPES server to be able to modify destination addresses. If we do not
want that, there is no need for OPES server to pass that info back to
processor, of course.

Note that source and destination information is meta-level information
that is often not completely available from HTTP headers. Take
interception and WCCP-controlled proxies for example. These
intermediaries often have to get destination address based on IP-level
details, not from HTTP headers. Similarly, the source information is
usually not available in the request headers but may be required to
route and modify the message.

Moreover, the protocol should make it possible to exchange other
meta-level information. For example, the time of the request may be
important ("no porn surfing before 6pm!").

> literally mean that a bid is a buffer id.  Otherwise, it should have

Yes, a terminology/naming problem: By "buffer" in Buffer ID you
probably mean "piece of memory that holds a data packet". I meant
"logical structure that holds all data associated with the application
connection/message".  That is, my-buffer may consist of many
your-buffers. Perhaps "buffer"  should be replaced with "connection"?
But "connection" is bad because OPES server does not really manage
application connections. "Message" seems too overloaded?
"Application message" (amid)? Will change bid to amid unless there are
better ideas.

This is just terminology though. "Bid/amid" is permanent for the
single application message (original or produced). This ID should be
used by processor and server to manage appropriate data structures
associated with the corresponding application connection.

> Also, the information about a bid should include its total length.

Not sure why that would be needed. Moreover, the "total length" of the
connection buffers (which is what bid identifies) may change at any
time. OPES server should not care how buffering is done at the OPES
processor side and vice versa. Perhaps your suggestion is a result of
my poor choice of the word "buffer"? See above.

> It should be possible to send the start and end messages on a
> separate transport connection for handling errors or congestion.

Yes, and it is possible. The start message is the first message for an
"amid"  so it can go on any connection (brand new or idle persistent).
The end messages, if they indicate an immediate abort, do not have to
be in order with data messages and, hence, can be sent on any OPES
connection as well. Recall that there is no protocol-mandated relation
between OPES connections and application transactions/connections. If
one wants to sent something "out of order", they can (and face the
consequences).

> be included.  To support it, one needs multiple service lists and an

I believe this is already supported, kind of. The "services" attribute
of the transaction start message can have a list of services. The OPES
server can initiate multiple consumer-start messages based on that
list. Each consumer-start message from OPES server has a unique bid
(amid).

What is not clear to me is how the OPES server would know whether the
services list is an OR, AND, or XOR, or something else. I suspect we
need to support If-header logic from ICAP if we want to go down this
route. I will polish the protocol once I understand the exact
requirement here. Do we need to support some kind of
service-to-response matching language here? I do not recall a clear
answer in available OPES IDs. Help?

> from several places in the bid.  This allows the OPES processor to
> omit huge cookies and other junk; the response, by including this
> information, helps the process limit the state and parsing.

Interesting. If I interpret your requirement correctly, we need an
indication that some data was skipped by the OPES processor when
forwarding original application message to OPES server. We also need
an ability to reinject the skipped data into produced application
message. Not sure this can be supported in a general way: OPES server
may modify application headers but it is not clear how it can tell
OPES processor to correctly inject skipped stuff into modified headers
if the server does not know exactly what was skipped.

We may be able to support the above for, say, header values but not
for header names (but this becomes too application-specific!).
Alternatively, we can document that OPES processor is responsible for
injecting skipped stuff the way it deems necessary. In the latter
case, OPES server should be informed that something was skipped, but
it would not care much except for message length interpretation code.
This still makes digital signatures and related concepts hard to
implement or verify. Am I making it more complicated than it is?
Comments?

Thank you,

Alex.

---

From: "Oskar Batuner" <[email-masked]>
Date: Wed, 19 Feb 2003 10:43:19 -0500

1. Terminology: I'd suggest using names "OPES processor" and "callout server",
as in the architecture document. OPES server is a bit confusing - too similar
to OPES processor (which in fact is also a server).

I'd also suggest using transaction-id (tid) instead of buffer-id.

2. Transaction boundaries: in your example OPES server does not send
xaction-end. This may result in protocol stack on the callout server keeping
some tid-related data for indefinite time - in order to correctly match
possible incoming messages to the protocol state.

I'd suggest keeping clear transaction structure: each side MUST open and close
transaction with predefined start-end messages. This will result in more
stable and reliable protocol state machine.

3. Client-server protocol versus independent message flow. I think that
client-server architecture (as in ICAP and HTTP) has certain advantages. It
fits well OPES dataflow model: OPES server always initiates transaction as
result of some message received from customer. Callout server provides
services that are initiated by request made by OPES processor. This structure
is simpler, and if we need transactions initiated by callout server
independent from OPES data flow (for example request for policy information,
address book, preferences, etc.) we may use a different unidirectional
channel.

In our case by-directional channel may be presented as two unidirectional
ones, and as far as I can see OPES protocol fits client-server model very
well. All exchanges are taking place in the context of some transaction, each
such transaction consist of request-response sequences, in each transaction we
have a clear division of roles: one side is a service requestor (client),
another - a service produce (server), and the protocol messages that each side
may produce are subordinate to it's role in the current transaction. Moreover,
there are distinct sets of requests that may be send by OPES server / callout
server.

- Oskar

> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]]On Behalf Of Alex Rousskov
> Sent: Tuesday, February 18, 2003 6:25 PM
> To: [email-masked]
> Subject: OPES protocol, pre-draft
>
>
>
>

---

From: Alex Rousskov <[email-masked]>
Date: Wed, 19 Feb 2003 09:48:36 -0700 (MST)

On Wed, 19 Feb 2003, Oskar Batuner wrote:

> 1. Terminology: I'd suggest using names "OPES processor" and
> "callout server", as in the architecture document. OPES server is a
> bit confusing - too similar to OPES processor (which in fact is also
> a server).

Will change.

> I'd also suggest using transaction-id (tid) instead of buffer-id.

Transaction ID is already used to identify application transaction
(xid). Xid is [the only thing] shared among messages from and to OPES
processor.

"Buffer ID" clearly causes confusion though. Do you like Application
Message ID (amid) instead of buffer-id? An application transaction
usually involves several application messages (original and produced),
which is reflected in how "amid" (currently "bid") is used by the
protocol.

> xaction-end. This may result in protocol stack on the callout server
> keeping some tid-related data for indefinite time - in order to
> correctly match possible incoming messages to the protocol state.
>
> I'd suggest keeping clear transaction structure: each side MUST open
> and close transaction with predefined start-end messages. This will
> result in more stable and reliable protocol state machine.

The text should probably be polished to make it more clear. There is
absolutely NO reason for the side that receives a xaction-end message
to send a xaction-end message back. Here is why:

	- a side MUST ignore messages for unknown transactions
	  (except for xaction-start messages, of course);
	  doing so requires keeping state for known transactions,
	  which is something that has to be done anyway; a simple
	  hash table would do

	- once a side sends a xaction-end message, it deletes all
	  corresponding data structures, and the corresponding
	  xaction ID immediately becomes "unknown" for that side

	- if the other side responds to the xaction-end message with
	  its own xaction-end message, the latter will simply be
	  ignored by the first side; sending a xaction-end message to
	  the side that already sent xaction-end message is a waste of
	  resources

As you can see, the state machine is deterministic and simple. Once a
side says that it does not care about the transaction, there is no
reason to tell it _anything_ about that transaction. There is no way a
correct implementation can end up with xid-related data for indefinite
time because of the above optimizations -- there is no xid-related
data at the side once xaction-end message is sent by that side.

Does this clarify? Did I miss anything?

> 3. Client-server protocol versus independent message flow. I think
> that client-server architecture (as in ICAP and HTTP) has certain
> advantages. It fits well OPES dataflow model: OPES server always
> initiates transaction as result of some message received from
> customer. Callout server provides services that are initiated by
> request made by OPES processor. This structure is simpler,

First of all, there is a terminology problem here (possibly introduced
by my own wording!). The proposed protocol uses a "pipeline" model. It
is still possible to call it a client-server protocol, I guess. The
key difference is the difference between strict "at most
one-outstanding-query" approach and a "pipeline of queries". But let's
use the client-server term to describe "at most one-outstanding-query"
approach, for now.

I do not see how a client-server model is simpler. Can you elaborate?
The proposed "pipeline" protocol has a very straightforward (simple?)
state-machine-like implementation.

Moreover, I argue that client-server model is inappropriate for OPES
because it does not handle data flow efficiently. It does not scale
with the size of the application message. Client-server model is good
for handling small atomic transactions. The proposed pipeline model is
good for handling data transfers. If we want OPES to be efficient then
we should optimize data flow, and client-server model does not allow
for that.

Client-server abstraction is OK when working on high-level and
treating each application message as a size-less blob of information:

	Time  Action
	----  ------
	0     Read the original application message
	1     Send query: "Please modify this message"
	2     Receive response: "Here is a modified message"
	3     Write the produced application message

Once we start looking at the implementation level, the above becomes
very inefficient both because of the resources spend keeping all the
state and because of the introduced delays. This traditional approach
does not scale with the message size. ICAP tries to solve the problem
with the "preview" feature, but that solution is partial and rigid. To
get rid of large buffers and staging delays one must use a pipeline
approach:

	Time  Action
	----  ------
	t     Read chunk #N of the original message
	t     Modify chunk #N-1 of the original message
	t     Write chunk #N-2 of the original message
	t+1   Send query: "Please modify chunk #N"
	t+1   Receive response: "Here is modified chunk #N-1
	...

Since most of the actions are performed on small chunks and in
parallel, the buffering requirements are minimal and so is the extra
delay. Response time overhead of a good OPES implementation should not
exceed that of a [second] proxy overhead.

Historical evidence in point: Optimized HTTP proxies do not use
client-server model internally. The proxies pipeline data from the
client to server side and back. OPES is also a proxy (just look at the
architecture diagrams in the release WG IDs).

> and if we need transactions initiated by callout server independent
> from OPES data flow (for example request for policy information,
> address book, preferences, etc.) we may use a different
> unidirectional channel.

Note that the proposed protocol does not define transaction-related
messages that are spontaneously initiated by callout server,
independent from OPES data flow. The callout server always "responds"
to the stream of messages initiated by the OPES processor.

As for transaction-independent messages, yes, those can be implemented
on separate channels. Having separate channels may make priority
handling of transaction-dependent messages easier. We can discuss this
(less important design decision) later.

> In our case by-directional channel may be presented as two
> unidirectional ones, and as far as I can see OPES protocol fits
> client-server model very well. All exchanges are taking place in the
> context of some transaction, each such transaction consist of
> request-response sequences, in each transaction we have a clear
> division of roles: one side is a service requestor (client), another
> - a service produce (server), and the protocol messages that each
> side may produce are subordinate to it's role in the current
> transaction. Moreover, there are distinct sets of requests that may
> be send by OPES server / callout server.

I mostly agree. As I said, there seems to be a terminology problem.
The proposed approach fits your description above relatively well; the
key [difference] is that the request-response pairs are not sequential
but pipelined. Without the pipeline, the protocol does not scale with
the message size (see illustrations above). Once you accept the
pipeline idea, everything should snap into place.

In other words, the proposed protocol is closed to a traditional
client-server approach at the level of message _chunks_. The important
part is, however, not handling of individual chunks, but making an
efficient pipeline from chunk-related "queries" and "responses".

Do you agree? Can you think of a pipeline-less alternative that still
scales well with the message size?

Thank you,

Alex.

---

From: jfcm <[email-masked]>
Date: Wed, 19 Feb 2003 18:38:50 +0100

>I'd also suggest using transaction-id (tid) instead of buffer-id.

Might we not think in term of dispatcher and services?

Does location of these functions change the logic of the relations between 
the dispatcher (applying the filtering rules) and the service(s)?

>stable and reliable protocol state machine.

I am not sure about this: can matters subject to rules of the filter be 
changed by the server? Canit result in loops?

>result of some message received from customer.

not if it acts as a sub-server. Question can we hide that sub-servicing?

Exemple: the OPES server must add a documentation inclduing
a graphic. That graphic mustbe obtained from another paying OPES
server. There is a need for the sub-transaction to be recorded
(payement, legal responsbility etc).

>Callout server provides

bi-directionnal if we want to have negociations?

jfc

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.454 / Virus Database: 253 - Release Date: 10/02/03

---

From: Alex Rousskov <[email-masked]>
Date: Wed, 19 Feb 2003 10:46:54 -0700 (MST)

jfc,

	We are talking about two different things here. You are
talking about directing application messages to appropriate callout
servers. I am talking about [re]directing application messages to
different data consumers.

For example, an HTTP request for www.w3c.com can be redirected (by the
callout server manipulations of HTTP headers and meta-information) to
www.w3.org. Another example: an e-mail to [email-masked] can be
redirected (by the callout server manipulations of SMTP headers and
meta-information) to [email-masked] _and_ to [email-masked]
(producing two application messages).

Some argued that such modification of application message destination
MUST NOT be allowed. Some wanted it to support the second example
above.

Hope this clarifies.

Thank you,

Alex.

On Wed, 19 Feb 2003, jfcm wrote:

> On 17:53 19/02/03, Alex Rousskov said:
> > > No.
>
>          - "No. There is no consensus: I think that a callout server
>            MAY change the destination of the application transaction,
>            but some other folks on the list disagree."
>
> I do not like the word transation in here. I prefer cession. It can be a
> lot more structured that just a transaction. Unless my Frenglish hits again.
>
> > > The OPES
>
> Server or Dispatcher system
>
> > > can sub-delegate a task temporarily or permanently. The
> > > first task that a server must be able to do is to qualify a request
> > > as a dispatcher.
>
> The Dispatcher qualifies a page as to be processed
>
> 1. it can send it to another/sub-dispatcher which is in charge
>      of handling that type of service (ex. DNS)
>
> 2. it can send it to a callout server which may decide
>      that this type of service will be handled by another
>      machine on the Internet or CPU in its own network.
>
> And this several times if loops are permitted on rules
> (the process permits to change something subject to
> a rule) or even if several rules are to be applied in
> sequence (but is that the same transaction according
> to your vision?).
>
> Thank you.
> jfc
>
>

---

From: jfcm <[email-masked]>
Date: Wed, 19 Feb 2003 19:16:43 +0100

At 17:48 19/02/03, Alex Rousskov wrote:
>Once you accept the pipeline idea, everything should snap into place.

total agreement. Takes good shape!

Now, in term of wording why pipes would they not link "dispatchers" and end 
into "services" (or service admins - which could include a dispatcher)?

An OPES domain would then end at services admins (which could belong to 
several domains). This architecture can be supported off the shelf in any 
message passing (network) protocol.

1. user data flows go through the first dispatcher
2. filters it to outboud socket or to a pipe
3. receiving dispatcher triggers the proper service admin
4. service admin tells the dispatcher which service to pipe it into
5. anytime the service can call the disptacher to get additional services
6. the output is sent back to first dispatcher

 From what you say there would be "data" pipes and "command" pipes, the 
command pipes being faster?

Reason is that there is a need for load management and therefore 
priorities. Pipes should permit that nicely but the sending dispatcher 
needs to control the throttle.

If you use a pipeline approach, would you support zappers in the pipes?

Reason of the importance of that is that if a service blocks, the preceding 
dispatcher must be able to zap the outband pipe and rebuild it in calling a 
new service admin.
jfc

---