Subject: OPES protocol, pre-draft 01

From: Alex Rousskov <[email-masked]>
Date: Fri, 21 Feb 2003 14:33:01 -0700 (MST)

There were many great suggestions on how to improve the protocol
pre-draft. Please find an updated document below. Here is the change
log:

	- replaced "bid" with "amid" attribute and fixed amid
	  definition due to many compliants and confusions

	- replaced OPES server with callout server (Oskar Batuner)

	- added "sizep", an optional message size prediction attribute
	  (Hilarie Orman)

	- added "modp", an optional modification prediction attribute
	  (Martin Stecher)

	- added <i-am-here> messages (Martin Stecher)

	- simplified document structure;
	  removed many general remarks and moved essential ones
	  into a short Introduction section;
	  polished text

	- added TODO section

Comments and help with the TODO list are requested.

Thank you,

Alex.

------------------------

Table of Contents:

	0. Introduction.
	1. Message properties
	2. Message types
	3. Examples
	4. Transport connections
	5. Synchronization and error handling
	6. About this document
	7. TODO

0. Introduction

draft-ietf-opes-protocol-reqs-03.txt defines the following
information flow:

  data provider --(original application message)-->
  -- [ OPES magic ] -->
  --(produced application messages)--> data consumers

The original and "produced" (forwarded) messages together
form an application protocol transaction. Note that there
may be more than one produced application message resulting
from a single original message.

When application protocol involves a request-response
sequence (e.g., HTTP), we treat it as two related OPES
transactions: request transaction and response transaction.

OPES processor and callout server exchange messages. The
exchange is bidirectional. There is no clear client or
server role.  There is no clear request/response message
category either.

OPES messages manipulate the state of these four
buffers/connections and associated meta-information:
    - data producer (incoming) buffer at the OPES processor
    - data producer (incoming) buffer at the callout server
    - data consumer (outgoing) buffer at the callout server
    - data consumer (outgoing) buffer at the OPES processor

The design prevents buffer overflows and allows to discard
buffered content as soon as possible. Note that we rely on
OPES transport protocol to be both reliable and to stop
sending us more data (eventually) if we stop reading it. TCP
has both properties.

[ Note: The XML-like syntax for describing protocol parts
does NOT imply that OPES messages should be implemented using
XML. Text or binary encodings can be used; the encoding
decision is out of scope of this document. ]

1. Message properties

Many OPES messages share the following properties.

    xid -- Application transaction identifier (Xaction ID)
	Uniquely identifies an application transaction
	among all OPES agents that may see this ID.

    amid -- Application Message IDentifier
	Uniquely identifies an application message within an
	application transaction. Amid can be interpreted in
	an application transaction context only.  Thus,
	either xid must be present whenever amid is used or
	amid must uniquely identify application transaction
	as well (e.g., by containing xid). [ @@@ we should
	decide one way or the other ]

    source -- Information about the data provider (i.e., the
	source of the application message). For messages
	originated from the OPES processor, the source
	describes the original data provider.  For messages
	originated from the callout server, the source
	describes what provider information should be
	presented to the data consumer; callout server may
	need to change how the original information looks to
	the other application side.

    destinations -- One or more destinations.
	Depending on the application, OPES processor may
	need to check that all original destinations have
	been covered by callout server.

    destination -- Information about the data consumer (i.e,
	the destination of the application message). For
	messages originated from the OPES processor,
	destination describes the consumer as intended by
	the producer. For messages originated from the
	callout server, the destination is the data consumer
	that should be used by the OPES processor; callout
	server may need to change the intended recipient.

    services -- One or more services.
        There will be a way to indicate desired order
	of service application, possibly including
	concurrent applications at the callout server

    data size -- Specific data size in octets OR a special
	token meaning "all" or "maximum". The all-token may
	only be used when requesting data, never when
	sending it.

    data offset -- non-negative number of octets
        relative to the beginning of the application message

    sizep -- size prediction
	An integer value of at least zero.  Size-prediction
	property carries remaining application message size
	prediction, in octets.  The value includes data in
	the current message, if any. This property can be
	used in any message with amid property.  This is a
	prediction, not a fact.

    modp -- modification probability prediction with
	An integer value from 0 to 100, indicating the
	probability (0 = will probably never happen, 100 =
	probably imminent) that some produced data following
	the prediction (including data in the current
	message, if any) will differ from the original data.
	A reading of 100 does not imply that the current (or
	any!) message data has been modified. This is a
	prediction, not a fact.

	This property can be present in any callout server
	message with amid property.  Absence of the property
	means absence of a [new] prediction, not that there
	will be no modifications.  Note that prediction is
	persistent for the given amid unless overwritten
	by a different value of modp in a later message.

	[ @@@ if OPES can change meta-info like destination
	address, should that be included in modification
	semantics? ]

    reason -- This should probably be a numeric status code with
	an optional information string. In examples, we will
	use just strings for now.

2. Message types

    An OPES processor may send the following messages to the
callout server.

    <xaction-start xid services ...>
	Informs callout server about a new application
	transaction. This message should probably identify OPES
	service(s) requested for this transaction and other
	transaction-global info unrelated to data buffering,
	sources, or destinations.

    <producer-start xid amid source destinations >
	Informs callout server about a new message from the
	data producer. Amid can probably be set to xid
	unless we expect to handle protocols that may merge
	messages before forwarding them.

    <data-have amid offset size [copied] >
	Sends [a portion of] application message from the
	data producer buffer to the callout server. If
	"copied" flag is set, the callout server may assume
	that the corresponding data is buffered at the
	processor and may refer to it using <data-as-is>
	messages described below. Copying commitment must
	last until the corresponding <data-as-is> message or
	<consumer-end> event.

    <data-pause amid>
	Notifies callout server that there will be no more
	data for this transaction (coming from the OPES
	processor) UNLESS callout server explicitly asks for
	it using <data-need> message described below. This
	message may be used if OPES processor suspects that
	callout server is not interested in the data and, hence,
	there is no reason to send it by default (e.g., a
	response content type indicates that it is unlikely
	to have a virus but only callout server can know for sure).

    <data-end amid reason>
	Notifies callout server that there will be no more data
	for this transaction (coming from the OPES processor)

    <producer-end amid reason>
	Notifies callout server that there will be no more messages
	for this amid (coming from the OPES processor)

    <xaction-end xid reason>
	Notifies callout server that there will be no more messages
	for this transaction (coming from the OPES processor)

    A callout server may send the following messages to the
OPES processor.

    <consumer-start xid amid source destinations />
	Informs OPES processor that callout server may want
	to send data from source to destination(s). There
	may be other messages (amids) associated with the
	same transaction (xid).  Xid comes from the
	corresponding xaction-start message send by the OPES
	processor.

    <data-have amid offset size>
	Tells OPES processor to send the attached data to the
	data consumer.

    <data-as-is amid offset size>
	Tells OPES processor to use processor's own copy of the
	specified data to send to the data consumer. This message
	can only specify data fragments previously marked with
	"copied" flag in a <data-have> message from OPES processor.

    <data-wont-need amid offset size>
	Tells OPES processor that the callout server will
	never send data-as-is message for the specified data
	range. This message can only specify data fragments
	previously marked with the "copied" flag in a
	<data-have> message from OPES processor. This
	message amid must match the <data-have> (producer)
	message amid, not the consumer amid. This optional
	message may help OPES processor to free its
	resources.

    <data-need amid offset size>
	Tells OPES processor to send the specified data
	segment to the callout server (probably in response
	to data-pause message from the callout server). This
	message amid must match the corresponding producer
	amid, not the consumer amid.

    <data-pause amid>
	Notifies OPES processor that it should not send more
	data for this transaction until callout server
	explicitly asks for it using data-need message
	described above. This message amid must match the
	corresponding producer amid, not the consumer amid.

    <data-end amid reason>
	Tells OPES processor that there will be no more data
	for this amid (coming from the callout server)

    <consumer-end amid reason>
	Notifies callout server that there will be no more messages
	for this amid (coming from the callout server)

    <xaction-end xid reason>
	Notifies callout server that there will be no more messages
	for this transaction (coming from the callout server)

Note: There needs to be a way for callout server to tell
OPES processor to terminate (or short-circuit) the
forwarding of a message. This feature needs to be added to
the protocol, but it should not change the overall design.
One way to support this feature is for callout server to
change the destination of the application message from
consumer to producer (and change source to itself?).

    OPES processor or callout server may send the following
messages.

    <i-am-here>
    <i-am-here xid>
    <i-am-here xid amid>
	The messages tell recipient that the sender is
	working, working on xid, or working on amid,
	respectively. The sender may not be able to send any
	other message (yet), but wants to inform the
	recipient that it knows of recipient's (or xid, or
	amid) existence. The sender MAY send more specific
	messages later.

3. Examples

Here is an example of (not) filtering an HTTP message based
on HTTP headers:

	processor: <xaction-start xid1 services ...>
	processor: <producer-start xid1 amid11 source destination>
	processor: <data-have amid11 offset=0 size=headers copied>
	processor: <data-pause amid11>

	server: <consumer-start xid1 amid12 source destination >
	server: <data-as-is amid12 offset=0 size=all>
	server: <xaction-end xid1 "end-of-HTTP-message">

Note that xaction-end implies consumer-end implies data-end, and
there is no reason for OPES processor to send a xaction-end
message to server if the server already sent xaction-end message.
The lines above are grouped about possible network I/O
boundaries; thus, only two network data packets may be required
to process a message if the callout server decides it does not care
based on the headers.

Here is an example of redirecting an HTTP request by changing its
destination info and corresponding HTTP headers:

	processor: <xaction-start xid2 services ...>
	processor: <producer-start xid2 amid21 source destination>
	processor: <data-have amid21 offset=0 size=headers copied>
	processor: <data-pause amid11>

	server: <consumer-start xid2 amid22 source other-destination >
	server: <data-have amid22 offset=0 size=new-headers>
	server: <xaction-end xid2 "end-of-HTTP-message">

Finally, here is an example of modifying the "middle" part of
HTTP message body. The callout server switches the message encoding
to chunked, to avoid buffering data to figure out new Content-Length.

	processor: <xaction-start xid3 services ...>
	processor: <producer-start xid3 amid31 source destination>
	processor: <data-have amid31 offset=0 size=headers copied>
	processor: <data-pause amid11>

	server: <consumer-start xid3 amid32 source destination >
	server: <data-have amid32 offset=0 size=new-headers>
	server: <data-wont-need amid31 offset=0 size=headers>
	server: <data-need amid31 offset=headers size=all>

	processor: <data-have amid31 offset=headers size=chunk1 copied>

	server: <data-as-is amid32 offset=headers size=chunk1>

	processor: <data-have amid31 offset=chunkOff1 size=chunk2 copied>

	/* send modified chunk, tell processor to ignore the original */
	server: <data-have amid32 offset=newheaders+chunk1 size=chunk2mod>
	server: <data-wont-need amid31 offset=chunkOff1 size=chunk2>

	processor: <data-have amid31 offset=chunkOff2 size=chunk3 copied>
	processor: <data-end amid31 "end-of-HTTP-message">

	server: <data-as-is amid32 offset=chunkOff2 size=chunk3>
	server: <xaction-end xid3 "end-of-HTTP-message">

Note that once the flow starts, there are no explicit synchronization
points or waiting. The above message order is not the only one
possible: most messages from the processor are not synchronized with
most messages from the server.

4. Transport connections

OPES transport connections would depend on the transport
protocol (HTTP, BEEP, etc.). It is important to note that
regardless of the transport protocol chosen, it is possible
to multiplex messages from the OPES processor (or from the
server) over several persistent connections. OPES messages
do not depend on "connection" properties except for the
basic requirement that order-dependent messages use the same
transport connection, in the right order.

5. Synchronization and error handling.

The protocol has very few explicit dependencies between messages.
It is trivial to imagine a case where incorrect processor or
server implementation would result in deadlocks or other bad
states.  All sorts of deadlocks are resolved using timeouts. If
there is no progress with the transaction for an
admin-configurable time, the transaction is aborted. Aborting at
callout server side is easy:

	server: <xaction-end xid3 "deadlock">

On the processor side, specific actions would depend on the
protocol and state. For example, if no response bytes have been
sent to an HTTP client yet, then an error response can be sent.

It would be also possible, in some states, to eliminate OPES
server from processing if it fails. Supporting this behavior
would require having a copy of entire application messages even
is callout server tells us it does not need a copy. The exact
behavior must be admin-configurable.

6. About this document

This document goal is to become a section in the future OPES
protocol specs, after a lot of editing. The OPES milestone
reads: "MAY 03 Initial protocol document for OPES services
including their authorization, invocation, tracking, and
enforcement of authorization".

7. TODO

	1. Decide whether callout server can change
	application message source and destinations

	2. Understand and support the following: "It should
	be possible to indicate that the transmitted data
	comes from several places in the amid.  This allows
	the OPES processor to omit huge cookies and other
	junk; the response, by including this information,
	helps the process limit the state and parsing."

	3. Document how one can write OPES extensions. Use
	"progress meter" as an example/motivation.

$Id: protocol.txt,v 1.2 2003/02/21 21:25:14 rousskov Exp rousskov $

---

From: "Oskar Batuner" <[email-masked]>
Date: Sat, 22 Feb 2003 15:08:21 -0500

Hi Alex,

>   --(produced application messages)--> data consumers

I think both protocol requirements and architecture support OPES
processing in bi-directional message flow, both for data consumer
requests and data producer responses. This means that this diagram
and subsequent association of incoming message with the data producer
and outgoing message with the data consumer should be changed to
reflect the bi-directional nature of the OPES flow.

> exchange is bi-directional. There is no clear client or

I do not agree with that. Per OPES architecture processing is initiating
by rules that are triggered by some event in the OPES flow (arrival of
request or response). As a result of some rule evaluation OPES processor
can make a request to OPES application (either in the same box or a callout
server). This creates a clear client-server roles: OPES processor makes a
request and callout server produces response.

OPES callout protocol should reflect these roles. Each transaction is
initiated by the client (OPES processor) and executed by the server
(callout server). There are certain message types that are produced by the
client and other (different) types produced by the server. Protocol state
transitions also should clearly reflect these roles. For example each
transaction starts from <xaction-start> message from the transaction client
and is terminated by <xaction-end> from the transaction server. BTW, my
initial
confusion was due to the absence of transaction roles. In bi-directional
exchange
without defined roles each side has to specify transaction boundaries, but
with predefined client-server roles it is enough to have one open and one
close message, as in your example.

Note that protocol requires definite roles only per transaction. Roles are set
by the opening message and may be different for different transactions.

For the OPES architecture when processing is initiated by incoming OPES
processor
messages (from producer or from consumer) OPES processor is always a client.
These
roles may be different only for metadata requests (e.g. callout server
requests
about consumer or provider preferences). But again each transaction has client
and server.

The question is: do we allow such requests in-band?

- Oskar

---

From: jfcm <[email-masked]>
Date: Sun, 23 Feb 2003 11:14:20 +0100

I am sorry, but I find this terminology confusing and limiting the analysis.

We are talking about services. Theses services are called by dispatchers 
according to rules. If there is a service there is a user the service is 
delivered to and a host the services is delivered on behalf of. My 
understanding from IAB and first draft is that OPES correspond to the 
following sheme under HTTP:

Service User ----> dispatcher -----> Service Organizer
                         ^
                         |  callout protocol
                         v
                      service
                      provider

As Oskar says, the traffic can be bidrectional. So I do not really know who 
the data consumer and the data producer are:

Service User <----> dispatcher <----> Service Organizer
                         ^
                         |  callout protocol
                         v
                      service
                      provider

Others want to support SMTP etc. Others want the system to be able to fork.

Service User <----> dispatcher <----> Service Organizer
                   /     ^       \
Other user      /       |         \    Other Host
                         v
                       service
                       provider

My own reading is that this dialog system conducted by the dispatcher 
should become a polylogal system, hence "open" in my "open extended network 
system" naming. This is also why I call them "network" services while IAB 
specifies clearly "edge". And why I call them "extended" (in comparison 
with "basic" and "enhanced", because they relate users with as well as what 
you call data producers and with other data users)

Because - as Oskar points it in saying it is bidireactional - I know who are:
- the user and the producer in an access connection
- the caller and the calleed party in a communication
- but I do not know who are the user/producer nor the caller and callee in 
a relation - all the more in a distributed networked relations among a 
community/market. I can at most differentiate different classes of users 
(of of their agents) and groups of virtual services (organizers). In this 
what counts is not what is installed or proposed, but what I am allowed to 
use at a given time. And the very organizing host (supervisor) may be 
itself subject to in the flow changes (example: the proposed service 
promotes me on-the-fly to an "admin" status).

To follow the same presentation as above:

                                        Supervisor or
Service User <-----> dispatcher <----> Service Organizer
                    /  ^      ^  \
Other user       /    |      |    \  Other users
                       v      v
                 service <--> server  <----> service
controler     /provider      dispatcher     provider
external user 
/                  ^            /^
etc.. etc..                      |          /  |
                                  v        /    v
                                 etc.    /   alarms,logs 

                                       /     etc
                                     /
                  other ONES domains

>reflect the bi-directional nature of the OPES flow.

True. But this only permits a dialog: ask the user to accept
a redirection. It should support polylog (example: conditional
access to be accepted. Ex. the redirection costs more and
depending on the caller, an external authority (bank) is to
permit it which may want to call the caller for extra warranties.

>request and callout server produces response.

There is a link between the dispatcher and the service. But none is
the boss. There may be negotiations (both ends) or flow control
(the service adding a rule in the dispatcher to block requests until
it is ready again), etc.. A service may be linked to serveral
dispatchers and a dispatcher to several identical/similar services.

>initial confusion was due to the absence of transaction roles. In 
>bi-directional
>exchange without defined roles each side has to specify transaction 
>boundaries, but

There is a client/server role within an accepted on-going
cession/transaction. But if the link is broken what are the
recovery decisions? The service may be to contact the
user and rebuild the link through another dispatcher?

>by the opening message and may be different for different transactions.

True.

>about consumer or provider preferences). But again each transaction has client

If I read you correctly you talk about two different client/server systems.
One is producer/consumer, the other one is processor/server.
And you say that depending on transactions the roles may be changed?

>The question is: do we allow such requests in-band?

May I suggest a question in response? How do we zap
and abort the transactions? Should not the response be
the same?

jfc

---

From: Alex Rousskov <[email-masked]>
Date: Mon, 24 Feb 2003 11:22:51 -0700 (MST)

On Sat, 22 Feb 2003, Oskar Batuner wrote:

> and subsequent association of incoming message with the data
> producer and outgoing message with the data consumer should be
> changed to reflect the bi-directional nature of the OPES flow.

Oskar,

	You are right. I apparently misinterpreted the
"consumer/producer" terminology in the architecture draft.

	Fortunately, the protocol itself requires no serious
adjustments because there are still application transactions, and
there are still application messages that belong to those
transactions. A couple of OPES messages names and definitions should
be changed to bring them in sync with the architecture terminology,
but these are cosmetic changes.

	In other words, the proposed protocol does not depend on how
many application messages there are per application transaction or on
their "direction".  Draft-01 implies there are usually at least two
application messages per application transaction (original and
produced). Draft-02 will remove that implication and will use
appropriate producer/consumer terminology from the architecture
draft..

> I do not agree with that. Per OPES architecture processing is
> initiating by rules that are triggered by some event in the OPES
> flow (arrival of request or response). As a result of some rule
> evaluation OPES processor can make a request to OPES application
> (either in the same box or a callout server).

I agree with that :-).

> This creates a clear client-server roles: OPES processor makes a

While I think this is not the only possible terminology (it may not be
enough to originate a transaction to be called a client), I do not
want to argue about this and will try to change the Introduction
wording according to your preferences. This terminology disagreement
does not change the protocol.

My original reasoning was to avoid "client" and "server" terms because
OPES processor and callout server exchange many messages in the
context of a single application transaction and those messages are not
pure request/response sequences. But we can still call OPES processor
a client; not a big deal.

> OPES callout protocol should reflect these roles. Each transaction
> is initiated by the client (OPES processor) and executed by the
> server (callout server). There are certain message types that are
> produced by the client and other (different) types produced by the
> server. Protocol state transitions also should clearly reflect these
> roles.

I agree with that. Keep-alive and maintenance messages may be
originated at the server, but this is minor. OPES transactions are
initiated by OPES processor and executed by the callout server.

> For example each transaction starts from <xaction-start> message
> from the transaction client

Yes.

> and is terminated by <xaction-end> from the transaction server.

This restriction is unnecessary and cannot be derived from the
client-server model. It is perfectly normal for a client-server
protocol to allow either side to terminate the transaction. Usually,
the transaction will be terminated once one party signals the "end"
and the other side receives that signal. In some protocols (e.g.,
SSL/TLS) the end of a transaction is a much more complex event.

In OPES context, the client (OPES processor), may terminate the
transaction by sending a <xaction-end> message to the callout server.
The callout server might send that message first (probably only under
unusual conditions like overload). The two sides might even send
<xaction-end> concurrently!

Also, please note that OPES protocol has to deal with three kinds of
"end" messages:
	- end-of-data from the data source
	  (does not imply that the work on the application message is
	  done)
	- end-of-work for the application message
	  (implies end-of-data but does not imply that the work on the
	  application transaction is done)
	- end-of-work for the application transaction
	  (implies end-of-data and end-of-message)

> For the OPES architecture when processing is initiated by incoming
> OPES processor messages (from producer or from consumer) OPES
> processor is always a client. These roles may be different only for
> metadata requests (e.g. callout server requests about consumer or
> provider preferences). But again each transaction has client and
> server.

I understand your desire to call OPES processor a client and callout
server a server in the context of an OPES transaction. I will adjust
predraft wording accordingly.

> The question is: do we allow such requests in-band?

If I understand the question correctly, this is purely an OPES
connection management issue. We can, for example, mark some
connections "for management messages only" by sending a special
message on that connection once the connection is open by the OPES
processor. Does anybody have a preference/recommendation? Should
transaction-independent messages be isolated to their own OPES
transport connections? If yes, should this be a MAY for OPES processor
to decide or a MUST that all processors will follow?

My suggestion is that it make it a MAY. I do not see a reason to
mandate presence of control-only (out-of-band) connections, but I can
see a reason to have such connections for priority message handling.

Thank you,

Alex.

---

From: Alex Rousskov <[email-masked]>
Date: Mon, 24 Feb 2003 11:37:10 -0700 (MST)

> controler     /provider      dispatcher     provider

My understanding is that the above is more-or-less allowed by OPES
architecture (as long as it can support OPES requirements). IIRC,
callout servers can delegate their functions and, hence, essentially
become "server dispatcher" on your diagram.

For a second, let's try to ignore what the terminology may or may not
imply. Can you point out specific desirable things that current OPES
architecture, OCP requirements, and predraft protocol do NOT allow you
to do? If you can name specific areas that cannot be supported by
OPES, we can discuss whether those areas should be supported (or
perhaps already are supported!).

Thank you,

Alex.

---

From: Alex Rousskov <[email-masked]>
Date: Mon, 24 Feb 2003 13:29:56 -0700 (MST)

On Sun, 23 Feb 2003, jfcm wrote:

> How do we zap and abort the transactions?

I assume you are talking about the situation where user request must
stop at the OPES processor or the callout server due to
rules/policies/etc.

The predraft document describes how a callout server can abort an
application transaction. The technique is based on changing the source
and destination addresses: the source address becomes the address of
the callout server. The destination address becomes the address of the
application client (it has to even point to the same application
connection if application protocol requires that). When OPES processor
receives a <message-start> OPES message with such addresses, it would
start responding to the original client request instead of forwarding
the request to the server.

As you know, Hilarie Orman (and probably others), objected to giving
callout servers ability to change source and destination addresses.
This objection would need to be resolved by either convincing
opponents that there are no good reasons to prohibit such a change OR
adding new OCP mechanisms to allow callout servers to tell OPES
processor:

	"you expected me to modify the request to be forwarded, but I
	am telling you to abort forwarding and respond with the
	following message instead"

Note that in the current proposal, the semantics is somewhat different
(abort is not an exceptional condition in the predraft):

	"you expected me to modify the application message; here is
	a modified version; forward it [back to the client]"

Does the above answer your question?

Thank you,

Alex.

---

From: "Oskar Batuner" <[email-masked]>
Date: Mon, 24 Feb 2003 16:53:57 -0500

> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]]On Behalf Of Alex Rousskov
> Sent: Monday, February 24, 2003 1:23 PM
> To: OPES WG
> Subject: RE: OPES protocol, pre-draft 01
>
>
>

[skip]

> 	  (implies end-of-data and end-of-message)

Here I'm looking at transaction boundaries that are defined by
<xaction-start> and <xaction-end>. You are right that in general
it is not necessary to limit the right to terminate transactions to
the server. But in the OPES case

> > ... processing is initiated by incoming
> > OPES processor messages (from producer or from consumer) ...

In such context OPES server has no apparent reason to terminate message
processing - there are no possible events associated with this transaction
coming from the OPES flow. After initial message is received and service
is initiated only service application (in this case callout server) knows
that transaction processing is finished. At least for the normal processing.
There may be abnormal situations (e.g. OPES data consumer has closed the
connection) when OPES processor becomes not interested in processing results
and may wish to stop request processing to avoid resource wasting.

Maybe we should have a special message for abnormal transaction termination?

> to decide or a MUST that all processors will follow?

In the architecture draft we are always separating OPES flow from
the metadata exchange. We state that metadata transfers are out-of-scope
for the OPES architecture. Of cause this statement does not preclude
using the same callout protocol - or any other - for the metadata
transfer. But if we permit mixing OPES data and metadata (like
preferences, etc.) on the same channel - well, this looks at least
inconsistent.

> see a reason to have such connections for priority message handling.

Oskar

---

From: Alex Rousskov <[email-masked]>
Date: Mon, 24 Feb 2003 15:54:17 -0700 (MST)

On Mon, 24 Feb 2003, Oskar Batuner wrote:

> In such context OPES server has no apparent reason to terminate
> message processing - there are no possible events associated with
> this transaction coming from the OPES flow. After initial message is
> received and service is initiated only service application (in this
> case callout server) knows that transaction processing is finished.
> At least for the normal processing. There may be abnormal situations
> (e.g. OPES data consumer has closed the connection) when OPES
> processor becomes not interested in processing results and may wish
> to stop request processing to avoid resource wasting.
>

Abnormal termination is exactly what motivated me to allow
<xaction-end> messages to be sent by OPES processor. We could have a
special <xaction-abnormal-end> message, but

	- the recipient (callout server) most likely does not
	  really care why the transaction is suddenly ended; the
	  reason field can be used for detailed logging/debugging
	  if desired

	- the reason field(s) can already be used to indicate
	  the kind of termination through application-specific
	  status codes (normal, client errors, internal errors,
	  overload, etc.)

	- there might be application protocols where "normal"
	  termination can be detected by OPES processor;
	  the notion of "normal termination" is, after all,
	  application [protocol] specific.

Overall, a more general <xaction-end> message seems to cause no
problems while having possibly desirable side-effects. Any objections?

> the metadata exchange. We state that metadata transfers are
> out-of-scope for the OPES architecture. Of cause this statement does
> not preclude using the same callout protocol - or any other - for
> the metadata transfer. But if we permit mixing OPES data and
> metadata (like preferences, etc.) on the same channel - well, this
> looks at least inconsistent.

I understand the intent -- keep OPES focused and small. Is there a
precise definition of "metadata" or "OPES flow" though? For example,
most will probably agree that service ID(s) should be attached to
<xaction-start> messages, but many will consider a service ID to be
METAdata because the application protocol does not carry those.

Another example: transferring a static set of preferences and rules
should probably be out of OPES protocol scope. On the other hand,
sending transaction-specific rules that affect, say, service
application order may be in OPES scope. Same for specific user
preferences and/or credentials? It seems to me that metadata specific
to application transaction may be in scope and may be transmitted
in-band.

Alex.

---

From: jfcm <[email-masked]>
Date: Wed, 26 Feb 2003 00:55:58 +0100

>stop at the OPES processor or the callout server due to

No. The question was about parallel command pipe or not.
My response was to try to see if we can have a consistent
response. Obviously triggering a command should probably
go the same way as aborting it.

The idea of a piped protocol being accepted, you must
zap the pipe if you want to abort. This means a priority
somewhere in the processus. Either in the pipe with a
zapper swallowing the data. Or a prioritized command
pipe ordering to clear the four buffers.

Then there is the message telling the server to abort and
and ack to the requester.

>the request to the server.

I am not sure I grasp all the implications in all the cases.
I need to make a .. model. And fully understand. The
real problem is also that I do not always figure out what
can be the osurce, the destination the processor, the
server, the client, the producer :-)

>         following message instead"

This is to abord forwarding. I was talking about command pipe,
hence abordting the service, not necessarilythe forwarding. This
is another option to add.

>         a modified version; forward it [back to the client]"

idem? However simpler.

The real issue is that at minimum there are four flows
involved user/dispatcher/service/dispatcher/organizer in my own words
instead of one. Following all the possiblities calls for an
Excell spreadshit.

jfc

---

From: Alex Rousskov <[email-masked]>
Date: Mon, 24 Feb 2003 21:30:59 -0700 (MST)

On Wed, 26 Feb 2003, jfcm wrote:

> No. The question was about parallel command pipe or not. My response
> was to try to see if we can have a consistent response. Obviously
> triggering a command should probably go the same way as aborting it.

If I understand the above, that is what the current predraft proposal
does. Triggering transaction processing on the other end uses the same
mechanism as aborting that processing (an OPES message with the
corresponding transaction ID xid).

> The idea of a piped protocol being accepted, you must zap the pipe
> if you want to abort. This means a priority somewhere in the
> processus. Either in the pipe with a zapper swallowing the data. Or
> a prioritized command pipe ordering to clear the four buffers.

Priority processing is not required to terminate a transaction.
Priority processing is an optimization -- it allows to terminate
transaction on the other side _faster_. How much faster? Depends on
how exactly one organizes priority queues and on how long those queues
are at the abortion time.

If you recall, the predraft protocol allows for out-of-order (i.e.,
prioritized) commands but does not document how the transport
connections are handled (yet). If a OPES processor opens more
connection to the callout server than there are concurrent application
messages, then a <xaction-end> message can be sent using "spare"
[control] connections, possibly triggering a faster abort.

I do not know what you mean by "zapper swallowing the data" in the
pipe. Can you clarify, please? Since we are talking about TCP-like
connections here, it may not be possible to "swallow" anything that
was sent. The only reliable option is to ignore messages with unknown
(e.g., already aborted) xid and/or terminate the corresponding
transport connection. Perhaps that's what you meant by swallowing?

> Then there is the message telling the server to abort and and ack to
> the requester.

An ACK would be a waste of resources unless you want to support OPES
transaction resumption after the event of transport-layer connectivity
loss. By "resumption", I mean resuming at the place of last
synchronization, transparently to the consumer and producer.

I doubt the benefits of transaction resumption outweigh increased
protocol complexity. It seems to me that for application protocols we
know about (HTTP and SMTP), an application transaction abort or an
automated retry would be more appropriate solutions. Do you think OPES
transactions should be resumable if the transport layer fails?

> The real issue is that at minimum there are four flows involved
> user/dispatcher/service/dispatcher/organizer in my own words instead
> of one. Following all the possiblities calls for an Excell
> spreadshit.

In such cases, a higher level simplification or abstraction often
works better than enumerating endless possibilities in a table (IMO).
The proposed OCP protocol is such an abstraction (at this point in
time) because it does not really care what the consumer, producer,
source, or destinations are. The number of flows is not important as
long as all flows are mostly independent and there is a well-known
"forking" or "initial scheduling" point (the OPES processor in our
case). The proposed protocol handles at least two flows per
application transaction and should scale well with the number of
flows.

Alex.

---

From: "Oskar Batuner" <[email-masked]>
Date: Tue, 25 Feb 2003 09:32:27 -0500

> Sent: Monday, February 24, 2003 5:54 PM
> To: OPES WG

Abnormal termination is different from normal termination (and
all other normal messages) in the sense that it switches context from
message processing to termination and cleanup. I think this difference
justifies presence of a separate message. In some sense it is a
question of preferences - as you have correctly pointed out
difference between normal and abnormal termination messages
can be determined by some additional reason code in the same
message. I prefer to keep an explicit difference. Another
difference is that abnormal termination message may arrive
at any moment, other messages are limited by the processing
context (protocol state machine).

> to application transaction may be in scope and may be transmitted

Yes, these is why I'm asking this question. We may go both ways.
In many cases it may be very natural to transfer in-band metadata related
to the current processing context. But as soon as you allow in-band metadata
transfer you may not set a clear border - what is specific to the transaction
and what is not. If we are allowed to send user preferences when message
to/from that user is processed - why not send to remember these preferences
on callout processor? And if callout processor keeps user's data - why not
to send it them just once, in advance?

Real decision is either yes (allow metadata transfer in-band) or no. Metadata
transfer in a limited context is not a consistent architectural decision: all
related protection measures mandated by the architecture MUST be implemented,
but if they are - why to limit the context?

Oskar

---

From: Alex Rousskov <[email-masked]>
Date: Tue, 25 Feb 2003 09:24:54 -0700 (MST)

On Tue, 25 Feb 2003, Oskar Batuner wrote:

> Abnormal termination is different from normal termination (and all
> other normal messages) in the sense that it switches context from
> message processing to termination and cleanup. I think this
> difference justifies presence of a separate message.

Both normal and abnormal termination switch callout server context
from message processing to termination and cleanup, don't they? I
agree that abnormal termination may have side-effects different from
normal termination. For example, if the callout server caches some
transaction information, it may want to purge the cache if the
transaction terminated abnormally.

> Another difference is that abnormal termination message may arrive
> at any moment, other messages are limited by the processing context
> (protocol state machine).

Good point. This distinction can be useful if an implementation wants
to ignore/log non-control messages sent on the control channel.

> In some sense it is a question of preferences - as you have
> correctly pointed out difference between normal and abnormal
> termination messages can be determined by some additional reason
> code in the same message. I prefer to keep an explicit difference.

You have convinced me that the difference may be big enough to deserve
explicit protocol support. However, since the implementation code
handling each message would be almost the same, it may be more
appropriate to indicate the difference via an explicit optional flag:

	<data-end xid amid [error] reason>
	<message-end xid amid [error] reason>
	<xaction-end xid [error] reason>

instead of having six very similar messages:

	<data-end-error xid amid reason>
	<message-end-error xid amid reason>
	<xaction-end-error xid reason>
	<data-end-ok xid amid reason>
	<message-end-ok xid amid reason>
	<xaction-end-ok xid reason>

The first approach provides explicit error notification while better
matching the code that is likely to be written to support the logic.
It also allows us to reuse the same "error" notification mechanism
with other message types if needed.

Do we have an agreement now? Would you still prefer the second
approach?

Thank you,

Alex.

---

From: Alex Rousskov <[email-masked]>
Date: Tue, 25 Feb 2003 09:57:56 -0700 (MST)

On Tue, 25 Feb 2003, Oskar Batuner wrote:

> Yes, these is why I'm asking this question. We may go both ways. In
> many cases it may be very natural to transfer in-band metadata
> related to the current processing context. But as soon as you allow
> in-band metadata transfer you may not set a clear border - what is
> specific to the transaction and what is not. If we are allowed to
> send user preferences when message to/from that user is processed -
> why not send to remember these preferences on callout processor? And
> if callout processor keeps user's data - why not to send it them
> just once, in advance?

We probably should not make all these choices because they are
application or even environment dependent. For example, if I have a
user population of 10,000 users but only 10 users can be active at a
time, it may not make sense to upload the entire user database to the
callout server, especially if the callout server [location] changes
with time. Some metadata transfers will always be out of OPES scope.
We should require support for only universally needed metathings (like
service IDs).

> Real decision is either yes (allow metadata transfer in-band) or no.
> Metadata transfer in a limited context is not a consistent
> architectural decision: all related protection measures mandated by
> the architecture MUST be implemented, but if they are - why to limit
> the context?

Exactly! IMO, there is no way we can avoid documenting/requiring
in-bound transmission of things other than application message octets.
The examples above illustrate that. Thus, metadata transfers must be
allowed (as a general architecture-level rule).

However, we should document/require metadata transmission on as-needed
basis. If later we notice a certain common pattern, then we may add a
general metadata transmission message/support to the protocol.
Otherwise, protocol extensions can be used for metadata transmissions
that we did not need or could not predict. Either way, it is OK to
violate the OPES architecture draft as long as our violations are
conscious and necessary. The architecture draft can be fixed later if
needed.

The original question was about requiring support for special
"control" channels. Those channels can be used for out-of-flow
transmission of "emergency" messages such as an abnormal transaction
end message. The same channel may [optionally] be used for metadata
transmissions, possibly unrelated to specific transactions. However,
since we do not have any specific need for such metadata transmissions,
I would not care about them for now. Perhaps those transmissions will
need their own channels; we will see.

For now, it seems like we should add support for emergency (priority)
channels (additional transport connections) as an optional
optimization feature. I will make a corresponding proposal and will
try to generalize the idea so that an OPES server can instruct callout
server to give certain transport connections a priority, to support
QoS features (e.g., application messages from Gold Members should be
processed before all other application messages).

Thank you,

Alex.

---

From: jfcm <[email-masked]>
Date: Thu, 27 Feb 2003 09:19:53 +0100

>         <xaction-end xid [error] reason>

Why not also just?
<zap xid [error] reason>

I just hit escape. We may have extensions of the protocol in the future 
needing to add other commands. I would hate to review all the code to add 
<linj-end xid [error] reason> etc. all the way through. jfc

---

From: jfcm <[email-masked]>
Date: Thu, 27 Feb 2003 09:12:27 +0100

>corresponding transaction ID xid).

Yes.  And this is what I underlined in asking that command and
abort be on the same pipe.

>[control] connections, possibly triggering a faster abort.

This is the issue. You can either think of a spare connection
or of a command connection. When you establish a relation
between servers you may decide that conenction 0 will be
the command pipe. I do prefer that approach as this is a
good pipe to signal the sever is still up and running. Also, in
my prefered wording, I would say that connection 0 would
go to the service administrator.

>transport connection. Perhaps that's what you meant by swallowing?

Yes and more.

Even under TCP/IP you transport data in blocks/messages
(IMHO due to TCP/IP there could be interest in multiplexed
datagrams to go faster and get easier acks). This means that
the non yet used data are in buffers and will be then processed.
When I say swallowing I mean that the buffers are cleaned (and
that the current processes are signaled to die). I cannot signal
processes but I can clean the I/O buffers (and on the path buffers
as the processes can be multi/machines).

Please recall that some processes can be quite CPU
consuming. Just ignoring the next protocol message
may result in a lot of unecessary finishing processing.

>synchronization, transparently to the consumer and producer.

This is a good idea (this puts a process on hold or to recover
from a bad situation). No, I meant that the interrupted service
acks that he died. Either when dying or through a service admin
obituary message.

The "waste of resources" comes from the trade-off. The whole
internet vision is based upon the TCP/IP trade-off you quote.
The acks do exist but they are implicit and come through
layer violations (the response of the application shows that
it received the data).

If you have a TCP/IP pipe you can use it as tansport for an
higher level block transport protocol and design a extremely
fast, reliable and secure value added virtual circuit switch
solution. Please remember that we are in very well
defined closed network. Please review my corrected draft.

I kept dispatcher/services/ends blocks as equal
because the command pipe issue has not been decided.
But, let assume that the ONES system is built only
of dispatchers and ends (and may be the ends also
with a dispatcher). This means that the "dispatcher"
is acting as a real time agent (like in X.400/e-mail but
real time).

The architecture for a virtual server would then be:

----> dispatcher -----> service admin
                        |----> service A occurence 1
                        |----> service A occurence 2
                        |----> service B occurence 1
                        |----> service B occurence 2
                        |----> service B occurence 3
                        | etc.

And that would be very near from the architecture I
introduced in my first mail about my prototype.

The inter-dispatcher protocol could be one of the
services and support soap, etc. under straight
TCP/IP or your OCP under a TCP/IP+ multiplexed
datagram solution with a behaviour and a response
time much closer of a message passing OS.

This could provide a very broad set of consistent
solutions?

>transactions should be resumable if the transport layer fails?

Let be clear. I discuss here what I name ONES of which
I think OPES is a fist level. I certainly do not think that
HTTP simple systems to replace a string in a page to
correct an URL or to change a .jpg into a .gif needs that.

But I think that an OPES may come to a point they have
to behave as a ONES because of the load or of the speed.
My site does not need much, but Amazone yes (we have
not discussed services running in parallel and the data
to keep from page to page to support a User cession?).

Then SOAP is useless. In talking about ONES, we know
which environement we might want to stay consistent with.
Also I do not see here a clear boarder defined yet.

IAB says HTTP. You add SMTP as I did. IAB says one
way, yet there seem to be some real support for both
ways. IAB does not say redirect and we are divided on
this.

Also, we may find that a more global vision may simlify
and make more robust the OPES. Or extend the field
of their immediate applictations.

Example. I wish to have a new vision of the DNS. I
send calls/mails using domain names and a 0.0.0.0
IP address that an OPES will replace with the proper
current IP of the DN savnig a lot of time: no more
first erroneous resolution and further redirect. And
a lot of flexibility in addressing.

>works better than enumerating endless possibilities in a table (IMO).

True. See above. But to make sure about it you must be
very basic first and see if the proposed abstraction resits
to the list of possible cases/conflicts. We fully agree.

>source, or destinations are.

True. This is why you should not embarass it with client/server
issues if it is bidi. And let consider seriously an inter dispatcher
network only. The disptacher as a smart plug everywhere,
including on the users side (what is a firewall?) where the minimum
access to a service is only a single (yes/no) rule (what is a
power switcher?).

>The number of flows is not important as
>case).

True. See above. But I prefer the word "dispatcher" from the dratf.
Because it is more generic and services are processed outside
of the dispatcher.

>The proposed protocol handles at least two flows per

Yes. Except that these two flows can be encapsulated
into a value added transport protocol for speed and
flow management. That way you may have cheap
and even virtual acks or service messages.

Please remember that we are real time. We cannot make
users wait because another user has big data to process.
We must permit to optimise dispatcher and services CPU
time. This certainly means priority assignement in transmitting
data under the protocol. Maximum flexibility can only come
from a service admin talking with the remote dispatcher. The
dispatcher must then act both as a link and as transport
dispatcher.

jfc

---

From: jfcm <[email-masked]>
Date: Thu, 27 Feb 2003 09:24:41 +0100

>processed before all other application messages).

This is a good point. But I would prefer the following wording and concept 
used/

1. command pipe (absolute priority) as a pipe 0. Establish the relation 
with the service admin in my model.
2. this prevents the confusion of the mata data wording which includes both 
command and ancillary information. I would treat ancillary as standard 
transactions with the service admin subject to the general prioritization 
service if any.
jfc

---

From: "Oskar Batuner" <[email-masked]>
Date: Wed, 26 Feb 2003 13:11:54 -0500

See below.

> Sent: Tuesday, February 25, 2003 11:25 AM
> To: OPES WG

I'd prefer something in between:

  <data-end xid amid [error] reason>
  <message-end xid amid [error] reason>
  <xaction-end xid reason>       -- issued by server,
                                    processing is done,
                                    "you know the results"

  <xaction-abort xid reason> -- issued by any party,
                                "just forget it",
                                results of previous processing 
                                can not be relied upon 
                                and should be discarded

Oskar

---

From: Alex Rousskov <[email-masked]>
Date: Wed, 26 Feb 2003 12:28:27 -0700 (MST)

>   <xaction-end xid reason>       -- issued by server,

Great! We have an agreement on the above messages. Note that processor
may need to send "normal" <xaction-end> messages as well: this is both
an application-dependent and OPES service dependent issue that we
probably should not restrict too much without good reasons.

>                                 results of previous processing

I agree with the definition. You prefer a specific error code for this
specific level of error. I prefer to keep it similar with the other
abnormal messages. Since processor may need to send "normal"
<xaction-end> messages too (see above), and since the final difference
between our approaches is minor (special message name versus a special
message flag), I would make the changes to have all <*-end> OPES
messages to use an optional "error" field for now and wait for more
objections/comments.

We have a semantics-level agreement. The remaining syntax-level
difference may even disappear once we start selecting specific
bindings and encodings for OPES.

Thank you,

Alex.

---

From: Alex Rousskov <[email-masked]>
Date: Wed, 26 Feb 2003 14:29:13 -0700 (MST)

> >         <xaction-end xid [error] reason>

What would be the meaning of the <zap> OPES message? The three old
messages above are based on three persistent "objects" or "states"
that OPES protocol has to maintain (data flow, application message, and
application transaction). See predraft for detailed definitions. What
does <zap> refer to? How is it different from, for example,
<xaction-end>?

> I just hit escape. We may have extensions of the protocol in the
> future needing to add other commands. I would hate to review all the
> code to add <linj-end xid [error] reason> etc. all the way through.

We cannot discuss or accommodate nonexistent extensions because we do
not even have an extension framework yet. What can be extended? How?
Without such a framework, I cannot (yet) say whether new <*-end>
messages may be needed.

"Hitting escape" in a user agent usually means the end of the
application transaction, which is already supported with the
<xaction-end> message. OPES is not about user actions or interfaces
though. It is about modifying application protocol messages.

In general, if OPES protocol deals with some kind of persistency (like
data flow or transaction state), there has to be a way to end that
persistency. In most cases, you want the flexibility of ending one
persistency without ending others that are external to it. For example,
ending an application message must not end transaction processing (in
general) because a transaction may include many semi-independent
application messages. That is why we have the three <*-end> messages
now.

If we discover that there are too many persistent "things" that need to
be terminated, we would have to redesign and/or, perhaps, introduce a
meta-message like the one below:

        <end what xid ... [error] reason>

Alex.

---

From: Alex Rousskov <[email-masked]>
Date: Wed, 26 Feb 2003 15:19:19 -0700 (MST)

> abort be on the same pipe.

The location of the message (i.e., the connection it is received on)
is a different story. To make aborts efficient, you may want to send
an abort message on a separate pipe that is not clogged with data
messages waiting to be processed (or to be discarded, actually).

> This is the issue. You can either think of a spare connection or of
> a command connection. When you establish a relation between servers
> you may decide that conenction 0 will be the command pipe. I do
> prefer that approach as this is a good pipe to signal the sever is
> still up and running. Also, in my prefered wording, I would say that
> connection 0 would go to the service administrator.

I think we can have a more general/flexible scheme that still allows
what you want: I would like OPES dispatcher to be able to give
connections (and possibly transactions/messages) optional priority. If
the callout server honors the priority setting, then we can have
high-priority "control" connection, for example. If the callout server
does not honor the priority setting or if there is no setting, then
all connections SHOULD be treated with the same priority. In the
latter case, a "control" connection is still possible to establish and
may still be "faster" if it has fewer messages.

It is relatively easy for modern implementations that manage multiple
connections to support rough connection priorities. It usually boils
down to polling a subset of all opened connections more often. Thus,
this optional and simple optimization is likely to be supported by
implementations that care about performance.

As an additional optimization, we could make it possible for either
side to establish a connection that only that side can send on. This
optimization will take care of the situations where a callout server
pollutes high-priority connection with statistics or other unimportant
messages.

It would be good to have a way for one side to reach the other side as
fast as possible, as long as both sides play by the rules. The
difficulty here is that the callout server is not allowed to open its
own connections and, hence, is at the dispatcher will as far as
connection optimization is concerned.

> Even under TCP/IP you transport data in blocks/messages (IMHO due to
> TCP/IP there could be interest in multiplexed datagrams to go faster
> and get easier acks). This means that the non yet used data are in
> buffers and will be then processed. When I say swallowing I mean
> that the buffers are cleaned (and that the current processes are
> signaled to die). I cannot signal processes but I can clean the I/O
> buffers (and on the path buffers as the processes can be
> multi/machines).

Clearing unprocessed (by application) TCP buffers might be possible
under a very limited set of conditions. For example, the buffers must
be known to contain no useful messages and no partial messages. The
application must have access to pretty low-level TCP controls. Also,
simply handling/keeping TCP packets is already a significant
performance hit and resource drain; discarding them without
application-level processing does not return back wasted CPU cycles.

Finally, TCP-level fiddling is possible if OPES protocol is
implemented directly on top of TCP, but may not be possible for
higher-level bindings.

> may result in a lot of unecessary finishing processing.

True. I do not think it is avoidable under general assumptions though.
See above. This is the price to pay for reliable message transmission
and other nice TCP properties. Fortunately, aborts should not happen
often. We optimize the common case, which is the right thing to do.

> If you have a TCP/IP pipe you can use it as tansport for an higher
> level block transport protocol and design a extremely fast, reliable
> and secure value added virtual circuit switch solution. Please
> remember that we are in very well defined closed network. Please
> review my corrected draft.
>
> I kept dispatcher/services/ends blocks as equal because the command
> pipe issue has not been decided. But, let assume that the ONES
> system is built only of dispatchers and ends (and may be the ends
> also with a dispatcher). This means that the "dispatcher" is acting
> as a real time agent (like in X.400/e-mail but real time).
>

And that is how OPES processor and callout server may be implemented!
A callout server may have an "admin" module that receives all messages
and directs them to appropriate services, doing load balancing if
desired.

> This could provide a very broad set of consistent

Since "this" is actually how OPES may be implemented, I am glad you
think that it could provide a very broad set of consistent solutions.

> Let be clear. I discuss here what I name ONES of which I think OPES
> is a fist level. I certainly do not think that HTTP simple systems
> to replace a string in a page to correct an URL or to change a .jpg
> into a .gif needs that.
>
> But I think that an OPES may come to a point they have to behave as
> a ONES because of the load or of the speed. My site does not need
> much, but Amazone yes (we have not discussed services running in
> parallel and the data to keep from page to page to support a User
> cession?).

You have to be more specific for me to follow your logic. What is it
exactly that Amazon needs and that OPES lacks?

> Then SOAP is useless. In talking about ONES, we know which
> environement we might want to stay consistent with. Also I do not
> see here a clear boarder defined yet.

I am not sure about SOAP, but the lack of clear boarder may be
attributed to the lack of understanding what ONES is. So far, I have
only seen abstract drawings that seem to match OPES architecture
pretty well. Perhaps it is just me, but I still do not know what ONES
is and how it is different from OPES.

> Also, we may find that a more global vision may simlify and make
> more robust the OPES. Or extend the field of their immediate
> applictations.

I agree. That is why I am trying hard to understand what ONES is and
what it has that OPES lacks.

> Example. I wish to have a new vision of the DNS. I send calls/mails
> using domain names and a 0.0.0.0 IP address that an OPES will
> replace with the proper current IP of the DN savnig a lot of time:
> no more first erroneous resolution and further redirect. And a lot
> of flexibility in addressing.

Now we are getting somewhere. An example! However, I am not sure I
fully understand the example: When I send e-mails, I use domain names,
not IP addresses. The name resolution happens when it is necessary to
find a mail handler for the named host. I am not sure how that can be
avoided. Can you clarify how the "new DNS" would differ from the
current one? How OPES would be better at resolving domain names than
current gethostbyname() system calls?

> ... But I prefer the word "dispatcher" from the dratf. Because it is
> more generic and services are processed outside of the dispatcher.

You are right. Most references to OPES processor in the current
protocol predraft should be replaced with OPES dispatcher, I guess.

> Yes. Except that these two flows can be encapsulated into a value
> added transport protocol for speed and flow management. That way you
> may have cheap and even virtual acks or service messages.
>
> Please remember that we are real time. We cannot make users wait
> because another user has big data to process. We must permit to
> optimise dispatcher and services CPU time. This certainly means
> priority assignement in transmitting data under the protocol.
> Maximum flexibility can only come from a service admin talking with
> the remote dispatcher. The dispatcher must then act both as a link
> and as transport dispatcher.

See above for a proposal on how to add priorities to OPES connections
(and, hence, possibly enable QoS discrimination you are talking
about). I hope it addresses your needs. If not, please specify what
"real time" needs are not addressed by the current scheme and the
above priority proposal.

Thank you,

Alex.

---

From: Alex Rousskov <[email-masked]>
Date: Wed, 26 Feb 2003 15:30:34 -0700 (MST)

On Thu, 27 Feb 2003, jfcm wrote:

> This is a good point. But I would prefer the following wording and
> concept used/
>
> 1. command pipe (absolute priority) as a pipe 0. Establish the
> relation with the service admin in my model.
>
> 2. this prevents the confusion of the mata data wording which
> includes both command and ancillary information. I would treat
> ancillary as standard transactions with the service admin subject to
> the general prioritization service if any.

My proposal allows creation of a connection with priority higher than
all other priorities. That connection can be used by OPES dispatcher
and service admin as the "command pipe" that you propose in item #1
above.

It would be up to OPES dispatcher what to send on that "command pipe".
Your implementation, for example, may avoid sending metadata
(auxiliary) messages on the command pipe if desired.

Do you agree that the priority-based scheme I proposed both addresses
your needs and is more general/flexible? Or does it lack some
necessary features?

Thank you,

Alex.

---

From: jfcm <[email-masked]>
Date: Wed, 26 Feb 2003 23:48:50 +0100

Alex,
we are talking about abort. And saving CPU time. And cleaning the mess 
possibly on may machines and pipes.

>does <zap> refer to? How is it different from, for example,

<zap> as I say would be for the three of them altoegether.
I want to get rid of everything about that transaction.
I do not want to consider anything left which may be ressource consuming or 
unclean anywhere.

>not even have an extension framework yet.

Well this will just introduce it :-). What I mean is that <zap> meaning
"to kill immediately whatever is related to the transaction", will also
kill whatever extension we may add in the future.

>What can be extended? How? Without such a framework, I cannot (yet) say 
>whether new <*-end>
>messages may be needed.

Right. But I do not want that. <zap> means that everything related to that 
transaction "never existed". You must understand that if you start 
considering messages and logger for every transaction of this protocol, in 
some kind of use, the protocol may be larger than the data, and disk access 
to take more of the machine time.

><xaction-end> message.

I said "also". Instead of your three messages, one zapper.

>OPES is not about user actions or interfaces

(1) it was an image. (2) that has direct implication on the protocol.

Let say that I call a web site through a translation OPES. If it is too 
long to wait or if text comes by parts, I want to be able to escape. This 
information is to hit the translation CPU asap. As a translation operator I 
want to save every CPU time I can to provide the fasted and smoothest 
service. And if it is paying not to be cheated.

>application messages. That is why we have the three <*-end> messages

Certainly. But here is not the case we discuss.

>         <end what xid ... [error] reason>

"what"?

BTW: we are in real-time multi-processor environement. If you want to keep 
track of all the "things" you may have to terminate at a given time without 
having any zombie, you may have to have a much more complex system.

Also, the servers must know they face an emergency situation because they 
may have to react differently from a normal termination. They do not expect 
any emergency in a standard <xaction end>.
jfc

---

From: Alex Rousskov <[email-masked]>
Date: Wed, 26 Feb 2003 16:43:25 -0700 (MST)

On Wed, 26 Feb 2003, jfcm wrote:

> <zap> as I say would be for the three of them altoegether. I want to
> get rid of everything about that transaction. I do not want to
> consider anything left which may be ressource consuming or unclean
> anywhere.

This is exactly what <xaction-end xid error reason> does.

> kill whatever extension we may add in the future.

This is exactly what <xaction-end xid error reason> does.

> I said "also". Instead of your three messages, one zapper.

<xaction-end> implies the other two, so only one message needs to be
sent, just like with <zap>. See examples in the predraft.

> Let say that I call a web site through a translation OPES. If it is
> too long to wait or if text comes by parts, I want to be able to
> escape. This information is to hit the translation CPU asap. As a
> translation operator I want to save every CPU time I can to provide
> the fasted and smoothest service. And if it is paying not to be
> cheated.

Looks like <xaction-end error> already supports this.

> Also, the servers must know they face an emergency situation because
> they may have to react differently from a normal termination. They
> do not expect any emergency in a standard <xaction end>.

That is exactly what Oskar Batuner pointed out after predraft 00 was
released. The problem is now fixed -- a special "error" flag has been
added to distinguish normal termination from an abort/emergency.
<xaction-end error ...> is not a "normal" termination.

Do you agree that <xaction-end error> is equivalent to <zap>? Or is
there more functionality in <zap> that <xaction-end error> is missing?

Thank you,

Alex.

---

From: jfcm <[email-masked]>
Date: Thu, 27 Feb 2003 14:08:22 +0100

At 00:43 27/02/03, Alex Rousskov wrote:
>there more functionality in <zap> that <xaction-end error> is missing

The more you add to <xaction-end error> the more you will make it looking 
it alike at a given time. But the more you add "error" the more you add 
possible error types confusion.

Let settles that <zap> means <xaction-end error nr "zap"> and let 
developpers use it. IMHO thei will result in <zap> being first tested and 
<action-end error nr "zap"> being the default in the action-end case, 
freely tested as per the developper gusto or priorities.

Also <xaction-end error> calls of reread/maintenance of the code when the 
protocol is enhanced and options added. <zap> not.

You will see that "nr zap" will complexify over time, while <zap> will stay 
an immediate clean and reset  command. Are you ok with that?

Anyway in my version there will be <zap> support :-)
Thank you.
jfc

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 27 Feb 2003 09:45:57 -0700 (MST)

jfc,

	It looks like you are still missing the point: There is
absolutely NO semantical difference between <xaction-end error> or
<xaction-end zap> or <zap>. Any compliant implementation will have
exactly the same code to handle any of the three messages. We cannot
argue about incompliant implementations, of course.

I already explained why I would prefer not to add [up to three] more
message types to OPES (which is what <zap> solution requires). We can
do that later if needed.

The remaining difference is only in visual formatting in the predraft
document: <xaction-end error> versus <xaction-end zap>. I am surprised
you want to spend time arguing about this difference, especially since
the OPES binding and message encoding may make the visual difference
disappear.

<philosophical-rant>
    If you insist, I can say that "zap" name is inferior to "error"
    because "zap" is an action and "error" is information. You may
    have noticed that the protocol is being designed so that each side
    provides information about its state. The other side will
    interpret that information according to protocol rules and its own
    state. Using action names like "zap" or "terminate" or
    "do-what-i-tell-you" is not the best approach here because it is
    too specific and may not match the other side capabilities or
    state.

    The simplest example is when the other side already zapped the
    transaction in question. It cannot zap it again, yet the message
    explicitly tells it to do so. From design point of view, this is
    bad because it creates an exceptional situation -- the side has
    been asked to do something that it cannot. Note that if the
    current design philosophy is followed, the problem does not exist
    -- the side is informed that there was a fatal error on the other
    side, and is free to take _any_ OPES-compliant action (which, in
    this case, is ignoring the message).
</philosophical-rant>

Alex.

P.S. I do not know what "nr" means in your
     <xaction-end error nr "zap"> message.

---

From: jfcm <[email-masked]>
Date: Thu, 27 Feb 2003 19:14:59 +0100

At 17:45 27/02/03, Alex Rousskov wrote:
>jfc,
>It looks like you are still missing the point:

I think you also does, due to my incapacity to explain it
in your own protocol words.

So, let stop this for the time being. We will keep it mind.

What I know is as a developper in my specific environment
I needed it and I suppose that in the project which takes
shape we will use it. But much to early.  Also you may be
right about it being ill construed. The development I refer
to is now 15/7 years old, initially very rustic and empirical.

I just know what I did with it :-)

<skip>
>     explicitly tells it to do so.

hmmmm. Sorry I lead you to a wrong understanding.

My use/understanding of zap is initial pipe oriented.
Clean the "thing" tree from that pipe.

Using 100 times zap is totally acceptable. Actually this is
the best way to synchronize. zap means swallow everything
related to this pipe, clean it. Nothing in there, what ever it was.
Clean buffers. Signal attached tasks to die propagate along
the pipes all over the network. Or the equivalent (I use the
image of my old QNX based system). May be I was wrong
in assuming that it corresponded to your commands.

Exemple of dispatcher rule:

    if (ball==zap)
    {
        send_zap(piper);
        clean_reset();
        exit();
    }

So for example zap 0 would kill the network and restart
the network establishment negociations, etc. Next to power
down it is the "reboot" of the network.

Is this clearer?
jfc

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 27 Feb 2003 12:14:36 -0700 (MST)

On Thu, 27 Feb 2003, jfcm wrote:

> My use/understanding of zap is initial pipe oriented. Clean the
> "thing" tree from that pipe.
>
> Using 100 times zap is totally acceptable. Actually this is the best
> way to synchronize. zap means swallow everything related to this
> pipe, clean it. Nothing in there, what ever it was. Clean buffers.
> Signal attached tasks to die propagate along the pipes all over the
> network. Or the equivalent (I use the image of my old QNX based
> system). May be I was wrong in assuming that it corresponded to your
> commands.

There are no connection-oriented commands in the predraft yet!

What you want is entirely different from a <xaction-end> message. You
want to kill every transaction that is associated with a given OPES
connection. You want <transactionS-end ...> message that has no xid
(because it applies to all transactions on the corresponding
connection).

<transactions-end> is an optimization. It is semantically equivalent
(almost) to sending multiple <transaction-end> messages, but is
smaller. I will add <transactions-end> to the TODO list. We need to
decide what to do with OPES connections in general before we can zap
them.

Moreover, you want to clear all unprocessed connection buffers which
should be a different command because it has a different semantics.
All implementations can support <transactions-end> message and only
very few (or none, depending on allowed OPES bindings and message
encodings) can support buffer flushing. For example, if OPES messages
are encoded in raw XML, then you may not be able to flush buffers
because you may create partial messages and may not be able to find a
beginning of the message after that.

IMO, this buffer-flushing command is too low-level for OPES. Simply
closing and re-opening the connection is probably more appropriate and
is not much more expensive. I suggest we wait for other opinions
before proceeding with this.

> So for example zap 0 would kill the network and restart the network
> establishment negociations, etc. Next to power down it is the
> "reboot" of the network.

This is even more than flushing buffers. You want to close and reopen
connections as well. I think this should wait until we add connection
management framework to the protocol. We need to decide whether and
how OPES connections can be reopened and what effect a suddenly closed
connection has on the other end.

Thank you,

Alex.