Subject: Comments on ocp-00

From: "Reinaldo Penno" <[email-masked]>
Date: Wed, 2 Apr 2003 08:33:37 -0800

1)

" application message: A sequence of octets that OPES processor
      designates for callout service processing.  Usually, an
      application message is the basic unit of application protocol
      communication, as defined by that application protocol (e.g.,
      HTTP/1.1 message).  Before OCP is applied, OPES processor may
      pre-process raw application messages to extract and manipulate
      well-known parts (e.g., HTTP message headers or SMTP message body)
      in order to subject just those parts to callout services. For the
      purpose of OCP, the result of such preprocessing is an application
      message. Naturally, OPES processor and callout services it is
      configured to use must agree on what application messages are
      acceptable. Establishing such an agreement is beyond OCP scope."

Everything after "Before OCP..." has nothing to do with the definition of
application message anymore. This has to do with the workings of the
protocol. 

Although I suggest to rip out the last part of this definition I should say
that I disagree (if I understood correctly) with your statement
"Establishing such an agreement is beyond OCP scope". Capability negotiation
can help you establish what application messages are acceptable on each
connection or callout server as a whole.

Suggested version.

" application message: A sequence of octets that OPES processor
      designates for callout service processing.  Usually, an
      application message is the basic unit of application protocol
      communication, as defined by that application protocol (e.g.,
      HTTP/1.1 message).  

2) 

"A callout server may send this data back to
   the OPES processor, with or without modifications."

Suggestion

"A callout server may send or not this data back to the OPES processor. When
the callout server sends data back to the OPES processor, it can be modified
or not."

3)

"The primary purpose of OCP communications is modification of
   application message payloads..."

Do not want to seem picky, but the primary purpose of OCP is to ship packets
back and forth between Callout Server and OPES Processor. Whatever is done
with message has nothing to do with OCP anymore. It is a function totally
contained within the callout server.

I guess you can say

"The primary purpose of the OCP is to send data to the callout server where
it will be examined and possibly changed"

4)

"   OCP transaction is a logical sequence of OCP messages processing a
   single original application message. The result of the processing may
   be zero or more application messages, adapted from the original. A
   typical transaction consists of two message flows: a flow from the
   CLIENT to the SERVER (sending original application message) and a
   flow from the SERVER to the CLIENT (sending adapted application
   messages). The number of application messages produced by the SERVER
   and whether the SERVER actually modifies original application message
   depends on the requested callout service. The CLIENT or the SERVER
   can terminate the transaction by sending a corresponding message to
   the other side."

Hummm...I guess I'm not confortable with the idea of mixing the OCP
semantics per se and whatever it carries..The problem with this approach is
that we will start asking things like "what is the original application
message?". "Is the the whole HTTP 1.1 message before TCP fragmentation or
after?" , "And if is protocol X..and Y", etc, etc. 

This is my (lean) suggestion using your definitions:

"  An OCP transaction is a logical exchange of OCP messages.

   A OCP transaction starts with a xaction-start message, followed 
   by zero or more data-xxx messages, zero or more appl-xxx and ends 
   with a xaction-end message. "

5) 

"xid: OCP transaction identifier.  Uniquely identifies an OCP
      transaction originated by a given CLIENT. Since each transaction
      deals with a single application message, "xid" also identifies
      that application message."

As you can guess, IMHO the part ""xid" also identifies that application
message." is troublesome. 

My suggestion

"xid: OCP transaction identifier.  Uniquely identifies an OCP
      transaction originated by a given CLIENT."

6)

"am-id: Application message identifier.  Uniquely identifies an
      application message within an OCP transaction."

I guess it is one more reason to make the modification I mention in number
5) above.

7)

"   am-source: Information about the source of an application message.
      For example, HTTP client or origin server IP address and TCP
      connection ID.

   am-destination: Information about the destination of an application
      message. For example, HTTP client or origin server address."

I'm not sure I follow this...TCP connection ID? Information about the source
of an application message?

8)

"size: Application data size in octets. The value either applies to
      the data attached to an OCP message or suggests data size to be
      sent in response to an OCP message."

Is this the original value (which I guess before reaseembly it cannot be
known)? The value of the OCP payload? 

9)

"modified: A boolean parameter indicating that the attached
      application data has been modified by he SERVER.  Only SERVER may
      send this flag. This parameter can be used with any OCP message
      that has am-id parameter."

Suggestion

"modified: A boolean parameter indicating that the attached
      application data has been modified by the SERVER. Only the SERVER may
      send this flag. This parameter can be used with any OCP message
      that has am-id parameter."

I guess the caveat is that if you but the flag in one OCP message containing
am-id, you have to put in all of them for the same am-id.

10)

"copied: A flag indicating that a copy of the attached application
      data is being kept at the CLIENT. Only the CLIENT may send this
      flag. This parameter can be used with any OCP message that may
      carry application message data. (XXX: it is yet unclear when
      CLIENT commitment to preserve the data may end.)"

Same caveat as above.

11)

"sizep: Remaining application data size prediction in octets. The
      value excludes data in the current OCP message, if any. The
      prediction applies to a single application message. This parameter
      can be used with any OCP message that has am-id parameter.

   modp: Future data modification prediction in percents. A modp value
      of 0 (zero) means the sender predicts that there will be no data
      modifications. A value of 100 means the sender is predicts that
      there will be data modifications.  The value excludes data in the
      current OCP message, if any.  The prediction applies to a single
      application message. This parameter can be used with any OCP
      message that has am-id parameter."

I guess we need to hash this out in much for detail...

12) 

"error: A flag indicating abnormal conditions at the sender that
      cannot be expressed via result parameter. "

Such as? I guess maybe we could maintain the HTTP/SIP approach where we have
status code and reason phrases for everything.

"It is RECOMMENDED that
      the recipient deletes all state associated with the corresponding
      OCP message. For example, if the message is 'app-message-end' and
      the application message contained user credentials, such
      credentials should be deleted."

My suggestion

""It is RECOMMENDED that
      the recipient deletes all state associated with the corresponding
      OCP message."

13)

"feature: A OCP feature identifier with optional feature parameters.
      Used to declare support and negotiate use of OCP optional features
      (e.g., copying of data chunks at the CLIENT)."

I guess this is another one we need to iron out...

14)

"If a malformed message is received, the recipient MUST terminate
   processing of the corresponding OCP connection using 'connection-end'
   message with an error flag"

Why not send a status code and reason phrase instead of this error flag?

Just say "400 Bad Request"...or "400 Malformed Message"

15)

"This is the only OCP message that may carry application data. There
   MUST NOT be any gaps in data supplied by data-have and data-as-is
   messages (i.e., the offset of the next data message must be equal to
   the offset+size of the previous data message) (XXX: we do not need
   offset then; should we keep it as a validation mechanism?) (XXX:
   document what to do when this MUST is violated).  Zero size is
   permitted and is useful for communicating predictions without sending
   data."

Well, although we must not have gaps, we might loose packets...we need to be
able to recover and not throw all the OCP messages received so far away...

	`
That's it for now..

I will send more comments later.

Regards,

Reinaldo

---

From: Alex Rousskov <[email-masked]>
Date: Wed, 2 Apr 2003 13:51:56 -0700 (MST)

> application message anymore. This has to do with the workings of the

Agreed. The second part is an explanation/motivation for the
not-so-obvious definition. Do you think it is worth keeping, by moving
it elsewhere? Or should we completely get rid of that prose?

> Although I suggest to rip out the last part of this definition I
> should say that I disagree (if I understood correctly) with your
> statement "Establishing such an agreement is beyond OCP scope".
> Capability negotiation can help you establish what application
> messages are acceptable on each connection or callout server as a
> whole.

I agree that it can help. I hope we can keep things simple (no
negotiation) in this context without loosing too much functionality or
interoperability. I suspect that callout implementation abilities
will be well known to those who add a callout server/service to an
OPES intermediary. These administrators would be able to configure
the system appropriately.

IMO, there are two cases where auto-negotiation is important:

	- dynamic config changes: a new service is automatically
	  added to the list of services available to an OPES
	  processor;

	- dynamic ability changes: the same service suddenly can no
	  longer handle complete HTTP messages and demands just
	  HTTP message bodies in certain encoding.

Both cases seem unusual and not worth supporting. Do you see other use
cases for auto-negotiation? Or do you think the above two cases are
worth supporting in OCP?

>       communication, as defined by that application protocol (e.g.,

Agreed. Please let me know whether you think we should still keep the
explanation/motivation part elsewhere in the specs.

> the callout server sends data back to the OPES processor, it can be modified

I think "may" implies "may not", but I will change to be explicit.

> it will be examined and possibly changed"

OK.

>    with a xaction-end message. "

Here I disagree. The whole purpose of having transactions is to group
together processing of a single application message (which is a
defined term). That grouping is not strictly necessary but it brings
structure and is actually required by the OCP requirements draft (see
the "3.3 Callout Transactions" section).

If we adopt your wording, there is no need to have a notion of OCP
transaction at all, right? Every app-message message from the callout
server would simply have to have an extra ID pointing to the original
application message it modifies (so that the OPES processor can put
things back together and forward the message as appropriate).

I agree that "original" qualifier should be removed, changed, or
defined.

As for "And if is protocol X..and Y", it is a valid question that we
will need to address, but it does not seem to be related to the
definition of an OCP transaction?

>       transaction originated by a given CLIENT."

OK.

> I guess it is one more reason to make the modification I mention in number

I am not sure I follow. Am-id exists so that a callout server can
modify one [original] application message into more than one [adapted]
application messages. Since there could be more than one adapted
message, we need am-id to be able to interleave their data and to
abort them.

> of an application message?

Callout server actions may depend on meta-information that is not
available in the message itself. For example, a callout service may
insert user-specific pieces of HTML but user authentication may happen
outside of the current message scope and, hence, user ID would need to
be relayed by OPES processor to the callout server (e.g., via the
destination parameter).

However, I consider current "am-source" and "am-destination" approach
inadequate because meta-information can include more than source and
destination descriptions. I will propose and argue for a more general
replacement shortly. Let's ignore this broken part for now.

> known)? The value of the OCP payload?

If it is the size of the data attached to an OCP message, then it is
OCP payload (e.g., "here are the next 100 bytes"). If it is a
suggestion (e.g., "give me 100 more bytes"), then it is not OCP
payload. See data-have and data-need messages. Note that "data" is a
defined term.

The "size" attribute has little to do with application message size.
It is about the size of an application message chunk being sent or
chunks being requested. The value of "size" would be equal to the
transfer-length of the application message only if the entire
application message is transmitted via one data-have OCP message.

> I guess the caveat is that if you but the flag in one OCP message
> containing am-id, you have to put in all of them for the same am-id.

Actually, not. The granularity may be one level lower. The SERVER
tells the CLIENT that a particular message data _chunk_ got modified.
This implies that the entire message got modified, but provides more
information and does not require repeating the same flag in subsequent
data-have messages.

Of course, if the flag is used with, say, app-message-start message,
then it applies to the entire application message without [yet] saying
which chunk got modified.

>       CLIENT commitment to preserve the data may end.)"

Same answer as above. Here, the chunk-level granularity is even more
important. See the long e-mail thread about copying commitment.
Copying a data chunk is relatively easy and painless. Commitment to
copy the entire application message is a very different animal.

Perhaps the "data" definition must be modified to stress the level of
granularity here?

> Such as? I guess maybe we could maintain the HTTP/SIP approach where
> we have status code and reason phrases for everything.

We could. Some argue that abnormal termination must be signaled more
explicitly. See messages from Oskar Batuner related to the
xaction-abort issue.

>       the recipient deletes all state associated with the corresponding

Do you mean deleting the example? Examples are not normative; do you
think the presence of an example hurts in this case?

> I guess this is another one we need to iron out...

Yes, definitely.

> Just say "400 Bad Request"...or "400 Malformed Message"

As Oskar argued, the semantics of "400 Malformed Message" may be
different from "400 Malformed Message, and the error on this side was
so bad that you probably want to delete all state associated with this
transaction even if that state looks valid to you". We could have a
special code for the latter, but we would need it for every error
response;  that is why Oskar proposed special *-abort messages and I
suggested a special "error" flag.

"Raw" HTTP ignores this problem because it does not document handling
of message-independent state (e.g., authentication information).

> Well, although we must not have gaps, we might loose packets...we
> need to be able to recover and not throw all the OCP messages
> received so far away...

I am under impression that section 3.1 of
draft-ietf-opes-protocol-reqs-03.txt prohibits OCP packet loss. Are
you talking about application protocol packet loss? If so, do you
really want to inform the SERVER of lost data on OCP level? Or should
we assume that if application protocol is lossy, the callout server
would wither know how to detect/handle the losses or will not care? If
possible, I would like data size and offset fields to represent
received application message octets, nothing else (because it keeps
things simple and application-agnostic).

Thanks a lot for all your comments!

Alex.