Subject: HTTP/OCP protocol binding

From: "Martin Stecher" <[email-masked]>
Date: Fri, 30 May 2003 12:32:43 +0200

Hi,

I think it is time to start thinking about application protocol bindings.
As we agreed, HTTP first.

This will help to validate OCP core and to detect some potential problems.
In addition we can then soon start to develop some prototypes that help verifying whether the procotol works.

Here are some first questions regarding HTTP/OCP:

- Transactions for HTTP requests/responses
How do OCP transactions look like and differ if they are used at activation points 1-2 and 3-4, i.e. OCP transactions for HTTP requests and responses; or REQMOD vs. RESPMOD for us ICAP guys.

- HTTP meta data
Will HTTP headers be simply the payload of a meta-have message?
Is the first line special? Will it be coded into named parameters of meta-have messages?
What about the empty line between HTTP header and body? Does it belong to the meta data?

- Message length and transfer encoding
How to handle HTTP message body in chunked transfer encoding? Remove the encoding before sending via OCP?
What is with the Content-Length header? Who is responsible for adding/changing/removing it?
Asynchronous OCP data handling and persistent HTTP/1.0 connections is not easy.

Do you have some ideas, comments, answers and additional questions?

Regards
Martin

---

From: Alex Rousskov <[email-masked]>
Date: Fri, 30 May 2003 10:10:31 -0600 (MDT)

On Fri, 30 May 2003, Martin Stecher wrote:

> I think it is time to start thinking about application protocol
> bindings. As we agreed, HTTP first.
>
> This will help to validate OCP core and to detect some potential
> problems. In addition we can then soon start to develop some
> prototypes that help verifying whether the procotol works.

I agree and will start putting an HTTP binding draft together unless
somebody else wants to take the lead. However, I suspect that we may
need a more general "Common OCP data encodings" draft (see below) as
well. Or, should common encodings become a part of OCP Core?

> How do OCP transactions look like and differ if they are used at
> activation points 1-2 and 3-4, i.e. OCP transactions for HTTP
> requests and responses; or REQMOD vs. RESPMOD for us ICAP guys.

OCP Core transactions will look identical regardless of the activation
points because activation points are application-specific things
beyond OCP Core scope. If activation point is important, it has to be
passed as metadata or as an [HTTP- or cache-specific] extension OCP
message/field. HTTP binding draft will have to document how this is
done, I guess. An extension field to app-message-start message seems
most appropriate to me:

	app-message-start ...
	...
	Activation-Point: request

or
	app-message-start ...
	...
	Activation-Point: response

What do you think?

> - HTTP meta data
>
> Will HTTP headers be simply the payload of a meta-have message? Is
> the first line special? Will it be coded into named parameters of
> meta-have messages? What about the empty line between HTTP header
> and body? Does it belong to the meta data?

I hope there will be no meta-have messages (see my posting titled "OCP
metadata == data"). I would use some very simple encoding to pass HTTP
messages in OCP payload. Something along these lines:

	<chunk-type> <chunk-length> <chunk-data>

where "chunk-type" can be either of
	"headers":  HTTP headers including the first line and the last
	            CRLF terminator
	"trailers": HTTP trailers including the last CRLF terminator
	"body":     raw HTTP message payload
	"all":      raw HTTP message
and one OCP payload may contain several chunks in the above format.

Each type can be transmitted via several data-have messages, of
course. There will be some ordering enforced. For example, one cannot
send "headers" after "body".

Also, if an OPES processor sends a particular chunk type to the
callout server, the callout server must send adapted chunk back using
the same chunk type or "all" type if appropriate. Similarly, if an
OPES processor does not send a particular chunk type to the callout
server, the chunks of that type cannot be adapted/returned.

I am not sure about the first line. Should it be separated? It may be
a good idea to separate it if we want to reuse the other parts for
other protocols that do not have a special first line (e.g., SMTP).

> the encoding before sending via OCP?

I think this should be left to implementations to decide. The
recipient MUST handle all valid HTTP encodings, but it is up to the
sender how to pre-process the message. Recall that, from OCP point of
view, any kind of preprocessing is out of scope.

OCP agents can negotiate more specific requirements, I guess. For
example, they can negotiate that chunked encoding is always used or
that identity encoding is used.

> adding/changing/removing it?

The sender of the header. For example:

	- If a callout server sends "headers" or "all" chunks
	  back to the OPES processor, then that callout server
	  must ensure that all headers it sends match the body it
	  sends. That includes adjusting or removing original
	  Content-Length as needed. The OPES processor may
	  further adapt the body and Content-Length, of course.

	- if an OPES processor sends just the "body" chunk to
	  the callout server, then the OPES processor is responsible
	  for matching the headers with the adapted body. This
	  mode lets us implement services that are not HTTP-dependent.
	  Note that data encoding will have to be negotiated in this
	  case.

> Asynchronous OCP data handling and persistent HTTP/1.0 connections

I do not see any complications. Note that we are not adapting
connections, only messages. Can you give an example where OCP and HTTP
persistency make things difficult?

One of the trickiest parts, IMO, is to document the above encoding and
communication rules so that they can be reused by other, similar
protocols, without losing efficiency. For example, the
"headers/body/trailers" structure is probably common to many
applications though specific encodings for each part will differ. We
need to find a way to document "general" or "common" application
bindings.

Alex.