Subject: HTTP as OCP transport, ICAP/2.0

From: Alex Rousskov <[email-masked]>
Date: Tue, 22 Apr 2003 15:22:04 -0600 (MDT)

Here are the reasons to use HTTP as OCP transport:

	+ ICAP uses HTTP as transport; we can make
	  OCP look like ICAP/2.0, which may
	  significantly improve OCP survival/adoption
	  chances

	+ HTTP applications are our primary target;
	  it would be easier to add OCP support to
	  HTTP intermediaries (OPES processors) if
	  OCP transport is also HTTP

	+ it is likely that many existing web services already
	  support HTTP (e.g., as a part of SOAP over HTTP);
	  it would be easier to add OCP support to
	  such services (OPES callout servers) if
	  OCP transport is also HTTP

	+ HTTP is fairly well-understood protocol as
	  far as basic transport functionality is
	  concerned

Here are the reasons to avoid HTTP as OCP transport:

	- HTTP makes it difficult to support a pipeline of OCP
	  messages without converting every message into
	  a request/response pair; this limits OCP flexibility
	  and/or performance

	- ICAP uses HTTP as transport; if we try to make OCP
	  to look like ICAP/2.0, we may have to sacrifice
	  some (many?) OCP properties and clean design to
	  retain some form of backward compatibility; for example,
	  OCP metadata should be distinct from HTTP metadata,
	  which may be difficult to achieve while
	  preserving compatibility with ICAP/1.1.

	- HTTP lacks appropriate connection management
	  primitives because persistent connections were
	  added on top of an existing protocol (HTTP/1.0)
	  and are just a performance optimization; HTTP is
	  not really design for long-lived connections many
	  OCP agents would enjoy

	- HTTP lacks appropriate native support for security
	  and authentication; compared to SOAP and BEEP,
	  little reuse of other protocols/mechanisms is
	  possible in this context

	- HTTP specification (RFC 2616) is a big, complicated
	  document; HTTP comes with many features that OCP
	  probably does not need

Did I miss anything important? ICAP/OPES issues are also discussed, in
part, in the draft-stecher-opes-icap-eval-00 document:
http://www.faqs.org/ftp/internet-drafts/draft-stecher-opes-icap-eval-00.txt

IMO, the primary reason to use HTTP as OCP transport would be
"compatibility" with ICAP. I wonder if ICAP proponents would consider
it possible for ICAP/2.0 or ICAP-NG to use something other than HTTP
as transport? Or will selection of other transports kill the
opportunity to merge ICAP/2.0 and OCP?

What do you think? I am especially interested in comments from Martin
Stecher and anybody else with first-hand ICAP experience.

Thanks,

Alex.

---

From: "The Purple Streak, Hilarie Orman" <[email-masked]>
Date: Tue, 22 Apr 2003 17:16:39 -0600

I thought BEEP's security was the same as HTTP's: use TLS.

Hilarie

---

From: Alex Rousskov <[email-masked]>
Date: Tue, 22 Apr 2003 17:34:44 -0600 (MDT)

> I thought BEEP's security was the same as HTTP's: use TLS.

My understanding is that BEEP can _negotiate_ security profile for a
BEEP session (so that we do not have to worry about that at all).
Please correct me if I am wrong.

I know that HTTP has no idea it is being (or can be) wrapped with TLS.
Thus, we, as OCP writers, would have to worry about negotiating and
then enabling TLS.

Alex.

---

From: Markus Hofmann <[email-masked]>
Date: Tue, 22 Apr 2003 22:23:57 -0400

> 	  significantly improve OCP survival/adoption

Hm, does ICAP really use HTTP as transport, i.e. run on top of HTTP? 
I'd say ICAP is a protocol that looks very much like HTTP and uses TCP 
as the transport. Indeed, RFC 3507 states "ICAP uses TCP/IP as a 
transport protocol.  The default port is 1344, but other ports may be 
used.".

In the early stages of ICAP, it was supposed to run over HTTP, but 
this was dropped at some later point.

> 	  far as basic transport functionality is

An additional pro might be firewall traversal, although this might not 
be that big of an issue in deployment scenarios OCP will play a role (?).

> 	  document; HTTP comes with many features that OCP

The kind of "streaming capabilities" of HTTP (i.e. chunking) is kind 
of limited.

If running over HTTP, would we expect problems with HTTP 
intermediaries sitting in-between the OPES processor and the callout 
server (for example if OPES processor and callout server ane in 
different adminsitrative domains).

-Markus

---

From: Alex Rousskov <[email-masked]>
Date: Tue, 22 Apr 2003 23:12:57 -0600 (MDT)

On Tue, 22 Apr 2003, Markus Hofmann wrote:

> I'd say ICAP is a protocol that looks very much like HTTP and uses
> TCP as the transport. Indeed, RFC 3507 states "ICAP uses TCP/IP as a
> transport protocol.  The default port is 1344, but other ports may
> be used.".

Minor terminology conflict, I believe. When I said "HTTP transport", I
meant what you mean above. Since HTTP is so extensible, it is
difficult to say what is "transport" and what is "looks very much
like". If we use HTTP as (the only) OCP transport we are likely to do
what ICAP folks had to do or something similar to that.

> An additional pro might be firewall traversal, although this might
> not be that big of an issue in deployment scenarios OCP will play a
> role (?).

Yes, though it becomes even less of an issue with firewalls becoming
smarter about the traffic they block (moving from L2-4 to L7).
Moreover, OCP servers, like ICAP servers, are not likely to run on
port 80 by default even if HTTP transport/model is used.

> The kind of "streaming capabilities" of HTTP (i.e. chunking) is kind

Yes, we would not be able to use chunking "as is" without giving up
current OCP flexibility to mix data chunks and control commands on the
same TCP connection.

> different adminsitrative domains).

This question is on the to-do list: do we want OCP to be proxy-able?
We can probably go either way, especially if OCP is based on HTTP. At
this time, I see no reason to allow/document OCP proxies explicitly,
but once the transport protocol is selected, we must revisit the
issue.

I've never heard of BEEP proxies. Do they exist?

What about SOAP proxies? Are they similar to (or even the same as) XML
switches?

Alex.

---

From: "The Purple Streak, Hilarie Orman" <[email-masked]>
Date: Wed, 23 Apr 2003 00:25:13 -0600

"Transport" in the classicial meaning is layer 4.  However, it's
possible for a protocol, like ICAP, to use a layer 5 protocol as its
transport.  That is, to fit entirely within the semantics of layer 5
and to introduce its own further semantics.  I think that is the best
way to view ICAP, from an architectural viewpoint.  The deviations from
HTTP semantics are trivial.  The intent was to have something that ran
on devices that already understood HTTP, so very little additional code
would be writtent to support ICAP.

Most multimedia streaming protocols run over HTTP fairly well.

We've stated several times that the OPES processor and callout server
have to be in the same administrative domain for security reasons.
But, let's look at the case where they aren't, just for fun.  Suppose
someone put an HTTP intermediary in their path, as you suggest, and
suppose it is an OPES process and it does HTTP content modification,
wouldn't that be a mess?

Even if the callout protocol is something else (SE), there might be an
OPES box doing SE modification in the path.

So, I'd think the callout protocol MUST carry a transport header with
"no OPES" in order to control what could turn into an uncontrollable
routing problem.

Hilarie

---

From: John Martin <[email-masked]>
Date: Wed, 23 Apr 2003 10:15:04 -0400

Hilarie is correct in that the aim was to make it friendly to HTTP 
developers and apps with existing HTTP interfaces rather than strict HTTP 
compliance.

If I recall correctly, the main reason that ICAP diverged from HTTP was 
that ICAP required chunking. Leaving chunking as optional made icap-enabled 
applications that dealt with the transfer of large amounts of data slow and 
inefficient. In particular virus scanning apps wanted a way to ack a small 
piece of data and tell the icap client to send more or not.

Originally, ICAP was defined as an HTTP vectoring protocol or "HTTP over 
HTTP" - a way of encapsulating one complete HTTP message within another 
HTTP message. It still looks and smells a lot like HTTP.

John

>"no OPES" in order to control what could turn into an uncontrollable

---------------------------------------------------------------
Please note new phone number: +1 347 613 2259
---------------------------------------------------------------

---

From: Marshall Rose <[email-masked]>
Date: Wed, 23 Apr 2003 07:54:55 -0700

[ again, not speaking as a co-chair... ]

> I thought BEEP's security was the same as HTTP's: use TLS.

no. you can negotiate the use of tls with beep. (in contrast, with most http
implementations, it's not a negotiation, it's "just there" or not, depending on
the port you open.)

in addition, tls uses sasl which can complement, or replace tls' functionality.
for example, want to authenticate/privatize using kerberos? no problem,
negotiate the use of the kerberos mechanism in sasl.

/mtr

---

From: Alex Rousskov <[email-masked]>
Date: Wed, 23 Apr 2003 09:24:34 -0600 (MDT)

On Wed, 23 Apr 2003, The Purple Streak, Hilarie Orman wrote:

> We've stated several times that the OPES processor and callout
> server have to be in the same administrative domain for security
> reasons.

I hope the above is a typo. Perhaps you meant "same trust domain"?
Requiring same administrative domain is totally unnecessary, even for
security reasons.

> But, let's look at the case where they aren't, just for fun.
> Suppose someone put an HTTP intermediary in their path, as you
> suggest, and suppose it is an OPES process and it does HTTP content
> modification, wouldn't that be a mess?

I am not sure what "as you suggest" refers to (you did not quote the
message you are responding to). If my ISP or an ISP further upstream
installed a TCP hijacking HTTP proxy and did not tell me about it
(explicitly or via a smallprint in a signed contract), then there is a
trust problem. If the installed HTTP proxy is OPES-compliant and my
browser is OPES-aware, I may be able to notice/debug it.

> Even if the callout protocol is something else (SE), there might be
> an OPES box doing SE modification in the path.

Yes, the specific callout mechanism is irrelevant here.

> So, I'd think the callout protocol MUST carry a transport header
> with "no OPES" in order to control what could turn into an
> uncontrollable routing problem.

I do not understand the above. Can you elaborate? IMO, the callout
protocol MUST NOT be involved if an OPES processor was instructed to
bypass OPES transformations (that instruction may come via the
application protocol being proxied). If callout protocol is involved,
we are not bypassing. If OPES processor cannot honor a bypass
instruction, then it is either incompliant or has to talk to an OPES
processor (and not just a callout server). Either case, is out of OCP
scope.

Alex.

---

From: "Oskar Batuner" <[email-masked]>
Date: Wed, 23 Apr 2003 14:49:45 -0400

> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]]On Behalf Of The Purple
> Streak, Hilarie Orman
> Sent: Wednesday, April 23, 2003 2:25 AM
> To: [email-masked]
> Cc: [email-masked]
> Subject: Re: HTTP as OCP transport, ICAP/2.0
> 

> "no OPES" in order to control what could turn into an uncontrollable

Hilarie, I do not quite understand your fears. An OPES box is (should be) 
a legitimate building block for network application architecture. If rules 
and protocols are designed correctly they should not require additional 
restrictions of this kind. Moreover, imposing such specific restrictions 
looks like introducing per-case patches. Unless there is a specific 
scenario that result in problem we should avoid patching.

Oskar