Subject: HTTP/OCP: All Content-* headers are special

From: Alex Rousskov <[email-masked]>
Date: Wed, 15 Oct 2003 16:40:06 -0600 (MDT)

Martin,

	Could you please check that HTTP Adaptations draft talks about
OPES processor responsibility for "correctness" of not just
Content-Length header but all(?) Content- headers including things
like Content-MD5? Perhaps we can include a general statement that
covers all headers?

	HTTP OPES processors MUST insure correctness of
	all HTTP headers documented in specifications
	that the processors intend to be compliant with.
	For example, the correctness of Content-Length
	and Content-MD5 headers have to be insured by
	processors claiming compliance with HTTP/1.1
	(RFC 2616).

Or would that be an overkill?

Motivation: We are running some ICAP tests here, and Polygraph is
detecting MD5 checksum errors on messages that went through an HTTP
proxy that uses an ICAP server. Apparently, neither the proxy nor the
ICAP server are adjusting Content-MD5 value when adapting content. I
checked ICAP RFC and did not find references to Content-MD5, only
Content-Length. I suspect an implementor may overlook the dependency
between adapted content and HTTP/MIME checksums if we are not being
more explicit about such dependencies.

It would still be up to the processor how to "insure correctness". As
discussed earlier, a given processor may have enough reasons to trust
a given service to adjust all headers accordingly. The processor is
judged by the final outcome only, not by the mechanism used to achieve
that outcome.

Sizep tricks will make Content-Length adjustments at the processor
cheap. I am not sure that Content-MD5 adjustments at the processor can
be cheap, but I think we have to document what the processor has to do
to avoid end-to-end checksum errors. Perhaps the only realistic
default for a processor would be to remove Content-MD5 header: absence
of truth is better than lies.

Alex.

---

From: "The Purple Streak, Hilarie Orman" <[email-masked]>
Date: Wed, 15 Oct 2003 16:51:57 -0600

This is the tip of the integrity iceberg.  If the point of the
Content-MD5 header is to ensure end-to-end integrity, then it cannot
be modified by intermediaries (even though it would generally but
pretty cheap to do so).  It would be more appropriate to add an
OPES-Integrity header with a checksum.

If the Content-MD5 header means "integrity from the last hop that
modified the data, so you can see that there are no transmission
errors", then modifying it is no big deal.

What does the community think it means?

Hilarie

>  I am not sure that Content-MD5 adjustments at the processor can

---

From: Alex Rousskov <[email-masked]>
Date: Wed, 15 Oct 2003 17:24:31 -0600 (MDT)

> What does the community think it means?

From HTTP point of view, Content-MD5 header is an end-to-end header.
There can be no questions about that. The true question is: what an
"end" means in the context where content is produced dynamically or in
stages, like what we have with OPES.

IMO, from content (and Content-MD5) point of view, an OPES proxy doing
content modification is not an intermediary or a hop, but an "end" or
an "edge" so "content integrity" arguments simply do not apply.  From
content consumer point of view, there is no integral content "behind"
an OPES system; content starts after OPES has done its job. That's
what Content-MD5 should represent.

OPES adaptations have to be authorized by content provider and traced
by others, of course, but that is irrelevant to HTTP or OCP issues
being raised in this thread.

We already agreed that OPES proxies can change content. Thus, we have
to give OPES the right to modify or delete content checksums or the
whole architecture would make no sense!

Alex.

P.S. Note that HTTP itself allows for similar modifications that
     some might mistakenly call "end-to-end" violations. For example,
     HTTP allows for aggregation and anonymization of Via: values.
     Such actions are HTTP-equivalent of OPES adjustment of
     Content-MD5 values.

---

From: Robert Collins <[email-masked]>
Date: Thu, 16 Oct 2003 16:46:16 +1000

> Or would that be an overkill?

I think its a necessary condition.

> Perhaps the only realistic

For rfc2617/digest signed message bodies and similar mechanisms, it's
probably intractable. 
For non verifiable headers - I think it depends on the advertisied
origin: that is if the OPES device is comparable to a reverse proxy, it
is considered authoritative for the entity and can safely correct such
headers. For non authoritative OPES devices - say ISP's performing local
ad replacement (ignoring the policy questions surrounding that) - then
removing it is probably the most accurate step.

Rob

-- 
GPG key available at: <http://members.aardvark.net.au/lifeless/keys.txt>.

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 16 Oct 2003 09:23:02 -0600 (MDT)

> I think its a necessary condition.

OK. Looks like we have consensus on the above. Martin, please add to
HTTP draft. I will try to add an OCP Core requirement for all OCP
bindings to address the problem so that other bindings do not miss the
issue.

> > Perhaps the only realistic default for a processor would be to
> > remove Content-MD5 header: absence of truth is better than lies.
>

Agreed. If an OPES system is adapting a securely signed message
without access to private keys, that OPES system is misconfigured or
is not authorized to do adaptations. In either case, secure signatures
will work well as long as the client checks for their presence and
validity (assuming they are essential).

> origin: that is if the OPES device is comparable to a reverse proxy,
> it is considered authoritative for the entity and can safely correct
> such headers. For non authoritative OPES devices - say ISP's
> performing local ad replacement (ignoring the policy questions
> surrounding that) - then removing it is probably the most accurate
> step.

Agreed.

Alex.