Subject: no-transform & Warning

From: Mark Nottingham <[email-masked]>
Date: Thu, 11 Apr 2002 13:21:45 -0700

Just curious,

Is anyone aware of an OPES, ICAP or other transcoding implementation 
that:

a) honors Cache-Control: no-transform
b) honors Warning: 214 Transformation applied

in either requests or responses?

--
Mark Nottingham
http://www.mnot.net/

---

From: Markus Hofmann <[email-masked]>
Date: Mon, 15 Apr 2002 16:41:09 -0400

Mark,

would you consider this part of an OPES/iCAP protocol implementation 
or rather the responsibility of the rule author (i.e. the rule author 
has to specify the appropriate rules, such as "if 'Cache-Control: 
no-transform', do NOT do...") or maybe even of the application service 
developer. I would expect that most implementations let you specify 
such rules, thus allowing the system to honor such controls.

-Markus

Mark Nottingham wrote:
> 
> Just curious,
> 
> Is anyone aware of an OPES, ICAP or other transcoding implementation that:
> 
> in either requests or responses

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 15 Apr 2002 14:15:59 -0700

These mechanisms are required in order to be HTTP-compliant; therefore, 
I'd say they should be part of the ICAP server's HTTP stack.

On Monday, April 15, 2002, at 01:41  PM, Markus Hofmann wrote:

> Mark,
>
> would you consider this part of an OPES/iCAP protocol implementation or 
> rather the responsibility of the rule author (i.e. the rule author has 
> to specify the appropriate rules, such as "if 'Cache-Control: 
--
Mark Nottingham
http://www.mnot.net/

---

From: Markus Hofmann <[email-masked]>
Date: Tue, 16 Apr 2002 10:43:03 -0400

> I'd say they should be part of the ICAP server's HTTP stack.

Hm, this assumes that the payload (i.e. the content-path protocol) is 
NOT transparent to the OPES/Callout protocol, i.e. the OPES/Callout 
protocol needs to have knowledge about the sematics of the 
encapsulated application protocol (HTTP in this case).

Do we want to make this assumption? Comments?

-Markus

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 16 Apr 2002 11:16:19 -0700

I was talking about placing a requirement on the OPES server 
implmentation, not the callout protocol itself. I.e., the protocol can 
be agnostic, but the thing that stuffs HTTP messages into it had better 
understand the semantics of HTTP.

On Tuesday, April 16, 2002, at 07:43  AM, Markus Hofmann wrote:

> Mark Nottingham wrote:
>
>> These mechanisms are required in order to be HTTP-compliant; 
>> therefore, I'd say they should be part of the ICAP server's HTTP stack.
>
> protocol needs to have knowledge about the sematics of the encapsulated 
> application protocol (HTTP in this case).
>
--
Mark Nottingham
http://www.mnot.net/

---

From: Markus Hofmann <[email-masked]>
Date: Tue, 16 Apr 2002 14:34:19 -0400

> understand the semantics of HTTP.

Yup, this I agree with, building the application semantic around a 
generic protocol, rather than building it into the protocol itself. It 
was not clear to me whether people thought of making the callout 
protocol itself aware of the encapsulated application protocol.

-Markus

---

From: Geetha Manjunath <[email-masked]>
Date: Wed, 17 Apr 2002 10:23:58 +0530

I would say, this is a requirement on the HTTP proxy servers rather than the
OPES/Callout protocol.

(a) Cache-Control: An implementation of the icap client module in the HTTP
proxy server (say squid) would have to just check for the Cache-Control
header before sending it to the ICAP server for a transformation
(reqmod/respmod). - Section 14.9.5 of RFC 2616 HTTP 1.1
(b) Warning 214: Along the same lines, the Warning header is required to be
inserted by the HTTP 1.1 complaint proxy server. ( as per section 14.46 of
RFC 2616)

Did I miss something?

regards,
Geetha

Markus Hofmann wrote:

> Masms are required in order to be HTTP-compliant; therefore,
> Hm, this assumes that the payload (irk Nottingham wrote:
>
> > These mechani.e. the content-path protocol) is

> Subject:  no-transform & Warning
>
> Date:Thu, 11 Apr 2002 13:21:45 -0700
> From: Mark Nottingham
>
> Just curious,
>

---

From: "Frank Berzau" <[email-masked]>
Date: Wed, 17 Apr 2002 23:28:03 +0200

Technically the callout protocol itself does not need to be aware of the
encapsulated payload and it's specific syntax or requirements. Question
is whether the callout client and/or server need to be. As far as the
client is concerned, it depends. The client could be implemented in a
switch, not a http proxy (to pick just one example). As such it is
unlikely that the client understands the protocol fully, it would just
forward all traffic sent to port 80 to a callout server. If we require
the client to understand those issues, it will make it far more complex
than needed. As far as the callout server is concerned, the answer can
only be Yes. The callout server is operating on the payload (headers or
body, or both or part of either, whatever), so it does need to
understand the protocol, and comply to all it's rules. Other examples
would be callout server that work on SMTP, NNTP or any other protocol.
Each one of those would need to understand the respective protocol it's
dealing with.

There may be cases, where putting "intelligence" into the client would
make sense. For example the no-transform HTTP header would make it
unnecessary to send a request to a callout server. In those very
specific and rare cases I'd still leave it up to the callout server to
handle it. Still a vendor could build that checking into the client in
order to improve performance by eliminating un-needed requests. That,
however, would be implementation specific. There is no need to require
it if we assume it's already handled at the server.

Thanks, Frank

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of 
> Geetha Manjunath
> Sent: Mittwoch, 17. April 2002 06:54
> To: Markus Hofmann
> Cc: Mark Nottingham; [email-masked]; [email-masked]
> Subject: Re: no-transform & Warning
> 
> 
>

---

From: Ian Cooper <[email-masked]>
Date: Wed, 17 Apr 2002 16:04:48 -0700

--On Wednesday, April 17, 2002 23:28 +0200 Frank Berzau 
<[email-masked]> wrote:

> than needed.

I was under the impression that the non-application layer intermediaries 
(e.g. things other than "proxies" or "surrogates") were out of scope for 
the work being considered by this WG.  Traffic interception/redirection 
operating at the layer you suggest above is widely agreed to be a Very Bad 
Thing in terms of the end-to-end architecture.

> As far as the callout server is concerned, the answer can

Er, but seeing as the application intermediary must, by definition, 
understand the protocol that it might be operating modifications on (with 
the help of the callout system) it should already have the appropriate code 
to operate on those directives that it needs to honour.  E.g. in this case 
if there's a Cache-control: no-transform the HTTP intermediary should 
already have a code path to stop it being vectored AT ALL.

Similarly, the HTTP intermediary is the one entity in the relationship that 
likely has the best knowledge as to whether a Warning: 214 should be added 
to the outbound response headers.

[And I think I just echoed Geetha there.]

---

From: "Oskar Batuner" <[email-masked]>
Date: Fri, 19 Apr 2002 17:08:16 -0400

I think Cache-Control headers and related response codes should be applied
exactly to the subject they claim - cache control. The extended use of HTTP
as a base protocol results in temptation to overload semantics of existing
headers. Mark Nottingham pointed one example of such overload about a year
ago (forgive me if I'm wrong with dates). Mark has posted a document
explaining why use of cache control headers is bad in origin-surrogate
context: semantic roles of HTTP client and server are different from the
surrogate-origin server case - they have a special trust relationship, and
cache control part of HTTP should better be preserved to manage downstream
caches.

The same approach should be taken for the callout protocol. The proxy that
is running OPES rules serves as a callout gateway. According to the OPES
intentions and IAB guidelines it has an explicit permission to perform
certain actions - that is, special relationship with either origin server or
the end user. Cache control directives should be preserved for caches that
may occur between OPES gateway and end user - otherwise there is no
mechanism left for content producer to control intimidate cache behavior.
Callout participants and callout protocol SHOULD NOT try to interpret such
directives.

One additional implication is that in some scenarios OPES server (and
implicitly OPES gateway) may act as an authorized surrogate content
producer. In this scenario cache control directives MAY by altered to
reflect the changed content semantics. The way to handle them was very well
described in Mark's document mentioned above. I do not remember what
happened to it, I'm afraid it just expired as a draft. Again, sorry if I'm
wrong. Mark MAY shed more light on the fate of this document.

- Oskar

> [mailto:[email-masked]]On Behalf Of Markus Hofmann
> Sent: Tuesday, April 16, 2002 2:34 PM
> To: Mark Nottingham
> Cc: [email-masked]; [email-masked]

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 19 Apr 2002 16:04:04 -0700

I think Oskar is referring to [1].

I agree that in a surrogate (gateway) scenario, where there is an out-of 
band contract regarding message handling between the origin and the 
intermediary, Cache-Control: no-transform in a request, as well as the 
generation of transform-related Warnings, can be safely ignored (if both 
parties desire that behavior, of course).

However, a proxy is a different kettle of fish; to be HTTP compliant, 
proxies encountering no-transform "MUST NOT change any aspect of the 
entity-body that is specified by these headers, including the value of 
the entity-body itself." [2]  I'd argue that even a proxy with an OPES 
ruleset from whoever's driving the User-Agent can't ignore this 
requirement; it places too much trust in what is often a heuristic - the 
ruleset - at the expense of directly associated, declared information to 
the contrary.

Looking at the IAB "considerations" for OPES [3], it seems that 
no-transform and Warning make significant inroads into the requirement 
to be non-blocking, as well as that for notification. I can think of no 
better way to allow clients to specify that a message should not be 
transformed, as well as notify them that an entity has been changed; 
using an already-specified mechanism assures that at least 
theoretically, OPES-ignorant HTTP implementations can still interoperate 
with OPES intermediaries.

Regarding the scope of "Cache-Control", "cache" isn't used with great 
precision in this part (and perhaps some others) of 2616; the language 
[2] clearly scopes no-transform as applying to caches *and* proxies, not 
just cache implementations.

1. http://www.terena.nl/conf/wcw/Proceedings/S4/S4-3.pdf
2. http://rfc2616.x42.com/ -  Section 14.9.5
3. http://rfc3238.x42.com/

--
Mark Nottingham
http://www.mnot.net/

---

From: "The Purple Streak (Hilarie Orman)" <[email-masked]>
Date: Fri, 19 Apr 2002 18:03:13 -0600

I don't see why a ruleset is less reliable than an implicit contract.

Hilarie

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 19 Apr 2002 18:46:16 -0700

I'd characterise CC: no-transform as an explicit directive, not as an 
implicit contract. Or are you saying that HTTP itself is an implicit 
contract?

On Friday, April 19, 2002, at 05:03  PM, The Purple Streak (Hilarie 
Orman) wrote:

>> I agree that in a surrogate (gateway) scenario, where there is an 
>> out-of band contract regarding message handling between the origin and 
>> the intermediary, Cache-Control: no-transform in a request, as well as 
>> the generation of transform-related Warnings, can be safely ignored 
>> (if both parties desire that behavior, of course).
>> requirement; it places too much trust in what is often a heuristic - 
>> the ruleset - at the expense of directly associated, declared 
>> information to the contrary.
>> to be non-blocking, as well as that for notification. I can think of 
>> no better way to allow clients to specify that a message should not be 
>> theoretically, OPES-ignorant HTTP implementations can still 
>> interoperate with OPES intermediaries.
>> [2] clearly scopes no-transform as applying to caches *and* proxies, 
>> not just cache implementations.
>> -- Mark Nottingham
--
Mark Nottingham
http://www.mnot.net/

---

From: "The Purple Streak (Hilarie Orman)" <[email-masked]>
Date: Sat, 20 Apr 2002 11:53:18 -0600

The message thread states that no-transform can be safely ignored
is there is an "implicit contract", but it cannot be ignored safely
if there is a ruleset explicitly stating that it can be ignored.
That's what I was taking issue with.

Hilarie

Mark Nottingham wrote:

>>> out-of band contract regarding message handling between the origin 
>>> and the intermediary, Cache-Control: no-transform in a request, as 
>>> well as the generation of transform-related Warnings, can be safely 
>>> ignored (if both parties desire that behavior, of course).
>>> entity-body that is specified by these headers, including the value 
>>> of the entity-body itself." [2]  I'd argue that even a proxy with an 
>>> OPES ruleset from whoever's driving the User-Agent can't ignore this 
>>> no-transform and Warning make significant inroads into the 
>>> requirement to be non-blocking, as well as that for notification. I 
>>> can think of no better way to allow clients to specify that a message 
>>> should not be transformed, as well as notify them that an entity has 
>>> been changed; using an already-specified mechanism assures that at 
>>> least theoretically, OPES-ignorant HTTP implementations can still 
>>> precision in this part (and perhaps some others) of 2616; the 
>>> language [2] clearly scopes no-transform as applying to caches *and* 
>>> proxies, not just cache implementations.

---

From: "Oskar Batuner" <[email-masked]>
Date: Mon, 22 Apr 2002 11:45:18 -0400

Again, for me this comes to the semantic overload: CC is an explicit
Cache Control directive. As such it has nothing to do with OPES
gateway behavior. I agree that it may me useful to have a
directive(s) for explicit per-message control of OPES gateway, but
we should not try to use existing HTTP headers - this is a
semantics overload (in the sense of C++ class member operators
overload).

For this particular operation (non-transform) our discussion looks
like pure academic - I am not aware of wide (or in fact any) use of
this header or 214 response. This leads to the temptation to use the
existing directive instead of introducing a new one. But a)this is
not a good practice (IMHO);
b) consider future development. If I understand correctly here we are
not looking at callout protocol itself - this kind of directive should
be interpreted by the application protocol gateway, not callout server.
We may need other directives to control gateway behavior. Adding
additional fields to Cache-Control header does not look a good idea
for extensions. For OPES gateway control we need an additional header
(x-opes-control). Use of CC just for one purpose creates a "legacy
branch" in a brand new protocol.

To summarize the issues:

1. Problems related to the application protocol interaction with
gateways should be considered separately from callout protocol
used by gateway to tunnel the request to the callout server.

2. We may need a set of directives to control a gateway included
in the application protocol. These directives are especially
useful to control per-message gateway behavior.

3. As a separate issue - while transport features of callout protocol
may be application-protocol agnostic, semantics of callout protocol and
server may need application protocol awareness. HTTP has some headers
(like Accept-Encoding, Accept-Language, Content-Language, Content-Encoding,
Content-Type, etc.) that define message properties
essential for some types of server processing.

- Oskar

> [mailto:[email-masked]]On Behalf Of Mark Nottingham
> Sent: Friday, April 19, 2002 9:46 PM
> To: The Purple Streak (Hilarie Orman)
> Cc: Oskar Batuner; Markus Hofmann; [email-masked]; [email-masked]