Subject: [apps-discuss] Request that the WG reconsider section 3.4: Content Negotiation

From: [email-masked] (Henry S. Thompson)
Date: Mon, 04 Nov 2013 16:53:30 +0000

It's my impression that content negotiation hasn't turned out to play
the kind of significant role in Web Architecture in general, or in
HTTP use in particular, that was expected for it.

I think the section on conneg in p2-semantics [1] is so out-of-step
with actually deployment, usage and expectations that to publish it as
it stands would be a serious mistake.

In particular, the discussion of the relative disadvantages of the
newly (re-)named 'proactive' and 'reactive' variants are not only
out-of-date, but also this discussion appears to at least this reader
to amount to a recommendation for 'reactive' negotiation.  Yet as far
as I can tell no user agents _or_ servers actually support this
approach today, as it's described here.

I was sufficiently concerned about this question to undertake a
moderately extensive empirical investigation [2].  To summarise
perhaps too briefly, I found _no_ evidence of the use of reactive
conneg in over 75 million HTTP request/response exchanges.

I think 3.4 can and should be substantially simplified, with all the
evaluative/speculative prose removed, focussing simply on the
semantics of the Accept... headers and the 300 and 406 status codes,
perhaps also making clear that 'proactive' conneg is the only form of
conneg with any signficant degree of server-side support.

[2] discusses all this at more length -- I hope it will be helpful.  I
am of course aware that personal experience, backed up by one small
study, may well be misleading, but I did look moderately hard to find
any other relevant experimental results without success.  It is less
likely, but not impossible, that what I report in [2] about what is
implemented in IIS, Apache and the major web browsers is also
mistaken.  On either count, I would welcome concrete evidence of where
I've misunderstood or misrepresented the actual situation.

ht

[1] http://tools.ietf.org/html/draft-ietf-httpbis-p2-semantics-24#section-3.4
[2] http://www.ltg.ed.ac.uk/~ht/reactive_conneg.html
-- 
       Henry S. Thompson, School of Informatics, University of Edinburgh
      10 Crichton Street, Edinburgh EH8 9AB, SCOTLAND -- (44) 131 650-4440
                Fax: (44) 131 650-4587, e-mail: [email-masked]
                       URL: http://www.ltg.ed.ac.uk/~ht/
 [mail from me _always_ has a .sig like this -- mail without it is forged spam]

---

From: Julian Reschke <[email-masked]>
Date: Mon, 04 Nov 2013 23:55:23 +0100

On 2013-11-04 17:53, Henry S. Thompson wrote:
> ...

Reactive conneg isn't just about 300s and 406s. Another example would be 
a representation returned with a 200 response that contains links to 
alternate versions of the content. That's what the

"If the user agent is not satisfied by the initial response 
representation, it can perform a GET request on one or more of the 
alternative resources, selected based on metadata included in the list, 
to obtain a different form of representation for that response. 
Selection of alternatives might be performed automatically by the user 
agent or manually by the user selecting from a generated (possibly 
hypertext) menu."

in 
<http://greenbytes.de/tech/webdav/draft-ietf-httpbis-p2-semantics-24.html#rfc.section.3.4.2.p.1> 
is about.

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 4 Nov 2013 15:12:34 -0800

What Julian says. Reactive negotiation is very common; formats are coming up with mechanisms for it all the time (e.g., the work on srcset, etc. in the W3C web perf WG).

Cheers,

On 4 Nov 2013, at 2:55 pm, Julian Reschke <[email-masked]> wrote:

> Reactive conneg isn't just about 300s and 406s. Another example would be a representation returned with a 200 response that contains links to alternate versions of the content. That's what the
> 
> "If the user agent is not satisfied by the initial response representation, it can perform a GET request on one or more of the alternative resources, selected based on metadata included in the list, to obtain a different form of representation for that response. Selection of alternatives might be performed automatically by the user agent or manually by the user selecting from a generated (possibly hypertext) menu."
> 
> in <http://greenbytes.de/tech/webdav/draft-ietf-httpbis-p2-semantics-24.html#rfc.section.3.4.2.p.1> is about.
> 
> Best regards, Julian

--
Mark Nottingham   http://www.mnot.net/

---

From: [email-masked] (Henry S. Thompson)
Date: Tue, 05 Nov 2013 12:18:23 +0000

Julian Reschke writes:

> Reactive conneg isn't just about 300s and 406s. Another example would
> be a representation returned with a 200 response that contains links
> to alternate versions of the content. That's what the

How is this in scope for discussion _in the HTTP spec._?  People (not
user agents, note) use 200 responses for a huge range of interesting,
powerful, innovative things.  We don't look in the HTTP spec. to find
a discussion of them.

> alternative resources, selected based on metadata included in the
> list, to obtain a different form of representation for that
> response. Selection of alternatives might be performed automatically
> by the user agent or manually by the user selecting from a generated
> (possibly hypertext) menu."

"based on metadata included in the list"!  That's a specific reference
to a 300 response.  There is no "list" in a 200 response, not that a
user agent can detect anyway.

And, as I said in the last message, I'm not aware of _any_ user agent
that does _anything_ specific to 300 responses.

ht

---

From: [email-masked] (Henry S. Thompson)
Date: Tue, 05 Nov 2013 12:45:04 +0000

Mark Nottingham writes:

> What Julian says.

What I said in reply to Julian :-).

> Reactive negotiation is very common; formats are coming up with
> mechanisms for it all the time (e.g., the work on srcset, etc. in
> the W3C web perf WG).

I'm still looking for a _single_ example of user agent support for
reactive conneg _at the HTTP level_.  srcset, as far as I can see, has
entirely HTML-language-level semantics, and has nothing to do with
HTTP.  It also, at first glance anyway, seems to be exactly the kind
of conneg anti-pattern which got such a bad review from the HTTP WG in
the thread on Mike Kelly's suggestions about changing the semantics in
HTML of a/@type and/or adding a/@accept, starting at [1].

ht

[1] http://lists.w3.org/Archives/Public/public-html/2009Oct/0527.html

---

From: "Rushforth, Peter" <[email-masked]>
Date: Tue, 5 Nov 2013 14:15:45 +0000

Hi Henry,

I see the reviews, but I don't know that I agree it's an anti-pattern. 

For example in atom, the existing ability to list several atom:link[@rel='alternate'] which only differ by
@type value is something similar to what Mike was suggesting for html.

Not that there are many user agents which support atom, but unless you *force* the
resource to return the media type suggested by @type (which couples format and identity/location), 
the only reasonable way to get that advertised media type back is by conneg, wherein the
user-agent reads the metadata from the hypertext and negotiates for that.

In HTML, the author can force the *display* of the links they want in the manner they want, but
they can't force the user agent to negotiate for anything, it is hard-coded to
'negotiate' for a list of media type preferences (based on the a or img), 
which I think is the anti-pattern.

Not that any of this affects the discussion of 300 Multiple Choices, I can see your point there. But
I would say that 200 OK with user-selectable alternates is a common pattern.  And it would
be nice if user-agents did 'help' with the interaction by negotiating for the advertised
media type.  By hard-coding the negotiation, they aren't supporting the web; as a result there
is no web which reaches across format (well across associated formats anyway). 

Regards,
Peter Rushforth

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Henry S. Thompson
> Sent: November 5, 2013 07:45
> To: Mark Nottingham
> Cc: Julian F. Reschke; [email-masked] WG; 
> [email-masked] Discuss
> Subject: Re: [apps-discuss] Request that the WG reconsider 
> section 3.4: Content Negotiation
> 
> Mark Nottingham writes:
> 
> > What Julian says.
> 
> > mechanisms for it all the time (e.g., the work on srcset, 
> etc. in the 
> > W3C web perf WG).
> 
> I'm still looking for a _single_ example of user agent 
> support for reactive conneg _at the HTTP level_.  srcset, as 
> far as I can see, has entirely HTML-language-level semantics, 
> and has nothing to do with HTTP.  It also, at first glance 
> anyway, seems to be exactly the kind of conneg anti-pattern 
> which got such a bad review from the HTTP WG in the thread on 
> Mike Kelly's suggestions about changing the semantics in HTML 
> of a/@type and/or adding a/@accept, starting at [1].
> 
> ht
> 
> [mail from me _always_ has a .sig like this -- mail without 
> it is forged spam] _______________________________________________
> apps-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/apps-discuss
>

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 04 Nov 2013 17:07:50 +0000

In message <[email-masked]>, Henry S. Thompson writes:

>I think 3.4 can and should be substantially simplified, 

Seconded.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 5 Nov 2013 15:45:08 -0800

Hi Henry,

> powerful, innovative things.  We don't look in the HTTP spec. to find

You seem to be thinking of HTTP as a separate layer from the application; as section 3.4 of p2 says, it’s defining a pattern of use, and that is certainly in scope for this specification, given that it was also in scope for RF2616.

> user agent can detect anyway.

If there is more than one link in the response in *some* format that’s tied together in some fashion (e.g., with a link relation, as per RF5988), there certainly is a list available.

We talked about this issue at the WG meeting yesterday. Based upon that discussion, we decided to close this issue. Looking at <https://svn.tools.ietf.org/svn/wg/httpbis/draft-ietf-httpbis/latest/p2-semantics.html#content.negotiation>, I can’t see any immediate clarifications that would help; if you can suggest some, please do bring them to our attention.

Regards,

---

From: [email-masked] (Henry S. Thompson)
Date: Wed, 06 Nov 2013 11:06:43 +0000

Mark Nottingham writes:

> You seem to be thinking of HTTP as a separate layer from the
> application; as section 3.4 of p2 says, it’s defining a pattern of
> use, and that is certainly in scope for this specification, given
> that it was also in scope for RF2616.

I guess it's statements such as "HTTP provides mechanisms for content
negotiation" (1st para of 3.4) and "patterns . . . visible within the
protocol" (2nd para) which suggest that we're talking about the HTTP
layer, not a more general functionality.

> We talked about this issue at the WG meeting yesterday. Based upon
> that discussion, we decided to close this issue. Looking at
> <https://svn.tools.ietf.org/svn/wg/httpbis/draft-ietf-httpbis/latest/p2-semantics.html#content.negotiation>,
> I can’t see any immediate clarifications that would help; if you can
> suggest some, please do bring them to our attention.

Thanks for your time.  Here's a quick redraft of 3.4 (the first para
is nearly unchanged) -- this is the direction I think would be a
helpful change.

--------
3.4 MSc Content Negotiation

When responses convey payload information, whether indicating a
success or an error, the origin server often has different ways of
representing that information; for example, in different formats,
languages, or encodings.  Likewise, different users or user agents
might have differing capabilities, characteristics, or preferences
that could influence which representation, among those available,
would be best to deliver.  For this reason, HTTP provides mechanisms
supporting content negotiation, that is, means for clients and servers
to select, or involve users and user agents in selecting, among
alternative representations.

Note that, in all cases, HTTP is not aware of resource semantics.  The
consistency with which an origin server responds to requests, over
time and over the varying dimensions of content negotiation, and thus
the "sameness" of a resource's observed representations over time, is
determined entirely by whatever entity or algorithm selects or
generates those responses.  HTTP pays no attention to the man behind
the curtain.

In particular, this specification defines headers and response codes
for use in exchanges where multiple representations are, or may be,
involved.  Accept headers in requests (Section 5.3) allow clients to
signal preferred alternatives.  Such explicit preferences may derive
from user agent capabilities (for example with respect to image format
support), user preferences (for example with respect to natural
language) or local context (for example with respect to media type).
Servers may also infer implicit preferences from other properties of
requests such as the network address or User-Agent header.
Response codes (Sections 6.4.1 and 6.5.6) and response headers
(Sections 7.1.4 and 7.1.2) allow servers (and proxies) to signal the
success or failure in any attempt to satisfy client preferences
(implicit or explicit) as well as the availability of alternatives.

Although the most common use of these mechanisms is for clients to
include Accept headers in requests, and servers (or proxies) to select
among available alternatives based on those headers, a wide range of
other patterns of use exist, including ones where the initiative is
all on the server side, where for example a 300 response may accompany
a payload offering what amounts to spelling correction, as an
alternative to a 404.  Many of these patterns depend on particular
media types and/or user agents.
------------------
and leave it at that.  Some corresponding changes to the referenced
sections would likely be necessary to pick up a few of the details
from the now-deleted 3.4.1 and 3.4.2.

ht

"Whereof one cannot speak, thereof one must be silent"

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 11 Nov 2013 17:31:30 +0800

Hi Henry,

Thanks for taking the time to make a suggestion. Based on the discussions in Vancouver, I don’t see consensus to make a change like this.

Regards,