Subject: [apps-discuss] APPSDIR review of draft-ietf-httpbis-p2-semantics-24

From: S Moonesamy <[email-masked]>
Date: Mon, 28 Oct 2013 01:07:31 -0700

I have been selected as the Applications Area Directorate reviewer 
for this draft (for background on APPSDIR, please see 
http://trac.tools.ietf.org/area/app/trac/wiki/ApplicationsAreaDirectorate ).

Please resolve these comments along with any other Last Call comments 
you may receive. Please wait for direction from your document 
shepherd or AD before posting a new version of the draft.

Document: draft-ietf-httpbis-p2-semantics-24
Title: Hypertext Transfer Protocol (HTTP/1.1): Semantics and Content
Reviewer: S. Moonesamy
Review Date: October 28, 2013
IETF Last Call Date: October 21, 2013

Summary: This draft is almost ready for publication as a Proposed Standard.

This document defines the semantics of HTTP/1.1 messages, as 
expressed by request methods, request header fields, response status 
codes, and response header fields, along with the payload of messages 
(metadata and body content) and mechanisms for content negotiation.

The draft is well-written and clear.

Major Issues:

In Section 3.1.1.5:

   'If a Content-Type header field is not present, the recipient MAY
    either assume a media type of "application/octet-stream" ([RFC2046],
    Section 4.5.1) or examine the data to determine its type.'

According to RFC 2046, the "octet-stream" subtype is used to indicate 
that a body contains arbitrary data.  The RFC 2119 "may" leaves it to 
the implementor to make an assumption.  I suggest turning this into a 
recommendation so that the assumption is clear to the 
implementor.  There is a discussion of MIME sniffing in 
draft-ietf-websec-mime-sniff-03.  There has been discussion about 
MIME or Content sniffing over the years.  I am aware that some 
browsers do MIME sniffing.  I understand that it is sometimes needed 
to make the Web work.  However, it can lead to security 
vulnerabilities.  The paragraph which follows the one quoted above 
discusses about that.  I listed this as a major issue for the 
attention of the Applications Area Directors.

Minor Issues:

In Section 5.5.1:

   'The "From" header field contains an Internet email address for a
    human user who controls the requesting user agent.  The address ought
    to be machine-usable, as defined by "mailbox" in Section 3.4 of
    [RFC5322]:'

The above text is similar to text which was in RFC 2068 and RFC 
2616.  The "mailbox" may also contain a display name whereas the 
"addr-spec" (see Section 3.4.1) is an Internet mail address.  I 
listed this issue as minor to flag it for the attention of the 
HTTPbis Working Group.

In Section 5.5.3:

   "a sender MUST NOT generate advertising or other non-essential information
    within the product identifier"

I suggest having the text as guidance together with an explanation 
instead of a RFC 2119 prohibition.

In Section 7.1.1:

   "The preferred format is a fixed-length and single-zone subset of
    the date and time specification used by the Internet Message Format
    [RFC5322]"

The ABNF imports obs-date from RFC 5322.  "GMT" is defined in 
"obs-zone" (see RFC 5322).  There is the following in the section:

   "For example, messages are occasionally forwarded over HTTP from a
    non-HTTP source that might generate any of the date and time
    specifications defined by the Internet Message Format."

I listed this as a minor issue.  I am not suggesting any change as 
"GMT" is used in other header fields.  I'll defer to the HTTPbis 
Working Group as to whether there should be a note about "GMT" with 
an explanation that although "GMT" is deprecated the use is widespread.

Section 8.1 defines a HTTP Method Registry where registration 
requires IETF Review.  I took a quick look at Issue #364.  Section 
4.2 discusses about common method properties, e.g. cacheable.  The 
fields in Section 8.1.1 does not include cacheable.

There are considerations for new methods in Section 8.1.2.  I gather 
that the working group understands that someone will have to review 
the specification and raise an issue if the considerations are not followed.

The table in Section 8.1.3 only mentions the section number.  There 
is an assumption that the specification text is in this draft.  I 
suggest also adding a reference for the RFC number.  As a note for 
the reader, draft-ietf-httpbis-method-registrations-13 also registers 
some HTTP methods.

The above assumption also applies to Section 8.2.  I suggest updating 
the existing registrations at 
http://www.iana.org/assignments/http-status-codes/ so that the
HTTP Status Code Registry is compliant with Section 8.2.1.

Nits:

In Section 5.5.2:

  "Referer allows servers to generate back-links to other resources for"

  "information disclosure in Referer ought to restrict their changes to"

I'll defer to the editors as to whether to add "header field".

I suggest changing the RFC 1305 reference in Section 7.1.1.1 to RFC 5905.

In Section 8.3.1:

   'Authors of specifications defining new fields are advised to keep the
    name as short as practical and to not prefix the name with "X-"
    unless the header field will never be used on the Internet.'

I suggest adding a reference to BCP 178.

Regards,
S. Moonesamy

---

From: Julian Reschke <[email-masked]>
Date: Mon, 28 Oct 2013 15:00:20 +0100

On 2013-10-28 09:07, S Moonesamy wrote:
> ...
> The ABNF imports obs-date from RFC 5322.  "GMT" is defined in "obs-zone"
> (see RFC 5322).  There is the following in the section:
>
> I listed this as a minor issue.  I am not suggesting any change as "GMT"
> is used in other header fields.  I'll defer to the HTTPbis Working Group
> as to whether there should be a note about "GMT" with an explanation
> that although "GMT" is deprecated the use is widespread.
> ...

Actually, HTTPbis has its own obs-date:

	obs-date     = rfc850-date / asctime-date

<http://greenbytes.de/tech/webdav/draft-ietf-httpbis-p2-semantics-24.html#rfc.section.7.1.1.1>

So there doesn't seem to be an issue here.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Mon, 28 Oct 2013 15:14:44 +0100

On 2013-10-28 09:07, S Moonesamy wrote:
> ...
> Section 8.1 defines a HTTP Method Registry where registration requires
> IETF Review.  I took a quick look at Issue #364.  Section 4.2 discusses
> about common method properties, e.g. cacheable.  The fields in Section
> 8.1.1 does not include cacheable.
> ...

Yes -- this is not necessarily a problem. There are many things that 
need to be defined for a new method, and not all of these fit into the 
template.

> that the working group understands that someone will have to review the
> specification and raise an issue if the considerations are not followed.

Yes.

> The table in Section 8.1.3 only mentions the section number.  There is
> an assumption that the specification text is in this draft.  I suggest

That's an assumption that is true for all "bare" Section references.

> also adding a reference for the RFC number.  As a note for the reader,
> draft-ietf-httpbis-method-registrations-13 also registers some HTTP
> methods.

The IANA Considerations are processed by the RFC Editor and IANA, and 
they will make sure that the registry is properly populated. There's no 
point in mentioning a still unknown RFC # here.

> HTTP Status Code Registry is compliant with Section 8.2.1.

What, precisely?

Best regards, Julian

-- 
<green/>bytes GmbH, Hafenweg 16, D-48155 Münster, Germany
Amtsgericht Münster: HRB5782

---

From: Julian Reschke <[email-masked]>
Date: Mon, 28 Oct 2013 15:24:56 +0100

On 2013-10-28 15:22, John C Klensin wrote:
> --On Monday, October 28, 2013 15:00 +0100 Julian Reschke
> <[email-masked]> wrote:
>
>>>     "The preferred format is a fixed-length and single-zone
>>>     subset of the date and time specification used by the
>>>      Internet Message Format [RFC5322]"
>> ...
> Julian,
> I've been reluctant to step into this mess, but, having had
> another bad experience over the weekend, and with the
> understanding that we already have multiple "obsolete" forms
> floating around that implementations are supposed to recognize,
> I'd like to see if it is still possible to think about moving to
> an ISO-compatible "preferred form" that would eliminate the
> difficulty in handling and ambiguities in month names (and the
> frequent violations where they are made upper-case or translated
> into local languages).   Doing so, and getting rid of "GMT"
> (which about half the world's population seems to think is a
> synonym for whatever time is being used around Greenwich), in
> favor of UT (which no one who has any understanding at all seems
> to think might change in the summer), would save a lot of
> problems long-term.  That would make the preferred form
>
>    [day-name ","] year "-" monthNumber "-" day SP time-of-day SP
> "UT"
>
>    with
>     monthNumber = 2DIGIT
>
>    and
>     	obs-date = rfc850-date / asctime-date / IMF-fixdate
> if that is necessary.
>
> I don't care whether day-name is optional or not, but there
> would be some small i18n charm in saying "either write it the
> way the spec says or leave it out" rather than the current rule,
> which is effectively "use those English-based abbreviations no
> matter how obnoxious they are in your environment".
>
> It is obviously late to be suggesting this, but it was also late
> a dozen years ago and will be a lot later five or ten years
> hence.
> ...

John,

that change would make almost every HTTP/1.1 code ever written 
non-compliant.

And yes, it would have been nice if a same date format would have been 
chosen back then.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Mon, 28 Oct 2013 17:06:54 +0100

On 2013-10-28 09:07, S Moonesamy wrote:
> recommendation so that the assumption is clear to the implementor.
> There is a discussion of MIME sniffing in
> draft-ietf-websec-mime-sniff-03.  There has been discussion about MIME

(which expired ~2 years ago)

> or Content sniffing over the years.  I am aware that some browsers do
> MIME sniffing.  I understand that it is sometimes needed to make the Web
> work.  However, it can lead to security vulnerabilities.  The paragraph
> which follows the one quoted above discusses about that.  I listed this
> as a major issue for the attention of the Applications Area Directors.
> ...

Could you clarify what exactly the issue is?

RFC 2616 said 
(<http://greenbytes.de/tech/webdav/rfc2616.html#rfc.section.7.2.1.p.4>):

> Any HTTP/1.1 message containing an entity-body SHOULD include a Content-Type header field defining the media type of that body. If and only if the media type is not given by a Content-Type field, the recipient MAY attempt to guess the media type via inspection of its content and/or the name extension(s) of the URI used to identify the resource. If the media type remains unknown, the recipient SHOULD treat it as type "application/octet-stream".

...which isn't that different.

Best regards, Julian

---

From: S Moonesamy <[email-masked]>
Date: Tue, 29 Oct 2013 06:26:34 -0700

Hi Julian,
>(which expired ~2 years ago)

I guess that the working group gave up.  Please note that I did not 
suggest adding a reference.

>>Any HTTP/1.1 message containing an entity-body SHOULD include a 
>>Content-Type header field defining the media type of that body. If 
>>and only if the media type is not given by a Content-Type field, 
>>the recipient MAY attempt to guess the media type via inspection of 
>>its content and/or the name extension(s) of the URI used to 
>>identify the resource. If the media type remains unknown, the 
>>recipient SHOULD treat it as type "application/octet-stream".
>

The 'If the media type remains unknown, the recipient SHOULD treat it 
as type "application/octet-stream' from RFC 2616 is not in 
draft-ietf-httpbis-p2-semantics-24.

The issue is the "or examine the data to determine its type".  I'll 
comment about the text (Section 3.1.1.5) again.  The server has to 
generate a Content-Type header field unless the media type is 
unknown.  There is then a RFC 2119 "may" which is applicable when the 
(previous) RFC 2119 "should" cannot be applied.  My reading of the 
"may" is that the usage is not entirely correct.  I am not raising 
that as an issue.  The issue is whether there is a security problem 
and whether there is adequate discussion of it (e.g. it is discussed 
in a Security Considerations section).

Regards,
S. Moonesamy

---

From: Julian Reschke <[email-masked]>
Date: Tue, 29 Oct 2013 15:12:29 +0100

On 2013-10-29 14:26, S Moonesamy wrote:
>>> and only if the media type is not given by a Content-Type field, the
>>> recipient MAY attempt to guess the media type via inspection of its
>>> content and/or the name extension(s) of the URI used to identify the
>>> resource. If the media type remains unknown, the recipient SHOULD
>>> treat it as type "application/octet-stream".
>>
> The 'If the media type remains unknown, the recipient SHOULD treat it as
> type "application/octet-stream' from RFC 2616 is not in

I consider that sentence to be useless - if I can't detect the type, 
what else but "treating as arbitrary data" is left as an option anyway?

> generate a Content-Type header field unless the media type is unknown.

Yes.

> There is then a RFC 2119 "may" which is applicable when the (previous)
> RFC 2119 "should" cannot be applied.  My reading of the "may" is that
> the usage is not entirely correct.  I am not raising that as an issue.

I still don't get what the issue is :-)

> The issue is whether there is a security problem and whether there is
> adequate discussion of it (e.g. it is discussed in a Security
> Considerations section).

The subsequent text is:

"In practice, resource owners do not always properly configure their 
origin server to provide the correct Content-Type for a given 
representation, with the result that some clients will examine a 
payload's content and override the specified type. Clients that do so 
risk drawing incorrect conclusions, which might expose additional 
security risks (e.g., "privilege escalation"). Furthermore, it is 
impossible to determine the sender's intent by examining the data 
format: many data formats match multiple media types that differ only in 
processing semantics. Implementers are encouraged to provide a means of 
disabling such "content sniffing" when it is used."

Do you think this is insufficient, or that it needs to move to a 
different part of the spec?

Best regards, Julian

---

From: Barry Leiba <[email-masked]>
Date: Tue, 29 Oct 2013 11:28:04 -0400

> I still don't get what the issue is :-)

Yeh, neither do I, and I'm pretty sensitive to those SHOULD+MAY
issues.  The one in Section 3.1.1.5 seems perfectly fine: it says that
the server SHOULD do something, and that if the server has not done
that the client MAY do something to try to compensate.  A-OK to me.
The tricky bits with SHOULD+MAY occur when both key words apply to the
same entity under the same conditions.  That's not the case here.

Barry

---

From: John C Klensin <[email-masked]>
Date: Mon, 28 Oct 2013 10:22:21 -0400

Julian,

  with 
   monthNumber = 2DIGIT

  and

best,
   john

---

From: John C Klensin <[email-masked]>
Date: Mon, 28 Oct 2013 15:22:16 -0400

--On Monday, October 28, 2013 15:24 +0100 Julian Reschke
<[email-masked]> wrote:

> And yes, it would have been nice if a same date format would
> have been chosen back then.

Julian,

I don't know enough of the issues in enough detail to argue
this, and won't.  My goal is only to be sure the question has
been asked.

My understanding is that the HTTPbis work is a rather major
revision with at least some cases for which "get it right" is
more important that complete backward compatibility, especially
if there is a clear migration path.  All of the usual arguments
for that are relevant, especially the ones that focus on how
many implementations and pages are likely to be created in the
future relative to those that exist today.   Here, the clear
migration path would be a narrow and restrictive "send syntax"
and a permissive "receive syntax" that requires support of all
known older forms, _especially_ those that were recommended by
HTTP 1.1.

It has been rather longer than I think we expected, but it was
clear when HTTP 1.1 was adopted (I am sure it was clear at least
to the relevant ADs) that there could be incompatible changes in
the future and that the concern should be a non-disruptive
migration path, not absolute forward compatibility.  That is one
of the several reasons why the successor to HTTP 1.0 was
identified as HTTP 1.1, not HTTP 2.

This sort of change would still make HTTP 1.1 implementations
non-compliant with HTTP 2.0, but that is ok -- they still comply
with HTTP 1.1 and I hope no one is going to claim that all HTTP
1.1 implementations are non-compliant with HTTP the day HTTP 2.0
is published.

If the WG has considered these options and rejected them after
an open discusssion, please ignore the comment/ suggestion.

> And yes, it would have been nice if a same date format would
> have been chosen back then.

Yes, even for the mail header syntax that inspired the current
form and that predates the web by more than a decade.  But that
was before there was an ISO Standard, few of us were worried
about the year 2000 transition, etc.  My practical concern is
that, as we internationalize more, those perfectly
well-specified ASCII month and day names will get mapped, by
implementations who consider the local (and sometimes paying)
customers far more important than any standard, into local names
that no one else can read.   That would be fine if the local
forms stayed local, but we have lots of experience, including
with email, to indicate that they will leak and cause at least
user confusion if not interoperability problems.

So, from my point of view, this isn't about introducing an
incompatibility with something that one merely regrets not being
done differently.  It is rather more about what might be the
last chance to get it right and, in the process, eliminate
completely predictable future confusion and interoperability
problems.

  best,
   john

   john

---

From: Amos Jeffries <[email-masked]>
Date: Tue, 29 Oct 2013 14:26:25 +1300

On 29/10/2013 8:22 a.m., John C Klensin wrote:
> if there is a clear migration path.

No. The focus on the WG has been to document what is actually *working*
and clarify existing HTTP behaviour in order to  encourage interoperability.

>    All of the usual arguments

However it is important to note that there is no clear migration path. Many
implementations are actively *rejecting* or ignoring date formats which do
not include the "GMT" moniker.

> identified as HTTP 1.1, not HTTP 2.

Indeed. This option was considered by the WG several times as people keep
bringing up this same point. (I myself even a few  years back). The WG 
charter
is explicitly prohibiting addition of a new version number in these 
documents
so there are several things like this which have had to be left unchanged.

> 1.1 implementations are non-compliant with HTTP the day HTTP 2.0

Those existing 1.1 implementations will certainly not be compliant with 2.0
binary traffic encoding!

But mapping between 1.1 with its useless GMT / day-of-week and the new
improved 2.0 date format (whatever that may be) can be explicitly specified
for the 1.1<->2.0 gateway translations in the new 2.0 implementations
without altering 1.1 compliance for any existing or future implementations.
The WG has chosen this approach over attempting a difficult upgrade within
1.1 itself.

Amos

---

From: S Moonesamy <[email-masked]>
Date: Tue, 29 Oct 2013 22:24:32 -0700

Hi Julian,
At 07:14 28-10-2013, Julian Reschke wrote:
>need to be defined for a new method, and not all of these fit into 
>the template.

A DISCUSS about this can be easily addressed. :-)

What is the intent behind the HTTP Method Registry (Section 
8.1.3)?  What information should that registry convey to the 
reader?  For example, is it a quick way for the reader to find out 
whether POST is cacheable (re. a choice that a browser vendor made 
some time back) or should the person read the relevant specification 
text to get accurate answer?

>That's an assumption that is true for all "bare" Section references.

The problem is that people copy and paste them blindly.  I suggest 
having information in the tables as the working group would like it 
to appear in the registry

>The IANA Considerations are processed by the RFC Editor and IANA, 
>and they will make sure that the registry is properly populated. 
>There's no point in mentioning a still unknown RFC # here.

It is up to the Responsible Area Director and the the document 
shepherd to make sure that the registry is properly populated.

>What, precisely?

I suggest adding "RFC XXXX" in the tables.  I read 
draft-reschke-http-status-308-07.  The reference is similar to what 
is in draft-ietf-httpbis-p2-semantics-24.  They are two different 
drafts and that makes referencing only the sections ambiguous.  RFC 
6585 updated the HTTP Status Code registry last year.  The current 
registry format only references the RFC number.  I suggest making the 
change clear so that, in some distant future, someone working on this 
can figure out the details of the registry.

Regards,
S. Moonesamy

---

From: S Moonesamy <[email-masked]>
Date: Tue, 29 Oct 2013 23:26:12 -0700

Yes.

>completely predictable future confusion and interoperability

The argument is about code complexity versus what the working group 
charter says.  It is difficult to review a specification when the 
current specification is still referencing RFC 822.  I don't recall 
whether there are any other application-related specifications using 
"GMT".  The HTTPbis drafts discusses about "GMT" when it actually means "UTC".

>not include the "GMT" moniker.

That can be a problem.

>bringing up this same point. (I myself even a few  years back). The WG charter
>is explicitly prohibiting addition of a new version number in these documents

People usually keep bringing up the same point when a choice is not 
documented.  I read what John Klensin wrote as during each revision 
of the specification the above argument is used to avoid making a change.

Regards,
S. Moonesamy

---

From: Julian Reschke <[email-masked]>
Date: Wed, 30 Oct 2013 09:48:18 +0100

On 2013-10-30 06:24, S Moonesamy wrote:
> What is the intent behind the HTTP Method Registry (Section 8.1.3)?
> What information should that registry convey to the reader?  For
> example, is it a quick way for the reader to find out whether POST is
> cacheable (re. a choice that a browser vendor made some time back) or
> should the person read the relevant specification text to get accurate
> answer?

The template only contains a subset of the information, so in general, 
the reader will have to read the referenced RFC.

The entries in the template serve the purpose of shortcutting this 
lookup for considerations that can be answered with a simple "yes" or "no".

> having information in the tables as the working group would like it to
> appear in the registry
>
>> they will make sure that the registry is properly populated. There's
>> no point in mentioning a still unknown RFC # here.
>
> It is up to the Responsible Area Director and the the document shepherd
> to make sure that the registry is properly populated.

And the authors during AUTH48. There is no problem here. We will make 
sure that the registry gets populated properly. Trust me.

Best regards, Julian

---

From: "Martin J. Dürst" <[email-masked]>
Date: Wed, 30 Oct 2013 17:51:34 +0900

> what else but "treating as arbitrary data" is left as an option anyway?

I have to say that I don't consider this sentence to be useless.

As far as I remember, there are other specs (mail?) that say that 
text/plain is the default. So some implementers may be used to this, and 
apply it to http, too.

Also, while every natural language text has to assume that the reader 
uses a certain amount of rational thinking, specs are usually written 
with a somewhat reduced expectation in that respect, not because the 
average reader is particularly dumb, but because the consequences of 
interpreting something wrong are different than the consequences of 
getting something wrong when e.g. reading a novel.

So I don't see any reason for not keeping that sentence. Even if it 
doesn't help, it definitely doesn't hurt.

Regards,   Martin.

---

From: "Martin J. Dürst" <[email-masked]>
Date: Wed, 30 Oct 2013 18:01:53 +0900

> and clarify existing HTTP behaviour in order to encourage interoperability.

To clarify for people not too involved in the HTTPbis work:

The HTTPbis WG has two main work items:

1) Moving HTTP 1.1 to Standard. That's what all the drafts are about 
that are currently under review and discussion in apps-discuss. For 
people familiar with Internet mail, that's somewhat similar to the work 
that happened when moving from RFCs 2821,... to RFCs 5321,... As such, 
it doesn't leave much room for innovation or even fixing stuff that 
looks broken from the outside, and the HTTPbis WG was particularly 
careful to avoid any such breakage.

2) HTTP 2.0: This is a completely new protocol design, so there is quite 
a bit of room for "getting things right", and stuff like using integers 
for representing dates (because the protocol is binary) is being 
considered. This on many fronts allows cleanup similar e.g. to what 
happened in EAI (Email Address Internationalization), because there's no 
deployed base to worry about. But HTTP 2.0 is not yet ready for general 
review.

Regards,   Martin.

---

From: Julian Reschke <[email-masked]>
Date: Wed, 30 Oct 2013 10:47:53 +0100

> doesn't help, it definitely doesn't hurt.

So what exactly does it mean in *practice* to treat something as 
"arbitrary data". What do you expect a browser to do in that case?

Best regards, Julian

---

From: S Moonesamy <[email-masked]>
Date: Wed, 30 Oct 2013 08:49:52 -0700

Hi Julian,
>what else but "treating as arbitrary data" is left as an option anyway?

I'll comment below.

>I still don't get what the issue is :-)

My preference is not to generate material which create more work for 
you.  It's better not to pursue this one. :-)

>payload's content and override the specified type. Clients that do 
>so risk drawing incorrect conclusions, which might expose additional 
>format: many data formats match multiple media types that differ 
>only in processing semantics. Implementers are encouraged to provide 
>a means of disabling such "content sniffing" when it is used."
>

The subsequent text is, to put it simply, about an operational issue 
and security considerations.  The recommendation in Section 3.1.1.5 
is to generate a Content-Type header field if the server knows the 
media type.  There are cases when the server does not know the media 
type.  In such cases the server sends the client 
"application/octet-stream".  There is where the user has to determine 
whether the server is operated by good person or a bad person (re. 
arbitrary data).  The user relies on the browser to perform some 
magic to determine that.  That magic does not always work well.

If it was my decision (and it is not) I would discuss about this 
under Security Considerations and mention that content sniffing can 
cause security problems.  Please note that there are different 
alternatives to tackle the issue.

Regards,
S. Moonesamy

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 31 Oct 2013 14:31:28 +1100

SM,

Consensus around this text was particularly hard-won; unless there is a very good reason to make a change, I'd rather not risk falling into that rat-hole again.

Regards,

On 31/10/2013, at 2:49 AM, S Moonesamy <[email-masked]> wrote:

>> I consider that sentence to be useless - if I can't detect the type, what else but "treating as arbitrary data" is left as an option anyway?
> 
> I'll comment below.
> 
> My preference is not to generate material which create more work for you.  It's better not to pursue this one. :-)
> 
>> The subsequent text is:
>> 
>> "In practice, resource owners do not always properly configure their origin server to provide the correct Content-Type for a given representation, with the result that some clients will examine a payload's content and override the specified type. Clients that do so risk drawing incorrect conclusions, which might expose additional security risks (e.g., "privilege escalation"). Furthermore, it is impossible to determine the sender's intent by examining the data format: many data formats match multiple media types that differ only in processing semantics. Implementers are encouraged to provide a means of disabling such "content sniffing" when it is used."
>> 
>> Do you think this is insufficient, or that it needs to move to a different part of the spec?
> 
> The subsequent text is, to put it simply, about an operational issue and security considerations.  The recommendation in Section 3.1.1.5 is to generate a Content-Type header field if the server knows the media type.  There are cases when the server does not know the media type.  In such cases the server sends the client "application/octet-stream".  There is where the user has to determine whether the server is operated by good person or a bad person (re. arbitrary data).  The user relies on the browser to perform some magic to determine that.  That magic does not always work well.
> 
> If it was my decision (and it is not) I would discuss about this under Security Considerations and mention that content sniffing can cause security problems.  Please note that there are different alternatives to tackle the issue.
> 
> Regards,
> S. Moonesamy 
> 

--
Mark Nottingham   http://www.mnot.net/

---

From: "Martin J. Dürst" <[email-masked]>
Date: Thu, 31 Oct 2013 13:32:01 +0900

> "arbitrary data". What do you expect a browser to do in that case?

Let's say ask the user. That's what they are doing currently, as far as 
I'm aware of. I haven't checked every latest version of every browser, 
but on average, I'd expect a dialog that asks whether (and then where) I 
want to save the file, or with what application I want to open it.

But of course HTTP cannot say that, because HTTP is about more than 
browsers. But then, HTTP doesn't say what to do with data that is marked 
as application/octet-stream, the same way as it doesn't say what to do 
with data that is marked as text/html or any other media type. And that 
seems just fine.

Put another way, the sentence is not useless because clients already 
have to figure out what to do with data marked as application/octet 
stream. We just tell them to reuse and leverage what they have figured 
out and apply it to unlabeled content. If a client implementer, e.g. for 
a debugging tool, has figured out that they want to show 
application/octet-stream as plain text assuming a 'charset' that shows 
each byte separately, then it's fine for them to use that for unlabeled 
data.

That also means that the requirement is testable. Just check whether the 
same happens for data labeled as application/octet-stream and for 
unlabeled data. It may be difficult to automate such a test, in 
particular in a test suite that is supposed to work across clients, but 
it's nevertheless testable.

Also, the sentence is not useless because it tells implementers to NOT 
treat as any of the other available content types, and that's what's 
important.

Regards,   Martin.

---

From: S Moonesamy <[email-masked]>
Date: Wed, 30 Oct 2013 22:24:58 -0700

Hi Mark,
At 20:31 30-10-2013, Mark Nottingham wrote:
>Consensus around this text was particularly hard-won; unless there 
>is a very good reason to make a change, I'd rather not risk falling 
>into that rat-hole again.

Understood.

I'll copy this message to the Application Area Directors in case they 
have any guidance to provide.

Regards,
S. Moonesamy