Subject: #1043 "header": Suggested resolution

From: Harald Tveit Alvestrand <[email-masked]>
Date: Fri, 24 Jun 2005 11:30:19 +0200

Since several people weighed in on the subject after my summary, we seem to 
be moving towards a rough consensus, and I want to try for a resolution.... 
Henry and Charles have continued to argue for the "news" syntax; I'd like 
to check that they find it possible to work with the "2822" syntax....

I suggest the following text change to section 2.1, with the necessary 
changes to the places where the terms occur in the rest of the text:

OLD:

 This specification uses the terms "header", "header name", and
 "header content" which are synonymous with the [RFC2822] terms
 "header field", "field name", and "field body" respectively.

NEW:

  NOTE: Older News specifications used the term "header" as a synonym
     for what [RFC2822] calls "header field". This document follows
     the terminology of RFC 2822.

I'd also like to add two more definitions, but those are less important:

  This specification also uses the following terms:

  Header line - a line as defined in [RFC2822] section 2.1 that forms
         part of a header field.

  Header field content - the field body of a header field, not including
         leading or trailing whitespace, or comments.

I'm not sure whether the last one is what Charles wanted for his "semantic 
content" construct - but I'm sure he'll tell us if it's not.

Comments?

                     Harald

---

From: Bruce Lilly <[email-masked]>
Date: Fri, 24 Jun 2005 09:25:38 -0400

>      the terminology of RFC 2822.

I suggest s/[RFC2822] calls/[RFC2822] and MIME call/ and I suggest
adding "and MIME [RFC2045], [RFC2046], [RFC2047], [RFC2049], [RFC2231]"
to the last sentence.  I'd also like to see an informative reference to
FYI 18 added, but for the sake of conciseness of the document, I can
live without it.
 
>   Header line - a line as defined in [RFC2822] section 2.1 that forms

I'd change "header field" to "message header [RFC2822] or MIME-part-header
[RFC2046]".  ("field" in this case is unnecessary because fields are
delimited by line endings (not followed by whitespace))
 
> content" construct - but I'm sure he'll tell us if it's not.

I'd lean more towards "Structured field body semantic content" for the
term, if we actually need such a term.  I'd also change "or comments" to
"with comments and their enclosing parentheses and any adjacent line
folding and/or whitespace replaced by a single space character" [see RFC
2822 section 3.2.3].

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Fri, 24 Jun 2005 17:34:17 +0200

> FYI 18 added, but for the sake of conciseness of the document, I can

I'd consider such a change to be at best useless and at worst harmful.

If there was ever a conflict discovered between MIME and 2822 about what a 
"header field" was, this would make our "note" ambiguous.
And if there is no conflict, the change adds no information.

---

From: Henry Spencer <[email-masked]>
Date: Fri, 24 Jun 2005 12:44:28 -0400 (EDT)

On Fri, 24 Jun 2005, Harald Tveit Alvestrand wrote:
> to check that they find it possible to work with the "2822" syntax...

I really think reducing our document's readability to its primary
audience, solely for the sake of consistency with RFC 2822, is a mistake. 
Especially since the usual news terminology is commonly heard even in the
mail community now. 

If we do it, then given that 2822speak is rare in the news world, was not
used in prior news specs, is not used in the NNTP draft, and has not been
seen previously in our work, we need to include an explanatory note saying
"sorry, we're using 2822 terminology, not the usual news terminology,
because we thought consistency was more important than readability".  I'm
serious -- this is a big enough disservice to our community that it calls
for at least an apology. 

I can live with it *if* we explicitly apologize for it.

                                                          Henry Spencer
                                                       [email-masked]

---

From: Graham Drabble <[email-masked]>
Date: Fri, 24 Jun 2005 18:35:27 +0100

On 24 Jun 2005 Henry Spencer <[email-masked]> wrote in
news:[email-masked]: 

>> Since several people weighed in on the subject after my summary,
>> we seem to be moving towards a rough consensus, and I want to try
>> for a resolution.... Henry and Charles have continued to argue
>> for the "news" syntax; I'd like to check that they find it
>> possible to work with the "2822" syntax... 
> 
> audience, solely for the sake of consistency with RFC 2822, is a
> mistake. Especially since the usual news terminology is commonly
> heard even in the mail community now. 

I would agree here. We could change but I would rate consistency with 
news (past standards and current practice) as more important than 
consistence with RFC2822 and MIME.
-- 
Graham Drabble
http://users.ox.ac.uk/~sjoh1646/

---

From: "Forrest J. Cavalier III" <[email-masked]>
Date: Fri, 24 Jun 2005 14:06:49 -0400

> Especially since the usual news terminology is commonly heard even in the

2822 is a NORMATIVE reference.  We refer to and require people to
use it.  The ENTIRE audience will need to refer and understand
RFC2822.

If our documents stood by themselves, your argument would have
merit.  But there is no way to understand it without using RFC2822.

I know you advocate that the document should stand alone, but
as written, it requires going back and forth.  Doing that is a
hassle.  Doing it with similar words but different terminology
is a nightmare.

I made this point before.  I do not recall any reply addressing
the point.  Maybe we shouldn't use RFC2822 as a base.

---

From: Bruce Lilly <[email-masked]>
Date: Fri, 24 Jun 2005 15:29:29 -0400

On Fri June 24 2005 12:44, Henry Spencer wrote:

> audience,

As you well know, that is not the intent, and it is frankly offensive
that you are implying that that is the intent.

> solely for the sake of consistency with RFC 2822,

Ditto.

> used in prior news specs,

RFC 1036:

    A standard USENET message consists of several header lines, followed
...
    continuation header lines (beginning with a blank or tab) is
...

are entirely consistent with RFC 822 (and 2822).  Granted there are also
inconsistencies.

> is not used in the NNTP draft,

Not our bailiwick.

> and has not been  
> seen previously in our work,

False assertion, presumably knowingly made (this has been discussed on
list):

draft-ietf-usefor-usefor-00:

   User agents SHOULD support on receipt and MAY generate MIME extension
   header fields, including but not limited to Content-Disposition
...
   This is the only header field definition that updates [RFC2822].  The
...

> because we thought consistency was more important than readability".

Again, this is absolutely not the case, and the assertion is offensive.

---

From: Bruce Lilly <[email-masked]>
Date: Sat, 25 Jun 2005 11:52:39 -0400

> And if there is no conflict, the change adds no information.

The information added relates to MIME-part headers and the fields contained
therein, as defined in RFCs 2045 and 2046 and used in other MIME
specifications, which as noted in 2822 itself (sect. 1.1) "are outside of
the scope of" 2822.  Although there is no conflict, there is additional
information.  [rather than asserting that a specific suggested change,
which can be assumed to have been suggested for a reason, is "useless",
"harmful", "ambiguous", and "adds no information", I suggest that in future
a more professional and productive approach would be to ask a question if
you are unclear about the reason for the suggestion]

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Sun, 26 Jun 2005 21:28:07 +0200

--On 25. juni 2005 11:52 -0400 Bruce Lilly <[email-masked]> wrote:

>> If there was ever a conflict discovered between MIME and 2822 about what
>> a  "header field" was, this would make our "note" ambiguous.
> The information added relates to MIME-part headers and the fields
> contained therein, as defined in RFCs 2045 and 2046 and used in other MIME
> "harmful", "ambiguous", and "adds no information", I suggest that in
> future a more professional and productive approach would be to ask a
> question if you are unclear about the reason for the suggestion]

After reading this a couple of times, I think I get the point.
When I suggested the note:

I thought I was making a note that concerned only the use of the term 
"header", while your suggestion of adding MIME to the mix seems appropriate 
if the purpose of the note was to say that we were going to use all the 
terminology from MIME and RFC 2822 (and others). To make it clearer what I 
intended to say, I'll amend my suggested note as follows:

  NOTE: Older News specifications used the term "header" as a synonym
     for what [RFC2822] calls "header field". This document follows
     the terminology of RFC 2822 in using the term "header field",
     based on a belief that consistent terminology among specifications
     that depend on each other makes the specifications easier to use
     in the long run.

That's also as far as I think it's appropriate to go in what Henry Spencer 
calls "apologizing for the change".

---

From: Bruce Lilly <[email-masked]>
Date: Sun, 26 Jun 2005 18:47:36 -0400

On Sun June 26 2005 15:28, Harald Tveit Alvestrand wrote:

> terminology from MIME and RFC 2822 (and others).

The purpose of adding MIME is to acknowledge that MIME RFCs also use the
terms "header" (as in MIME-part[ -]header) and "header field".  Usefor
draft -04 also refers to MIME fields in section 2.3 and 3.2.  There are
differences between a message header and a MIME-part header, and the two
sets of RFCs are disjoint (I have already pointed out 2822s "out of scope"
text; RFC 2045 says "these documents are largely orthogonal to [...] RFC
822").  I don't know that we're going to use *all* of the MIME terminology,
but we certainly do refer to:
o "the character set for headers [...] MIME mechanisms"
o "MIME conformance [...] internationalization of headers"
o "MIME extension headers"
o "The MIME headers MIME-Version [...]"
o "MIME message and multipart parts"
o "[...] headers and bodies [...] using MIME mechanisms"
RFC 2047 specifically (sect. 5) pertains to MIME-part (therein called
"body part") header fields.  MIME extension fields by RFC 2045 rules begin
with "Content-" and are specifically permitted in a MIME-part header (indeed,
only those fields explicitly have meaning there; RFC 2046 sect. 5.1).
MIME-part headers of course appear in what is considered an RFC 822/2822
body.  So specifically related to use of the terms "header" and "header
field", we do use (directly and indirectly) the relevant MIME terms as
they relate to MIME-part headers.

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Mon, 27 Jun 2005 10:56:47 +0200

--On 26. juni 2005 18:47 -0400 Bruce Lilly <[email-masked]> wrote:

>> "header", while your suggestion of adding MIME to the mix seems
>> appropriate  if the purpose of the note was to say that we were going to
>> use all the  terminology from MIME and RFC 2822 (and others).
>
> terms "header" (as in MIME-part[ -]header) and "header field".

Uses. Does not define. And (naturally) refers to RFC 822, not 2822.
RFC 2045 section 1:

   All of the header fields defined in this document are subject to the
   general syntactic rules for header fields specified in RFC 822.  In
   particular, all of these header fields except for Content-Disposition
   can include RFC 822 comments, which have no semantic content and
   should be ignored during MIME processing.

We're dealing with definitions here. We should just refer to 2822.

Your argument has been made. Unless others strongly wish to support the 
inclusion, this change will not be made.

                 Harald

---

From: Bruce Lilly <[email-masked]>
Date: Mon, 27 Jun 2005 09:17:14 -0400

> Uses. Does not define.

I believe you are mistaken.  RFC 2045:

    (2)   In a MIME body part header within a multipart
          construct.

   The formal definition of these header fields is as follows:

[...]
     MIME-part-headers := entity-headers
                          [ fields ]
                          ; Any field not beginning with
                          ; "content-" can have no defined
                          ; meaning and may be ignored.
                          ; The ordering of the header
                          ; fields implied by this BNF
                          ; definition should be ignored.

That says it is a formal definition and is not covered by [2]822.

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Mon, 27 Jun 2005 18:18:29 +0200

--On 27. juni 2005 09:17 -0400 Bruce Lilly <[email-masked]> wrote:

>> > The purpose of adding MIME is to acknowledge that MIME RFCs also use
>> > the terms "header" (as in MIME-part[ -]header) and "header field".
>>
>> Uses. Does not define.
>

Defines "MIME-part-headers", not "header field". The sentence you quote 
uses the term "header field", it does not define it.
 The paragraph I'm asking to have inserted in USEFOR talks about the 
definition of the term "header field".

Since nobody else appears interested in this debate, I'll stop responding 
now.

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 27 Jun 2005 19:57:40 GMT

>  Header line - a line as defined in [RFC2822] section 2.1 that forms

I think that depends on whether and how the term gets used when the
problems you identified earlier get fixed (I hope Ken is owrking on that
one).

>content" construct - but I'm sure he'll tell us if it's not.

Please leave me to define the term in USEPRO, where it will be used. My
next job is to trawl through all the places in USEPRO that would use the
term, in order to see exactly what properties I need in order to make it
work.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 27 Jun 2005 18:45:17 GMT

>use it.  The ENTIRE audience will need to refer and understand

But I suspect that the entire audience is not going to refer to RFC 2822
(because they will just use what they "remember from last time they read
it"). And I doubt they will fully understand it, much less the
ramifications of all the other MIME etc documents that we refer to
normatively.

That is a sad fact of life. If you write a document that is not free
standing, and/or does not contain sufficient explanatory material to
assist the "less than omniscient reader", then people are going to cut
corners when they come to read it.

That is why Son-of-1036 was so long, and it is also why our earlier
article-nn drafts were so long, and it is a lesson I and my colleagues
learnt the hard way many years ago when writing the Revised Algol 68 Report.

And yet Usefor is excessively short and brief (with no examples, and few
NOTEs). Some people seem to think that is a Good Thing. We shall see.

>the point.  Maybe we shouldn't use RFC2822 as a base.

There is much merit in that, but we have already changed horses in
mid-stream once. Let us not do it again.

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 27 Jun 2005 18:53:37 GMT

>     that depend on each other makes the specifications easier to use

Add "even if it confuses the rest of the world, and leads them to complain
you are stuck in your ivory tower and would rather you had a document that
was pedantically correct rather than one of use to the people working at
the coal face".

And I suspect that "in the long run" the terminology that the marketplace
uses will eventually prevail, even for email.

I am still with Henry on this one, and I see that Graham Drabble has now
joined us, if head counting is required.

You could also argue that, since this WG has consistenly used the "header"
terminology since the earliest days, that it should require an actual
consensus to make the change before reversing that decision.

---

From: "Forrest J. Cavalier III" <[email-masked]>
Date: Mon, 27 Jun 2005 23:37:12 -0400

> corners when they come to read it.

This is a preposterous position for a WG participant, and absolutely
unacceptable position for a WG editor.

Can I ask the chairs to issue clear statements that we are NOT
writing "free-standing" documents, and that it will be impossible to
create a conforming implementation without reading RFC2822?  In evidence
that this must be so, I offer the wording in Section 3 of USEFOR-USEFOR-04.

---

From: "Forrest J. Cavalier III" <[email-masked]>
Date: Mon, 27 Jun 2005 23:39:03 -0400

> corners when they come to read it.

I object to all parts of the drafts written or inserted because
of an assumption that implementors are uncooperative and lazy.

Adopting this position officially will clear the way for removal of
inappropriate uses of MUST/NOT imperatives, result in a shorter document,
and I believe will improve the tone and usefulness of several sections.

(This mostly affects USEPRO, I think.)

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Tue, 28 Jun 2005 10:12:50 +0200

--On 27. juni 2005 23:37 -0400 "Forrest J. Cavalier III" 
<[email-masked]> wrote:

> create a conforming implementation without reading RFC2822?

We are not writing "free-standing" documents, and it will be impossible to 
create a conforming implementation without reading RFC 2822. (1)

As I've remarked in another thread: Sometimes you need to state the obvious.

                        Harald

(1) nitpicking mode ON: it's possible that some freestanding piece of 
software that performs a single function, and which uses libraries written 
by someone who's carefully studied RFC 2822, can be conforming. It's just 
quite unlikely.

---

From: Bruce Lilly <[email-masked]>
Date: Tue, 28 Jun 2005 09:45:14 -0400

On Mon June 27 2005 14:53, Charles Lindsey wrote:

> if head counting is required.

May I remind the co-chairs that there is still the unresolved
question as to the method that is to be used to determine
consensus; is it in fact a head count (in which case those of us
who have been using reasoned arguments can adopt the simpler
method of making a non-sequitur comment), or is it to be based
on reasoned arguments as was the case in the Resnick-Gierth era,
or something else?

---

From: Alexey Melnikov <[email-masked]>
Date: Tue, 28 Jun 2005 14:53:23 +0100

>consensus to make the change before reversing that decision.
I was under impression that this is exactly what this thread is all about.

Alexey

---

From: Bruce Lilly <[email-masked]>
Date: Tue, 28 Jun 2005 10:19:39 -0400

On Mon June 27 2005 14:53, Charles Lindsey wrote:

> since this WG has consistenly used the "header"
> terminology since the earliest days

Charles, please stop promulgating unsupported lies; text from
our past drafts has been cited which uses standard terminology.
Moreover, it has been stated (with supporting evidence) that
such use was self-inconsistent.

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 28 Jun 2005 20:47:01 +0200

Bruce Lilly wrote:
 
> consensus; is it in fact a head count

It's a "judgement call".  You have repeatedly posted articles
explaining why one sound argument can be more important than
several "heads".  Now what's new about this, are you testing
Scott's "trouble" filter ?  Then you have to write "voting" ;-)

                            Bye, Frank

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 28 Jun 2005 17:17:19 GMT

>that this must be so, I offer the wording in Section 3 of USEFOR-USEFOR-04.

Eh? WHen did I say that we should be writing a "free standing" document?

I pointed out that in the real world implementors and other readers DO
take shortcute when reading documents. That is a fact of life. You have to
live with it.

Therefore, people writing specifications should take care to look at what
they have written through the eyes of those likely to read them, and ensure
they are written in a manner sympathetic to those readers.

We are not trying to write a specification which is a challenge for
everyone who comes to read it, so that we can say, with great glee when
they misinterpret what we have written, "you are a moron because of your
inability to decipher what (with superhuman attention to every detail) was
plainly stated in the document.

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 28 Jun 2005 18:28:36 GMT

>As I've remarked in another thread: Sometimes you need to state the obvious.

But many more times you also need to state the "not so obvious".

Which was my whole point.

---

From: "Forrest J. Cavalier III" <[email-masked]>
Date: Wed, 29 Jun 2005 07:26:49 -0400

> Which was my whole point.

No.  You crossed the line by saying implementors could not be expected to
read and refer to a normative draft, despite section 3 refering
to RFC2822 so many times.

There will still be occasions to highlight and repeat text from
normative references.   But now there is no merit to any argument
that the draft can be written as if people can read USEFOR-USEFOR-04
and not understand RFC2822.

Are there any other substantative arguments against matching
RFC2822 terminology?  How long it takes to edit has been debunked.
How long text has been in the draft predating MESSFOR is irrelevant
now that they finished first.

---

From: "Forrest J. Cavalier III" <[email-masked]>
Date: Wed, 29 Jun 2005 08:18:37 -0400

> take shortcute when reading documents. That is a fact of life. You have to

No I do not have to live with it.  As has been said so many times: we
need a clear, complete, unambiguous specification that is as easy to read
as possible, BUT WE HAVE NO CONCERN for people who refuse to read and
understand NORMATIVE references.  We can totally ignore such people when
writing, because they are irrelevant.

A specification is never read by unthinking sheep who have to be
bullied into compliance.  And frankly, if you write that way you
insult the people who actually do read it.

Until you understand this subtle point, this is going to keep coming up
as long as you are editor.

> they are written in a manner sympathetic to those readers.

Right.  The disagreement and the chair's rule was in limiting the set
of who "is likely to read them."

Now we know that EVERY READER will be expected to understand RFC2822, and we are
sympathetic to them.  NO ONE will not have read RFC2822, so we do not have to
be sympathetic to people who are from "news only and do not expect the RFC2822
terminology.   Accomodating people who will not read the document is
unecessary and of no concern.  Doing so cannot justify any wording or text
of the drafts.

> plainly stated in the document.

Excluded Middle, hostile, and insulting.  Can't you please make a dignified
argument instead?

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 29 Jun 2005 13:25:11 GMT

>Charles, please stop promulgating unsupported lies; text from
                                               ^^^^
					       ????
>such use was self-inconsistent.

One example has been cited, from USEFOR-00, which was the initial attempt
at the new format-only document, and had not been previously been reviewed
by the WG. It was corrected in USEFOR-01.

All our previous drafts had consistently used the "header", rather than
the "header field", notation, as had Son-of-1036 before that.

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 30 Jun 2005 19:10:00 GMT

>writing, because they are irrelevant.

To describe a substantial part of your audience as "irrelevant" is no way
to write any document, whether technical or otherwise.

If that is your attitude, then we will just have to agree to differ.

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 30 Jun 2005 19:10:00 GMT

---

From: "Forrest J. Cavalier III" <[email-masked]>
Date: Thu, 30 Jun 2005 17:58:48 -0400

> If that is your attitude, then we will just have to agree to differ.
No, we don't have to agree to differ on this either.

I request the chairs issue a clear statement that we will use RFC2119
imperatives and when used we expect ALL implementors to understand them as
defined in RFC2119.  We will also expect them to understand English.

I request the chairs issue a clear statement that we expect all implementors
claiming conformance are expected to read and understand NORMATIVE references.
(I thought we already had that declaration of the obvious, but guess not...)

I request the chairs issue a clear statement that we will repeat and highlight
text from NORMATIVE references only when we have reason to believe that our
draft text is so difficult to write clearly that it is likely to mislead
an implementor into violating a normative reference.