Subject: Headers folding

From: Julien ÉLIE <[email-masked]>
Date: Sat, 18 Apr 2009 20:19:36 +0200

Hi,

As you are speaking about headers and compliance of USEFOR with
RFC 3977 and RFC 5322, I wonder what happens to folding.

In RFC 3977, we have:

9.7.  Articles

     header = header-name ":" [CRLF] SP header-content CRLF
     header-content = *(S-CHAR / [CRLF] WS)
     S-CHAR   = %x21-FF
     WS = 1*(SP / TAB)
     SP  = %x20
     TAB = %x09

A.1.  Header Folding

   NNTP allows a header line to be folded (by inserting a CRLF pair)
   before any space or TAB character.

   Both email and Netnews articles are required to have at least one
   octet other than space or TAB on each header line.  Thus, folding can
   only happen at one point in each sequence of consecutive spaces or
   TABs.  Netnews articles are further required to have the header name,
   colon, and following space all on the first line; folding may only
   happen beyond that space.  Finally, some non-conforming software will
   remove trailing spaces and TABs from a line.  Therefore, it might be
   inadvisable to fold a header after a space or TAB.

   For maximum safety, header lines SHOULD conform to the following
   syntax rather than to that in Section 9.7.

     header = header-name ":" SP [header-content] CRLF
     header-content = [WS] token *( [CRLF] WS token )

     token     = 1*P-CHAR
     P-CHAR    = A-CHAR / UTF8-non-ascii
     A-CHAR    = %x21-7E
     UTF8-non-ascii = UTF8-2 / UTF8-3 / UTF8-4
     UTF8-2    = %xC2-DF UTF8-tail
     UTF8-3    = %xE0 %xA0-BF UTF8-tail / %xE1-EC 2UTF8-tail /
                 %xED %x80-9F UTF8-tail / %xEE-EF 2UTF8-tail
     UTF8-4    = %xF0 %x90-BF 2UTF8-tail / %xF1-F3 3UTF8-tail /
                 %xF4 %x80-8F 2UTF8-tail
     UTF8-tail = %x80-BF

Well, my question is for 9.7:

    header = header-name ":" [CRLF] SP header-content CRLF

Is it a valid header for USEFOR?  and for RFC 5322?

In A.1, it looks strange to have written the syntax this way:

     header-content = [WS] token *( [CRLF] WS token )

Hopefully it works!  (token does not contain WS but it comes
afterwards with "WS token"!)

-- 
Julien ÉLIE

« Rubor, tumor, dolor, calor et functio laesa. »

---

From: Harald Alvestrand <[email-masked]>
Date: Tue, 21 Apr 2009 12:58:32 +0200

> Is it a valid header for USEFOR?  and for RFC 5322?
I'm sure it is possible to construct a header that conforms to this 
syntax but isn't valid in USEFOR (the header "Date: nonsense" is illegal 
both in USEFOR and in RFC 5322, but conforms to the syntax above).

The syntax above also (and perhaps more importantly) allows UTF-8 in 
headers, something that is illegal in RFC 5322 and USEFOR. You should 
check for conformance with the EAI spec.

---

From: Julien ÉLIE <[email-masked]>
Date: Tue, 21 Apr 2009 18:07:13 +0200

Hi Harald,

> I'm sure it is possible to construct a header that conforms to this syntax but isn't valid in USEFOR (the header "Date: nonsense" 
> is illegal both in USEFOR and in RFC 5322, but conforms to the syntax above).
>
> The syntax above also (and perhaps more importantly) allows UTF-8 in headers, something that is illegal in RFC 5322 and USEFOR.

OK, thanks a lot.
Have a nice day,

-- 
Julien ÉLIE

« Je viens de rencontrer Isocèle : il a une idée pour un nouveau
  triangle. » (Woody Allen)

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 21 Apr 2009 15:15:03 GMT

>     header-content = [WS] token *( [CRLF] WS token )

I think that comes to the same thing as USEFOR (except it is less
restrictive as regards the quoted-strings and whatnot).

>Is it a valid header for USEFOR?  and for RFC 5322?

I think it is valid in RFC 5322, but NOT in RFC 3977 and NOT in USEFOR.

>afterwards with "WS token"!)

I think that is just to allow extra WS once you have got past the
obligatory first SP. I think it works.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131            Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: Julien ÉLIE <[email-masked]>
Date: Tue, 21 Apr 2009 19:24:38 +0200

Hi Charles,

> I think it is valid in RFC 5322, but NOT in RFC 3977 and NOT in USEFOR.

This syntax is taken from RFC 3977, which says:

   Header lines SHOULD NOT be folded before the space
   after the colon that follows the header name.

It is not a MUST NOT :)
Of course it is better not to do such a folding.

> obligatory first SP. I think it works.

OK.  Yes it works.

-- 
Julien ÉLIE

« Je viens de rencontrer Isocèle : il a une idée pour un nouveau triangle. » (Woody Allen)

---

From: Russ Allbery <[email-masked]>
Date: Tue, 21 Apr 2009 17:10:53 -0700

> check for conformance with the EAI spec.

Yes, this was done intentionally for NNTP with the understanding that
for the time being articles carried by NNTP will probably have to
satisfy a more restrictive standard that doesn't allow UTF-8.  Since
NNTP doesn't, for the most part, care about the contents of the message,
we took this approach so that should UTF-8 be added to a message format
standard in the future, NNTP will hopefully not require revision.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>