Subject: Issue in ABNF syntax for the User-Agent: header

From: Julien ÉLIE <[email-masked]>
Date: Mon, 2 Nov 2009 23:52:28 +0100

Hi,

I read in USEFOR:

   user-agent      =  "User-Agent:" SP 1*product [CFWS] CRLF
   product         =  [CFWS] token [ [CFWS] "/" product-version ]
   product-version =  [CFWS] token

and in RFC 2045:

     token := 1*<any (US-ASCII) CHAR except SPACE, CTLs,
                 or tspecials>

     tspecials :=  "(" / ")" / "<" / ">" / "@" /
                   "," / ";" / ":" / "\" / <"> /
                   "/" / "[" / "]" / "?" / "="
                   ; Must be in quoted-string,
                   ; to use within parameter values

However, a quick "HDR User-Agent -" on news.software.nntp gives at least:

7710 Gnus/5.13 (Gnus v5.13) Emacs/23.1 (gnu/linux)
7704 slrn/pre1.0.0-15 (Linux)
7660 tin/1.9.5-20091003 ("Rieclachan") (UNIX) (Linux/2.6.24-24-generic (i686))
7659 Thunderbird 2.0.0.23 (Windows/20090812)
7600 MacSOUP/2.8.2 (Mac OS X version 10.5.8 (x86))
7594 Turnpike/6.07-M (<wztiu2+cqs7vk6mlt6f268lB5p>)
7587 Pan/0.133 (House of Butterflies)

All of them are not USEFOR-compliant!

Is it normal?!?

Isn't something missing in the ABNF syntax?  If I read RFC 2626 for HTTP, I see:

   User-Agent     = "User-Agent" ":" 1*( product | comment )
   comment        = "(" *( ctext | quoted-pair | comment ) ")"
   ctext          = <any TEXT excluding "(" and ")">

which seems to be more accurate thant what USEFOR mentions.

Incidentally, it is normal we have [CFWS] for a few headers and not others?
No initial folding for Path:
   path            =  "Path:" SP *WSP path-list tail-entry *WSP CRLF
   path-list       =  *( path-identity [FWS] [path-diagnostic] "!" )
but a possible one for Archive:
   archive         =  "Archive:" SP [CFWS] ("no" / "yes")
                      *( [CFWS] ";" [CFWS] archive-param ) [CFWS] CRLF

I think that's OK, but better be sure...

-- 
Julien ÉLIE

« A man inserted an 'ad' in the classifieds: "Wife wanted".
  Next day he received a hundred letters.  They all said
  the same thing:  "You can have mine." »

---

From: Antti-Juhani Kaijanaho <[email-masked]>
Date: Tue, 3 Nov 2009 06:58:38 +0200

>     tspecials :=  "(" / ")" / "<" / ">" / "@" /

Do you mean "None of them are" or "At least one of them is"?

In any case, as far as I can seem they all comply.  Can you explain in more
detail why you think they don't?

> Incidentally, it is normal we have [CFWS] for a few headers and not others?

I believe it is intentional.

-- 
Antti-Juhani Kaijanaho, Jyväskylä, Finland
http://antti-juhani.kaijanaho.fi/newblog/
http://www.flickr.com/photos/antti-juhani/

---

From: Julien ÉLIE <[email-masked]>
Date: Tue, 3 Nov 2009 18:31:42 +0100

Hi Antti-Juhani,

> detail why you think they don't?

Because a parenthesis, for instance "(", is NOT allowed in a token.

I believe USEFOR should introduce the notion of a comment here (which allows
such a construction, as RFC 2626 does), besides the product.

-- 
Julien ÉLIE

« La différence entre un chanteur et une paire de chaussures est que le chanteur
  doit partir avant de lasser. La paire de chaussures, il vaut mieux les lacer
  avant de partir. » (Philippe Geluck)

---

From: Bill McQuillan <[email-masked]>
Date: Tue, 3 Nov 2009 10:11:28 -0800

> Because a parenthesis, for instance "(", is NOT allowed in a token.

CFWS allows a <commment>

-- 
Bill McQuillan <[email-masked]>

---

From: Antti-Juhani Kaijanaho <[email-masked]>
Date: Tue, 3 Nov 2009 20:15:27 +0200

> Because a parenthesis, for instance "(", is NOT allowed in a token.

Of course not - but the parenthetical statements are not tokens, they are
CFWSs, which are allowed between tokens in the grammar quoted above.

> such a construction, as RFC 2626 does), besides the product.

No need to introduce - it already allows them.

---

From: Julien ÉLIE <[email-masked]>
Date: Tue, 3 Nov 2009 19:35:19 +0100

Hi,

[Bill]
>> Because a parenthesis, for instance "(", is NOT allowed in a token.

Ah, I've missed that!  And CFWS allows consecutive comments too.

[Antti]
> CFWSs, which are allowed between tokens in the grammar quoted above.

Well, sorry for the disturbance...

-- 
Julien ÉLIE

« Je suis capable du meilleur comme du pire,
  mais pour le pire, c'est moi le meilleur. » (Coluche)