Subject: Case-sensitivity in newsgroup names and message-IDs

From: Julien ÉLIE <[email-masked]>
Date: Fri, 5 Jun 2009 07:39:39 +0200

Hi,

Following a discussion between Mateusz Viste, Russ Allbery, Urs Janßen
and me on news.software.nntp (see <[email-masked]>),
it appears there is a problem with case-sensitivity of a newsgroup name
and a message-ID.

The following paragraph from RFC 3977 applies:

3.1.  Commands and Responses

   Keywords are case insensitive; the case of keywords for commands MUST
   be ignored by the server.  Command and response arguments are case or
   language specific only when stated, either in this document or in
   other relevant specifications.

In practice, news servers implement:

GROUP news.software.nntp
211 3219 149 6892 news.software.nntp

GROUP news.software.NNTP
411 No such group news.software.NNTP

that is to say that news.software.nntp and news.software.NNTP are *not*
the same newsgroup.  The same behaviour occurs if I use:

Newsgroups: news.software.NNTP

in the headers of an article.  The post is rejected.

Message-IDs are also parsed case-sensitively but that fact is not
said in RFC 3977.

Besides, I read in RFC 5536 (USEFOR):

   Some software will try to match the <id-right> of a <msg-id> in a
   case-insensitive fashion; some will match it in a case-sensitive
   fashion.  Implementations MUST NOT generate a Message-ID where the
   only difference from another Message-ID is the case of characters in
   the <id-right> part.

   NOTE:  All-digit <component>s conflict with one widely used storage
   scheme for articles.  Mixed-case groups cause confusion between
   systems with case-sensitive matching and systems with case-
   insensitive matching of <newsgroup-name>s.

It does not say pretty well what is the expected behaviour!  For
message-IDs, only <id-right>s are dealt with.  What for <id-left>?!?
(In RFC 5322?  but I did not find it clearly.)
And nothing explicitly states that newsgroups are case-sensitive...

In Son of 1036, it was said:

| 2.5. Case-Sensitivity
|
|          Text  in  newsgroup  names, header parameters, etc. is case-
|          sensitive unless stated otherwise.
|
|               NOTE: This is at  variance  with  MAIL,  which  is
|               case-insensitive  unless  stated otherwise, but is
|               consistent  with  news  historical  practice   and
|               existing news software.  See the comments on back-
|               ward compatibility in section 1.

|          As with From addresses, a message ID's local part  is case-
|          sensitive  and  its domain is case-insensitive.  The "<" and
|          ">" are parts of the message ID, not  peculiarities  of the
|          Message-ID header.

|               NOTE: Possibly newsgroup names  should  have  been
|               case-insensitive, but all existing software treats
|               them as case-sensitive.   (RFC  977  [rrr]  claims
|               that they are case-insensitive in NNTP, but exist-
|               ing implementations are believed to ignore  this.)
|               The simplest solution is just to ban use of upper-
|               case letters, since no widespread  newsgroup  name
|               uses  them  anyway; this avoids any possibility of
|               confusion.

Such comments are no longer present in USEFOR.
Shouldn't something be done on that subject?

-- 
Julien ÉLIE

« Les idées fausses ne sont pas toujours mauvaises. » (Marcel Aymé)

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 8 Jun 2009 11:24:14 GMT

>And nothing explicitly states that newsgroups are case-sensitive...

That is because various systems treat them in various ways :-( .

USEFOR makes it clear that all newsgroups names SHOULD be in LC, but
recognizes that some existing ones might have UC in them, in which case
offering to NNTP (or to whatever else) exactly what the 'official' name of
the group is (as it appears in a checkgroups, if you are lucky enough to
be in a hierarchy that publishes those) is likely to succeed (mostly). But
USEFOR makes it clear that all _new_ groups MUST be LC throughout.

So the only thing you are entitled to "expect" is that if you offfer LC
names in order to retrieve LC groups, then that will _always_ work.
Anything else is problematical, so don't do it.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131            Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: Russ Allbery <[email-masked]>
Date: Mon, 08 Jun 2009 11:25:19 -0700

"Charles Lindsey" <[email-masked]> writes:

> recognizes that some existing ones might have UC in them, in which
> case offering to NNTP (or to whatever else) exactly what the
> 'official' name of the group is (as it appears in a checkgroups, if
> you are lucky enough to be in a hierarchy that publishes those) is
> likely to succeed (mostly). But USEFOR makes it clear that all _new_
> groups MUST be LC throughout.
>
> So the only thing you are entitled to "expect" is that if you offfer
> LC names in order to retrieve LC groups, then that will _always_ work.

The problem is, we don't actually say this anywhere that I can see, and
historically several NNTP documents have claimed that newsgroup names
are case-insensitive.  The current NNTP standard says:

which means in the absence of some specific statement to the contrary,
newsgroup names are case-insensitive.  However, in practice, I thought
they were always case-sensitive.  However, I've not done a survey of
what news software does outside of INN.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 9 Jun 2009 11:28:07 GMT

>are case-insensitive.  The current NNTP standard says:

I think it is what section 3.1.4 of USEFOR is _trying_ to say (at least,
that is what my analysis was based on). Maybe a little further
word-smithing in USEFOR would help clarifiy things?