Subject: Re: AUTH48 is stuck

From: Harald Alvestrand <[email-masked]>
Date: Mon, 02 Nov 2009 06:05:23 +0100

I've woken up now, and will try to deal with this.

Charles, I have your diffs from September 21. I will try to inspect 
them, and if I approve them, I think we can get this unstuck.

                       Harald

Charles Lindsey wrote:
> RFC 5536 (aka our USEFOR) draft is stuck in AUTH 48.
>
> The problem is that it is agreed it needs to refer to RFC 5532 rather than
> RFC 2822, but RFC has changed the syntax of Message-ID (considerably
> simplified it) and this has consequential effects on the corresponding
> syntax in RFC 5536.
>
> I published the changes I considered desirable (a considerable
> simplification in fact) in this WG months and months ago, and I thought we
> had rough consensus that they should go ahead - certainly several people
> seemed happy with the idea, and none opposed it.
>
> But I am only the second editor. Ken Murchison is the primary editor and
> holds the responsibility for communicating the changes to the RFC Editor,
> unless either he or Harald decides or requests otherwise. But there has
> been total silence from Ken, apart from two messages to say he would do it
> "RSN".
>
> Every month, the RFC Editor circulates to the editors and others concerned
> an enquiry as to whether the changes are yet available, and each month I
> reply saying we are still waiting for Ken.
>
> In desperation a few weeks ago, I prepared the diffs I considered
> necessary in the required format, and sent them to Ken (CC to Harald)
> with a request that he either adopted them, or did his own, or authorized
> me to submit them myself. But there has been no reply.
>
> How can we proceed if neither Ken nor Harald are replying to emails? And
> how else can we proceed?
>
>

---

From: Julien ÉLIE <[email-masked]>
Date: Tue, 3 Nov 2009 19:17:26 +0100

Hi Charles,

>> If I understand well the new syntax in USEFOR, the following message-IDs
>> are invalid:
>>
>> <"test"@test>
>> <te"st@test>
>> <test@[te\st]>
>> <test@[te\\st]>
>>
>> whereas the following message-ID is valid:
>>
>> <test@[te"st]>
>
> Correct. And the same is true in RFC 5322 also.

OK.
INN is currently only seeing the second one as invalid.  The four other
are considered to be valid.

CHECK <"test"@test>
238 <"test"@test>
CHECK <te"st@test>
438 <te"st@test>
CHECK <test@[te\st]>
238 <test@[te\st]>
CHECK <test@[te\\st]>
238 <test@[te\\st]>
CHECK <test@[te"st]>
238 <test@[te"st]>
CHECK <bad mid@test>
438 <bad mid@test>

Note that 438 is sent (refusal to receive the article) instead of 501
(syntax error) because of backwards compatibility with existing news
servers, contrary to what RFC 3977 suggests.

> But some strange variants
> such as <"te..st@test> are allowed in the present USEFOR syntax.

INN currently considers it to be invalid.  So that's fine because
the new USEFOR syntax disallows it.

CHECK <"te..st@test>
438 <"te..st@test>

Is it OK for news servers to switch to the new USEFOR syntax right
now?  or would it cause problems?

We can use that new syntax, starting with INN 2.5.2.

-- 
Julien ÉLIE

« La différence entre un chanteur et une paire de chaussures est que le chanteur
  doit partir avant de lasser. La paire de chaussures, il vaut mieux les lacer
  avant de partir. » (Philippe Geluck)

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 5 Nov 2009 16:51:23 GMT

>are considered to be valid.

I think that is more or less what I would expect, since those four would
be valid under RFC 2822, but not under RFC 5322. I don't think we need to
care whether servers accept the invalid cases. Indeed I am surprised that
INN bothers to test them at all. The normal implementation should just
extract eveything within <...>, and see if it is matched in the history
file.

As for sites that attempt to _generate_ msg-ids, then they certainly
should not be doing it. But the plain fact is that nobody ever generates
msg-ids of those formats (have you ever seen one in the wild?). So there
is really no problem.

>CHECK <"test"@test>

CHECK is not a standard NNTP command BTW.

>We can use that new syntax, starting with INN 2.5.2.

Yes, because in practice the awkward cases will never arise.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131            Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: Julien ÉLIE <[email-masked]>
Date: Thu, 5 Nov 2009 18:43:25 +0100

Hi Charles,

> I am surprised that

Because first of all we check the syntax of NNTP commands:

IHAVE <te"st@test>
435 Syntax error in message-ID

(We *reject* the message with 435, not 501.)

which is also done for readers:

ARTICLE bad-id
501 Syntax error in message-ID

ARTICLE 12a
501 Syntax error in article number

Secondly, the Message-ID: header will be checked in the message:

IHAVE <valid@mid>
335 Send it
From: "Demo User" <[email-masked]>
Newsgroups: fr.test
Subject: I am just a test article
Date: 5 Nov 2009 04:38:40 -0500
Path: me
Message-ID: nasty-one!

Test.
.
437 Bad "Message-ID" header

RFC 3977:

   If the server is also conforming
   to another specification that contains a definition of message-id
   compatible with this one, the server SHOULD use those message-ids.

I also see in USEPRO that:

   [A relaying agent] MAY reject any article that contains header
   fields that do not have valid contents.

so I believe that's OK.

> CHECK is not a standard NNTP command BTW.

The result would have been the same with IHAVE.  See my examples above.

> Yes, because in practice the awkward cases will never arise.

OK, thanks.

-- 
Julien ÉLIE

« La science consiste à passer d'un étonnement à l'autre. » (Aristote)

---

From: Russ Allbery <[email-masked]>
Date: Thu, 05 Nov 2009 09:48:43 -0800

"Charles Lindsey" <[email-masked]> writes:
> =?Windows-1252?Q?Julien_=C9LIE?= <[email-masked]> writes:

> CHECK is not a standard NNTP command BTW.

RFC 4644 is standards track.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 6 Nov 2009 14:38:31 GMT

>RFC 4644 is standards track.

OK, I got that now. It essentially means "I have an article with this
msg-id; would you like me to send it?" But in fact there is no great need
for the server to check the syntax of the msg-id (though there is nothing
wrong or harmful in doing so). The client has no business offering a bad
msg-id in the first place (and wherever it got it from should not have
sent it, right back to who/whatever constructed it in the first place).

If a bad msg-id _does_ enter the system, then it make well propagate
perfectly well. Then again it may not (some servers may not like it; some
servers might munge it; garbage-in -> garbage-out and all that). What the
standards say is that a well-formed msg-id MUST/SHOULD progagate correctly
(modulo the right of any site to reject anything for reasons of local
policy, of course).

---

From: Russ Allbery <[email-masked]>
Date: Fri, 06 Nov 2009 11:44:50 -0800

"Charles Lindsey" <[email-masked]> writes:

> msg-id; would you like me to send it?" But in fact there is no great
> need for the server to check the syntax of the msg-id (though there is
> nothing wrong or harmful in doing so). The client has no business
> offering a bad msg-id in the first place (and wherever it got it from
> should not have sent it, right back to who/whatever constructed it in
> the first place).

Yes, there's no major need, but it's not a bad idea anyway.  There is some
interoperability justification in rejecting malformed message IDs, since
some servers behave poorly if you send them one, in ways that can cause
problems in correctly identifying the error and understanding which
article to not send on to the badly behaving server.  Rejecting them
up-front is in some ways cleaner.

This is particularly true of excessively long message IDs, which can cause
all sorts of weird problems.

Message IDs containing whitespace are in general just a bad idea, too,
since they're likely to break NNTP command parsing in numerous places down
the line.