Subject: Just got AUTH48 notice for USEPRO

From: Russ Allbery <[email-masked]>
Date: Fri, 10 Apr 2009 14:58:01 -0700

I just got the AUTH48 notice for USEPRO.  I'll try to look at it soon.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 15 Apr 2009 13:54:02 GMT

>I just got the AUTH48 notice for USEPRO.  I'll try to look at it soon.

As a joint author of both documents, I get to see the AUTH48 comments of
both of them :-( .

I have looked as USEFOR so far, and the RFC-Editor has changed a host of
nits to meet his "house style" and they mostly seem fine.

But there is one big issue, and that is he wants to replace RFC 2822 with
RFC 5322 throughout. That is fine by me, but if we do it for one we need
to do it for both, and in the case of USEFOR it would require a
considerable pruning of the syntax of <msg-id>.

Essentially, the USEFOR syntax is a strict subset of the RFC 2822 syntax,
but NOT of the RFC 5322 syntax. And since the changes of syntax between
2822 and 5322 were made at the express request of us Usenet folk, it would
seem churlish not to adopt them (especially since doing so would remove a
singularly long and ugly set of now unnecessary syntax rules).

Therefore, I propose that we should do that. I will try to produce the
necessary textual changes in the next few days.

BTW, our two drafts will become RFC 5536 and 5537, which seems kind neat!

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131            Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 17 Apr 2009 15:36:34 GMT

>necessary textual changes in the next few days.

OK, I think the issue of whether to rebase the documents on RFC5322 rather
than RFC2822 requires some input from the WG. The Rfc-Editor seems to
think it should be done, and she has provisionally edited both documents
that way. And it it will indeed get us out of various awkward corners
regarding the IANA registrations of the various header fields and other
such editorial confusions, so I personally would support it.

In USEPRO (now RFC5537) it is easily done, but in USEFOR (now RFC5536)
there are some consequential changes that would be needed. Here is what I
have reported to the other editors for their consideration. Some comment
on my proposals from the WG is desirable.

-----------------------------------------------------------------------------

At the time the USEFOR draft was submitted for Last Call, RFC 2822 was
still current. For severe technical reasons, the syntax of <msg-id> was
totally unsuited to Netnews, hence the cumbersome and ugly syntax you see
in 3.1.3 which contained the minimal change that would work and yet still
allow us to say, in 2.2 (and also in Appendix C):

   ....  The syntax allowed for Netnews
   article headers is a strict subset of the "Internet Message Format"
   headers, making all headers compliant with this specification
   inherently compliant with [RFC2822].  ...

When the update to RFC 2822 was being discussed, some of us lobbied for
changes to the <msg-id> syntax that would avoid those technical problems
with Netnews and, much to our surprise, we managed to get 95% of what we
wanted. In fact, they went further than we asked, by entirely removing
NO-WS-CTL, <quoted-string>s and <quoted-pair> from <msgid>, which was
fine, except that now Netnews has become, in some parts, a superset of the
new RFC 5322, so the sentence quoted above is no longer true.

Personally, I would be happy to move the whole thing to be based on RFC
5322, but it would require some consequential changes to restore that
'subset' situation. But I think we would need to check back with the WG
before going ahead. Here is what would be needed:

A. RFC 5322 no longer defines <utext>. Instead, it uses VCHAR from RFC
5234, which comprises the old <utext> with all the control characters
(NO-WS-CTL) omitted (and we don't really want those control characters
either). So, in our section 2.2 you would now write:

   unstructured    =  *WSP VCHAR *( [FWS] VCHAR ) *WSP

(which still differs from <unstructured> in RFC 5322 by requiring at least
one visible character). You also need to remove <utext> from 1.4 and add
VCHAR in its place.

B. I see that <date-time> in RFC 5322 has added an extra FWS that was not
there before, but that is compensated by removal of an FWS in <year>, so I
do not think it affects us. Likewise various other shuntings around of FWS
within <date-time>.

C. Section 3.1.3 would need to be written as follows:

3.1.3.  Message-ID

   The Message-ID header field contains a unique message identifier.
   Netnews is more dependent on message identifier uniqueness and fast
   comparison than Email is, and some news software and standards
   [RFC3977] might have trouble with the full range of possible
   <msg-id>s permitted by [RFC5322].  This section therefore restricts
   the syntax of <msg-id> as compared to Section 3.6.4 of [RFC5322].
   The global uniqueness requirement for <msg-id> in [RFC5322] is to be
   understood as applying across all protocols using such message
   identifiers, and across both Email and Netnews in particular.

   message-id      =  "Message-ID:" SP *WSP msg-id *WSP CRLF

   msg-id          =  "<" msg-id-core ">"
                      ; maximum length is 250 octets

   msg-id-core     =  id-left "@" id-right

{{EDITORIAL NOTE: Those still remove the possibility of <comment>s; it
also introduces <msg-id-core> which is a useful concept (though I don't
think either document uses it any more).}}

   id-left         =  dot-atom-text

   id-right        =  dot-atom-text / no-fold-literal

{{EDITORIAL NOTE: Those are actually identical to RFC 5322 if you apply
the blanket removal of obs-syntax in 2.1; but it might be kinder to leave
them in.}}

   no-fold-literal =  "[" *mdtext "]"

   mdtext          =  %d33-61 /        ; The rest of the US-ASCII
                      %d63-90 /        ; characters not including
                      %d94-126         ; ">", "[", "]", or "\"

{{EDITORIAL NOTE: <mdtext> replaces <dtext> from RFC 5322 (or you could
simply redefine <dtext>); the effect is to remove ">".}}

   The msg-id MUST NOT be more than 250 octets in length.

      NOTE: The length restriction ensures that systems that accept
      message identifiers as a parameter when referencing an article
      (e.g.  [RFC3977]) can rely on a bounded length.

   Observe that msg-id includes the < and >.

   Observe also that in contrast to the corresponding header field in
   [RFC5322]:

   o  The syntax does not allow comments within the Message-ID header
      field,

   o  There is no possibility for ">" to occur inside a <msg-id>.

{{EDITORIAL NOTE: I have removed the bullet about <id-rights>s now being
case-sensitive because I cannot see that they ever were so even in RFC
2822, and if they were then we have done nothing to change that.}}

   This is to simplify processing by news servers and to ensure
   interoperability with existing implementations and compliance with
   [RFC3977]. Moreover, a simple comparison of octets will always suffice
   to determine the identity of two <msg-id>s.

{{EDITORIAL NOTE: I think that last sentence is now also true of RFC 5322,
but it is still useful to emphasise it.}}

   Also note that this updated ABNF applies wherever <msg-id> is used,
   including the References header field discussed in Section 3.2.10 and
   the Supersedes header field discussed in Section 3.2.12.

   Some current software may try to match the <id-right> of a <msg-id> in
   a case-insensitive fashion; some may match it in a case-sensitive
   fashion.  Implementations MUST NOT generate a Message-ID where the only
   difference from another Message-ID is the case of characters in the
   <id-right> part.

   When generating a <msg-id>, implementations SHOULD use a domain name
   as the <id-right>.

      NOTE: Section 3.6.4 of [RFC5322] recommends that the <id-right>
      should be a domain name or a domain literal.  Domain literals are
      troublesome since many IP addresses are not globally unique;
      domain names are more likely to generate unique Message-IDs.

Comparison of all that with the present text will show that it is
considerably shorter, and much easier to understand.

-----------------------------------------------------------------------------

---

From: Julien ÉLIE <[email-masked]>
Date: Sat, 18 Apr 2009 13:19:03 +0200

Hi Charles,

> NO-WS-CTL, <quoted-string>s and <quoted-pair> from <msgid>

That's a nice improvement.  It then also simplifies what RFC 3977 states
about such quoted strings:

   This specification states that message-ids are the same if and only
   if they consist of the same sequence of octets.  Other specifications
   may define two different sequences as being equal because they are
   putting an interpretation on particular characters.  RFC 2822
   [RFC2822] has a concept of "quoted" and "escaped" characters.  It
   therefore considers the three message-ids:

      <[email-masked]>
      <"ab.cd"@example.com>
      <"ab.\cd"@example.com>

   as being identical.  Therefore, an NNTP implementation handing email
   articles must ensure that only one of these three appears in the
   protocol and that the other two are converted to it as and when
   necessary, such as when a client checks the results of a NEWNEWS
   command against an internal database of message-ids.  Note that
   RFC 1036 [RFC1036] never treats two different strings as being
   identical.  Its successor (as of the time of writing) restricts the
   syntax of message-ids so that, whenever RFC 2822 would treat two
   strings as equivalent, only one of them is valid (in the above
   example, only the first string is valid).

It is now clearer with RFC 5322 and... 5536!

> one visible character). You also need to remove <utext> from 1.4 and add

It still a subset of RFC 5322 even though our <unstructured> is not contained
in theirs (we do not allow "Distribution:\r\n" whereas RFC 5322 does
for instance).

I think your new definition is good.

> do not think it affects us. Likewise various other shuntings around of FWS

But we still keep obsolete GMT-like constructions.

> considerably shorter, and much easier to understand.

Absolutely.  I believe your suggested changes should be applied during AUTH48.

-- 
Julien ÉLIE

« Rubor, tumor, dolor, calor et functio laesa. »

---

From: Harald Alvestrand <[email-masked]>
Date: Tue, 21 Apr 2009 12:51:17 +0200

I think I like this, but would definitely want other people to take a 
look too.

              Harald

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 21 Apr 2009 15:17:00 GMT

>But we still keep obsolete GMT-like constructions.

Of course? It is there because CNEWS, and maybe others, still does it.

>Absolutely.  I believe your suggested changes should be applied during AUTH48.

OK, that makes two of us. Anyone else?

---

From: Russ Allbery <[email-masked]>
Date: Tue, 21 Apr 2009 17:09:15 -0700

> I think I like this, but would definitely want other people to take a

I agree with rebasing the documents on RFC 5322 and Charles's text looks
good to me.

(I'm completely swamped and will probably not be able to get to other
AUTH48 comments or concerns until next weekend at the earliest.)

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 22 Apr 2009 11:50:25 GMT

>I agree with rebasing the documents on RFC 5322 and Charles's text looks

OK, there semms to be something close to Rough Consensus on that. So where
is Ken Murchison? I have not seen him on this list for a long time, and he
has not yet appeared on the thread of emails that has been passing
between myself, the Rfc-Editor, and the various authors.

---

From: Ken Murchison <[email-masked]>
Date: Fri, 24 Apr 2009 09:10:46 -0400

> between myself, the Rfc-Editor, and the various authors.

I'm here.  I've been putting out some fires with our campus in Qatar.  I 
will carve out some time to work on this next week.

-- 
Kenneth Murchison
Systems Programmer
Carnegie Mellon University