Subject: Re: msg-id

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 1 Dec 2004 12:58:01 GMT

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:

> [id-right]
>> there are other ways that can be done. Look at Agent, which
>> routinely uses the same id-right for everybody.

>AFAIK they use a domain reserved for this purpose, within the
>syntax of 2821, no problem for other domains.

Yes, but in order to ensure that the id-left is unique, they rely on
generating a random number long enough that the probability of getting an
accidental duplicate is reduced to once per thousand years, or somesuch. I
have no problem with that as an adequate implementation technique.

>Message-IDs are essential, we're supposed to get this right,
>it's no fight with RfC 2822bis, let them fix their own texts.

>BTW, where is this mythical 2822bis ?  Google doesn't find a
>2822bis at ietf.org, there's no active WG listed at ietf.org,
>any WG even remotely related to mail or MIME is apparently
>closed, we're the last survivors.

It never got beyond some preliminary discussions on the ietf-822 mailing
list.

>> The IETF is too packed with people who believe that "The
>> Internet Message Format" is some inviolable edifice that we
>> "News" people are refusing to bow down before

>Now you're confusing Bruce with "the IETF", and he finds all
>bugs, we're lucky to have him.  He finds even bugs he doesn't
>want to find. ;-)

>And you're listed in the RfC 2822 credits, together with Russ,
>Alexej, Paul, and Graham, no conspiracy to screw-up net news.

Hey! I never noticed that!

Yes, we found some problems with <msg-id) which they fixed. We found some
further problems just as they were going to last call, and we found some
more after that. As you say, the basic concept of the RFC 2822 <msg-id> is
broken, but there it is.

>> But if the WG really wants to follow that route ..........

>...as long as you restrict 2822 constructs to a proper subset
>you're free to do "the right thing".  Back to Message-IDs, you
>have:

Yes, there are lots of ways it might be fixed, but I don't want to do it,
and it now seems Russ doesn't want to do it. And without some signs of a
consensus to do it it isn't going to happen.

But if people have other views to express, then let us hear them.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 02 Dec 2004 05:17:17 +0100

Charles Lindsey wrote:

> they rely on generating a random number long enough that the
> probability of getting an accidental duplicate is reduced to
> once per thousand years, or somesuch.

Not good enough for the MUST in 2822, and it won't be allowed
anymore with your drafts.  At the moment it's at least dubious.

> I have no problem with that as an adequate implementation
> technique.

But I have, it's really annoying to find these problems if an
article is apparently lost, or if Groups.Google accepts dupes
keeping only the newest instead of older identical Message-IDs.

Okay, in that particular case it's possible to say "fix your
newsreader" or "ignore deja.news", but that's not the general
idea of a Message-ID:

>> Message-IDs are essential, we're supposed to get this right

 [2822bis] 
> It never got beyond some preliminary discussions on the
> ietf-822 mailing list.

Okay, 2822bis is a fata morgana, but your drafts are real, and
they should be convincing for s-o-1036 fundamentalists.  The
Message-ID doesn't convince me.
  
Some days ago somebody asked in de.admin.news.misc about his
Message-ID <xyz@news-fe-01>, and the consensus was that it's
wrong.  The ..!news-fe-01!.. in his Path: wasn't much better.

> the basic concept of the RFC 2822 <msg-id> is broken, but
> there it is.

If it's broken let's fix it, with the same strategy for both
sides.  If the strategy for the RHS is "case sensitive", then
it should be "keep quoted pairs literally" in the LHS.

If the strategy is to get rid of any quoted pairs in the LHS,
then it's also okay to treat the RHS as "case insensitive".

But at the moment you want a canonical form for the LHS, and
a magical token for the RHS, I just don't understand it.  Why
not use the same strategy for both sides, whatever it is ?

> it now seems Russ doesn't want to do it.

He said something in the general direction of "if it ain't
broken don't fix it", but a NO-WS-CTL in Message-IDs is not
only broken = forbidden by RfC 1036, it's IMNSHO ridiculous.

     NOTE:  News message IDs are a restricted subset of
     MAIL message IDs.  In particular, no existing news
     software  copes properly with MAIL quoting conven-
     tions within the local part, so they  are  forbid-
     den.

If that was true 1994, why do you think that it's wrong now ?
Is there any evidence that s-o-1036 is completely obsolete, a
statistics about Message-ID syntax in news after 2822 maybe ?

                            Bye, Frank

---

From: "Charles Lindsey" <[email-masked]>
Date: Sat, 4 Dec 2004 20:36:55 GMT

>anymore with your drafts.  At the moment it's at least dubious.

It's a MUST in RFC 2822, and the current USEFOR draft inherits that MUST,
no more and no less.

It has been discussed peridically in this WG, and the opinion expressed
has always been that one failure per thousand years (whatever - I haven't
calculated the exact probability for the Agent scheme) could be regarded
as satisfying that MUST for all practical purposes. Nobody ever asked for
special wording to cover that case.

>keeping only the newest instead of older identical Message-IDs.

If Google makes a mistake once every thousand years for this particular
cause, then that is as nothing compared to all the stupidities that Google
regularly makes every day of the week :-( .

Has anyone else anything to say on this topic? My preference is to leave
it as it is, and turn a blind eye once every thousand years (whevever).

>wrong.  The ..!news-fe-01!.. in his Path: wasn't much better.

Well I suppose "xyz" COULD be the first in a long sequence of strings all
provably different, but it is not a convincing example, as you say.

>it should be "keep quoted pairs literally" in the LHS.

But the id-right is NOT case-sensitive, whether it looks like it came from
a domain-name or not (I know Bruce will disagree with this, but he is
wrong). You will also see some wording on this in 7.3 of the latest
USEPRO.

>not use the same strategy for both sides, whatever it is ?

I would have no problem changing it if the WG consensus is to do so. But I
hear no other support for the idea.

>statistics about Message-ID syntax in news after 2822 maybe ?

I don't think it matters particularly. What happens in practice is that
news agents just take everything from the first '<' to the next '>' or SP
and do a simple comparison of the resulting string. I doubt any agent,
except an injecting agent, every looks inside to see what is there, and I
expect many injecting agents do little more than check there is a single
'@' somewhere in it.

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 10 Dec 2004 06:44:03 +0100

Charles Lindsey wrote:

> one failure per thousand years
[...]
> could be regarded as satisfying that MUST for all practical
> purposes

That might be true, or let's say I don't care.  But we have a
practical problem with some very old UAs like my "Mozilla 3":

This monster creates Message-IDs based on the RHS of the From:
address.  Now we tolerate something like spamhater@invalid as
an "address".  With these UAs that would result in Message-IDs
like <41AE974D.1DCF@invalid>.  We can't have that.

Of course the UA is broken, but it's another argument for RHS =
domain as specified in 2821.

> Well I suppose "xyz" COULD be the first in a long sequence of
> strings all provably different, but it is not a convincing
> example, as you say.

The discussion wasn't about the xyz, and the real LHS was one
of the usual constructs, it was about the RHS.  This RHS was
"identified" as invalid, because it's no domain.

> But the id-right is NOT case-sensitive, whether it looks like
> it came from a domain-name or not (I know Bruce will disagree
> with this, but he is wrong).

No, I disagree, and you are wrong, it _is_ now case-sensitive,
you said so in another article. ;-)  That's fine, I like the
opaque "magic token" concept for Message-IDs.  If the UA uses a
decent domain as RHS, not invalid stuff, or if it accepts the
proposed Message-ID from its injecting agent.

So the last thing I'm unhappy with is the very complex special
syntax for the LHS, especially the idea to allow any NO-WS-CTL
where it never before was allowed in news.  It's just bad for
the news URL.
               Bye, Frank

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 13 Dec 2004 13:17:34 GMT

>like <41AE974D.1DCF@invalid>.  We can't have that.

But sadly you have got it :-( . That implementation is broken (i.e. the
probability that it will go wrong is way way worse than once in a
thousand years). It already contravenes our draft (more specifically, it
contravenes RFC 2822), so changing our draft cannot fix it.

>domain as specified in 2821.

Why? You are asking for further changes. Others are asking for the minimal
change from RFC 2822. Which am I supposed to do?

>...  This RHS was

But it is quite likely to be unique to that site, given that it was
considered suitable as a path-identity. It is not necessarily wrong.
Please bear in mind also that Usenet sites are not obliged to possess
domain names. A lot of news (and even some email) is still transported by
UUCP and other protocols.

>No, I disagree, and you are wrong, it _is_ now case-sensitive,

Yes, those two words "sensitive" and "insensitive" are just too similar. I
should have said

 But the id-right is NOT case-insensitive, whether it looks like
 it came from a domain-name or not (I know Bruce will disagree
 with this, but he is wrong).

Consider the two message-ids:

   <[email-masked]>
   <[email-masked]>

They are DIFFERENT message-ids. Yes?

>where it never before was allowed in news.  It's just bad for

Then persuade someone else on this list to agree to the further
restrictions you are asking for. If there is consensus, then I shall
change it.

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 17 Dec 2004 14:06:08 +0100

Charles Lindsey wrote:

> That implementation is broken

Indeed, fortunately I found an ISP allowing me to fix it... :-)

> changing our draft cannot fix it.

True, but if you'd say "domain" instead of "id-right" it's more
obvious.  Address literals are already on the border to utter
dubious, with different algorithms for "id-left", different
time zones, and dynamic IPs.  But <whatever@invalid> isn't only
a bad idea, it's broken.

>> Of course the UA is broken, but it's another argument for
>> RHS = domain as specified in 2821.

> Why? You are asking for further changes. Others are asking
> for the minimal change from RFC 2822. Which am I supposed to
> do?

Let's just find a minimal _working_ model for Message-IDs.  In
RfC 2822 you were not interested in these details, you only
wanted some simple syntax covering wild and wonderful things
like mail, s-o-1036, and X.400.

Here and today we know it better, we want something working in
Usenet, in news URLs, in cancel messages, and as many existing
UAs as possible.  IMHO that disqualifies <whatever@invalid>,
and it also doesn't allow NO-WS-CTL.

For the purists like Russ and Bruce the "working model" is most
probably RfC 1036 <unique@full_domain_name>.  For the heretics
like me the "working model" is s-o-1036, essentially the same:

|    message-id          = "<" local-part "@" domain ">"

RfC 2822 is too broad, they wanted a "working model" covering
obsolete and broken software for mail, where Message-IDs aren't
essential.  I'm only guessing why RfC 2822 got it wrong, you
were there:  Why did they replace the old concept of "like an
addr-spec, only unique" by a new "id-left @ id-right" ?

The RfC 2822 "addr-spec" is still fine, it mentions "domain":

| addr-spec  =       local-part "@" domain

But the RfC 2822 Message-ID invents an obscure "id-right" out
of thin air:

| msg-id     =       [CFWS] "<" id-left "@" id-right ">" [CFWS]

I just don't know why you did this in 2822, but it was wrong.
It's incompatible with 1036 and s-o-1036, we can't use it here.

We need the good old "<" unique "@" domain ">" minus RfC 2822
oddities like obs-domain or CFWS.  The RfC 2821 domain syntax
makes sense, it even includes address literals, just forget the
1 in its domain = (subdomain 1*("." subdomain)) / address-literal

Or copy the s-o-1036 definition of domain adding the address
literals as defined in 2821:

| domain        = unquoted-word *( "." unquoted-word )
| unquoted-word = 1*unquoted-char
| unquoted-char = <ASCII printable character except !()<>@,;:\".[]>

It's no big difference from 2822 or your text, and the syntax
still allows an insane <whatever@invalid>, but at least the
intention "domain" is clear and not obscured by an "id-right".

 [dubious <whatever@news-fe-01> and ...!news-fe-01!...]
> it is quite likely to be unique to that site, given that it
> was considered suitable as a path-identity.

It was more likely an error in the configuration of this news
server.

> It is not necessarily wrong.

Show me the UUCP world map entry for news-fe-01, and I consider
it :-)  That's what I said in the de.admin,news.misc.discussion
about it, all others said "unconditionally invalid Message-ID".

> A lot of news (and even some email) is still transported by

s-o-1036 mumbles something about using .uucp in this case, and
to stop this nonsense a.s.a.p.  That was 1994, and the dubious
"domain" and path entry was news-fe-01 (not news-fe-01.uucp) in
2004.

> They are DIFFERENT message-ids. Yes?

Yes.  And I would use the same reasoning for id-left:

     <[email-masked]>
     <"12345678"@foo.example.com>

Also different.  I'm a KISS extremist.  Some gateways trying
to "fix" the second version while others don't would be hell.

 [NO-WS-CTL in Message-ID]
> persuade someone else on this list to agree to the further
> restrictions you are asking for

If Russ really is a 1036 purist, and Henry still is a s-o-1036
heretic, then I simply assume that they also hate any NO-WS-CTL
in Message-IDs.  Only Bruce loves it, but I'm not sure why.

You have already said that you hate it, because NO-WS-CTL is a
PITA for the news URL.
                      Bye, Frank

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 20 Dec 2004 15:12:47 GMT

>like mail, s-o-1036, and X.400.

That idea is very appealing, but it is not what we have been instructed to
do, which is to follow RFC 2822 wherever possible. Now if there were
several more members of this WG in addition to yourself campaigning for a
more radical approach, then it might be considered; but there are not.

>and it also doesn't allow NO-WS-CTL.

Now NO-WS-CTL is an interesting case, because I have just spotted the
following in the new NNTP draft:

   o  A message-id MUST NOT contain octets other than printable US-ASCII
      characters.

and I would not like to be proposing something that could not be
transported by NNTP.

So, since Russ is the Chair of the NNTP WG, could he please speak up on
this one? I think we would all agree that NO-WS-CTLs in message-ids are
ugly, but they are allowed in RFC 2822.

>addr-spec, only unique" by a new "id-left @ id-right" ?

Because both <local-part> and <domain> have built-in [CFWS] before and
after them (and even in the middle), and the last thing we wanted (even
for email) was comments and folding in the middle of a message identifier.

Essentially, <id-left> and <id-right) are just <local-part> and <domain>
with all CFWS possibilities excluded.