Subject: Re: [EAI] UTF-8 in Message-IDs

From: Julien ÉLIE <[email-masked]>
Date: Mon, 03 Oct 2011 23:51:29 +0200

Hi Charles,

> The Message-ID is crucial to the correct operation of the 
> Netnews transport, and the effect of using utf-8 in it has not been 
> investigated (but should be).  It is expected that some ancient transport 
> agents might not like it but even that might not be a show stopper, 
> since Netnews is very good at navigating around such obstacles.

For what is worth, I do not think there would be any major issue with INN
(a news server) because message-IDs are internally "hashed" after
having parsed them.  They can be retrievable.

The problem is the transition period.  INN currently rejects any message
whose Message-ID: header field does not comply with RFC 5536.
It also does not want to search for such message-IDs.

200 news.trigofacile.com InterNetNews NNRP server INN 2.6.0 (20111003 prerelease) ready (posting ok)
ARTICLE <Â©@test>
501 Syntax error in message-ID
QUIT
205 Bye!

> If then, as is likely, they start to appear, then we will see how they 
> cope and decide how best to regulate them in a future standard.

Meanwhile, the propagation of such articles will be very bad.

-- 
Julien ÉLIE

« Le travail n'est pas une bonne chose. Si ça l'était, les riches
  l'auraient accaparé. »

---

From: "John Levine" <[email-masked]>
Date: 4 Oct 2011 01:42:57 -0000

>Meanwhile, the propagation of such articles will be very bad.

News servers aren't going to be thrilled about messages with
unencoded UTF-8 in other headers, either.

They'll have to upgrade to handle EAI, just like mail servers will.  I
suppose we might want to invent some new NNTP verb that lets hosts
tell each other whether they can accept EAI messages, but this is most
definitely not the current EAI group's problem.

R's,
John

---

From: John C Klensin <[email-masked]>
Date: Mon, 03 Oct 2011 22:34:35 -0400

--On Monday, October 03, 2011 23:51 +0200 Julien ÉLIE
<[email-masked]> wrote:

>> Netnews transport, and the effect of using utf-8 in it has
>> not been  investigated (but should be).  It is expected that
>> some ancient transport  agents might not like it but even
>> that might not be a show stopper,  since Netnews is very good
>> at navigating around such obstacles.
> 
> For what is worth, I do not think there would be any major
> issue with INN (a news server) because message-IDs are
> internally "hashed" after having parsed them.  They can be
> retrievable.
> 
> The problem is the transition period.  INN currently rejects
> any message whose Message-ID: header field does not comply
> with RFC 5536. It also does not want to search for such
> message-IDs.
>...
 
Julien,

The edge case of a message with a non-ASCII Message-ID but no
other non-ASCII header fields aside, how would the news servers
you and Charles are familiar with respond to non-ASCII addresses
or header fields more generally?

Note also that, to some extent, this is not a news reader
problem at all but a gateway issue between Internet mail and
news.  If articles originate in news and move to mail, there is
no issue because they will be ASCII.  So the question is what
the gateways in the "to news" direction do.  If they reject
messages with non-ASCII headers (Message-IDs or otherwise), the
readers won't see such messages... and the problem is really no
different from an attempt to send an extended mail message to a
server that is not UTF8SMTP-capable.

   john

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 04 Oct 2011 21:41:12 +0100

> or header fields more generally?

They would propagate fine. User agents that could display them would do  
so. Making EAI an official extension to Netnews would be quite simple -  
mainly a matter of inventing a Newsgroups: header and doing something  
about normalization of Message-IDs if there were to contain UTF-8 (which  
is why I opposed allowing them in EAI without some normalization rules  
being provided at the same time).
> server that is not UTF8SMTP-capable.

I suspect that even if EAI does not find its way 'officially' into  
Netnews, it will still just happen anyway (probably in selected newsgroups  
where its use will become common). The transport mechanism it already  
8-bit clean, so it should "just work".

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131                       
   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: Julien ÉLIE <[email-masked]>
Date: Tue, 04 Oct 2011 23:21:06 +0200

Hi John,

> or header fields more generally?

It depends on the header field.  Basically, they should be valid per RFC 
5536 when a syntax is specified (for instance, a Date: header field 
should be parsable, and should not contain non-ASCII chars where they 
are not expected).

Regarding the Newsgroups: header field, it would work, as Charles says. 
  However, I am unsure that a moderated newsgroup with non-ASCII chars 
works well (that is to say I am unsure the conversion from an 
internationalized newsgroup name to an internationalized e-mail address 
works out of the box right now).

The Distribution: header field may also be affected by the use of 
non-ASCII chars.  (Besides, MIME is not expected for this header.)

No problem with non-ASCII addresses (From:, Sender:, Approved: ... 
header fields) as long as external programs using them comply with EAI 
(for instance programs processing NoCeM notices [a kind of cancel 
articles], control messages to create/remove newsgroups, moderation 
software, etc.).

-- 
Julien ÉLIE

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 05 Oct 2011 11:10:32 +0100

> definitely not the current EAI group's problem.

No they won't. User agents will have to be upgraded if people want to read  
the utf-8 stuff, or to reply by mail to articles From: people with utf-8  
addresses. But the existing servers will handle all this just fine except  
insofar as they might be rejecting (unnecessarily) utf-8 in some headers  
where it did not matter.

Servers will expect to be able to compare Message-IDs for equality by a  
simple octet-by-octet comparison, and they will expect Dates to be  
comparable (<,=,>), but I don't think EAI has changed that. Even Newsgroup  
names with utf-8 in them will work, as has alredy been demonstrated on the  
existing network, (so long as octet-by-octet comparison works, which means  
some severe normalization).

As to whether it is any of this WG's business, it is certainly NOT the  
business of this WG to put obstacles in the way of other mail-like  
protocols which might want to follow the EAI path.

---

From: John C Klensin <[email-masked]>
Date: Wed, 05 Oct 2011 08:40:08 -0400

--On Wednesday, October 05, 2011 11:10 +0100 Charles Lindsey
<[email-masked]> wrote:

>...
> As to whether it is any of this WG's business, it is certainly
> NOT the business of this WG to put obstacles in the way of
> other mail-like protocols which might want to follow the EAI
> path.

Charles,

The job of this WG is to make a specific set of extensions to
allow internationalized addresses, envelopes, and headers work
well in Internet mail.   That job includes making the extensions
as obvious, internally consistent, and deployable as possible
for Internet email use on the public Internet.  Like any other
change, including the original adoption of the SMTP extension
model and MIME, almost anything that is done is likely to cause
some difficulties for gateways to other mail systems (and
mail-like protocols) and/or, depending on how the gateways
behave, on user agents in those other systems.

Trying to consider the perceived needs of every other mail-like
protocol in order to avoid doing something that its implementers
might consider an obstruction leads to paralysis and possibly
madness.  Because there are IETF Standards-Track documents, NNTP
gets some extra consideration, but not much -- the job of the WG
remains Internet email, not, in your words, "mail-like
protocols".

Speaking personally, I would be much more sympathetic to the
position you are taking had I not seen all sorts of chaos caused
over the years by news systems believing that netnews
Message-IDs could be compared, uncritically, to Internet email
Message-IDs and actions taken on the results.  But those
disruptions have occurred and presumably continue to occur.
People live with them, and I don't see the disruptions this
change to Message-IDs are likely to cause being any worse.  YMMD.

Again speaking personally, I would have preferred a "don't
change anything that cannot be proven to be necessary" approach
by the WG, even if it resulted in somewhat more complex rules
about what header fields could have non-ASCII content and which
could not.    That opinion was almost certainly influenced by my
having spent a lot of my life in the gateway world (although not
with NNTP).  The WG disagreed, favoring a less complex model in
which substantially any header field can contain non-ASCII
material, and Joseph has declared consensus.  I've accepted that
conclusion; unless you have something that is actually new to
add to the discussion, I suggest you do too.

best,
   john

---

From: Frank Ellermann <[email-masked]>
Date: Wed, 5 Oct 2011 22:28:47 +0200

> remains Internet email, not, in your words, "mail-like

I'm not sure what Charles and you are discussing here.  Please
correct me if I'm wrong, but the "Internet message format" is not
only for e-mail, in essence it is also used by NetNews.  And that
is already the complete list.  Other mail formats are not based
on 822+, and other protocols (e.g., XMPP) are not about Internet
messages.  HTTP From: header fields not withstanding, "they" can
adopt EAI when they feel like it.  The mailto: URLs are almost
ready for EAI, and the news: URLs can be "fixed" by an erratum.

What can't be "fixed" might be thousands of NNTP servers, if the
operators pick the wrong of three alternatives:  (1) Do nothing,
(2) upgrade, or (3) uninstall and give up on NetNews.

> change to Message-IDs are likely to cause being any worse.  YMMD.

A message-ID is a message-ID:  Any chaos you have seen presumably
involved naive or broken gateways from NetNews to e-mail, or more
obscure sources, e.g., Fido (FTN) echomail.  EAI won't make that
worse.  Sadly SASL killed it, otherwise CRAM-MD5 might be now in
theory ready for "EAI-IDs".  In practice I'd guess that CRAM-MD5
implementations never bother to check Message-ID syntax details.

-Frank

---

From: "John Levine" <[email-masked]>
Date: 5 Oct 2011 20:44:51 -0000

>only for e-mail, in essence it is also used by NetNews.

Mail message format is described in RFC 5322, while news article
format is described in RFC 5536.

They're deliberately fairly similar, but it is just wrong to assert
that you can turn one into the other without using a gateway that
deals with the differences.   I've written such gateways (indeed, I
read this list via a gateway that turns it into a local newsgroup)
and there's a lot of little nits they have to deal with.

R's,
John

---

From: John C Klensin <[email-masked]>
Date: Wed, 05 Oct 2011 17:13:59 -0400

--On Wednesday, October 05, 2011 22:28 +0200 Frank Ellermann
<[email-masked]> wrote:

First, thanks to John Levine for the response to your first
question.  I have nothing to add to his response.  These are two
systems that use very similar formats but that communicate
through gateways, not one system.

>> position you are taking had I not seen all sorts of chaos
>> caused over the years by news systems believing that netnews
> A message-ID is a message-ID:  Any chaos you have seen
> presumably involved naive or broken gateways from NetNews to
> e-mail, or more obscure sources, e.g., Fido (FTN) echomail.

A Message-ID is a Message-ID until something (usually a gateway)
decides it needs or wants to improve on it.  Those decisions can
be due to misunderstanding or stupidity, they can be because
other things change and the gateway believes that makes a new
message, they can be because the gateway wants to be able to
guarantee a unique identifier in the other environment.
Sometimes changes are made to protect systems further downstream
that are believed to be fragile, sometimes the gateways
themselves are fragile and change things to recover from
breakage.  The changes are usually made with the best of
intentions; whether they are naive, broken, or really useful and
creative is in the minds of the beholders and the purpose to
which they are supposed to be put.  I would not presume to try
to categorize all of the transformations I've seen, but some of
them probably don't fall under your description above.

This isn't trivial.  Remember that simple transcoding of header
fields from net-ASCII into, e.g., EBCDIC or what we call
ISO-2022-JP means that dumb octet-string comparisons of
Message-IDs (and other things) will fail.  There is a rather
long history of those issues.

If someone asked my opinion about whether a gateway should mess
with Message-IDs, I'd say "no, unless it is absolutely required
by the systems on the other end".  But lots of people don't ask
and even fewer listen.   Neither RFC 5321/5322 nor RFC 5536
offer any reciprocal guarantees.

> EAI won't make that worse.

If there are fragile systems out there -- and there almost
certainly still are-- how can you possibly know that?

>  Sadly SASL killed it, otherwise
> CRAM-MD5 might be now in theory ready for "EAI-IDs".  In
> practice I'd guess that CRAM-MD5 implementations never bother
> to check Message-ID syntax details.

In spite of being co-author of CRAM-MD5, I don't understand this
comment at all. Certainly the CRAM-MD5 spec, regardless of its
other strengths and weaknesses, doesn't even mention Message-IDs
(or any other signature over headers or message content for that
matter).

    john

---

From: Julien ÉLIE <[email-masked]>
Date: Wed, 05 Oct 2011 23:36:28 +0200

Hi John,

> offer any reciprocal guarantees.

RFC 5536 defines a stricter subset than RFC 5322 for Message-IDs; they 
therefore need being checked.  I agree with your saying that they should 
not be modified if they have a valid syntax for the system on the other end.

According to RFC 5537:

3.10.2.  Duties of an Incoming Gateway

    If there is a message identifier that fills a role similar to that of
    the Message-ID header field in news, it SHOULD be used in the
    formation of the message identifier of the news article, perhaps with
    transformations required to meet the uniqueness requirement of
    Netnews and with the removal of any comments so as to comply with the
    syntax in Section 3.1.3 of [RFC5536].  Such transformations SHOULD be
    designed so that two messages with the same identifier generate the
    same Message-ID header field.

       NOTE: Message identifiers play a central role in the prevention of
       duplicates, and their correct use by gateways will do much to
       prevent loops.  Netnews does, however, require that message
       identifiers be unique, and therefore message identifiers from
       other media may not be suitable for use without modification.  A
       balance must be struck by the gateway between preserving
       information used to prevent loops and generating unique message
       identifiers.

-- 
Julien ÉLIE

« Rien, ce n'est pas rien ! La preuve, c'est que l'on peut le
   soustraire. Exemple : rien moins rien = moins que rien ! »
   (Raymond Devos)

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 6 Oct 2011 00:12:51 +0200

> format is described in RFC 5536.

Yeah, I'm aware of 5536, it is firmly based on 5322.  An earlier
Usefor draft already had UTF-8, but it did not make it (roughly
at the time when I felt that the "making of RFCs" is fascinating):
<URL:http://purl.net/xyzzy/usefor07.htm>

5536 is an application of 5322 + MIME.  Son-of-1036 (1849) already
had 822 + MIME, so this was not really a new idea after UTF-8 got
no consensus back in 2002 or 2003.

> it is just wrong to assert that you can turn one into the other
> without using a gateway that deals with the differences.

Nobody said that there are no gateways.  About twenty years ago I
ended up maintaining a list of gateway software for some time (at
that time RFC822 <-> NetNews was an *apparently* simple case, and
consequently this caused more troubles than the more convoluted
scenarios involving other networks).   IIRC that was in one of
the "gatebau GABELs" (Group, Area, Boards, Echo, List; this was
a "politically correct" acronym using the German word "gabel",
i.e., it was no Usenet newsgroup also available in Fido or v.v.,
it was a neutral territory wrt the transport technologies and the
resulting different admin structures.)

> I read this list via a gateway that turns it into a local
> newsgroup) and there's a lot of little nits they have to deal
> with.

For more than six years I read all IETF mailing lists on Larsi's
GMaNe (Gateway MAil NEws), at the moment I try my luck with Gmail.

Somewhere the EAI drafts should mention that the first ideas for
UTF-8 in Internet message header fields was created by Usefor or
more precisely by Charles about ten years ago.

-Frank

---

From: Julien ÉLIE <[email-masked]>
Date: Thu, 06 Oct 2011 00:39:41 +0200

Hi Frank,

> more precisely by Charles about ten years ago.

Definitely.

-- 
Julien ÉLIE

---

From: Julien ÉLIE <[email-masked]>
Date: Thu, 06 Oct 2011 00:53:25 +0200

Hi Frank,

> The mailto: URLs are almost

Wouldn't it be proper to update the RFCs instead of adding an erratum? 
(which would probably be "held for document update")

It is not an error; it is a technical update.

(Incidentally, we could try to do some work on USEAGE too, if Charles 
agrees.  Maybe a new draft would be worthwhile, so that we could comment 
and make progress on it.  -- Discussions on that subject should continue 
on the USEFOR mailing-list; it is not related to IMA.)

> (2) upgrade, or (3) uninstall and give up on NetNews.

I hope it will not split Usenet into two distinct "feeding areas":  the 
UTF-8-aware one (with UTF-8 moderated newsgroup names, message-IDs, 
etc.) and the legacy one.

-- 
Julien ÉLIE

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 6 Oct 2011 01:20:39 +0200

On 5 October 2011 23:13, John C Klensin wrote:

 [Naive or broken]
> I would not presume to try to categorize all of the
> transformations I've seen, but some of them probably don't
> fall under your description above.

> This isn't trivial.

ACK, but thinking that it's trivial would be covered by naive.

I never had anything to do with MIXER.  All I vaguely recall
is related to "dupes" (= the same message arriving more than
once where it is expected at most once) and "nopes" (= if an
expected message arrives less than once, i.e., never, null,
nada, nope).  For Netnews "same message" means by definition
"same Message-ID", it is a fundmental concept.

Because I had a hard time to grok this years ago (in FTN the
Message-ID is only optional, the real thing in FTN echos are
the "Seen-By" thingies) I'm now reluctant to touch this "holy
cow" of what used to be the "RFC822" side from my FTN (Fido
technology network) POV.

> Remember that simple transcoding of header fields from
> net-ASCII into, e.g., EBCDIC or what we call ISO-2022-JP
> means that dumb octet-string comparisons of Message-IDs
> (and other things) will fail.

Actually I don't remember this issue, the "gatebau" problems
with Message-IDs were limited to ASCII-compatible networks.

But I can imagine how "ASCII-incompatible" offers wild and
wonderful ways to get it wrong.  If "they" can't do UTF-8 or
at least UTF-EBCDIC they have only one EAI-option:  Reject.

The corner cases where only the EAI-ID causes this "reject",
because everything else happens to work for say EBCDIC, are
too odd to consider.

> offer any reciprocal guarantees.

RFC 1849 has a very good advice:  "Avove all, prevent loops."

The Usefor folks, or rather Russ, considered to write a gateway
memo after 5537 (not the same as 5536).  Some of the topics you
didn't find in 5322 & 5536 are covered in 5321 & 5537.  Sadly you
can't put more about this in 5321bis, and actually it should get
its own memo -- 5321 & 5537 aren't the place for gateway details.

> certainly still are-- how can you possibly know that?

It is a guess:  Gateway operators who manage to get it right for
message/rfc822 <-> something based on EBCDIC could also manage
to get it right for message/global <-> something UTF-EBCDIC, or
simply reject message/global.  In other words, when I'm opposed
to UTF-8 in message-IDs I'm not worried about breaking gateways.

> other strengths and weaknesses, doesn't even mention Message-IDs

Quoth 2195:  "The syntax of the unencoded form must correspond to
that of an RFC 822 'msg-id' [RFC822] as described in [POP3]."

APOP and CRAM-MD5 use the syntactical form of a Message-ID for
their challenges.  You didn't write "globally unique forever", but
the Message-ID challenges are still supposed to be "unique".

-Frank

---

From: John C Klensin <[email-masked]>
Date: Wed, 05 Oct 2011 20:11:53 -0400

--On Wednesday, October 05, 2011 23:36 +0200 Julien ÉLIE
<[email-masked]> wrote:

> Hi John,
> 
>> If someone asked my opinion about whether a gateway should
>> mess with Message-IDs, I'd say "no, unless it is absolutely
>> required by the systems on the other end".  But lots of
>> people don't ask and even fewer listen.   Neither RFC
>> 5321/5322 nor RFC 5536 offer any reciprocal guarantees.
> 
> RFC 5536 defines a stricter subset than RFC 5322 for
> Message-IDs; they therefore need being checked.  I agree with
> your saying that they should not be modified if they have a
> valid syntax for the system on the other end.
> 
> According to RFC 5537:
> 

Yes.  This is exactly the point I was trying to make -- thanks
for digging out the text.

I believe it leaves us with the following cases for Message-IDs
in mail<->news gateways:

Direction news -> mail

	  Since 5322 Message-IDs are less restrictive than 5536
	ones, there should be no need to change anything.  If a
	change is made, it would presumably be to better assure
	uniqueness on the Internet side, but still would not
	violate Internet rules for either unextended or 
	UTF8SMTPbis messages.  Note that 5335bis permits 
	UTF-8 in Message-IDs, but certainly does not require any
	non-ASCII characters there.

Direction mail -> news

 Case 1: The Message-ID is all-ASCII and otherwise
	conforms to the 5536 requirements.  No transformation
	other than copying is necessary.  If a change is made,
	it would presumably be to assure uniqueness on the news
	side.  Nothing new here.
	
 Case 2: The Message-ID is all-ASCII but does not conform
	to the 5536 requirements.  The gateway is required to
	change the mail Message-ID to a 5536 conforming one.
	Whatever happens, the two Message-IDs are no longer the
	same.
	
 Case 3: The Message-ID contains some non-ASCII
	characters.  That doesn't conform to 5536, so the
	gateway is required to change it to something else,
	something that is 5536 conforming.  Whatever happens,
	the two Message-IDs are no longer the same.  

Case 3 is the only case that is dependent on or influenced by
EAI/ UTF8SMTPbis.  Everything else is the same.  And, possible
implementation details aside, there is really no difference
between Case 2 and Case 3.  Any competent existing gateway has
to contain a test for 5536 conformance and has to contain code
to generate a new Message-ID if the mail one does not conform.
The only difference between the two cases is that the source of
non-conformance under Case 2 is presumably some lexical problem,
while the source under Case 3 is presumably the presence of one
or more non-ASCII characters, which is still a lexical problem.
No difference, really.

Of course, if the mail-> news gateway doesn't enforce 5536, one
can have all sorts of interesting problems at the news reader.
But that is also true with or without the EAI work.

Right?

    john

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 6 Oct 2011 02:37:31 +0200

On 6 October 2011 00:53, Julien ÉLIE wrote:

>> The mailto: URLs are almost  ready for EAI, and the
>> news: URLs can be "fixed" by an erratum.

> It is not an error; it is a technical update.

The EAI mailto: IRIs are documented in an existing I-D:
<http://tools.ietf.org/html/draft-duerst-eai-mailto-01>

news: IRIs are documented in 5538, modulo one erroneous
statement (if this WG gets its amended will in an IETF
Last Call.)  I'm not aware of "implementation reports"
or any other reasons justifying more work than a simple
"erratum":  The secrets of percent-encoding are already
covered by 3986 + 3987 + 5538.

Maybe if Usefor and/or NNTP are updated for EAI it could
make sense to work on a 5538bis.  But as long as Usenet
sticks to 5322/5536 Message-IDs that point is moot.

> Incidentally, we could try to do some work on USEAGE
> too, if Charles agrees.

Well, I'm willing to review drafts, IIRC there were some
pending IANA issues to be covered in USEAGE or "manually".

Otherwise I'm not much tempted by serious IETF work at
the moment, unless it is about 4408bis, 5321bis, 6068bis.

> I hope it will not split Usenet into two distinct
> "feeding areas":  the UTF-8-aware one (with UTF-8
> moderated newsgroup names, message-IDs, etc.) and the
> legacy one.

My best wishes for this hope,
 Frank

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 06 Oct 2011 10:40:46 +0100

On Thu, 06 Oct 2011 01:11:53 +0100, John C Klensin <[email-masked]> wrote:

> 	ones, there should be no need to change anything....

Actually, the difference between 5322 and 5536, as it finally ended up, is  
miniscule (5536 does not permit a '>' inside a Message-ID, but it is  
wildly improbable that one would see such a '>' in a real-world email).
> 	the two Message-IDs are no longer the same.

Actually no. The best thing for a gateway to do (in the sense that it will  
cause the least damage/misrouting/looping/whatever) is to leave the utf-8  
untouched. If some subsequent server gets upset, then its connected  
clients will suffer, but the article will still route its way through  
other servers anr thus propagate throughout the network. Maybe not  
standards conforming, but that is what will work best. There might just be  
some slight benefit in NFC normalization at that point. Standards might  
eventually catch up, but in the meantime servers that refused such  
articles would get heavily leant upon to let them through.

As you say, gatewaying is a messy business, and attempts to fix "broken"  
messages usually make things worse. What I have suggested is well within  
the spirit of the gatewaying guidelines from 5537 that were quoted.

My only complaint is that if EAI had required that NFC normalization in  
any utf-8 message-id, then that would have been one issue less for those  
qateways. Indeed it would have also been one issue less for _email_ user  
agents that like to do threading.

---

From: Julien ÉLIE <[email-masked]>
Date: Thu, 06 Oct 2011 21:53:02 +0200

Hi John,

> to generate a new Message-ID if the mail one does not conform.

Existing gateways conforming to RFC 5322 may also decide to reject mails 
containing non-ASCII chars in their Message-IDs.  We would then notice a 
loss of articles in Netnews.
Such gateways need being updated.  (Which is also the case for any mail 
agent not knowing EAI / UTF8SMTPbis.)
So, yes, I agree that Case 2 and Case 3 are the same for EAI-aware gateways.

> But that is also true with or without the EAI work.

Yes, I agree.
The issue also affects Supersedes:, Control: and References: header 
fields, to mention only the most frequently used.

> Right?

I think you're right.

-- 
Julien ÉLIE