Subject: [EAI] ASCII tunneling hack standardization

From: Shawn Steele <[email-masked]>
Date: Thu, 31 Jul 2014 19:14:15 +0000

Back when we were still being active in discussion on this list I ran into scenarios where we needed to put EAI addresses in non-UTF-8 slots.  There was some discussion about whether or not fallback should be permitted, and I had some conversations with Mark Davis where we came up with a draft way of doing that.  However the group headed in the direction that addresses should NEVER be manipulated into some sort of ASCII slot, so I didn’t really follow up.

Anyway, in recent conversations with others, several people have mentioned putting the EAI addresses in non-UTF-8 aware slots for various reasons, so I’d like to bring this up again.  Basically the attached doc says 2 things:

A)     You SHOULD never shove EAI into UTF-8 slots...  but sometimes we don’t have that choice, so this is a hack until those cases get fixed.

B)      If you really, really need to, then punycode the raw local part, using the xl-- prefix to be clear that this isn’t IDN.  (no mapping step)

For various reasons about ship cycle and stuff, this is what’s implemented in Windows’ IdnToAscii() when passed the IDN_EMAIL_ADDRESS flag.

Note that you cannot ‘just’ use IDN on the local part because 1) local parts aren’t like DNS labels, and 2) IDN does all sorts of strange mapping, whereas mail explicitly doesn’t provide standard mappings; even case mapping is optional and can be left up to the server.

Our intent at the time was to publish this as an informational RFC, eg: “this is what we’re doing in the even we really need to abuse stuff and shove EAI addresses into non-EAI aware slots”.

It seems like I should go ahead and get it posted since implementers are diverging in the way they do things.  (There are currently 3 techniques I know about, at least one of which clearly damages the local part).

It’d be really cool if we could agree on this technique for when someone feels they need to hack the address into an ASCII slot.  Again, people should be using Unicode, but in the event something had to tunnel through an ASCII slot at least consistency on the methodology would give a chance of having fewer problems.

-Shawn

 
http://blogs.msdn.com/shawnste
http://L3-G0.blogspot.com

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 1 Aug 2014 08:54:57 +0200

I've had the same discussion, although not with you. But the result was a 
different one: just send UTF-8 even if the channel doesn't say that's OK. 
It'll either work or it won't.

Arnt