Subject: [Emailcore] Unicode

From: Borden <[email-masked]>
Date: Wed, 16 Feb 2022 01:06:02 +0100 (CET)

I apologise that my first question to the IETF is out of scope. However, since there is so much activity, I think the best person to answer will be reading:

Under 2.1. of RFC 5322, why does it say ANSI instead of UTF-8 (or -16 or -32)? Most technology uses Unicode (or at least supports it), so why does the core e-mail standard still rely on the basic Latin alphabet? How will the Internet break if the standard switches to Unicode?

As not to clutter the list, you may respond to me privately. I'd appreciate links to previous discussions or analysis that can help me understand this issue. As I get more familiar with the IETF process, I would like to prepare an argument to submit when the circumstances are appropriate.

With thanks,

---

From: Dave Crocker <[email-masked]>
Date: Tue, 15 Feb 2022 16:12:14 -0800

On 2/15/2022 4:06 PM, Borden wrote:
> Most technology uses Unicode

The origins of RFC 5322 date back to the 1970s.  Quite a bit earlier 
than Unicode.

Successors to that original work have been careful to maintain 
continuity, to protect the installed base.

d/

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: "John Levine" <[email-masked]>
Date: 15 Feb 2022 21:47:36 -0500

>continuity, to protect the installed base.

MIME allows you to use UTF-8 in nearly every bit of visible text in an
e-mail messsage, sometimes encoded in base64 or quoted-printable in
the internal form of the message, but every current mail program takes
care of that transparently. The one place MIME doesn't allow UTF-8 is
in the actual e-mail addresses. That was dealt with by
Internationalized Email, usually known as EAI. See RFCs 6530, 6531,
6532, 6533, 6855, 6856, 6857, and 6858.

Many large mail operators including Google's Gmail and Microsoft's various
offerings can correspond using EAI addresses even though they only offer
ASCII addresses on their own systems.

Having done a lot of EAI conformance and compatibility testing, I can
report that allowing UTF-8 in addreses and actually making it work is
way more complicated than you might expect.

R's,
John

---

From: John C Klensin <[email-masked]>
Date: Tue, 15 Feb 2022 22:09:50 -0500

--On Wednesday, February 16, 2022 01:06 +0100 Borden
<borden_c=[email-masked]> wrote:

> I apologise that my first question to the IETF is out of
> scope. However, since there is so much activity, I think the
> best person to answer will be reading:
> 
> Under 2.1. of RFC 5322, why does it say ANSI instead of UTF-8
> (or -16 or -32)? Most technology uses Unicode (or at least
> supports it), so why does the core e-mail standard still rely
> on the basic Latin alphabet? How will the Internet break if
> the standard switches to Unicode?
> 
> As not to clutter the list, you may respond to me privately.
> I'd appreciate links to previous discussions or analysis that
> can help me understand this issue. As I get more familiar with
> the IETF process, I would like to prepare an argument to
> submit when the circumstances are appropriate.

Completely agreeing to Dave Crocker's comments, but to add to
them, one does not just get to say "UTF-8" or "Unicode" -- there
are all sorts of additional issues about comparisons, matching,
special issues with particular scripts (including but certainly
not limited to so-called "bidirectional" issues), etc., etc.
If you are interested in what is involved, see RFC 6530 through
6533 for hints, both about what is involved and what breaks.   I
recommend directing further discussions to [email-masked] if you
have additional questions or concerns.

    john