Subject: [IPP] Fwd: [art] Modern Network Unicode [consensus]

From: Ira McDonald via ipp <[email-masked]>
Date: Wed, 17 Jul 2019 19:05:25 -0400

Hi,

The excellent consensus after more discussion - READ THIS!

Cheers,
- Ira

---------- Forwarded message ---------
From: Carsten Bormann <[email-masked]>
Date: Thu, Jul 11, 2019 at 7:47 PM
Subject: Re: [art] [I18ndir] Modern Network Unicode
To: Ira McDonald <[email-masked]>
Cc: John C Klensin <[email-masked]>, <[email-masked]>, Asmus Freytag (c) <
[email-masked]>, <[email-masked]>

This is a great discussion.

To me, it seems to converge on the following.

(1) Sending sane data is the job of the data originator.

(2) Do not include gratuitous normalization steps in your processing, once
the data have been originated in a sane form.

(2a) If you broke it, you fix it (as far as possible): If your processing
steps did involve gratuitous normalization, you have to renormalize to NFC
before sending.

Here, “sane” is defined as:

(0) Data SHOULD be originated in NFC, unless that would be inappropriate
for the specific script, in which case the community consensus rules for
the script govern.

For Latin script, this happens to collapse to what 5198 says.

This set of rules places the onus on the place where the data is generated,
which is usually the place that knows most about the specific script and
about the intent of the originator.  If you know that place isn’t doing its
job, add the rule:

(1a) If the data originator does not do (0), the software placing the data
on the network may need to sanitize (normalize towards sane).

1a is similar to 2a in that it doesn’t create perfect results, so both
SHOULD be avoided — there is no way to, after the fact, perfectly sanitize
data that weren’t originated sane or that were gratuitously normalized on
the way.

With these definitions, MNU can direct towards:
(A) Senders: send sane data
(B) Recipients: break as little as reasonable when data received isn’t sane
(C) B is not a valid excuse not to do A, and specifically: recipients are
not expected to clean up after senders (because there is no correct way to
do that).

(Rule C is the often forgotten third rule of the Postel principle.
It also means that an entity that is a recipient of MNU and then sends the
data on as MNU has no need to gratuitously normalize, but it does not
entirely get rid of rule 1a for recipients of data from places known not to
be sane.)

Grüße, Carsten
_______________________________________________
ipp mailing list
[email-masked]
https://www.pwg.org/mailman/listinfo/ipp

---

From: Uli Wehner via ipp <[email-masked]>
Date: Thu, 18 Jul 2019 01:40:57 +0000

The basic rules of all communication. The sender is responsible for making the message such that their communication goal is met. The recipient just needs to be helpful to that end.

Sent from my Sprint Samsung Galaxy S9.

-------- Original message --------
From: Ira McDonald via ipp <[email-masked]>
Date: 7/17/19 19:06 (GMT-05:00)
To: [email-masked], Ira McDonald <[email-masked]>
Subject: [IPP] Fwd: [art] Modern Network Unicode [consensus][EXTERNAL]

Hi,

Cheers,
- Ira

---------- Forwarded message ---------
From: Carsten Bormann <[email-masked]<mailto:[email-masked]>>
Date: Thu, Jul 11, 2019 at 7:47 PM
Subject: Re: [art] [I18ndir] Modern Network Unicode
To: Ira McDonald <[email-masked]<mailto:[email-masked]>>
Cc: John C Klensin <[email-masked]<mailto:[email-masked]>>, <[email-masked]<mailto:[email-masked]>>, Asmus Freytag (c) <[email-masked]<mailto:[email-masked]>>, <[email-masked]<mailto:[email-masked]>>

(2) Do not include gratuitous normalization steps in your processing, once the data have been originated in a sane form.

(2a) If you broke it, you fix it (as far as possible): If your processing steps did involve gratuitous normalization, you have to renormalize to NFC before sending.

(0) Data SHOULD be originated in NFC, unless that would be inappropriate for the specific script, in which case the community consensus rules for the script govern.

This set of rules places the onus on the place where the data is generated, which is usually the place that knows most about the specific script and about the intent of the originator.  If you know that place isn’t doing its job, add the rule:

(1a) If the data originator does not do (0), the software placing the data on the network may need to sanitize (normalize towards sane).

1a is similar to 2a in that it doesn’t create perfect results, so both SHOULD be avoided — there is no way to, after the fact, perfectly sanitize data that weren’t originated sane or that were gratuitously normalized on the way.

With these definitions, MNU can direct towards:
(A) Senders: send sane data
(B) Recipients: break as little as reasonable when data received isn’t sane
(C) B is not a valid excuse not to do A, and specifically: recipients are not expected to clean up after senders (because there is no correct way to do that).

(Rule C is the often forgotten third rule of the Postel principle.
It also means that an entity that is a recipient of MNU and then sends the data on as MNU has no need to gratuitously normalize, but it does not entirely get rid of rule 1a for recipients of data from places known not to be sane.)

Grüße, Carsten