Subject: Re: [precis] Enforcement as an Idempotent operation

From: Peter Saint-Andre <[email-masked]>
Date: Tue, 21 Mar 2017 19:30:02 -0600

On 2/26/17 5:48 PM, Peter Saint-Andre wrote:
> On 2/13/17 12:04 AM, William Fisher wrote:
>> On Sun, Feb 12, 2017 at 12:27 PM, Peter Saint-Andre <[email-masked]> wrote:
>>> Did you mean U+212A (KELVIN SIGN)? That decomposes to U+004B (LATIN CAPITAL
>>> LETTER K).
>>>
>>>> The full example is:
>>>> "\U0001f11aevin" => "(K)evin" => "(k)evin"
>>
>> I'm talking about 'PARENTHESIZED LATIN CAPITAL LETTER K' (U+1F11A).
>> Sorry it's not clear that the A is part of the unicode escape.
> 
> Thanks for the clarification.
> 
>> With casefold or tolower, the result is the same for these Nicknames:
>>
>> Not idempotent: "\U0001f11A" => "(K)" => "(k)"
>> Not idempotent: "\U0001f13A" => "K" => "k"
>> Not idempotent: "\u210c" => "H" => "h"
>> Not idempotent: "\u210d" => "H" => "h"
>> Not idempotent: "\u20a8" => "Rs" => "rs"
>>
>> When you apply the comparison steps from RFC 7700, Section 2.4, you
>> still get something that is upper case. If you apply the comparison
>> steps again, you now get lower case.
> 
> I see what you mean. I'm now leaning toward moving the case mapping rule
> after the normalization rule, but first I want to think about the
> implications for all of the PRECIS profiles (e.g., when using NFC vs.
> using NFKC). If we go down this road, we will also want to describe the
> reasoning in Section 5.2.3 of 7564bis.

Thinking about this further, I now lean against making this change in
the PRECIS processing rules, for several reasons:

1. Existing PRECIS implementations would need to be modified, resulting
in a behavioral difference between older and newer implementations (or
older and newer versions of the same implementation).

2. The order of operations in PRECIS was intended to be consistent with
IDNA2008 (in which case mapping is performed before normalization,
albeit in the application before the protocol is invoked), and with
IDNA2003 and Stringprep prior to IDNA2008 (note also that several PRECIS
profiles were designed as modernized replacements for Stringprep
profiles). Making PRECIS inconsistent with IDNA might make it harder to
reuse code and might lead to unexpected and undesirable consequences.

3. Idempotence, although a desirable quality, in my opinion falls into
the category of "nice but not necessary". (If we were designing PRECIS
anew, my opinion might be different.)

A safer approach would be to add an implementation note to the effect
that PRECIS processing might not be idempotent, and that implementations
might need to apply the rules more than once to the same string.

Peter

---

From: William Fisher <[email-masked]>
Date: Thu, 23 Mar 2017 12:46:00 -0700

I agree with the "implementation note" strategy. In all my testing,
only the "Nickname" profile can fail to be idempotent for some inputs.
I have not found any inputs that fail the idempotent test using the
Username or OpaqueString profiles.  I believe "Nickname" has problems
because it uses NFKC.  I would add an implementation note/warning to
the Nickname profile.

Thanks,
Bill

---

From: Peter Saint-Andre <[email-masked]>
Date: Thu, 23 Mar 2017 17:52:04 -0600

> because it uses NFKC.  I would add an implementation note/warning to

Hi Bill, thanks for your feedback. I propose the following text.

1. At the end of Section 7 ("Order of Operations") of 7564bis, add this
note:

   Because of the order of operations specified here, applying the rules
   for any given PRECIS profile is not necessarily an idempotent
   procedure (e.g., under certain circumstances, such as when Unicode
   normalization form KC is used, performing Unicode normalization after
   case mapping can still yield uppercase characters for certain code
   points); therefore, implementations might need to apply the rules
   more than once to an internationalized string.

2. At the end of Section 4 ("Use in Application Protocols") of 7700bis,
add this note:

   Implementation experience has shown that applying the rules for the
   Nickname profile is not an idempotent procedure for all code points.
   Therefore, implementations might need to apply the rules more than
   once to a nickname string.

3. I see no harm in also adding the following note to the end of Section
5 ("Use in Application Protocols") of 7613bis:

   Applying the rules for any given PRECIS profile is not necessarily
   an idempotent procedure for all code points. Therefore,
   implementations might need to apply the rules more than once to an
   internationalized string.

Peter

---

From: William Fisher <[email-masked]>
Date: Fri, 24 Mar 2017 11:40:29 -0700

Looks good to me.

-Bill