Subject: [apps-discuss] Last Call: <draft-faltstrom-5892bis-04.txt> (The Unicode code points and IDNA - Unicode 6.0) to Proposed Standard

From: The IESG <[email-masked]>
Date: Mon, 23 May 2011 15:19:03 -0700

The IESG has received a request from the Applications Area Working Group
WG (appsawg) to consider the following document:
- 'The Unicode code points and IDNA - Unicode 6.0'
  <draft-faltstrom-5892bis-04.txt> as a Proposed Standard

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
[email-masked] mailing lists by 2011-06-06. Exceptionally, comments may be
sent to [email-masked] instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract

This document specifies IETF consensus for IDNA derived character
properties related to the three code points, existing in Unicode 5.2,
that changed property values when version 6.0 was released.  The
consensus is that no update is needed to RFC 5892 based on the
changes made in Unicode 6.0.

The file can be obtained via
http://datatracker.ietf.org/doc/draft-faltstrom-5892bis/

IESG discussion can be tracked via
http://datatracker.ietf.org/doc/draft-faltstrom-5892bis/

No IPR declarations have been submitted directly on this I-D.

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 23 May 2011 16:26:41 -0600

The first paragraph of the Introduction contains a few infelicities and
grammatical errors. I suggest modifying it as follows:

   RFC 5892 [RFC5892] specifies an algorithm that was defined when
   the current version of The Unicode Standard was Unicode 5.2
   [Unicode5.2], and also defines a derived property value based on
   that algorithm.  Unicode 6.0 has changed the GeneralCategory of
   three code points that were allocated in Unicode 5.2 or earlier.
   This implies that the derived property value differs depending on
   whether the property definitions used are from Unicode 5.2 or
   Unicode 6.0.

On 5/23/11 4:19 PM, The IESG wrote:
> _______________________________________________
> apps-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/apps-discuss

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: John C Klensin <[email-masked]>
Date: Mon, 23 May 2011 22:22:13 -0400

Peter,

Personally, I think it would be a big mistake to spend a lot
more time on this.  It just isn't worth it.  And I continue to
object to the IESG -- or even responsible ADs -- doing
copy-editing.  We pay the highly qualified RFC Editor staff for
doing that and should let them do their jobs.  That said, if
this is worth fussing with...

--On Monday, May 23, 2011 16:26 -0600 Peter Saint-Andre
<[email-masked]> wrote:

> The first paragraph of the Introduction contains a few
> infelicities and grammatical errors. I suggest modifying it as
> follows:
> 
>    RFC 5892 [RFC5892] specifies an algorithm that was defined
> when    the current version of The Unicode Standard was
> Unicode 5.2    [Unicode5.2], and also defines a derived
> property value based on    that algorithm.  Unicode 6.0 has
> changed the GeneralCategory of    three code points that were
> allocated in Unicode 5.2 or earlier.    This implies that the
> derived property value differs depending on    whether the
> property definitions used are from Unicode 5.2 or    Unicode
> 6.0.

Strictly speaking, IDNA2008 (including RFC 5890 and 5892) were
defined when the current version of the Unicode Standard was
5.0.  It was then verified (and would have been modified if
needed, but it wasn't) for Unicode 5.2.  The table-generating
algorithm was rerun for 5.2, producing new non-normative
reference tables.  Because of the General Category changes that
were part of Unicode 6.0, the IDNA category values for those
code points changed and rerunning the table-generating algorithm
produces different results.

     john

---

From: John C Klensin <[email-masked]>
Date: Wed, 25 May 2011 13:32:22 -0400

Just for the record,..

I favor the publication of this document with a minimum of
further fuss.

I read and studied this document before the first I-D was
published, participated in discussion of it on the IDNABIS WG
list and again on the APPSAWG list.  In each case, essentially
the same arguments were repeated and the same conclusion
reached.  While the consensus is definitely rough, we are
putting a tremendous amount of of review cycles and energy into
a document whose essence is "we have agreed to do nothing".  We
rarely publish documentation of a decision to do nothing in the
RFC Series.  Instead, we simply do nothing and move on.

   john

_Background and Reprise (in case needed or wanted)_

The argument for doing something --that this document makes and
describes the wrong decision-- is rooted in a set of decisions
the IDNABIS WG made (also with rough, rather than perfect,
consensus) a rather long time ago.  One of the key differences
between IDNA2003 and IDNA2008 was the decision to try to be
independent of Unicode versions rather than tied to a specific
one.  That decision, in turn, required that the IDNA properties
of characters -- whether they are Protocol-VALID in IDNs,
DISALLOWED, or valid only in particular contexts -- became a
matter of rules that, in turn, utilize the values of selected
Unicode properties.  If Unicode never changed the properties for
a particular character once it was allocated (and avoided
introducing new characters that required contextual evaluation),
the rule-set would never need to be changed and we would never
have to debate, and write, documents keyed to those property
changes: new versions of Unicode would add new characters but
not change the IDNA properties of previously-assigned ones.

The world is not that perfect: Unicode does change properties of
already-assigned characters.  It is rare (how rare depends a bit
on perspective), but it does happen.  The reason is typically to
correct errors made in earlier property assignments, but other
reasons are possible (or one could have a
philosophically-interesting, but otherwise useless, debate about
how far the concept of "error" can extend).  IDNA2008 recognized
that such changes might occur and might be disruptive, so
provided for a special "exception" rule that could be used to
list characters that needed special treatment because of changes
in Unicode properties.  

The WG discussed the question of how to fill the table
associated with that rule in, a discussion that included the
realization that having the  having the table become large would
simply create a new, and hard-to-explain, Unicode-version
dependence.  One option was to populate it automatically: if
Unicode made changes, the table would be filled in to preserve
the behavior at the time the character was first allocated or
that appeared in Unicode 5.2, whichever came later, whatever
that was.  Another was to turn the decision-making over to the
Unicode consortium, which might have resulted in either that
same rule or a rule that treated changes from DISALLOWED to
PVALID differently from changes from PVALID to DISALLOWED.  The
WG rejected both of those ideas as well as the idea that some
changes were inherently more attractive than others.

Instead, the WG concluded that Unicode changes of character
properties that affected IDNA needed to be evaluated in the IETF
on a case-by-case basis as to whether they were important enough
to justify an addition to that exceptions table or whether the
balance fell on the side of keeping the table small (and IDNA
reflecting as short and obvious an exception list as possible).  

In this particular instance, rough consensus has been reached in
multiple groups that it is better to keep the rules (including
that exception table stable and compact than to change the rules
in the interest of preserving the character classification
associated with Unicode 5.2.  Had the characters involved been
less obscure, or more obviously used in domain names for other
than demonstrations and equivalent purposes, perhaps the
decision would have been different (or, if they were less
obscure, perhaps Unicode either would not have made the mistake
in the first place or would have decided to not correct it).

But wanting to consider the tradeoffs and balance between those
alternatives, probably with a slight bias toward not making
changes in the rules,a is precisely why the WG decided that
Unicode property changes needed to be considered by the IETF and
on a case-by-case basis.   

There is no perfect answer to the tradeoff involved unless
Unicode never makes an error and concludes that a correction is
needed.  Because no perfect answer exists, it is possible to
reopen the issue, bring up variations on the same old arguments,
and debate them endlessly.  In that case, we've done that at
least twice, and, depending on how one counts, probably several
more times.  We've decided.  Let's get the document published
rather than seeing how many words and person-hours we can devote
to saying "we considered the issue and decided to do nothing".

     john

--On Monday, 23 May, 2011 15:19 -0700 The IESG
<[email-masked]> wrote:

> 
> The IESG has received a request from the Applications Area
> Working Group WG (appsawg) to consider the following document:
> The IESG plans to make a decision in the next few weeks, and
> solicits final comments on this action. Please send
> substantive comments to the [email-masked] mailing lists by
> 2011-06-06.

---

From: Mark Davis ☕ <[email-masked]>
Date: Wed, 25 May 2011 10:54:20 -0700

I also favor publication of the document with a minimum of further fuss.

[For a somewhat different reason. I do believe that
draft-faltstrom-5892bis-04.txt
makes an incorrect choice, breaking compatibility when that isn't at all
necessary. However, I don't think that any further discussion will change
the decision-maker's minds. Since further discussion appears to just be a
waste of everyone's time,  the document might as well just get issued.]

Mark

*— Il meglio è l’inimico del bene —*

On Wed, May 25, 2011 at 10:32, John C Klensin <[email-masked]> wrote:

> Ietf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ietf
>

---

From: Patrik Fältström <[email-masked]>
Date: Sun, 29 May 2011 13:25:38 +0200

FWIW, although I agree with John and expect me getting a fair number of editorial corrections because of English not being my native language when this document hit the RFC editor, I have slightly changed the first paragraph in the upcoming new version to the following:

(sorry, can not generate TXT now, but you get the point...)

If you have other suggested changes, let me know.

<t><xref target="RFC5892">RFC 5892</xref> specifies an algorithm that
   was defined when <xref target="Unicode5.2">version 5.0 (later
   updated to version 5.2)</xref> was the current version of Unicode,
   and it also defines a derived property value based on that algorithm.
   Unicode 6.0 has changed GeneralCategory of three code points that
   where allocated in Unicode 5.2 or earlier. This imply the derived
   property value differs depending on whether the property definitions
   used are from Unicode 5.2 or 6.0.
</t>

   Patrik

---

From: John C Klensin <[email-masked]>
Date: Sun, 29 May 2011 15:14:00 -0400

--On Sunday, May 29, 2011 13:25 +0200 Patrik Fältström
<[email-masked]> wrote:

> FWIW, although I agree with John and expect me getting a fair
> number of editorial corrections because of English not being
> my native language when this document hit the RFC editor, I
> have slightly changed the first paragraph in the upcoming new
> version to the following:
> 
> <t><xref target="RFC5892">RFC 5892</xref> specifies an
> algorithm that    was defined when 

Please say "rule set" or something like that, not "algorithm".
While "algorithm" is strictly correct, some people will read it
as pseudo-code, and our principle should remain that anything
that produces the save results is correct.

>  <xref> target="Unicode5.2">version 5.0 (later 
> updated to version 5.2)</xref> 
> was the current version of Unicode, and it also
> defines a derived property value based on that algorithm.

Again, "rule set" or some such thing.   In addition, IMO, I
think that, if you are going to make that change, you should add
an extra sentence that says something like "That derived
property value is, in turn, used to generate the tables shown in
Appendix B and their successors".   Neither the derived property
value nor the tables are themselves normative: they depend on a
particular version of Unicode, which the rules do not."

> Unicode 6.0 has changed GeneralCategory of three code points
> that    where allocated in Unicode 5.2 or earlier. This imply
> the derived    property value differs depending on whether the
> property definitions    used are from Unicode 5.2 or 6.0.
> </t>

Editorial recommendation (no change in meaning):

Unicode 6.0 changed the GeneralCategory of three code points
that had been allocated in Unicode 5.2 or earlier. The
consequence of that change is that the derived property value
for those code points differs depending on whether the property
definitions used are from Unicode 5.2 or 6.0. </t>

  --john

---

From: Paul Hoffman <[email-masked]>
Date: Sun, 29 May 2011 12:24:39 -0700

> that produces the save results is correct.

Do you think this change follows the definition of "rule set" in RFC 5892? To me, "algorithm" seems much better. From the introduction to 5892:
   This document reviews and classifies the collections of code points
   in the Unicode character set by examining various properties of the
   code points.  It then defines an algorithm for determining a derived
   property value.  It specifies a procedure, and not a table, of code
   points so that the algorithm can be used to determine code point sets
   independent of the version of Unicode that is in use.
The term "rule set" is used specifically as part of the context registry, and not at all what we are discussing here.

> definitions used are from Unicode 5.2 or 6.0. </t>

Seems like a good change to me.

--Paul Hoffman

---

From: John C Klensin <[email-masked]>
Date: Sun, 29 May 2011 15:50:32 -0400

--On Sunday, May 29, 2011 12:24 -0700 Paul Hoffman
<[email-masked]> wrote:

>...
>> While "algorithm" is strictly correct, some people will read
>> it as pseudo-code, and our principle should remain that
>> anything that produces the save results is correct.
> 
> Do you think this change follows the definition of "rule set"
> in RFC 5892? To me, "algorithm" seems much better. From the
> introduction to 5892:    This document reviews and classifies
> the collections of code points    in the Unicode character set
>...

Sorry, had forgotten that sentence in the Introduction.   Given
that, you are right: "algorithm" is better and we should stick
with it.

    john