Subject: [xmpp] Comments on draft-ietf-xmpp-6122bis-02

From: Florian Zeitz <[email-masked]>
Date: Sun, 05 Aug 2012 00:51:27 +0200

Hello everyone,

I reread the 6122bis draft in preparation for the WG meeting this week
and based on that have some comment that have not yet been brought up on
the list as far as I can remember.

=General=
For the three parts of a JID we still say «The *part of a JID MUST be
treated as follows». I think this sounds somewhat vague (when?, why?, by
whom?).
We might want to change this text to "For normalization of a *part the
following steps MUST be performed" and include a forward reference to
Section 3.

It would likely also be appropriate for Section 3 to explicitly state that:
a) "preparation" refers of this normalization steps
b) "enforcing the rules" means normalization, followed by validation

=Section 2.2=
The current text says «A domainpart consisting of a fully-qualified
domain name MUST be an "internationalized domain name" as defined in
[RFC5890]».
RFC 5890 in turn says «An "internationalized domain name" (IDN) is a
domain name that contains at least one A-label or U-label».
However, a domainpart can consist solely of NR-LDH labels. Considering
the normalizations we enforce I think it would be better for us to
instead say that a domainpart «MUST only contain NR-LDH labels, and
U-labels».

Concerning the normalization steps described in this section I have two
concerns.

The first thing I noticed was that conversion from A-labels to U-labels
is last. However, this could produce Unicode characters from the
punycoded version that would have changed under steps 1 to 3.

Thinking about this some more I started wondering whether an A-label
could actually contain punycoded characters that would change under the
first 3 normalization steps.
What struck me about this is that we are specifying a normalization of
something that already has normalization rules.
E.g. IDNA2008 defines a U-label as a «IDNA-valid string of Unicode
characters, in Normalization Form C (NFC)», so there is no need to ever
perform NFC on it.
We also point people to the PRECIS mappings when a mapping document for
IDNs already exists.
I think it is likely preferable to just defer to IDNA2008 on this and
only mandate the conversion of A-labels to U-labels. Thoughts?

=Section 3=
The list of elements and attributes that a server should consider as JID
slots includes items like data form fields, which also exist from client
to client.
I think it would be appropriate to clarify that
normalization/verification of these JIDs is only required when the
server is responsible for handling the containing stanza (i.e. is not
just acting as a "dumb" router for that stanza).

=Section 4=
The second paragraph starting «For backward compatibility» might be more
appropriate as a Interoperability Note.

Regards,
Florian Zeitz

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 08 Aug 2012 14:44:36 -0600

Hi Florian, thanks for the feedback. I also need to update 6122bis to
track changes to the PRECIS framework spec, so your comments are
timely. Replies inline.

On 8/4/12 4:51 PM, Florian Zeitz wrote:
> Hello everyone,
> 
> I reread the 6122bis draft in preparation for the WG meeting this
> week and based on that have some comment that have not yet been
> brought up on the list as far as I can remember.
> 
> =General= For the three parts of a JID we still say «The *part of a
> JID MUST be treated as follows». I think this sounds somewhat vague
> (when?, why?, by whom?).

The "why" probably seems obvious to me, so I haven't provided much
justification for doing something other than ASCII addresses. Or do
you mean providing justification for each rule?

The "when" and "by whom" are covered in Section 3.

> We might want to change this text to "For normalization of a *part
> the following steps MUST be performed" and include a forward
> reference to Section 3.

Normalization refers to Unicode normalization, so I think your
proposed text would be confusing.

We could say something like this:

OLD
   The localpart of a JID MUST be treated as follows, where the
   operations specified MUST be completed in the order shown:

NEW
   The normalization and mapping rules for the localpart of a JID
   are as follows, where the operations MUST be completed in
   the order shown:

That seems better to me.

> It would likely also be appropriate for Section 3 to explicitly
> state that: a) "preparation" refers of this normalization steps b)
> "enforcing the rules" means normalization, followed by validation

I think that "preparation" means generating JIDs and JID-parts that
are in conformance with the rules, whereas "enforcement" means
actively checking that JIDs and JID-parts are in conformance and
returning an appropriate error if they are not.

Thus I propose the following modification.

OLD
   Enforcement of the XMPP address format rules is the responsibility of
   XMPP servers.  Although XMPP clients SHOULD prepare complete JIDs and
   parts of JIDs in accordance with these rules before including them in
   protocol slots within XMPP streams, XMPP servers MUST enforce the
   rules wherever possible.

NEW
   Enforcement of the XMPP address format rules is the responsibility of
   XMPP servers.  Although XMPP clients SHOULD prepare complete JIDs and
   parts of JIDs in accordance with the rules before including them in
   protocol slots within XML streams (such that JIDs and parts of JIDs
   are in conformance), XMPP servers MUST enforce the rules wherever
   possible and reject stanzas and other XML elements that violate the
   rules (for stanzas, by returning a <jid-malformed/> error to the
   sender as described in Section 8.3.3.8 of [RFC6120]).

> =Section 2.2= The current text says «A domainpart consisting of a
> fully-qualified domain name MUST be an "internationalized domain
> name" as defined in [RFC5890]». RFC 5890 in turn says «An
> "internationalized domain name" (IDN) is a domain name that
> contains at least one A-label or U-label». However, a domainpart
> can consist solely of NR-LDH labels. Considering the normalizations
> we enforce I think it would be better for us to instead say that a
> domainpart «MUST only contain NR-LDH labels, and U-labels».

That is indeed more accurate. Thanks for the suggested text.

OLD
   A domainpart consisting of a fully-qualified domain name MUST be an
   "internationalized domain name" as defined in [RFC5890] and MUST
   consist only of Unicode code points that conform to the rules
   specified in [RFC5892].

NEW
   A domainpart consisting of a fully-qualified domain name MUST contain
   only NR-LDH labels and U-labels as defined in [RFC5890] and MUST
   consist only of Unicode code points that conform to the rules
   specified in [RFC5892].

> Concerning the normalization steps described in this section I have
> two concerns.
> 
> The first thing I noticed was that conversion from A-labels to
> U-labels is last. However, this could produce Unicode characters
> from the punycoded version that would have changed under steps 1 to
> 3.
> 
> Thinking about this some more I started wondering whether an
> A-label could actually contain punycoded characters that would
> change under the first 3 normalization steps. What struck me about
> this is that we are specifying a normalization of something that
> already has normalization rules. E.g. IDNA2008 defines a U-label as
> a «IDNA-valid string of Unicode characters, in Normalization Form C
> (NFC)», so there is no need to ever perform NFC on it.

Right, by specifying things in that way, IDNA2008 assumes that
normalization occurs first.

> We also point people to the PRECIS mappings when a mapping document
> for IDNs already exists. I think it is likely preferable to just
> defer to IDNA2008 on this and only mandate the conversion of
> A-labels to U-labels. Thoughts?

You raise a very good point.

I think we're all in agreement that we want to say as little as
possible here above what's in the IDNA2008 specifications.

Currently we say:

###

   The domainpart of a JID MUST be treated as follows, where the
   operations specified MUST be completed in the order shown:

   1.  Uppercase and titlecase characters MUST be mapped to their
       lowercase equivalents.

   2.  Additional mappings MAY be applied, such as those defined in
       [MAPPINGS].

   3.  All characters MUST be mapped using Unicode Normalization Form C
       (NFC).

   4.  Each A-label MUST be converted to a U-label.

   With regard to directionality, applications MUST apply the "Bidi
   Rule" defined in [RFC5893] (i.e., each of the six conditions of the
   Bidi Rule must be satisfied).

###

Let's take these one at a time.

   1.  Uppercase and titlecase characters MUST be mapped to their
       lowercase equivalents.

IDNA2008 does not specify casemapping, so EXAMPLE.org is the same as
example.org. That's because the DNS has this (to me, strange) property
of preserving case on output but ignoring it during comparison. It
seems to me that we have two options here:

(a) Preserve case in domainparts but ignore it during comparison.

(b) Map uppercase and title case code points to their lowercase
equivalents.

I realize that (a) follows DNS, but (b) seems safer to me since in
XMPP we do that mapping for localparts (whereas for resourceparts we
preserve case but don't ignore it during comparison).

   2.  Additional mappings MAY be applied, such as those defined in
       [MAPPINGS].

The topic of "additional mappings" is opened by RFC 5895. However, I
think it would be better to point to draft-yoneya-precis-mappings (and
its WG version once published) because it is more specific. However,
such additional mappings would be purely optional in any case.

   3.  All characters MUST be mapped using Unicode Normalization Form C
       (NFC).

The normalization rule (NFC) is covered by IDNA2008, so we don't need
to include it here.

   4.  Each A-label MUST be converted to a U-label.

This is something we want (so that we're not dealing with Punycode).

   With regard to directionality, applications MUST apply the "Bidi
   Rule" defined in [RFC5893] (i.e., each of the six conditions of the
   Bidi Rule must be satisfied).

The directionality rule (BiDi) is covered by IDNA2008, so we don't
need to include it here.

As a result of the foregoing analysis, I suggest the following text:

###

   In general, the content of a domainpart is an Internationalized
   Domain Name ("IDN") as described in the specifications for
   Internationalized Domain Names in Applications (commonly called
   "IDNA2008") [RFC5890], and a domainpart is an "IDNA-aware domain name
   slot".  The following rules apply to a domainpart that consists of a
   fully-qualified domain name:

   o  The domainpart MUST contain only NR-LDH labels and U-labels as
      defined in [RFC5890] and MUST consist only of Unicode code points
      that conform to the rules specified in [RFC5892].

   o  The domainpart MUST NOT include A-labels as defined in [RFC5890];
      each A-label MUST be converted to a U-label during preparation of
      a domainpart and comparison MUST be performed using U-labels, not
      A-labels.

   o  After conversion of A-labels to U-labels if necessary, all
      uppercase and titlecase code points within the domainpart MUST be
      mapped to their lowercase equivalents.

   o  After (and in addition to) casemapping, other mappings MAY be
      applied to the domainpart, such as those defined in [MAPPINGS] or
      [RFC5895].

   After any and all conversion, normalization, and mapping of code
   points, a domainpart MUST NOT be zero bytes in length and MUST NOT be
   more than 1023 bytes in length.  (Naturally, the length limits of
   [RFC1034] apply, and nothing in this document is to be interpreted as
   overriding those more fundamental limits.)

###

> =Section 3= The list of elements and attributes that a server
> should consider as JID slots includes items like data form fields,
> which also exist from client to client. I think it would be
> appropriate to clarify that normalization/verification of these
> JIDs is only required when the server is responsible for handling
> the containing stanza (i.e. is not just acting as a "dumb" router
> for that stanza).

Section 3 already says:

   In general, servers are responsible for enforcing the address format
   rules when receiving protocol elements from clients where the server
   is expected to process or act on such elements ....

   However, servers are not responsible for enforcing the rules when the
   protocol elements are intended for communication among other
   entities ....
   .... in such cases, clients SHOULD enforce the rules, and client
   implementers need to understand that not enforcing the rules can lead
   to a degraded user experience or security vulnerabilities.

Do you think that we need to make clear in each instance whether a
given element or attribute can appear in both situations (handled by
the server or routed to another entity)?

Note also the following statement in RFC 6120:

"Each server implementation will contain its own logic for processing
stanzas it receives. Such logic determines whether the server needs to
route a given stanza to another domain, deliver it to a local entity
(typically a connected client associated with a local account), or
handle it directly within the server itself."

Thus I suggest...

OLD
   In general, servers are responsible for enforcing the address format
   rules when receiving protocol elements from clients where the server
   is expected to process or act on such elements; two examples from
   [RFC6120] are the 'to' attribute on XML stanzas (which is a JID slot
   used by XMPP servers for routing of outbound stanzas) and the
   <resource/> child of the <bind/> element (which is a resourcepart
   slot used by XMPP servers for binding of a resource to an account for
   routing of stanzas between the server and a particular client).
   However, servers are not responsible for enforcing the rules when the
   protocol elements are intended for communication among other
   entities; two examples are the 'initiator' attribute in the Jingle
   extension [XEP-0166] (which is a JID slot used for client-to-client
   coordination of multimedia sessions) and the 'nick' attribute in the
   Multi-User Chat extension [XEP-0045] (which is a resourcepart slot
   used for administrative purposes in the context of XMPP chatrooms);
   in such cases, clients SHOULD enforce the rules, and client
   implementers need to understand that not enforcing the rules can lead
   to a degraded user experience or security vulnerabilities.

NEW

   In general, a server is responsible for enforcing the address format
   rules when receiving protocol elements from clients where the server
   is expected to handle such elements directly or to use them for
   purposes of routing a stanza to another domain or delivering a stanza
   to a local entity; two examples from [RFC6120] are the 'to' attribute
   on XML stanzas (which is a JID slot used by XMPP servers for routing
   of outbound stanzas) and the <resource/> child of the <bind/> element
   (which is a resourcepart slot used by XMPP servers for binding of a
   resource to an account for routing of stanzas between the server and
   a particular client).  However, a server is not responsible for
   enforcing the rules when the protocol elements are intended for
   communication among other entities, typically within the payload of a
   stanza that the server is merely routing to another domain or
   delivering to a local entity; two examples are the 'initiator'
   attribute in the Jingle extension [XEP-0166] (which is a JID slot
   used for client-to-client coordination of multimedia sessions) and
   the 'nick' attribute in the Multi-User Chat extension [XEP-0045]
   (which is a resourcepart slot used for administrative purposes in the
   context of XMPP chatrooms); in such cases, clients SHOULD enforce the
   rules themselves and not depend on the server to do so, and client
   implementers need to understand that not enforcing the rules can lead
   to a degraded user experience or security vulnerabilities.

> =Section 4= The second paragraph starting «For backward
> compatibility» might be more appropriate as a Interoperability
> Note.

Agreed.

I will immediately submit -03 of draft-ietf-xmpp-6122bis so the
changes are easier to track.

Peter

- -- 
Peter Saint-Andre
https://stpeter.im/

---

From: Florian Zeitz <[email-masked]>
Date: Thu, 09 Aug 2012 04:38:41 +0200

Am 08.08.2012 22:44, schrieb Peter Saint-Andre:
> Hi Florian, thanks for the feedback. I also need to update 6122bis 
> to track changes to the PRECIS framework spec, so your comments are
> timely. Replies inline.
> 
>> =General= For the three parts of a JID we still say «The *part
>> of a JID MUST be treated as follows». I think this sounds
>> somewhat vague (when?, why?, by whom?).
> 
>  justification for doing something other than ASCII addresses. Or 
> do you mean providing justification for each rule?
> 
Agreed, my point was basically that a forward reference to Section 3
would be good here, to improve readability.

>> We might want to change this text to "For normalization of a 
>> *part the following steps MUST be performed" and include a 
>> forward reference to Section 3.
> 
Right, the intention here was to avoid the overly general "treated".

> OLD The localpart of a JID MUST be treated as follows, where the 
> NEW The normalization and mapping rules for the localpart of a JID
>  are as follows, where the operations MUST be completed in the
> order shown:
> 
> That seems better to me.
> 
That seems fine to me.

>> state that: a) "preparation" refers of this normalization steps 
I tend to disagree. I might of course be mistaken, but my current
understanding is that even after performing all "normalization and
mapping" steps we might still end up with something that is not
NAME_PVAL. So we can act according to the rules for preparation, but
still end up with something that does not conform the rules for a
localpart. (I.e. what are "the rules" exactly)

Also, as long as clients are not required to perform preparation, a
server must IMHO always do a combination of preparation and
enforcement. I think returning <jid-malformed/> to someone who sent
U+006F LATIN SMALL LETTER O + U+0308 COMBINING DIAERESIS instead of
U+00F6 LATIN SMALL LETTER O WITH DIAERESIS is something we want to
avoid by all means. It might be worthwhile to make that abundantly clear.

>>  can consist solely of NR-LDH labels. Considering the 
>> normalizations we enforce I think it would be better for us to 
>> instead say that a domainpart «MUST only contain NR-LDH labels, 
>> and U-labels».
> 
> OLD A domainpart consisting of a fully-qualified domain name MUST 
> be an "internationalized domain name" as defined in [RFC5890] and 
> MUST consist only of Unicode code points that conform to the rules
>  specified in [RFC5892].
> 
> NEW A domainpart consisting of a fully-qualified domain name MUST 
> contain only NR-LDH labels and U-labels as defined in [RFC5890]
> and MUST consist only of Unicode code points that conform to the
> rules specified in [RFC5892].
> 
Looks good.

>> Concerning the normalization steps described in this section I 
>> have two concerns.
> 
> [...] Let's take these one at a time.
> 
> IDNA2008 does not specify casemapping, so EXAMPLE.org is the same 
> as example.org. That's because the DNS has this (to me, strange) 
> property of preserving case on output but ignoring it during 
> comparison. It seems to me that we have two options here:
> 
>  XMPP we do that mapping for localparts (whereas for resourceparts 
> we preserve case but don't ignore it during comparison).
> 
That might be me misunderstanding IDNA2008/DNS, but:
What ends up in DNS are always only A-labels and NR-LDH labels. DNS is
not aware of Unicode characters as such, everything in there is ASCII.
RFC 5895 talks specifically about mapping uppercase characters to
their lowercase equivalent in Section 2.
My understanding is that uppercase letters (which can be mapped to
lowercase) are not PVALID in IDNA2008 U-labels, which is archived by
exclusion of the Unstable (B) category.

As such I don't think we need to care about this one.
(Disclaimer: This is from my somewhat limited understanding of
IDNA2008, I'll gladly bow to anyone with a better understanding)

> The topic of "additional mappings" is opened by RFC 5895. However, 
> I think it would be better to point to
> draft-yoneya-precis-mappings (and its WG version once published)
> because it is more specific. However, such additional mappings
> would be purely optional in any case.
> 
I couldn't find anything about additional mappings in RFC 5895, could
you point me to some specific text?
The reason I'm very uncomfortable with even making this OPTIONAL is
the following:
We do DNS lookup based on the domainpart. That means we prepare it
according to these rules, perform punycode it, prepend
"_xmpp-server._tcp." and do a SRV lookup based on that. If we map
certain characters (other than what is specified for IDNA2008) to
other ones, or nothing, what we look up in DNS changes.
I don't think that is the case for anything currently in
draft-yoneya-precis-mappings, but I think it is dangerous terrain to
tread on.

> 3.  All characters MUST be mapped using Unicode Normalization Form 
> C (NFC).
> 
> The normalization rule (NFC) is covered by IDNA2008, so we don't 
> need to include it here.
> 
I agree with all of these.

>> should consider as JID slots includes items like data form 
>> fields, which also exist from client to client. I think it would 
>> be appropriate to clarify that normalization/verification of 
>> these JIDs is only required when the server is responsible for 
>> handling the containing stanza (i.e. is not just acting as a 
>> "dumb" router for that stanza).
> 
> Section 3 already says:
> 
> In general, servers are responsible for enforcing the address 
> format rules when receiving protocol elements from clients where 
> the server is expected to process or act on such elements ....
> 
> However, servers are not responsible for enforcing the rules when 
> the protocol elements are intended for communication among other 
> entities .... .... in such cases, clients SHOULD enforce the
> rules, and client implementers need to understand that not
> enforcing the rules can lead to a degraded user experience or
> security vulnerabilities.
> 
>  given element or attribute can appear in both situations (handled 
> by the server or routed to another entity)?
> 
I had seen this text, I just feel that it addresses a marginally
different issue.
The misunderstanding I foresee (partially due to the Jingle example)
is that there are protocols that servers don't ever care about (e.g.
Jingle) and some that they need to parse anyway (e.g. dataforms for
ad-hoc commands directed at the server). Thus, someone could conclude
that dataforms always need to be checked, because servers needs to
"handle *such* elements", even though it will never handle this
specific element.

I don't think we need to say which slots can occur in what contexts.
However, I do think we might be better off if we add some text that
makes it very clear that whether validation needs to be performed is
dependent on the context and not the type of slot.

Florian

---

From: Peter Saint-Andre <[email-masked]>
Date: Thu, 09 Aug 2012 09:42:00 -0600

Comments inline, areas of agreement snipped...

>>>  state that: a) "preparation" refers of this normalization
>>> steps b) "enforcing the rules" means normalization, followed by
>>>  validation
> 
>> I think that "preparation" means generating JIDs and JID-parts
>> that are in conformance with the rules, whereas "enforcement"
>> means actively checking that JIDs and JID-parts are in
>> conformance and returning an appropriate error if they are not.
> 
> NAME_PVAL. So we can act according to the rules for preparation,
> but still end up with something that does not conform the rules for
> a localpart. (I.e. what are "the rules" exactly)

If a client receives an input string, prepares it according to the
rules, and the output doesn't conform to the rules (e.g., a localpart
contains a code point not allowed by NAME_PVAL), then IMHO it needs to
bounce it back to the user (if any) for correction. However, I see
your point that such behavior is, de facto, enforcement. :)

> Also, as long as clients are not required to perform preparation,
> a server must IMHO always do a combination of preparation and 
> enforcement. I think returning <jid-malformed/> to someone who
> sent U+006F LATIN SMALL LETTER O + U+0308 COMBINING DIAERESIS
> instead of U+00F6 LATIN SMALL LETTER O WITH DIAERESIS is something
> we want to avoid by all means. It might be worthwhile to make that
> abundantly clear.

Yes, precisely so. I'll try to work on proposed text.

>> I realize that (a) follows DNS, but (b) seems safer to me since
>> in XMPP we do that mapping for localparts (whereas for
>> resourceparts we preserve case but don't ignore it during
>> comparison).
> 
> That might be me misunderstanding IDNA2008/DNS, but: What ends up
> in DNS are always only A-labels and NR-LDH labels. DNS is not aware
> of Unicode characters as such, everything in there is ASCII.

Correct.

> their lowercase equivalent in Section 2. My understanding is that
> uppercase letters (which can be mapped to lowercase) are not PVALID
> in IDNA2008 U-labels, which is archived by exclusion of the
> Unstable (B) category.

Yes, that is the IDNA2008 workaround. Personally I find that a bit of
an odd way to solve the problem, but that's what the IDNA2008 design
team chose.

> As such I don't think we need to care about this one. (Disclaimer:
> This is from my somewhat limited understanding of IDNA2008, I'll
> gladly bow to anyone with a better understanding)
> 
>> The topic of "additional mappings" is opened by RFC 5895.
>> However, I think it would be better to point to 
> I couldn't find anything about additional mappings in RFC 5895,
> could you point me to some specific text?

Section 2 of RFC 5895 talks about mapping uppercase characters (but
not titlecase characters(?)) to lowercase, mapping fullwidth and
halfwidth characters to their decomposition mappings, etc. Strangely,
RFC 5895 notes that the steps are ordered and application of NFC is
not the first step. This seems to be inconsistent with the advice in
the other IDNA2008 RFCs. I'll need to investigate further and think
about it some more.

> The reason I'm very uncomfortable with even making this OPTIONAL
> is the following: We do DNS lookup based on the domainpart. That
> means we prepare it according to these rules, perform punycode it,
> prepend "_xmpp-server._tcp." and do a SRV lookup based on that. If
> we map certain characters (other than what is specified for
> IDNA2008) to other ones, or nothing, what we look up in DNS
> changes. I don't think that is the case for anything currently in 
> draft-yoneya-precis-mappings, but I think it is dangerous terrain
> to tread on.

So, let's get down to cases (see draft-yoneya-precis-mappings, Section 4).

Are you worried about mapping of uppercase and titlecase characters to
lowercase? It certainly seems that *not* mapping Example.Com to
example.com would violate the principle of least user surprise.

The fullwidth and halfwidth characters are less familiar to most
people on this list, but they are common in certain East Asian
scripts. Mapping them to their decomposition mappings would, I think,
make sense to most users of those scripts.

For domainparts, mapping of whitespace characters to SP (U+0020) does
not apply.

The local case mappings in Section 4.3 of draft-yoneya-precis-mappings
essentially capture the exceptions from the SpecialCasing.txt file in
the Unicode character database. Applying those rules also makes sense
to me (e.g., to handle "dotless i" in certain Turkic locales).

>> 3.  All characters MUST be mapped using Unicode Normalization
>> Form C (NFC).
> 
>>  Rule" defined in [RFC5893] (i.e., each of the six conditions of
>> the Bidi Rule must be satisfied).
> 
>> The directionality rule (BiDi) is covered by IDNA2008, so we
>> don't need to include it here.
> 
>>> fields, which also exist from client to client. I think it
>>> would be appropriate to clarify that normalization/verification
>>> of these JIDs is only required when the server is responsible
>>> for handling the containing stanza (i.e. is not just acting as
>>> a "dumb" router for that stanza).
> 
>> Section 3 already says:
> 
>> Do you think that we need to make clear in each instance whether
>> a given element or attribute can appear in both situations
>> (handled by the server or routed to another entity)?
> 
> different issue. The misunderstanding I foresee (partially due to
> the Jingle example) is that there are protocols that servers don't
> ever care about (e.g. Jingle) and some that they need to parse
> anyway (e.g. dataforms for ad-hoc commands directed at the server).
> Thus, someone could conclude that dataforms always need to be
> checked, because servers needs to "handle *such* elements", even
> though it will never handle this specific element.

BTW, see 6122bis-03 -- it now reads:

   o  The <value/> element for Data Forms [XEP-0004] communicated to the
      server, when the 'type' attribute is "jid-single" or "jid-multi".

I could add further clarification, such as:

   o  The <value/> element for Data Forms [XEP-0004] communicated to the
      server itself (e.g., as in [XEP-0133]), when the 'type'
attribute is "jid-single"
      or "jid-multi".

> I don't think we need to say which slots can occur in what
> contexts. However, I do think we might be better off if we add some
> text that makes it very clear that whether validation needs to be
> performed is dependent on the context and not the type of slot.

I might be missing something, but I'm not completely clear on what you
mean by the constrast between context and type of slot.

In any case, do you find the text in -03 to be a bit clearer? Can you
make suggested improvements to that text? I'll look at it again, as well.

Thanks again for your close review of this document!

Peter

- -- 
Peter Saint-Andre
https://stpeter.im/

---

From: Florian Zeitz <[email-masked]>
Date: Thu, 09 Aug 2012 22:06:38 +0200

> Comments inline, areas of agreement snipped...
Same :)
>>>> Concerning the normalization steps described in this section
>>>> I have two concerns.
> 
> [...]
>>  their lowercase equivalent in Section 2. My understanding is
>> that uppercase letters (which can be mapped to lowercase) are not
>> PVALID in IDNA2008 U-labels, which is archived by exclusion of
>> the Unstable (B) category.
> 
> Yes, that is the IDNA2008 workaround. Personally I find that a bit
> of an odd way to solve the problem, but that's what the IDNA2008
> design team chose.
> 
>>> 2.  Additional mappings MAY be applied, such as those defined
>>> in [MAPPINGS].
> 
>>> draft-yoneya-precis-mappings (and its WG version once
>>> published) because it is more specific. However, such
>>> additional mappings would be purely optional in any case.
> 
> Section 2 of RFC 5895 talks about mapping uppercase characters
> (but not titlecase characters(?)) to lowercase, mapping fullwidth
> and halfwidth characters to their decomposition mappings, etc.
> Strangely, RFC 5895 notes that the steps are ordered and
> application of NFC is not the first step. This seems to be
> inconsistent with the advice in the other IDNA2008 RFCs. I'll need
> to investigate further and think about it some more.
> 
I think the fact that titlecase characters are not mentioned is just
an oversight. The mapping process further described a bit down would
also catch titlecase characters.
Also note that while IDNA2008 mandates normalization before validation
what we have here is mapping before normalization. That is appropriate
since mappings could cause the string to no longer be in NFC.

>> means we prepare it according to these rules, perform punycode
>> it, prepend "_xmpp-server._tcp." and do a SRV lookup based on
>> that. If we map certain characters (other than what is specified
>> for IDNA2008) to other ones, or nothing, what we look up in DNS 
>>  draft-yoneya-precis-mappings, but I think it is dangerous
>> terrain to tread on.
> 
> Are you worried about mapping of uppercase and titlecase characters
> to lowercase? It certainly seems that *not* mapping Example.Com to 
> scripts. Mapping them to their decomposition mappings would, I
> think, make sense to most users of those scripts.
> 
> For domainparts, mapping of whitespace characters to SP (U+0020)
> does not apply.
> 
> The local case mappings in Section 4.3 of
> draft-yoneya-precis-mappings essentially capture the exceptions
> from the SpecialCasing.txt file in the Unicode character database.
> Applying those rules also makes sense to me (e.g., to handle
> "dotless i" in certain Turkic locales).
> 
As I said it does not seem that draft-yoneya-precis-mappings in it's
current state would cause any problem. In fact all mappings you listed
above are already specified by RFC 5895 (including the ones from
SpecialCasing.txt as part of the case folding). So this is at best
redundant. My worry however is that we say you MAY perform any
mappings. I'm not sure what the motivation for that is. If any
mappings that are not in line with what IDNA2008 specifies are
performed DNS lookup *will* fail. Either that or I'm completely
missing something.

> [...]
> o  The <value/> element for Data Forms [XEP-0004] communicated to
> the server, when the 'type' attribute is "jid-single" or
> "jid-multi".
> 
> o  The <value/> element for Data Forms [XEP-0004] communicated to
> the server itself (e.g., as in [XEP-0133]), when the 'type' 
> attribute is "jid-single" or "jid-multi".
> 
That is better, but we'd need that all over the place. E.g. also for
disco. I'd think a general statement up front would be better, see below.

>> contexts. However, I do think we might be better off if we add
>> some text that makes it very clear that whether validation needs
>> to be performed is dependent on the context and not the type of
>> slot.
> 
> I might be missing something, but I'm not completely clear on what
> you mean by the constrast between context and type of slot.
> 
Okay, I had feared that this might be unclear, so let me try to
rephrase that as proposed text:

"Note that while a server might be capable of detecting a certain
element or attribute is a JID slot, enforcing the rules on the
contained JID is only REQUIRED if the server needs to perform further
operations based on the value of the JID".

It may however be possible to integrate the same information within
the existing text.

> In any case, do you find the text in -03 to be a bit clearer? Can
> you make suggested improvements to that text? I'll look at it
> again, as well.
> 
Yes, I do find the text in -03 is definitely an improvement. Thank you
for working on this.

- --
Florian

---

From: Peter Saint-Andre <[email-masked]>
Date: Thu, 09 Aug 2012 16:24:18 -0600

>> Comments inline, areas of agreement snipped...

Ditto. :)

>>>>> Concerning the normalization steps described in this
>>>>> section I have two concerns.
> 
>>>> 1.  Uppercase and titlecase characters MUST be mapped to
>>>> their lowercase equivalents.
>> [...]
>>> RFC 5895 talks specifically about mapping uppercase characters
>>> to their lowercase equivalent in Section 2. My understanding
>>> is that uppercase letters (which can be mapped to lowercase)
>>> are not PVALID in IDNA2008 U-labels, which is archived by
>> Yes, that is the IDNA2008 workaround. Personally I find that a
>> bit of an odd way to solve the problem, but that's what the
>> IDNA2008 design team chose.
> 
>>>> 2.  Additional mappings MAY be applied, such as those
>>>> defined in [MAPPINGS].
> 
>> (but not titlecase characters(?)) to lowercase, mapping
>> fullwidth and halfwidth characters to their decomposition
>> mappings, etc. Strangely, RFC 5895 notes that the steps are
>> ordered and application of NFC is not the first step. This seems
>> to be inconsistent with the advice in the other IDNA2008 RFCs.
>> I'll need to investigate further and think about it some more.
> 
> I think the fact that titlecase characters are not mentioned is
> just an oversight. The mapping process further described a bit down
> would also catch titlecase characters. Also note that while
> IDNA2008 mandates normalization before validation what we have here
> is mapping before normalization. That is appropriate since mappings
> could cause the string to no longer be in NFC.
> 
>>> The reason I'm very uncomfortable with even making this
>>> OPTIONAL is the following: We do DNS lookup based on the
>>> domainpart. That means we prepare it according to these rules,
>>> perform punycode it, prepend "_xmpp-server._tcp." and do a SRV
>>> lookup based on that. If we map certain characters (other than
>>> what is specified for IDNA2008) to other ones, or nothing, what
>>> we look up in DNS changes. I don't think that is the case for
>>> anything currently in draft-yoneya-precis-mappings, but I think
>>> it is dangerous terrain to tread on.
> 
>> Are you worried about mapping of uppercase and titlecase
>> characters to lowercase? It certainly seems that *not* mapping
>> Example.Com to example.com would violate the principle of least
>> user surprise.
> 
>> from the SpecialCasing.txt file in the Unicode character
>> database. Applying those rules also makes sense to me (e.g., to
>> handle "dotless i" in certain Turkic locales).
> 
> As I said it does not seem that draft-yoneya-precis-mappings in
> it's current state would cause any problem. In fact all mappings
> you listed above are already specified by RFC 5895 (including the
> ones from SpecialCasing.txt as part of the case folding). So this
> is at best redundant. My worry however is that we say you MAY
> perform any mappings. I'm not sure what the motivation for that is.
> If any mappings that are not in line with what IDNA2008 specifies
> are performed DNS lookup *will* fail. Either that or I'm
> completely missing something.

OK. So, one approach would simply be to say that a domainpart needs to
conform to IDNA2008 and all A-labels need to be represented as
U-labels. Full stop. Would that allay your concerns?

>> o  The <value/> element for Data Forms [XEP-0004] communicated
>> to the server, when the 'type' attribute is "jid-single" or 
>> "jid-multi".
> 
>> o  The <value/> element for Data Forms [XEP-0004] communicated
>> to the server itself (e.g., as in [XEP-0133]), when the 'type' 
> That is better, but we'd need that all over the place. E.g. also
> for disco. I'd think a general statement up front would be better,
> see below.
> 
>>> some text that makes it very clear that whether validation
>>> needs to be performed is dependent on the context and not the
>>> type of slot.
> 
>> I might be missing something, but I'm not completely clear on
>> what you mean by the constrast between context and type of slot.
> 
> contained JID is only REQUIRED if the server needs to perform
> further operations based on the value of the JID".

Yes, that sums it up nicely. Thanks for the proposed text.

> It may however be possible to integrate the same information
> within the existing text.

Right, I think it belongs in the section we're referring to. I'll do
so in -04.

>> In any case, do you find the text in -03 to be a bit clearer?
>> Can you make suggested improvements to that text? I'll look at
>> it again, as well.
> 
> Yes, I do find the text in -03 is definitely an improvement. Thank
> you for working on this.

And thank you for the continued feedback.

Peter

- -- 
Peter Saint-Andre
https://stpeter.im/

---

From: Florian Zeitz <[email-masked]>
Date: Fri, 10 Aug 2012 01:49:46 +0200

>>> Comments inline, areas of agreement snipped...
Idem.

>> you listed above are already specified by RFC 5895 (including
>> the ones from SpecialCasing.txt as part of the case folding). So
>> this is at best redundant. My worry however is that we say you
>> MAY perform any mappings. I'm not sure what the motivation for
>> that is. If any mappings that are not in line with what IDNA2008
>> specifies are performed DNS lookup *will* fail. Either that or
>> I'm completely missing something.
> 
> OK. So, one approach would simply be to say that a domainpart needs
> to conform to IDNA2008 and all A-labels need to be represented as 
I think that was in fact my initial suggestion, so yes, that would
make me happy.
However, seeing how we specify preparation and validation for all
other JID parts, I would amend that slightly and also suggest saying
that any mapping and normalization steps specified by IDNA2008 should
be performed on user input, or data send by a client.

- --
Florian

---

From: [email-masked]
Date: Thu, 9 Aug 2012 20:14:07 -0400

> performed DNS lookup *will* fail. Either that or I'm completely

I had a question about this case mapping issue, which I've brought up at
previous IETFs, and with some more reading of the source documents I think I
can make the comment more intelligently.

RFC 5895 distinguishes between "user interface" and "protocol", because
a number of transformations need to take into account "culture, context, and
locale."  Protocol needs to be specified normatively, but user interface
shouldn't be.

In the XMPP context, enforcement rules need to be about protocol, but JID
creation should (I think) be in the realm of user interface.  I think this
distinction is currently not terribly clear in 6122bis; the algorithm in
Section 2 reads like it's trying to do JID creation, whereas I think what we
actually need to specify is how to do enforcement, and make sure that
enforcement can be entirely locale-independent.

To make this more concrete, when enforcing that a JID localpart is lower
case, a server doesn't need to care whether a LATIN SMALL LETTER I was
originally LATIN CAPITAL LETTER I, LATIN CAPITAL LETTER I WITH DOT ABOVE, or
actually entered directly by a user; it's fine regardless.  Conversely, if
that localpart has a LATIN CAPITAL LETTER I in it the JID is malformed, and
it doesn't matter what the equivalent normalized JID would be.

-- 
Jonathan Lennox
[email-masked] / [email-masked]

---

From: Florian Zeitz <[email-masked]>
Date: Fri, 10 Aug 2012 14:22:07 +0200

Am 10.08.2012 02:14, schrieb [email-masked]:
>> perform any mappings. I'm not sure what the motivation for that
>> is. If any mappings that are not in line with what IDNA2008
> I had a question about this case mapping issue, which I've brought
> up at previous IETFs, and with some more reading of the source
> documents I think I can make the comment more intelligently.
> 
> RFC 5895 distinguishes between "user interface" and "protocol",
> because a number of transformations need to take into account
> "culture, context, and locale."  Protocol needs to be specified
> normatively, but user interface shouldn't be.
> 
> In the XMPP context, enforcement rules need to be about protocol,
> but JID creation should (I think) be in the realm of user
> interface.  I think this distinction is currently not terribly
> clear in 6122bis; the algorithm in Section 2 reads like it's trying
> to do JID creation, whereas I think what we actually need to
> specify is how to do enforcement, and make sure that enforcement
> can be entirely locale-independent.
> 
> To make this more concrete, when enforcing that a JID localpart is
> lower case, a server doesn't need to care whether a LATIN SMALL
> LETTER I was originally LATIN CAPITAL LETTER I, LATIN CAPITAL
> LETTER I WITH DOT ABOVE, or actually entered directly by a user;
> it's fine regardless.  Conversely, if that localpart has a LATIN
> CAPITAL LETTER I in it the JID is malformed, and it doesn't matter
> what the equivalent normalized JID would be.
> 
Hi,
thank you a lot for jumping in on this discussion. You helped me get
some things a bit clearer in my head.

First of all, I think you are precisely right.
We have two separate things here:
a) Trying to prepare user input such that it becomes a valid JID
b) Asserting whether a JID is valid

Let's start with b) (enforcement), because that is easier.
We generally would do (not necessarily in this order):
1. isNFC(part)
2. is PVALID (for NameClass+restrictions, IDNA, and FreeClass
respectively)
3. check the Bidi Rule is fulfilled
(4. for the domainpart only, check if each label is a U-label)
If any of these fails the JID is not valid.

I think these conditions are somewhat scattered across the current
draft, e.g. the Bidi Rule is always separated from the rest of these
conditions, by a paragraph on preparation.
I suspect that is what you perceive as a missing distinction in the
current draft.

So what about a) (preparation).
I'd like to note that I had a big misconception about this before.
As far as I remember people had previously argued that they want to
write simple clients, that basically forwards user input to the server
right away, possibly dealing with the fact that they get a
<jid-malformed/> back.
Conversely the current draft says:
«XMPP clients SHOULD prepare complete JIDs and parts of JIDs in
accordance with the rules before including them in protocol slots
within XML streams (such that JIDs and parts of JIDs are in conformance)»

With that text in place I wonder what we would want to say about
servers performing preparation.
It generally should not be needed, yet (overly) simple clients that
don't perform preparation would get a terrible user experience (i.e.
typing a single uppercase character leads to a <jid-malformed/> and
very confused users).
I have no strong opinion about this, I suspect poor overloaded servers
would prefer to only do enforcement.

It also is a lot clearer to me why I was so baffled by the text about
mappings now. The problem here is not necessarily allowing other
mappings, it's allowing arbitrary ones.
We have a goal with this mappings, which is to get user input in a
form that is a valid JID. So if our input contains codepoints that are
disallowed it seems sensible to allow (possibly locale specific)
additional mappings that map these disallowed codepoints to allowed
ones. However, I'd want the text to include that restriction.
To be clear, this is not specific to domainparts.

(BTW I'm still intrigued by the fact that locale specific mappings
seems to imply that a Turkish user typing e.g. STPETER.IM would get
that mapped to something that has a U-label as TLD, but that's beside
the point here)

So, for preparation I think I'd suggest:
For nodeparts and resourceparts:
1. case-folding
2. mappings from draft-yoneya-precis-mappings and other goal-oriented
mappings
3. NFC

(I think NFC last is actually correct, since mappings might lead
combining characters appearing in the wrong order, but that is
admittedly more of a hunch)

For domainparts:
1. goal-oriented mappings
2. procedure from RFC 5895 (includes case-folding and NFC)

Where goal-oriented means that it is a mapping from the set of
disallowed codepoints to the set of allowed codepoints.

- --
Florian