From nobody Mon Dec 4 15:06:13 2017 Return-Path: X-Original-To: precis@ietfa.amsl.com Delivered-To: precis@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6A250128D69 for ; Mon, 4 Dec 2017 15:06:12 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.62 X-Spam-Level: X-Spam-Status: No, score=-2.62 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pCz1KPJSK2YI for ; Mon, 4 Dec 2017 15:06:10 -0800 (PST) Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1C119128D64 for ; Mon, 4 Dec 2017 15:06:09 -0800 (PST) Received: from christihudtsmbp.fritz.box ([88.77.154.196]) by mail.gmx.com (mrgmx001 [212.227.17.190]) with ESMTPSA (Nemesis) id 0MC4R6-1eDENz2jtg-008sRd for ; Tue, 05 Dec 2017 00:06:07 +0100 From: Christian Schudt Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Message-Id: <0207F020-32D2-4181-A020-3143BD8E88FE@gmx.de> Date: Tue, 5 Dec 2017 00:06:06 +0100 To: precis@ietf.org X-Mailer: Apple Mail (2.3273) X-Provags-ID: V03:K0:1wBLZ28tEMb1r4bxsBRVj+ME0z4ZNl7jlTcHMwL6sfEfpL5BrG5 s5B7aoImqaTgPoZEfVHmVXi26/79DcqlxRLtp0XqMdtUo1U/sj1GrEUmeKGGFNiSUTc7WMO OmT6753nrx7dc+aq6bLOCQHq9Bz2qMjNgAmyINn8PkOdRs0NgMG0mE4B0Qj+TlA/6fmgfh+ 33jCzqccZrq6OloF1bKRQ== X-UI-Out-Filterresults: notjunk:1;V01:K0:L0ScTfCVKRM=:ge1Vm6JMelSycfLsNQCP2A OhmVBnzLmbl14E3MsvxF8t8XP5xiddUm87W7S4+/K/XoOq958fyB1Ajoi5G6lx7Rhm8b3zBsm Hcvz5b+qP7EWWljxRPPx0CSPvE8dh9asdGraZPc+Hy6QCbzUIkZ4ebqhHfzNPKaT5DqL5+OiY sqkLdzfMF9lzVbebGyGdbAATx+QqpxFmXD0FWQlbzm5rDpkinIYiA6ZaadMJZDNEhR4P8y+rt gp6sWaXATivsze8DrYiPJoGgZu5ioJFiAfOQ0cjY7o6X2xb/XmAGQbS4bbyc0WCCTPAq+KInk g6NkQI5KE7SSFHd//QCvSaoVrc+kJesJY6HnBBE4cDoVXqjYRYTtWDrs/op/TDhYYP11OT4Ht C/suxJpS61DkAE9opwX6WPZRRwJqPjEQyVdSufbrXKagj2hofsnnG+NWIQb211o7/4pIYHfjF pu+E6Bf4aCExUdwab7HjIpQcpP/stn/bJ+8jEShdU1vZloeKG94hEjtYu82ggBz/3B2z0BA+6 NktIPmrq7QjkJTsLUtJASKjsHDmfWViocjQWs7xQWexbB550+n7MqNcXPPtekpfbGenqhoTOQ w6ejBRdwUcIGzHYCbfMNqKaooaUmsLOWl9Fd23T9OjR4lBoPB/T1rEUQm7UKFAp/ag7ogSlnn B6/PN5D26PDgHKwor9IL1MJ3Mv2IuI8yJf8vgzXwBAP0Y0skv4L6Ri/j8LnKJ3DQEL8OJEAyC C33VXueZfNPulGsA3MPQBtofc70KW2DBhD24wnh68tRfsYq/c8OVvw4LCILlDFZda8jDLptWM P61bOTrd1FcHdVosRKt7ECTvnFiqnxbKhMl5QYQ2iDXwiqqKLAsEwAiYW1HXo4Q9NSrbFVp Archived-At: Subject: [precis] RFC 8264 / 8265 Order of rules X-BeenThere: precis@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Preparation and Comparison of Internationalized Strings List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Dec 2017 23:06:12 -0000 Hi all, now that the new RFC 8264 / 8265 are out, I wanted to update my = implementation, which was based on the older RFCs. Unfortunately the order of rules still confuses me. During enforcement and comparison of a string, do I first validate a = string, then apply the rules (as written in = https://tools.ietf.org/html/rfc8265#section-3.3.4), or do I first apply the rules and then validate it? (as in = https://tools.ietf.org/html/rfc8264#section-7) With "validate" I mean checking the "Behavioral rules for determining = whether a code point is valid, allowed under a contextual rule, = disallowed, or unassigned=E2=80=9C. The Appendix says: "Corrected the order of operations for the UsernameCaseMapped profile to = ensure consistency with [RFC8264].=E2=80=9C but I just can=E2=80=99t see that: "MUST prepare then MUST enforce" Concrete example (I asked it here already): If a user wishes to create a username with U+212B (Angstrom sign), = should an application reject it (because it=E2=80=99s disallowed) or = allow it, because enforcement converts the character to a valid code = point first? Thanks! Kind regards, =E2=80=94 Christian= From nobody Mon Dec 4 19:58:30 2017 Return-Path: X-Original-To: precis@ietfa.amsl.com Delivered-To: precis@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E5676124207 for ; Mon, 4 Dec 2017 19:58:28 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.699 X-Spam-Level: X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nUqR5DlJyuuH for ; Mon, 4 Dec 2017 19:58:27 -0800 (PST) Received: from mail-lf0-x232.google.com (mail-lf0-x232.google.com [IPv6:2a00:1450:4010:c07::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id ACB5F120454 for ; Mon, 4 Dec 2017 19:58:26 -0800 (PST) Received: by mail-lf0-x232.google.com with SMTP id i2so21615423lfe.9 for ; Mon, 04 Dec 2017 19:58:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=rpmACn/V74PU3rCxoIN4QL82nQkw6GF4XhXp57O9myk=; b=si1cR4PEENUS7GveON/b7ssjRyB32eKDxOL+AMc9Tug2lF26GQWuc/hROhNPEHd32S A6VjCfL7DCHwTq5jQkgwBQb2+adnZPoqhvePywOnck/RdH3qdg9D9JH+pXfgtTbUkCdG j6fFuwUzjDXDZMjUJC7mptMITOFkT+pYpR5O/9D6SWk6/L0JJxajlYy/Ga1aurXL9JZz nWz31mcJsXMHgfXlVn9OZa/ncksyFhVVGkcksl2ve2RNwmJiFbgo9QmNdEbwSk2K1Sg0 4YkMIQ1RIkICPi0Nkt8+47E/reE2VQHNVw9gI16O2aE33iA6sTF/hnt/vVc0K+tIoDSn rHBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=rpmACn/V74PU3rCxoIN4QL82nQkw6GF4XhXp57O9myk=; b=sogT/xHQtu2fVlsq5rfB0oev2gUuDkJpO26ZZ7bsZ9Wt6fyNnO6nPU4eEis+clSxER G6DS+rw3FiuR4sWsZ+tT3W1XDJKid+C4HXtTHEZZGAuTWmvDwipKA6YP0Uqn7Y/3fYc0 TdJFPeRy0v6QuYqfVdYG5WGxArBX2z/kGBD9rilTBt/4W2O1mvW1FmFEqZejoLLmpChA 0KqN2Klw/8CJ20vYnmNLupVEWzXJ7pf4GnU+GQL6w7i0/G/ls8YxqxyU6f2INKC0acgF BdHzTveMSdOKPxq1qQYB3NkN5JSgj+T/RIGQy9JXGDzlWNPDpY2uDMPUtFwJgXB9z8dY 25Gg== X-Gm-Message-State: AJaThX65cKZLtST+lO23HOiRzoDc+lI8wYwGOP3HC1OwfIkZlL5c28QG 0EZFVSQmCwZZvMBRjqqvnDa3CD525AaU4fVrJJ6oxuhF X-Google-Smtp-Source: AGs4zMYAGF3xCLx/Vb86l/CdYUA0SmnamsFdwy/6+RcRGoqtQgPrhB/UrNShfQlMNO56T8xAnOCLpHKW95Qra/nJ7is= X-Received: by 10.46.91.129 with SMTP id m1mr9651600lje.185.1512446304765; Mon, 04 Dec 2017 19:58:24 -0800 (PST) MIME-Version: 1.0 Received: by 10.179.26.33 with HTTP; Mon, 4 Dec 2017 19:58:24 -0800 (PST) In-Reply-To: <0207F020-32D2-4181-A020-3143BD8E88FE@gmx.de> References: <0207F020-32D2-4181-A020-3143BD8E88FE@gmx.de> From: William Fisher Date: Mon, 4 Dec 2017 20:58:24 -0700 Message-ID: To: Christian Schudt Cc: precis@ietf.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Archived-At: Subject: Re: [precis] RFC 8264 / 8265 Order of rules X-BeenThere: precis@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Preparation and Comparison of Internationalized Strings List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Dec 2017 03:58:29 -0000 On Mon, Dec 4, 2017 at 4:06 PM, Christian Schudt wrote: > If a user wishes to create a username with U+212B (Angstrom sign), should= an application reject it (because it=E2=80=99s disallowed) or allow it, be= cause enforcement converts the character to a valid code point first? I implemented enforcement based on the rules in RFC 8264 section 7. The behavioral rules (IdentifierClass) are applied last. >>> from precis_i18n import get_profile >>> ic =3D get_profile('IdentifierClass') >>> ic.enforce('\u212b') Traceback (most recent call last): ... UnicodeEncodeError: 'IdentifierClass' codec can't encode character '\u212b' in position 0: DISALLOWED/has_compat >>> un =3D get_profile('UsernameCasePreserved') >>> un.enforce('\u212b') '=C3=85' See also: https://www.ietf.org/mail-archive/web/precis/current/msg01225.html You're right that the references to preparation in RFC 8265 (sections 3.3.2, 3.4.2) are at odds with RFC 8264 and the mailing list. -Bill From nobody Wed Dec 6 14:07:56 2017 Return-Path: X-Original-To: precis@ietfa.amsl.com Delivered-To: precis@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8C921127869 for ; Wed, 6 Dec 2017 14:07:55 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.701 X-Spam-Level: X-Spam-Status: No, score=-2.701 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=mozilla.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wZQjH8V3_b2y for ; Wed, 6 Dec 2017 14:07:53 -0800 (PST) Received: from mail-it0-x236.google.com (mail-it0-x236.google.com [IPv6:2607:f8b0:4001:c0b::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 62E0E126BF3 for ; Wed, 6 Dec 2017 14:07:53 -0800 (PST) Received: by mail-it0-x236.google.com with SMTP id 68so9861405ite.4 for ; Wed, 06 Dec 2017 14:07:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mozilla.com; s=google; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to; bh=ldw0OoNg9BRSo+mf78OC8AohEORgA9lotFeaPVm9/XQ=; b=UaItSS0Xd5BVuFLarzU/3x4B03/ZREDyZjA6BBxIADcjzOHRQz7R66XKg1+DXveHkc uKkBaBJOVGSTpQDTVtsZ4sIZCwIUBrO+RrSq3Mi4xmf51X3q+h2CZwqc98Nshk252qCM r7jvwXqUfiQGZVe4MvFULVzrhsE8lvIMv/fMY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to; bh=ldw0OoNg9BRSo+mf78OC8AohEORgA9lotFeaPVm9/XQ=; b=i+TcE2rrriSuHFcBmD18eB2uttG3GL7rFKQyX5ihyFOJiU4HK1Cgj5/bXZszec2kCF pwXrtZEm9IKCZwbG/QtNkrQw1rof/Hn5UTE18oKg7U5Xck2PYlJoxjMr5vav9udCaXfn yWT7SEcSeNABWG2pn8UmmnV8YWPtU1h6BYQy2CTJ+KxFnt10h7v7MpP6mH4zDEEMfsSH +83f9eeeACRD3TyP0xVt+8EF/4AXmuWlGHi4vydT6iLMq9mtVTh74kNa0BzdEViJrE8p ksfbXy0cnOkP58UOyk6mEayyZRKSajdqq9ovLjcMvju6f5woFIaTlMrvGz8OvX6fe44c YxbQ== X-Gm-Message-State: AKGB3mJpAAYTO5nORAiIJpJhKI+q2Fj9PDr+up/V+eHXRLYPt18O4kag 0YJ4S1xbpV06BJISWZJ0PwqoYw== X-Google-Smtp-Source: AGs4zMZcqFurxfdOWZVOUyGh31sTJLXwyGSSMNU+yowhTWZyFkRfwDnKdutjyCvxKRXHJ/03cAlDxA== X-Received: by 10.36.147.3 with SMTP id y3mr26853273itd.82.1512598072655; Wed, 06 Dec 2017 14:07:52 -0800 (PST) Received: from dragon.local ([76.25.3.152]) by smtp.gmail.com with ESMTPSA id g79sm2038281itb.29.2017.12.06.14.07.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 06 Dec 2017 14:07:51 -0800 (PST) To: William Fisher , Christian Schudt Cc: precis@ietf.org References: <0207F020-32D2-4181-A020-3143BD8E88FE@gmx.de> From: Peter Saint-Andre Message-ID: <3c7ba81b-190d-639b-d992-8d0125e8b33e@mozilla.com> Date: Wed, 6 Dec 2017 15:07:39 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="X3ifBbFfKkclo1Q7MPq0NUfRi3na14mLp" Archived-At: Subject: Re: [precis] RFC 8264 / 8265 Order of rules X-BeenThere: precis@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Preparation and Comparison of Internationalized Strings List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Dec 2017 22:07:55 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --X3ifBbFfKkclo1Q7MPq0NUfRi3na14mLp Content-Type: multipart/mixed; boundary="JHHQeoXiAXHNb6oN4poStGpK9tXlvS5tv"; protected-headers="v1" From: Peter Saint-Andre To: William Fisher , Christian Schudt Cc: precis@ietf.org Message-ID: <3c7ba81b-190d-639b-d992-8d0125e8b33e@mozilla.com> Subject: Re: [precis] RFC 8264 / 8265 Order of rules References: <0207F020-32D2-4181-A020-3143BD8E88FE@gmx.de> In-Reply-To: --JHHQeoXiAXHNb6oN4poStGpK9tXlvS5tv Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 12/4/17 8:58 PM, William Fisher wrote: > On Mon, Dec 4, 2017 at 4:06 PM, Christian Schudt > wrote: >> If a user wishes to create a username with U+212B (Angstrom sign), sho= uld an application reject it (because it=E2=80=99s disallowed) or allow i= t, because enforcement converts the character to a valid code point first= ? >=20 > I implemented enforcement based on the rules in RFC 8264 section 7. > The behavioral rules (IdentifierClass) are applied last. >=20 >>>> from precis_i18n import get_profile >>>> ic =3D get_profile('IdentifierClass') >>>> ic.enforce('\u212b') > Traceback (most recent call last): > ... > UnicodeEncodeError: 'IdentifierClass' codec can't encode character > '\u212b' in position 0: DISALLOWED/has_compat >>>> un =3D get_profile('UsernameCasePreserved') >>>> un.enforce('\u212b') > '=C3=85' >=20 > See also: > https://www.ietf.org/mail-archive/web/precis/current/msg01225.html >=20 > You're right that the references to preparation in RFC 8265 (sections > 3.3.2, 3.4.2) are at odds with RFC 8264 and the mailing list. Sigh. I'm sorry that we failed to make things clear and consistent. I agree with Bill that implementations should follow the order of rules in Section 7 of RFC 8264. Let me think about how we can clarify things. That might involve filing an erratum against RFC 8265. Peter --JHHQeoXiAXHNb6oN4poStGpK9tXlvS5tv-- --X3ifBbFfKkclo1Q7MPq0NUfRi3na14mLp Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEENVUj07j078lgnb70ZWGMGH9oFKkFAlooaisACgkQZWGMGH9o FKnAsBAA0pGAEwAxZyxHG6ZzkXnG6ZXla1SmmSq/O+7cXqbeOry1WkpVelt9w0w8 7V9PVFcYdaNBPaXeOM+l9R3S8wybNAvyEuOQuQj6fxgiMTiWxOKqBecpphC5Ekr6 WAoRo4EWWD85N20w1TqUBqq2nrfdS4sVBfy5tNRLGuaFe5hPGcv+tcKY8UNMQETB VqAhqOGkNklKLjxVKjnBBt1EQM/xXBiC26HbLXDb0PZK5+x5Ifx3VjrdyDnK9Y// 7qeG8gG8l7Q0LifLFxwvQttYYH35y3i2MBiHPFwtsptfDKbMJRcskCTblOzpCJxn E+nb4nMKUO/rdo3q8OBr3QXtVBtwS3ayi6F3hLOtCF/BOyA8sOQzYLW8sfHjtJoF g080fljBu25gNOuK39tVkqiDp+gbhFqf9Lq3yKl/Xjwjoxqx+D1skeByWii4dkzx BydLy6vXI15tiPpHgZarCRDWf5t34u5+YRMaVSv/gHI7FkJEArfJaojwPKS/eIOa XJxozVhW9Kt31gxYKQrQcN+N454oKKEXnzt5SfFpQjyZ8Ns+bpCukBTsXVSW6LUk e2nKC5W8fOrVBmfUy1GCJR3t42GF4pjQvKoHLbMLVXDo8q/Mf1lvn5ghMl086mUM 06n+Qd8ICKNi3kBSby0bxjFRI14ymJmK5zH5NhQ6NNirmpPzvvo= =CQhG -----END PGP SIGNATURE----- --X3ifBbFfKkclo1Q7MPq0NUfRi3na14mLp-- From nobody Sat Dec 9 03:39:57 2017 Return-Path: X-Original-To: precis@ietfa.amsl.com Delivered-To: precis@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BC4FE127342 for ; Sat, 9 Dec 2017 03:39:56 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.62 X-Spam-Level: X-Spam-Status: No, score=-2.62 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id D1xGsq09Wmie for ; Sat, 9 Dec 2017 03:39:55 -0800 (PST) Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D7E281201F8 for ; Sat, 9 Dec 2017 03:39:54 -0800 (PST) Received: from christihudtsmbp.fritz.box ([88.77.188.33]) by mail.gmx.com (mrgmx001 [212.227.17.190]) with ESMTPSA (Nemesis) id 0M2L2a-1fFKWx3iwW-00s5HW; Sat, 09 Dec 2017 12:39:51 +0100 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) From: Christian Schudt In-Reply-To: <3c7ba81b-190d-639b-d992-8d0125e8b33e@mozilla.com> Date: Sat, 9 Dec 2017 12:39:55 +0100 Cc: precis@ietf.org Content-Transfer-Encoding: quoted-printable Message-Id: <4388F9AE-82CA-4365-A505-FC2AEA5B6747@gmx.de> References: <0207F020-32D2-4181-A020-3143BD8E88FE@gmx.de> <3c7ba81b-190d-639b-d992-8d0125e8b33e@mozilla.com> To: Peter Saint-Andre X-Mailer: Apple Mail (2.3273) X-Provags-ID: V03:K0:skWDL50YYOKM9ksiDTCCrUByqLGsxcLZ5XW+VEOonhL7683OpvC XfiadN6tHmx5M6gB/HbKM5TsBrMtNJfe933lCRjLqakqJ4jJQfPGtvsiHtPdRKvEFNGelIN 9nSZ6duHYqu98QBz0BGhYDjBKbu3NQEMTFojrG9nQ7jRTOlE/oltHwRzubRk1hf+NjlJiGf ftQlSca+lzrXiaYNUQTkA== X-UI-Out-Filterresults: notjunk:1;V01:K0:6zulAuK+Emg=:E7hiHKSELvKcc/41XIw1Oh 0W4LoXihW/zs1roQnR2zyhiIgJ1GJ9gVmj+WNJ8ZzX1HN1pbW95ml5KDnMRyLJbh5+EFgWFsA RUTaSLhqG0EP0zBSRmMYtcI1mf88+P25yp5vzDHXwxyDADeKNmEU8P/wutaabUvAdP+3duICJ fNiy8q1H8y2alqb1zUSM8IBCeKNak4Xz1uhxuIV+Ef+R1OtH3NiUUF1aKBW0im/99Sc2TBZE8 VO7j6ExDCXUfqprlL4F9QzoRO39Hmqkuin++b5EzJHAoZFxKzbQzTIh0eVssGJy4PKXheZbHs px/sn91ucbjpFwhysG3NbqPV/y/a9K2ukcY7i7XONz0tG8WmzrpOtZcVn56uuRdSju8g+QdEb P3EJ5cRcHZiUEleafCINIIFHZCrFJ1QjKXQr4qYQaraJE4YU44Sly33QTulzVGkK6OPyGZJCt x8dreu41ed/VdSvjpNW7H0kljPT166EuOHUfCwmeKD0iuyPqts3gI5+dr2jKxrOhGeGE5/Iwg ndVIOM2CTzAj55uttNnX5CR63N+whtMDyMiWEccj7XvGRdF1wlCF5OF00CS5Kq5KW7wNN1TvU Ng7Ib2WunPG6E7Y8yZGUfZddEs6cyp95nGrek03c9tm8wDksgm+10SIClDgISWkM9kvAXSGlv Iqr/tJczRuv6ZsYIRBoU4VBcLfKHpSiCuf+W7uKMouTPle/QTJ7ZEqiWXzNrnOpmYhA7yQafh yfZRy/TqMCvmf77+ayKroegwjK7j6kGpWpJyVkPecF1ddxVM0pqyUoVL4xYRKS6QjlOn7j/9w U6IQJHLvAZ/t/tRrkNx00RMYhR/8US4T5bDJMAV7WX+4PsFlQQ= Archived-At: Subject: Re: [precis] RFC 8264 / 8265 Order of rules X-BeenThere: precis@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Preparation and Comparison of Internationalized Strings List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Dec 2017 11:39:57 -0000 >=20 > Sigh. >=20 > I'm sorry that we failed to make things clear and consistent. >=20 > I agree with Bill that implementations should follow the order of = rules > in Section 7 of RFC 8264. >=20 > Let me think about how we can clarify things. That might involve = filing > an erratum against RFC 8265. >=20 > Peter Ok, thanks for clarifying it (again). My implementation will keep on = following the rules in 8264 then. =E2=80=94 Christian= From nobody Sat Dec 9 07:37:04 2017 Return-Path: X-Original-To: precis@ietfa.amsl.com Delivered-To: precis@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0705A1293E8 for ; Sat, 9 Dec 2017 07:37:03 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.62 X-Spam-Level: X-Spam-Status: No, score=-2.62 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vyG3qZa2vYz0 for ; Sat, 9 Dec 2017 07:37:01 -0800 (PST) Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 16F371293E4 for ; Sat, 9 Dec 2017 07:37:00 -0800 (PST) Received: from christihudtsmbp.fritz.box ([88.77.188.33]) by mail.gmx.com (mrgmx003 [212.227.17.190]) with ESMTPSA (Nemesis) id 0LzLJR-1fAcoN2xjl-014SaA for ; Sat, 09 Dec 2017 16:36:58 +0100 From: Christian Schudt Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Message-Id: Date: Sat, 9 Dec 2017 16:37:01 +0100 To: precis@ietf.org X-Mailer: Apple Mail (2.3273) X-Provags-ID: V03:K0:QsAs3YD/fsSOsONMVElWuG7kGinbBnqNGvbZo3pSWfYGWhtfVI3 k8WNuyFsUKEk50vcdi67xlHSwvc78iIdJPN0/DDPzeE04CK7fm7As7V9nwPSjyLWtaD97uZ XpaIPK0d/CWsq9azeNoVgZ/t/vb2F0xdZG06YYlFWSKKhPsvLr4dmtefD6jirbVpD0iEfk2 IyiQ3y3oES3hhNymKOGMA== X-UI-Out-Filterresults: notjunk:1;V01:K0:wpaFypZLOyw=:08EyK6VMBF5WDSiDRreEmi 8FYA3rq0tI/8LDO57Bd1cwG9Tp7bBQLUtki8JliCowrmWKhjz0xiMOGkYYg4ZCjuIkBooMmxA /PFHtHfgKktajZX37HmB7GaI2uANYd+b5I7ONcNLAY2DzyBoUs+Q6d8xBDa0uqhnGikGb0bDm SfUQ1G8zlWp33vYoDXrjOKv6BRhx/0Urv/MElSWKFmlbUaXGaSvCixmEd9llu1aSkE6a1PNmo YdqB1rmtHnOPV7qoB2bbbS178mJgfR7/MRQzizAQyR3x763LGYws99dDqy5b72+sg+wMGJIq+ YpqZUADnfm9amzgH4j1rJLD5lSFYIGtNSSdAbVqcAsTfEqrpyb5uH8sz+uU2lIZv6Z/AHXW57 ybUX6ivyhVCCeWRSdCpcdjbLZfB2p/sBGxxClcZhHMba2877Zas/wbVBl7ZO7cooaCPJIhILp 9gayue8sNU7+J5CSAJwfignlQEJ4+mOhPHFy7WkeHqReLO+agX5i+MsrjH1GdeMXSnRAqw04B EJjZEfKnV7V0a8sgMwklyZIyKrBv0ctW25NZmcR+Z/aCsYNGZMhhH7wcQnPfLEyBecJ9iYxB3 6cUh9YjRl5eBaSUfNrJ5MENNIKwphNnMcjoUAoVvOiuVrQcRAzcBQeAh8WdXGMkHT/U+nw3wh C7z/3D8ER9Y02UcsFdqGgFixbZtUzL1CkRabo31xCeTfPyxilTFmSywY8VURnkv2RSwc4QxDN 4EQNL3WIOI7n1vKfpHxsycLz2lNVY1LxtybxSmq2FNtkq8WXw2jZ6TtggDdOaj5Gf6WFCTNTp nvhd73/GhVpOcCBvSB5JrhACeNxzhUvUFOrq5gaxZKEQa5Z2EE= Archived-At: Subject: [precis] Applying the rules three times to get a stable output string? X-BeenThere: precis@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Preparation and Comparison of Internationalized Strings List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Dec 2017 15:37:03 -0000 Hi, RFC 8264 introduced these new sentences: under certain circumstances, such as when Unicode Normalization Form KC is used, performing Unicode normalization after case mapping can still yield uppercase characters for certain code points Therefore, an implementation SHOULD apply the rules repeatedly until the output string is stable I could imagine these sentences refer to code points of the = =E2=80=9EUnstable=E2=80=9C category, but this category is unused. Are there any concrete code points or input strings which show this = unstable behaviour? I am asking for some test vectors, i.e. an input string, which doesn=E2=80= =99t have the expected output string after the first rule application, = but after the second one. Thanks, =E2=80=94 Christian= From nobody Sat Dec 9 11:34:08 2017 Return-Path: X-Original-To: precis@ietfa.amsl.com Delivered-To: precis@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B63EE127843 for ; Sat, 9 Dec 2017 11:34:06 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.699 X-Spam-Level: X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OBl3o8Ebj1ux for ; Sat, 9 Dec 2017 11:34:05 -0800 (PST) Received: from mail-lf0-x22d.google.com (mail-lf0-x22d.google.com [IPv6:2a00:1450:4010:c07::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CA36912778D for ; Sat, 9 Dec 2017 11:34:04 -0800 (PST) Received: by mail-lf0-x22d.google.com with SMTP id f13so15100651lff.12 for ; Sat, 09 Dec 2017 11:34:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=urkq4C8zqntjEg/DI7oYacJNFIpjc0+7/N57IMkdVOA=; b=WKCZkJO7Fzmt9PEGyduOFjXkkLD9XJXLlqkKv/1ldMjohXKnIilVJI3WJDEw0yPrO5 FxnuNC//tOFkzMHvMTevX/kcH1k1NyjZQKTW9ariVI0+UtkRo4gRGrnOsSCqMerx848b Ys/YFcpWALG6Ed/XMqF72dw87yBNNCFE3sitWiJ7cgtLHqhcz+Rjyu15SB7HcBGCRIQs duRG7zBOwoIqQLKuUTt+PLjsSIhwkCiyXFhEBOEusdC+J2UTkFoH9tInJbp1LWOPXNSr vJ07r497NKCXRB5F8g2A/FOf5avufImz/cvCh63t9lXJ6iIe/vM5iaQ+dw1ifo/b6f7s VJ7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=urkq4C8zqntjEg/DI7oYacJNFIpjc0+7/N57IMkdVOA=; b=h0Tb5JXVy6ZnI21zEf448i0VZU0Ha9Kt3vNe1fBf45yBWEqoNp5WEygDlZOuKi/LN/ 2kYiVgFfXZxx16swVXHKwlqJ+YE2yUt+DOiurxGvOJeP6252BH8W0ru9PtLuVJP7+A7p PGUwJSAZLp7VUec8YbX2YUtYG+2k3sF1vpZEz9v6TQZUn8YQHGaTHeNDGgBmXpF4+t6G +8dz61cUOG/opXG8yOOTOKGwJuosviRSxX816KyQbbcZWhM995KLggqwshX4fdqmYzOH FGeDx399D97OjumUgBv4MlaylCodf7ETNIsmlKwZzGWk4X35+Dw1wSkA2ENO5GyAGZli LuYQ== X-Gm-Message-State: AJaThX6/OIKXSgX+9xCVpOKSF3mRukPugxGFjjHkZUCUnIOyX5nqjOK2 pqEBNocl3RuYmuzRPX4VOytb2tikM9yU9NJW53cuOQ5j X-Google-Smtp-Source: AGs4zMbYesxks6IQZs+1wm/eauelRyQ19w80G6Og5t8mE+HIYHCDACxQ2VQfBWR4bLVaFTMvQOKgIKbAUiUhMEf6j3Q= X-Received: by 10.46.9.14 with SMTP id 14mr17541000ljj.175.1512848042970; Sat, 09 Dec 2017 11:34:02 -0800 (PST) MIME-Version: 1.0 Received: by 10.179.26.33 with HTTP; Sat, 9 Dec 2017 11:34:02 -0800 (PST) In-Reply-To: References: From: William Fisher Date: Sat, 9 Dec 2017 12:34:02 -0700 Message-ID: To: Christian Schudt Cc: precis@ietf.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Archived-At: Subject: Re: [precis] Applying the rules three times to get a stable output string? X-BeenThere: precis@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Preparation and Comparison of Internationalized Strings List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Dec 2017 19:34:07 -0000 Where it makes a difference for NicknameCaseMapped: "\u210c" "\u20a8" Where it makes a difference for Nickname due to spaces: "\u00a8" "\u02dc" On Sat, Dec 9, 2017 at 8:37 AM, Christian Schudt wrote: > Hi, > > RFC 8264 introduced these new sentences: > > under certain circumstances, such as when Unicode > Normalization Form KC is used, performing Unicode normalization after > case mapping can still yield uppercase characters for certain code > points > > Therefore, an implementation SHOULD apply the rules > repeatedly until the output string is stable > > > I could imagine these sentences refer to code points of the =E2=80=9EUnst= able=E2=80=9C category, but this category is unused. > > Are there any concrete code points or input strings which show this unsta= ble behaviour? > I am asking for some test vectors, i.e. an input string, which doesn=E2= =80=99t have the expected output string after the first rule application, b= ut after the second one. > > Thanks, > =E2=80=94 Christian > _______________________________________________ > precis mailing list > precis@ietf.org > https://www.ietf.org/mailman/listinfo/precis From nobody Sat Dec 9 13:27:57 2017 Return-Path: X-Original-To: precis@ietfa.amsl.com Delivered-To: precis@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BBDCB1242EA for ; Sat, 9 Dec 2017 13:27:56 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.899 X-Spam-Level: X-Spam-Status: No, score=-4.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-2.8, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JTnl24P7xEw6 for ; Sat, 9 Dec 2017 13:27:55 -0800 (PST) Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7D4C412009C for ; Sat, 9 Dec 2017 13:27:54 -0800 (PST) Received: from christihudtsmbp.fritz.box ([88.77.188.33]) by mail.gmx.com (mrgmx102 [212.227.17.168]) with ESMTPSA (Nemesis) id 0MKbZD-1eNVPU3ESV-0020jS; Sat, 09 Dec 2017 22:27:51 +0100 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) From: Christian Schudt In-Reply-To: Date: Sat, 9 Dec 2017 22:27:49 +0100 Cc: precis@ietf.org Content-Transfer-Encoding: quoted-printable Message-Id: References: To: William Fisher X-Mailer: Apple Mail (2.3273) X-Provags-ID: V03:K0:okttsnBCRgY7X/nV9CHSAO6NFljArbJKd7xeu+KdDXCHRFk/nL5 Xl4YckVbUTon2hfz84taGPcKh8P25fS5PQcLNkUmW8hNlG7h6Wftf8Nbp+Bfd4UejskHJRf UYDx1YQCNMNxAP2sORxovXKw6VUZv3iYDfxKTNc49NaEY4ATSdQnUocIgVATwbAJkMYvjqI cMmMYSw4NERwWyBsFmYlA== X-UI-Out-Filterresults: notjunk:1;V01:K0:jAMArVPk/1o=:2G8FVLhS4VLLQlmoBt3+LF 2qXMqvaFuQuK12pfOwerFTWGclIE66H2jMnVYOghfeUBa9fNm5x5gRJ9LOP9FWIY2YsnDl9ZQ 0DHsw0ji0hfh7tEGvFNAt5tedC/FK47nIkp8Wp75VeID/sU8vueZwSB19wC7EVEMfTWK5Ki4q 0N+qx6F0IOXmetbwaxufBCpvFk63dERGxZikiP3k+aYR1MqyDhWLt2F8gaxAOnKGkJLdba8U/ /hYI6zgldFKL41EokhssSp9spc1GJHhOatwItXEXUNTNPQhtGB+fIpIWN3Tjba2ubYjSMVaV6 QiYkwBKFPkfEI8kiklMxDuJx0YNH/oBmU3zEVgIgU5LTroxSvo0w/wH6dvgM1HvcJKyR6IbXy BLFUc6K2cAp6z+VW8ck9I07BUs3yay9bhs2W2Ai4BH1tncpv7PefWsLsHF3Qa1heCVKtK7QRW KGXRjixrnCf+Vslx+HVnan6hODRa54PFj9Z+BnqdmnAktUorH76+mavPbzv0ckixHGrEBbjqR ayPyk0qFoKjX1NUF4rf3lBv7+YEeU/XA07+IP0SL/pTlwPixTYynPV97qnSTBjK0w5SdiJlUV BPA22nY60/zmKKEj6SiZsqM/1KXXqBqd4Woz+n5ZCdHPlxqFbBFYwjl3UC0Q7aM2so5xd+Z4T 6ZA29N8g1xEprQn7rhGLiEjltpYW7AGdIVVvQnZPpkKiqZuzBkAlAUJMKBuUY5KC87e/45xs6 JTDyQ6qS/VicifLja428ODVCoD6cw+C8Q5/mbOuDCf4fNgoVvWKWZEy0eNHGW+8j9ZVq/Cekh I+OaT6hsCMNjMcS0zYaZmvAWqXrpgQat5eqS66dOf5hAq39mgk= Archived-At: Subject: Re: [precis] Applying the rules three times to get a stable output string? X-BeenThere: precis@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Preparation and Comparison of Internationalized Strings List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Dec 2017 21:27:57 -0000 Great, thanks! These code points revealed some bugs :-). They should = have been included in the Examples. Are there any known code points for the IdentifierClass / Usernames as = well? Seems like all these code points are disallowed anyway. If not, implementations could save 1-2 iterations and only apply the = =E2=80=9E3-times=E2=80=9C-rule for FreeformClass. > Am 09.12.2017 um 20:34 schrieb William Fisher = : >=20 > Where it makes a difference for NicknameCaseMapped: >=20 > "\u210c" > "\u20a8" >=20 > Where it makes a difference for Nickname due to spaces: >=20 > "\u00a8" > "\u02dc" >=20 >=20 > On Sat, Dec 9, 2017 at 8:37 AM, Christian Schudt > wrote: >> Hi, >>=20 >> RFC 8264 introduced these new sentences: >>=20 >> under certain circumstances, such as when Unicode >> Normalization Form KC is used, performing Unicode normalization = after >> case mapping can still yield uppercase characters for certain code >> points >>=20 >> Therefore, an implementation SHOULD apply the rules >> repeatedly until the output string is stable >>=20 >>=20 >> I could imagine these sentences refer to code points of the = =E2=80=9EUnstable=E2=80=9C category, but this category is unused. >>=20 >> Are there any concrete code points or input strings which show this = unstable behaviour? >> I am asking for some test vectors, i.e. an input string, which = doesn=E2=80=99t have the expected output string after the first rule = application, but after the second one. >>=20 >> Thanks, >> =E2=80=94 Christian >> _______________________________________________ >> precis mailing list >> precis@ietf.org >> https://www.ietf.org/mailman/listinfo/precis From nobody Sat Dec 9 14:09:26 2017 Return-Path: X-Original-To: precis@ietfa.amsl.com Delivered-To: precis@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C580E120454 for ; Sat, 9 Dec 2017 14:09:24 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.699 X-Spam-Level: X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ET8-FR8-qpdg for ; Sat, 9 Dec 2017 14:09:22 -0800 (PST) Received: from mail-lf0-x22e.google.com (mail-lf0-x22e.google.com [IPv6:2a00:1450:4010:c07::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 98979120227 for ; Sat, 9 Dec 2017 14:09:22 -0800 (PST) Received: by mail-lf0-x22e.google.com with SMTP id 94so15316044lfy.10 for ; Sat, 09 Dec 2017 14:09:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=aSjHrEm2I7GDFrc/77BqPwNcqjKHi4nGELrNu2DTCXw=; b=Tu5aiqg/39mNJdp5raV0bq2gJ8JvOTLm456Mh5QrCx816/jc0+4qItpGAQ1kQCwg7U eL0wCdmOmhOxw6oSy9NSyHmys44js2FDxjObk8fWAH5UxilAXO99ceaERuVJpEW9UIhu SnUk0kOyeiJuFzBwn1RUEAkzrHTuZmGwHeP1HMCCtE9RvY0ckrxP1DM9MDHO+IfYIOGT HNtyOZfyS9TLNa2nEXnxga0TaQmHyw/1qq/BXXomURypoQtZhEaoIOYj1LcSOMgj7bs8 ZKSYgHcwtXXcvUnNlnJPIVxqKw5/YctZZgiuspF/Y8hokihid1r+4ryYpqCR0dJ5rB6+ bHKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=aSjHrEm2I7GDFrc/77BqPwNcqjKHi4nGELrNu2DTCXw=; b=oKq+hQEIkmsNl4Z1dHNYeLzLVauUZxSGs9q3jbT2Tp3c6MoB2LKvbi7UiyXwT7soq0 y0jn2v6gEW80HeLLFSUAWqvQ9xJqRDtXmMkkOOo1sXStgZ7FPGw5xQQh/KPDUvcWfZxM R9CGwUdA76cWh91dqu3+nlGeN87C4tZsZR0YAWlxScu9EfkPlx8JIB+fbjNOwW4Nrqt7 edm/yx7rpIFnnk9meyFbYg2T/Ou7MP1pq81Dvkz0dzeqCtnuA9X+dE1FwshRgSf2YU3U Za+nThaLfytwoRCzUDffWgeKvKVqDglZI0+gaH1VCMjnaJTR1KpPpcxYno/flwKPbIjm by7w== X-Gm-Message-State: AJaThX5fVOK9TZ0kTZaVZizUWpx33JMChTplWb3SA6GiDcBlt3XaizfF VD5DB/Ke8npmiYRGabKOPFDkJ2QfiXIU53QV35c= X-Google-Smtp-Source: AGs4zMb3wsguIoGA731azEFvgReGcmb6um3EGYX4VZE0P4940WOcHs/XstQniHm+4YlCcclkM8ryNqIiAAErOOOkN1w= X-Received: by 10.46.20.5 with SMTP id u5mr19384781ljd.9.1512857360629; Sat, 09 Dec 2017 14:09:20 -0800 (PST) MIME-Version: 1.0 Received: by 10.179.26.33 with HTTP; Sat, 9 Dec 2017 14:09:20 -0800 (PST) In-Reply-To: References: From: William Fisher Date: Sat, 9 Dec 2017 15:09:20 -0700 Message-ID: To: Christian Schudt Cc: precis@ietf.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Archived-At: Subject: Re: [precis] Applying the rules three times to get a stable output string? X-BeenThere: precis@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Preparation and Comparison of Internationalized Strings List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Dec 2017 22:09:25 -0000 I did not come across any code points where IdentifierClass/Usernames required multiple passes to make the result idempotent. Only the Nickname profile is affected, due to the interaction between NFKC and the case/space rules. My implementation applies an extra iteration for the Nickname profile. The other profiles verify that the result is idempotent and raise a DISALLOWED/not_idempotent error if this is violated. I do not believe there are legal inputs for Usernames which violate the idempotency requirement, so this is purely defensive. On Sat, Dec 9, 2017 at 2:27 PM, Christian Schudt wrote: > Great, thanks! These code points revealed some bugs :-). They should have= been included in the Examples. > > Are there any known code points for the IdentifierClass / Usernames as we= ll? > Seems like all these code points are disallowed anyway. > > If not, implementations could save 1-2 iterations and only apply the =E2= =80=9E3-times=E2=80=9C-rule for FreeformClass. > > > >> Am 09.12.2017 um 20:34 schrieb William Fisher : >> >> Where it makes a difference for NicknameCaseMapped: >> >> "\u210c" >> "\u20a8" >> >> Where it makes a difference for Nickname due to spaces: >> >> "\u00a8" >> "\u02dc" >> >> >> On Sat, Dec 9, 2017 at 8:37 AM, Christian Schudt >> wrote: >>> Hi, >>> >>> RFC 8264 introduced these new sentences: >>> >>> under certain circumstances, such as when Unicode >>> Normalization Form KC is used, performing Unicode normalization after >>> case mapping can still yield uppercase characters for certain code >>> points >>> >>> Therefore, an implementation SHOULD apply the rules >>> repeatedly until the output string is stable >>> >>> >>> I could imagine these sentences refer to code points of the =E2=80=9EUn= stable=E2=80=9C category, but this category is unused. >>> >>> Are there any concrete code points or input strings which show this uns= table behaviour? >>> I am asking for some test vectors, i.e. an input string, which doesn=E2= =80=99t have the expected output string after the first rule application, b= ut after the second one. >>> >>> Thanks, >>> =E2=80=94 Christian >>> _______________________________________________ >>> precis mailing list >>> precis@ietf.org >>> https://www.ietf.org/mailman/listinfo/precis > From nobody Sat Dec 9 14:41:18 2017 Return-Path: X-Original-To: precis@ietfa.amsl.com Delivered-To: precis@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 70750126CC7 for ; Sat, 9 Dec 2017 14:41:17 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.619 X-Spam-Level: X-Spam-Status: No, score=-2.619 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FbYGhDpNsWPm for ; Sat, 9 Dec 2017 14:41:15 -0800 (PST) Received: from mout.gmx.net (mout.gmx.net [212.227.15.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id F0DEE1243F6 for ; Sat, 9 Dec 2017 14:41:14 -0800 (PST) Received: from christihudtsmbp.fritz.box ([88.77.188.33]) by mail.gmx.com (mrgmx001 [212.227.17.190]) with ESMTPSA (Nemesis) id 0LfkUs-1er4pR3NYw-00pNAB; Sat, 09 Dec 2017 23:41:11 +0100 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) From: Christian Schudt In-Reply-To: Date: Sat, 9 Dec 2017 23:41:11 +0100 Cc: precis@ietf.org Content-Transfer-Encoding: quoted-printable Message-Id: <81CE0A33-A6B8-464B-80E4-F2F94F44CB28@gmx.de> References: To: William Fisher X-Mailer: Apple Mail (2.3273) X-Provags-ID: V03:K0:IOr6IGvmUvCMQmxrfihHda9mSVoeq3n3PCC0vreIHSvrtwLLPZK Ltf6QDXL6kPcbfXABlHaxz8BwJw0pelLAXy/+f4DHF7ecVugb0uRZ3k9Dk7s04WC6sNSlcs KjMvTbGhxiJ/zmhUwyvDVy5xLEkFMYiqr60G+ci4eCrdiJH1dvTyPQ5lFF4dFNZyL5TBLFG TV1CDnDMyvj34Fk6wBkmA== X-UI-Out-Filterresults: notjunk:1;V01:K0:WXa+EVfdWrA=:tSvmIhzH22eDv9hoqOgMjs 3befrd6RlQmxCpQB8ICdGWDokvcKKbOTF6RWsH9ODYA/GulbIi9JV4cFxJFpXZ6OUdTWG7u70 Y04D1/QG1FcUakLwdLodXq9y4BidCdRFrWj9m39w3hYt3qMIR2lzUD+byhK/4MOat2kevHZTF Rb6fsRjsrZDvvwGcf7H7a7aaAGPTXLBCIY774XmiiS1J4lZzPUoG4SFPQugKYvqX7uIaAw+o0 15oTo+Z/IfbarKfnncSUMCiO1L1AwWO9ceYe4nB8nVJ72y7plebkPR2B4197DJDDXLjXJFTPc Od+s1dXVe+FeXK2X37ZR/6OEL9tCXkZ5Ejf5X59HvpiaLt1Pex6CpPn50jM4mEB0II0N92tnM GhoBPpLQX9ZSHIg76cfs5dxXVF4St4Z18RP24bu8J3Na58L4jS2EgtpL74jj4RHm7kPMShEhm bU6UWPvgBXZUaAIsh543xVRGq15Gctij5H3hlEI+KnSb7IJmI1JcsXaA7sH53xuNvTVae0oCY wBa2kgrOEEVmTsNrLQ8UKLRewb1JZ1jfSaAmsqA+ps2TLj+89eLX8zqkPKWse0YiammgW8kk+ PVAvsukHnYC4V3LAUqYCkkK3y+7+V/w6nVAX4XMgBvJPcwtu5tUxrjsdLK/eu5pbr2XXMh29f RTqYK5G2ov/14X++zHetk/XQOrsPOyMDHFUdjiE2ZU0gkQMoDTAeuNOC1vrQihiSXBVUWQido tYNm5HKVwP+gxPJEgah2rFp0V868VAhzxeWMemjfp5eZuJF3EZC2XD3yRRQ2aT98ZfbTq1mlz ijuoCS65KDquqVuo4TQJ4WJHdEh2uaL4olQFnj+iJf0cRw3POo= Archived-At: Subject: Re: [precis] Applying the rules three times to get a stable output string? X-BeenThere: precis@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Preparation and Comparison of Internationalized Strings List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Dec 2017 22:41:17 -0000 I just wrote a Java test, which checks the idempotency for all code = points (0 - 0X10FFFF) for all 4 profiles (opaque, username, username = preserved, nickname). The result is as you suspected: Only the Nickname profile requires additional application of the rules = in order to stabilize the output string. But there=E2=80=99s is no code point which requires more than one = iteration. The other 3 profiles are idempotent on the first run. I tested with Java 8, which uses Unicode 6.2.0. If the tests fail on Java 9 (Unicode 8.0), I=E2=80=99ll report back. =E2=80=94 Christian > Am 09.12.2017 um 23:09 schrieb William Fisher = : >=20 > I did not come across any code points where IdentifierClass/Usernames > required multiple passes to make the result idempotent. Only the > Nickname profile is affected, due to the interaction between NFKC and > the case/space rules. >=20 > My implementation applies an extra iteration for the Nickname profile. > The other profiles verify that the result is idempotent and raise a > DISALLOWED/not_idempotent error if this is violated. I do not believe > there are legal inputs for Usernames which violate the idempotency > requirement, so this is purely defensive. >=20 >=20 > On Sat, Dec 9, 2017 at 2:27 PM, Christian Schudt > wrote: >> Great, thanks! These code points revealed some bugs :-). They should = have been included in the Examples. >>=20 >> Are there any known code points for the IdentifierClass / Usernames = as well? >> Seems like all these code points are disallowed anyway. >>=20 >> If not, implementations could save 1-2 iterations and only apply the = =E2=80=9E3-times=E2=80=9C-rule for FreeformClass. >>=20 >>=20 >>=20 >>> Am 09.12.2017 um 20:34 schrieb William Fisher = : >>>=20 >>> Where it makes a difference for NicknameCaseMapped: >>>=20 >>> "\u210c" >>> "\u20a8" >>>=20 >>> Where it makes a difference for Nickname due to spaces: >>>=20 >>> "\u00a8" >>> "\u02dc" >>>=20 >>>=20 >>> On Sat, Dec 9, 2017 at 8:37 AM, Christian Schudt >>> wrote: >>>> Hi, >>>>=20 >>>> RFC 8264 introduced these new sentences: >>>>=20 >>>> under certain circumstances, such as when Unicode >>>> Normalization Form KC is used, performing Unicode normalization = after >>>> case mapping can still yield uppercase characters for certain code >>>> points >>>>=20 >>>> Therefore, an implementation SHOULD apply the rules >>>> repeatedly until the output string is stable >>>>=20 >>>>=20 >>>> I could imagine these sentences refer to code points of the = =E2=80=9EUnstable=E2=80=9C category, but this category is unused. >>>>=20 >>>> Are there any concrete code points or input strings which show this = unstable behaviour? >>>> I am asking for some test vectors, i.e. an input string, which = doesn=E2=80=99t have the expected output string after the first rule = application, but after the second one. >>>>=20 >>>> Thanks, >>>> =E2=80=94 Christian >>>> _______________________________________________ >>>> precis mailing list >>>> precis@ietf.org >>>> https://www.ietf.org/mailman/listinfo/precis >>=20 From nobody Mon Dec 11 03:42:41 2017 Return-Path: X-Original-To: precis@ietfa.amsl.com Delivered-To: precis@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 263E8126B6D for ; Mon, 11 Dec 2017 03:42:40 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.62 X-Spam-Level: X-Spam-Status: No, score=-2.62 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Zo1xVWgrMKIp for ; Mon, 11 Dec 2017 03:42:38 -0800 (PST) Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1BA2F1201FA for ; Mon, 11 Dec 2017 03:42:37 -0800 (PST) Received: from christihudtsmbp.fritz.box ([94.221.106.221]) by mail.gmx.com (mrgmx101 [212.227.17.168]) with ESMTPSA (Nemesis) id 0M97Nh-1eCUMw3nlV-00CUDR for ; Mon, 11 Dec 2017 12:42:36 +0100 From: Christian Schudt Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Message-Id: <18C096A5-3EAC-4F93-86BE-725758E22E3D@gmx.de> Date: Mon, 11 Dec 2017 12:42:34 +0100 To: precis@ietf.org X-Mailer: Apple Mail (2.3273) X-Provags-ID: V03:K0:efF5D+Ys65o+K2tsQzcOLlgG0/1ifhGKaURtHnVGEF8ZkoNsrb5 4UXbqhyr+ZjZR1ydFBsHvh9jPrEcY9syaLJ3ak2YT5hSgVL6ltWFOzntlYIwkgkhxFX3x4i ZQAyuYjXNx4wArMtFOtB0vc5A4DagUsEz7/BNlde/N2RqWfNnIu3aLcoN8NRjWPK2LWgOKE SskqD+qlJZYmyedsoAUJQ== X-UI-Out-Filterresults: notjunk:1;V01:K0:D/xJ8+EurA0=:OQL9NHF51rSU0rhHN9/HRz 0xAOFlFvCbC+jluKC/JeHKYoI8CMWZ9y/4+8DIvRVClRvSt75lEkgn6xLjF0QKHm5gKlYWW1D 73CvLCAB98DTYKwrenLL61eaP9kc7xo4e3F7SIDqWjEa18gZLvEHXLIIXI9qaIc99zviNPfoV VxGyHA5Ls22TAZ09GNxAhk9+UGsmJq+XStNOKGQRO7kdMR6qiMe8i1gI7qDhVq4ALWwk3ms/i 3dF/KrnFrMWxrQJVITE2un7/pw4O5xKacAUqaOxxSGnfX5C/NWisyz2tykm7foHenUWo6XdZv SbhBPOTZswe9S2hrfSF+flHd2xP1YkFV1nyyXX3JR8P+SHW/ZKsQsmKYKeSUKs8nJaNo4gXwd kWK9wJr89w4DIB3koF+Ljb1mwOlP6B8Ec5FbGKKFniBuyqzkKQFb2a8DAKHy+pFABDWf3kXzM W/sOv5CoyB31ZNz0QOb7Sri2TZWLVSbDI2SJrXquch9nzlbtXhc3gJnLfr+1Hcele61Z5UVER Ae2BQWHrPKKvXHpnPOYrgCGBDspEcRm71PyuuUc+MAB+1snkcNcZONDL5rw0wohSa+CK8W98T zNPAc270BBettxpp4cMrTh/mtaH+UPNRZR8S1Lk4dmib7ho2eE7JARwkyPEl6amFWlCla4VaL 33jjm85iDphrTOCfBl1ARozK6WffimkOCmUpDcV7M9k0bkoN3Yvr244RwjC+oCj3CbrOP7TMP nonEBioQd+gF+OcuhXy4rXRKd6vLFI/YlqbLqP1MUoGFa+eeji5QNSA8SPK/tCYNl91Sip4ej Ci/VItx0eg8AMR6SiUJnL+u60lpIwGEOMw5Vjx9Ep02wUTVazw= Archived-At: Subject: [precis] Updated PRECIS Java library to RFC 8264, 8265 and 8266 X-BeenThere: precis@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Preparation and Comparison of Internationalized Strings List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Dec 2017 11:42:40 -0000 Hi, I=E2=80=99ve updated my PRECIS Java library to the new RFCs 8264, 8265 = and 8266. You can find it on Maven Central: rocks.xmpp precis 1.0.0 https://bitbucket.org/sco0ter/precis Changes: - use toLowerCase() for case mapping - ensure that methods are idempotent - added toComparableString() method, in order be able to put comparable = nicknames into a HashMap. - prepare build for Java 9 (but it=E2=80=99s still targeted and built = for Java 8) - updated Maven plugins - added more unit tests - ensure/increase code quality with Google ErrorProne Kind regards, =E2=80=94 Christian=