Subject: [EAI] Correct LDAP matching rule for EAI addresses?

From: [email-masked]
Date: Wed, 05 Feb 2014 09:39:07 -0800 (PST)

So what's the matching rule/syntax to use for EAI addresses stored in LDAP?
AFAICT there isn't an appropriate one: At a minimum you want something that
does ASCII case mapping while allowing the full UTF-8 repertoire. (Better still
would be one that does the right sorts of normalization, but that's probably
too much to ask.)

I note in passing that Sieve's i;ascii-casemap comparator does more or less
the right thing. The thing that's missing in Sieve is a way to decode
encoded domains as part of the address test.

				Ned

---

From: John C Klensin <[email-masked]>
Date: Wed, 05 Feb 2014 13:21:19 -0500

--On Wednesday, February 05, 2014 09:39 -0800
[email-masked] wrote:

> So what's the matching rule/syntax to use for EAI addresses
> stored in LDAP? AFAICT there isn't an appropriate one: At a
> minimum you want something that does ASCII case mapping while
> allowing the full UTF-8 repertoire. (Better still would be one
> that does the right sorts of normalization, but that's probably
> too much to ask.)

Ned, the problem is that the EAI WG, after considering a number
of alternatives and the forest of rat holes to which they led,
decided on a very strict reading of the "no one but (or before)
the delivery server can determine whether two addresses match"
rule that is implicit in RFC 5321 and its predecessors.  The net
result is that, for local parts, even
   NFC(local-part-in-address) ? NFC(local-part-in-database)
cannot be guaranteed to represent the same destination address
and things are sort of downhill from there.  Of course, if one
were strictly following 5321, then 
   toLowerCase(local-part-in-address) ?
toLowerCase(local-part-in-database)
can't be guaranteed to represent the same address either.

The domain part is easy by comparison: each non-ASCII label
should be converted to A-label form (and rejected as nonsense if
the conversion fails), lower-cased if needed, and then compared
on a bitstring basis.

We probably agree on what ought to happen, but, if we want
consistency, a number of statements to the effect that "if you
want this to work in LDAP or Sieve, then the delivery server
better be configured to treat these as equivalent" probably need
to be scattered in various places.

> I note in passing that Sieve's i;ascii-casemap comparator does
> more or less the right thing. The thing that's missing in
> Sieve is a way to decode encoded domains as part of the
> address test.

Ack.

best,
   john

---

From: [email-masked]
Date: Wed, 05 Feb 2014 11:41:28 -0800 (PST)

> > that does the right sorts of normalization, but that's probably

The good news is that caseIgnoreMatch appears to be sufficient for
this purpose. It doesn't handle spaces correctly, but anyone that puts
sequences of spaces into an address and expects it to work is losing anyway.

> rule that is implicit in RFC 5321 and its predecessors. 

Well, that's actually total nonsese, as use-cases abound where addresses have
to be compared during message transport. Various antispam measures are only the
most obvious example, there are many, many others.

This may be another one of those head-in-the-purity-sand situations the IETF
sometimes gets itself into, like the notion that messages never undergo
transformations in transit (despite the fact that we have standards that
specifically sanction such activity).

But even if this was a reasonable rule, it's irrelevant, because it is
*precisely* the delivery server that's mostly commonly looking up addresses in
LDAP. So the rule, even if valid, in no way lets us skirt the problem.

> The net result is that, for local parts, even
> and things are sort of downhill from there.

I don't actually care about that. My view of the local-part-normalization punt
EAI ended up with is that it forces clients and provisioning systems to deal
with the local part normalization issue.

My issue is that for better or worse the expectation is that the ASCII subset
of characters in addresses needs to be compared case-insensitively. So you have
to use a matching rule that handles that while not restricting the syntax to
IA5.

A normalizing matching rule would be better, but getting such a thing designed
and implemented is beyond my purview.

> Of course, if one

Yep. But the reality on the ground is that nobody wants that. We actually have
support for that in our product, but to the best of my knowledge we've only
ever had one customer who ever used it.

> the conversion fails), lower-cased if needed, and then compared

Sorry, that's not going work. People are going to want to enter pure utf-8
addresses into the directory. Asking provivisioning systes to enter addresses
with the local part in utf-8 and the domain part in A-label is a pretty
clearly a losing proposition.

The only approach that makes sense is properly normalized utf-8 addresses in
the directory. So you actually want the opposite: Convert all A-labels to utf-8
prior to lookup, and normalize any utf-8 already present in the domain part,
then search. Coding this is tedious but straightforward.

Domains stored in the directory are another matter. An argument can be for
using A-levels there. But since there's no encoding defined for local parts,
the simplest thing is to just use normalized utf-8 again.

				Ned