Subject: Re: [EAI] I-D Action: draft-ietf-eai-5738bis-01.txt

From: Joseph Yee <[email-masked]>
Date: Mon, 1 Aug 2011 10:56:08 -0400

Hi Chris, Alexey and all,

joseph ENABLE HAT=IMPELMENTOR

On 2011-07-31, at 9:28 PM, Alexey Melnikov wrote:

> Hi Chris,
> 
> Chris Newman wrote:
>> --On July 11, 2011 3:04:44 -0700 [email-masked] wrote:
>>>    Title           : IMAP Support for UTF-8
>>>    Author(s)       : Pete Resnick
>>>                          Chris Newman
>>>                          Sean Shen
>>>    Filename        : draft-ietf-eai-5738bis-01.txt
>>>    Pages           : 16
>>>    Date            : 2011-07-11
>> I re-read this and tried putting on my somewhat dusty "IMAP implementer hat" rather than my "protocol designer" hat. This unfortunately resulted in a lot of substantive issues :-(
>> 
>> There is one change I feel would be most helpful to make. I have learned that LIST extended is quite difficult to implement in a distributed server deployment and I believe that requirement will be a barrier to deployment of this extension. I suggest the document be changed so that the UTF8 LIST selection and return options can be used without requiring full implementation of the LIST extended extension. If the working group has rough consensus to go this direction, then I can provide the ABNF and text so that capability can be provided with or without the LIST extended extension.
>> 
>> This benefits servers because servers that make simpler implementation choices (such as UTF-8-only mailboxes) would no longer be forced to implement the entire LIST extended structure to allow interoperability. It also removes a "conditional" from the client (the code the client uses presently has to be different based on the presence of the LIST-EXTENDED capability).
> I might be agreeable to this change, but I would like to see a bit more details before unconditionally supporting it. Can you suggest some text/show an example on how this would look like?

Same to Alexey, would love to see some text and example.

>> I then subsequently asked myself what features could be removed from this document while keeping the document sufficiently complete. There are three options here:
>> 
>> 1. Remove the "UTF8=USER" capability. Since there's no password change or account creation command in IMAP, this information is not strictly necessary for clients. UTF-8 usernames and passwords will either work or not based on the identity subsystem.
> I tend to agree.
>> The client doesn't have to know, so unless there's a client that really needs to know for some reason, this should probably be removed just to make the protocol less cluttered. The same could be done for POP3, FYI.
> Base IMAP (RFC 3501) doesn't require support for non-ASCII usernames. In practice I suspect that many server implementations are ignoring the "ASCII-only" restriction and that UTF-8 usernames are just going to work.
> 
> I am undecided on whether this capability should be removed or not.  I think it is basically advertising whether the identity subsystem supports non-ASCII usernames/passwords or not, so adding advertisement of this capability if a particular identity subsystem is compliant should be easy.
> If we are going to remove this capability, I would like to at least have a SHOULD about supporting UTF-8 username/passwords. As email addresses are frequently used as IMAP usernames, this can even be a MUST.

I would assume "UTF8=USER" meant to notify client about SASL on username & password rather than support of UTF8 on username & password.

If it's about SASL, I think we are safe to remove the tag, with client needs to check the SASL tag.  If it's about supporting UTF8 in username & password, then I think it's ok to remove "UTF8=USER", but only stating "UTF8=ACCEPT" MUST imply the support of UTF8 to username & password.

>> 2. Remove the up-conversion requirements. The goal was to make it simpler for clients by allowing clients without 2047/2231 to work sooner. But this might be an additional barrier to server implementers, especially now that down-conversion has become more prominent than in the previous model. I'm thinking the deployment concern may be more important now than the hope of a protocol-designer to get rid of the ugly encodings sooner.
> +1.

+1.

>> 3. Remove the UTF8=APPEND capability. This is a direct server-implementer vs. client-implementer tradeoff. Not having that capability and requiring servers to support this in any mailbox that supports SELECT/EXAMINE UTF8 makes things simpler and more predictable for clients, but at the expense of the server implementer. In hindsight, I think this is one where the server implementers should suffer to make things easier for the clients and to make the protocol look simpler.
> Just to clarify: are you saying that this functionality should always be supported by compliant servers? If yes, do you also propose to remove the UTF-8 signaling in the APPEND command?

I think this is good, but haven't think deep enough on this, including the UTF8  signalling in APPEND command.  Haven't thought completely yet where we may need the signalling.

-Joseph

>> And yes, you can be annoyed at me proposing simplifications/changes that remove things I put into the original document several years ago ;-) But I must admit that working in the more constrained post-bubble environment has given me a greater respect and attention to the nuances of simplicity.
>> 
>> Note that I did not see any technical errors in the current document, so if the WG decided to ignore my guilt-in-hindsight-for-unnecessary-complexity, I would not object to publishing the current document. But I do think it could be made more deployable and that is a significant concern. I'd really appreciate if others spoke up on this topic.
> 
> 
> _______________________________________________
> IMA mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ima

---

From: Chris Newman <[email-masked]>
Date: Mon, 01 Aug 2011 11:17:39 -0700

--On July 31, 2011 21:28:18 -0400 Alexey Melnikov 
<[email-masked]> wrote:
> I might be agreeable to this change, but I would like to see a bit more
> details before unconditionally supporting it. Can you suggest some
> text/show an example on how this would look like?

Basically, I'd copy a subset of the LIST EXTENDED support into the draft 
and say that subset is activated by UTF8=ACCEPT even if LIST EXTENDED is 
not advertised.

That subset would not include multiple list patterns, remote, childinfo, or 
recursivematch. I think the subset would have to include SUBSCRIBED to 
avoid mucking with LSUB.

So the result would be syntactically upwards-compatible with list extended 
(the client could send commands within the subset regardless of whether 
LIST-EXTENDED was advertised), and would support the SUBSCRIBED, UTF8 and 
UTF8ONLY selection options and the UTF8 and SUBSCRIBED return options.

>> 3. Remove the UTF8=APPEND capability. This is a direct
>> server-implementer vs. client-implementer tradeoff. Not having that
>> capability and requiring servers to support this in any mailbox that
>> supports SELECT/EXAMINE UTF8 makes things simpler and more predictable
>> for clients, but at the expense of the server implementer. In
>> hindsight, I think this is one where the server implementers should
>> suffer to make things easier for the clients and to make the protocol
>> look simpler.
> Just to clarify: are you saying that this functionality should always be
> supported by compliant servers? If yes, do you also propose to remove the
> UTF-8 signaling in the APPEND command?

I think the answer should be YES and NO. That is UTF8=ACCEPT implies what 
is presently implied by UTF8=APPEND, but no changes to the document 
otherwise.

I think the signaling in the APPEND command needs to be left for symmetry 
with the signaling we have in SMTP. But if someone wants to make the 
contrary argument, that would interest me.

		- Chris

---

From: Chris Newman <[email-masked]>
Date: Mon, 01 Aug 2011 11:51:19 -0700

--On August 1, 2011 10:56:08 -0400 Joseph Yee <[email-masked]> wrote:
> I would assume "UTF8=USER" meant to notify client about SASL on username
> & password rather than support of UTF8 on username & password.
>
> If it's about SASL, I think we are safe to remove the tag, with client
> needs to check the SASL tag.  If it's about supporting UTF8 in username &
> password, then I think it's ok to remove "UTF8=USER", but only stating
> "UTF8=ACCEPT" MUST imply the support of UTF8 to username & password.

The IMAP AUTHENTICATE command already supports UTF-8 if the SASL mechanism 
selected does. The PLAIN mechanism supports UTF-8.

So UTF8=USER was only about the IMAP LOGIN command. So I'd replace section 
5 with something like:

5.  LOGIN Command

   If the "UTF8=ACCEPT" capability is advertised, that indicates the
   server understands UTF-8 user names and passwords in the LOGIN
   command. This is not a guarantee that they underlying identity
   system will allow the creation of accounts with UTF-8 user names
   and/or passwords. However, if the identity system does allow such
   accounts, then the server MUST apply SASLprep [RFC4013] to both
   arguments of the LOGIN command. The server MUST reject UTF-8 that
   fails to comply with the formal syntax in RFC 3629 [RFC3629] or
   if it encounters Unicode characters disallowed by SASLprep.

		- Chris

---

From: Joseph Yee <[email-masked]>
Date: Wed, 3 Aug 2011 15:23:38 -0400

On 2011-08-01, at 2:51 PM, Chris Newman wrote:

> The IMAP AUTHENTICATE command already supports UTF-8 if the SASL mechanism selected does. The PLAIN mechanism supports UTF-8.
> 
> So UTF8=USER was only about the IMAP LOGIN command. So I'd replace section 5 with something like:
> 
> 5.  LOGIN Command
> 

as individual, this works with a minor suggestion to the end
"disallowed by SASLprep in Section 2.3 of RFC4013 [RFC4013]."

-Joseph