Subject: Re: [Jmap] Review of draft-ietf-jmap-mail-05

From: Neil Jenkins <[email-masked]>
Date: Sat, 09 Jun 2018 02:45:32 -0400

On Thu, 31 May 2018, at 4:25 PM, Neil Jhaveri wrote:
>> Types signatures are given for all JSON objects in this document…
> Since this document is presumably dependent upon JMAP Core, can we reference it and avoid repeating the same 4 conventions?  Something like "Type signatures are given for all JSON objects in this document. The conventions follow those in Section 1 of [RFC XXXX]”.

Sure, sounds reasonable. I'll do that.

>> 2. Mailboxes
>> …
>> Servers *SHOULD* forbid sibling Mailboxes with the same name.
> Should this be be a MUST? I thought this is expressly forbidden in IMAP - RFC 3501 6.3.3. says "It is an error to attempt to create INBOX or a mailbox with a name that refers to an extant mailbox”.

For compatibility with IMAP it needs to be a MUST – so yes, I'll change it.

> The current definition of EmailAddress object doesn’t seem sufficient to cover an RFC 5322 Group Address? e.g.:
>> To: A Group:Ed Jones <[email-masked]>,[email-masked],John <[email-masked]>;

The current mapping would be:

[
    { "name": "A Group", email: null },
    { "name": "Ed Jones", email: "[email-masked]" },
    { "name": "John", email: "[email-masked]" }
]

This gives the group name, but doesn't give the position of the end of the group. I believe the thinking was groups are normally used for the whole set, so the end wasn't important. However, we could add an end marker (name=null and email=null?)

I would really prefer this stayed a flat list rather than a hierarchical structure though (and remember you are not allowed to have nested groups).

Thoughts?

>> 4.1.2. Header fields
>> …
>>    Any syntactically correct [RFC2047] encoded sections with a known encoding MUST be decoded, following the same rules as for the _Text_ form...
> It looks to me like the indentation of this entire section is 1 level too low?  The same issue applies to the comment right after the *MessageIds* property definition. Ditto for *URLs*.

Yeh, I got the markdown indentation wrong. Will fix.

>> 4.1.2. Header fields   
>> ...
>> In addition, the client may request/send properties representing individual header fields of the form:
>> 
>>                         header:{header-field-name}
>> 
>> Where "{header-field-name}" means any series of one or more printable ASCII characters (i.e. characters that have values between 33 and 126, inclusive), except colon. 
> I am not really sure what the conventions are... would this be better expressed with ABNF, or is this textual explanation sufficient?

I don't know; I'll see what more experienced editors say. My view is as long as it's clear to the average interested person reading the document, it's OK.

> When I first read this, I was a little confused by the text “including sub parts”, and thought maybe it meant that the tree of parts was flattened into a single array. If others also found this confusing, maybe it could be clarified into something like this:
>> This is the full MIME structure of the message body, represented as an array of the message’s top-level MIME parts, without recursing into “message/rfc822” or “message/global” parts. Note that EmailBodyParts may have subParts if they are type “multipart/*”.

Sounds good, will do.

>>    o  *bodyValues*: "String[*BodyValue*]" (immutable) This is a map of...
> Should this be EmailBodyValue instead of BodyValue?

Yes, will fix.

> 4.1.3. Body parts
>> ...
>> MIME structures are arbitrary nested trees of documents, but the majority of email clients present a model of an email body...
> This entire section is a solid in-depth explanation, but it comes after the property definitions, so it read a little bit confusing to me, and felt like the properties were being re-defined. I think it might be worthwhile considering an edit where the high-level explanation of MIME-tree-flattening comes before the list of Email properties, and then in each property, add more detail as to what each one is, so that it reads a bit more linearly and each property definition is a bit more self-contained. Thoughts?

Yes, I think it needs to be moved to the very top of the Email description, providing a non-normative overview of the structure before the actual specification. I'll do this.

> I personally think it may be worthwhile considering consolidating back down to a single attachedFiles property rather than having to have two properties and merge them. Thoughts?

Yes, I agree. Looking at it again, this is simpler to understand and gives you the full ordering. I'll do this (and name the property just "attachments").

>> 4.2. Email/get
>> …
>>    The following properties are expected to be fast to fetch in a quality implementation:
> Resent-XX are commonly shown by 

… by what?! And importantly, are they commonly shown in the mailbox listing or when you open the message (because the latter will be fetched with the body so doesn't have to be fast)?

>> 4.7. Email/import
>> ...
>> The server MAY forbid two email objects with the same exact [RFC5322] content, or even just with the same [RFC5322] Message-ID, to coexist within an account.  In this case, it MUST reject attempts to import an email considered a duplicate with an "alreadyExists" SetError.  
> Allowing a rejection purely based on a Message-ID match seems a little bit restrictive to me — it’s author-generated, so I have seen the global-uniqueness requirement violated by naively written scripts. 

This text is there because I believe Gmail does this (but please correct me if I'm wrong; maybe it's only on delivery, not append?). It's a MAY, so it's up to the server to define what it wants to do, but the spec needs to be compatible as much as possible with existing systems.

>> 5. Email Submission
>> …
>>    o  *identityId*: "String" (immutable) The id of the identity to associate with this submission.
> It’s a little bit confusing that Identity is used here before it’s defined in section 6. Maybe a slight note? 

I think I'll just reorder it so the Identity section comes before EmailSubmission. That should read better.

>> 5. Email Submission
>>   *  *mailFrom*: The email in the _Sender_ header, if present, otherwise the _From_ header, if present, and no parameters.  If multiple addresses are present in one of these headers, or there is more than one _Sender_/_From_ header, the server SHOULD reject the email as invalid but otherwise MUST take the first address in the last _Sender_/_From_ header in the [RFC5322] version of the message.  *If the address found from this is not allowed by the identity associated with this submission, the _email_ property from the identity MUST be used instead*.
> Instead of silently swapping the user’s requested _From_ to the identity address, I'd prefer returning a "forbiddenFrom” error and give the user a chance to correct their mistake. Thoughts?

This is the RFC5321 SMTP envelope From remember, not the RFC5322 message From. This passage of spec is also only referring to what to do when the client doesn't explicitly set an envelope. I think therefore this passage is OK, but you raise a good point that we probably need a separate error for a forbidden RFC5321 From in the envelope. I'll add:

- `forbiddenMailFrom` – The server does not permit the user to send an email
  with the [@!RFC5321] envelope From.

which is in addition to:

- `forbiddenFrom` – The server does not permit the user to send an email
  with the [@!RFC5322] From header of the email to be sent.

>> 5. Email Submission
>> … 
>>          For emails relayed via an alternative to SMTP, the server MAY
>>          generate a synthetic string representing the status instead.
>>          If it does this, the string MUST be of the following form:
>> 
>>          +  A 3-digit SMTP reply code, as defined in [
>> RFC5321], section
>>          +  Then an SMTP Enhanced Mail System Status Code as defined in [RFC3463], with a registry defined in [RFC5248].
>>          +  Then a single space character.
>>          +  Then an implementation-specific information string with a human readable explanation of the response.
> Not very familiar with the conventions here, but would ABNF be more appropriate to specify this?

Again, happy to take advice from more experienced editors here.

>> 5.5 EmailSubmission/set
>>    Standard _/set_ method, with the following two extra arguments:
>>    o  *onSuccessUpdateEmail*:...
> I think a lot of clients will want to move the message from Draft to Sent using this block. It might be worth explicitly calling that out, or even giving a full example.  Thoughts?

Yes, that's exactly what it's for. I'll add an example.

>> 7. Search Snippets
>> …
>>    o  *attachments*: "String|null" If text from the filter matches the text extracted from an attachment, this is the relevant section of the attachment (converted to plain text), with matching words/phrases wrapped in "<mark></mark>" tags.  It MUST NOT be bigger than 255 octets in size.  If it does not match, this is "null".
> I could imagine a MUA wanting to present to the user exactly which attachment the search string was found in, so it might be worth including an attachment ID property. Thoughts? 

That does sound like a useful feature to me. It will probably have to be optional, any suggestions on the best way to represent this? Could make the property a map of partId to matching section, but then what to do if the server doesn't know which part had the match?

>> 8. Vacation Response
>> The *VacationResponse* object represents the state of vacation-response related settings for an account.  It has the following properties:
> I’m not too knowledgeable about prior work here, but:
> 1. Exchange Server offers the ability to configure a separate “Internal" and “External" vacation response — which is useful if you work in a large organization and want to have your internal reply include things like “you can contact my manager at XXX”, but you want your external client-facing response to expose less information.
> 2. I also think that there may need to be some mention of loop prevention — so that two accounts with vacation responses don’t continuously reply to each other. 

The spec here was not intended to provide a complete consideration of everything that should be considered when building a vacation response system (which is already covered in previous RFCs), merely a mechanism by which it can be configured by the client. The functionality was chosen as a common subset supported by most major systems, to make adoption feasible. Individual systems (including Exchange, Gmail, FastMail etc.) will have extended options for vacation responses, but then which options do you support in the spec, and how do you make it so other systems can be compliant? Lots of optional features are a pain for everyone. The real question is whether being able to configure the common subset from arbitrary 3rd party clients is useful or not?

> I am starting to think this should be deferred to a JMAP extension, so that prior work in this area can be more carefully considered (such as RFC 5230). Thoughts?

So, I agree the vacation response could be neatly separated into a separate specification rather than part of the core mail. The main reason to include it here is to make sure there is concrete extra functionality that an existing IMAP client implementing JMAP would be able to provide that it couldn't with just IMAP. This (sadly) can make it an easier sell to management than just "it will make everything work faster, more efficiently and more reliably than before". However, I'm very interested to hear if you don't think this is likely to be so useful a hook based on your previous employment experience.

Neil.