Subject: [Jmap] Eric Rescorla's Discuss on draft-ietf-jmap-mail-14: (with DISCUSS and COMMENT)

From: Eric Rescorla <[email-masked]>
Date: Sun, 17 Feb 2019 16:06:14 -0800

Eric Rescorla has entered the following ballot position for
draft-ietf-jmap-mail-14: Discuss

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.

The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-jmap-mail/

----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------

Rich version of this review at:
https://mozphab-ietf.devsvcdev.mozaws.net/D4198

IMPORTANT: It's a bit hard to tell what the server is required to do.
Which of the many properties of emails (headers, the like) is the
server required to provide?

DETAIL
S 2.
>   
>         *  *mayReadItems*: "Boolean" If true, the user may use this
>            mailbox as part of a filter in a _Email/query_ call and the
>            mailbox may be included in the _mailboxIds_ set of _Email_
>            objects.  If a sub-mailbox is shared but not the parent
>            mailbox, this may be "false".  Corresponds to IMAP ACLs "lr".

This doesn't have any impact on Email/get? that seems odd.

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

S 2.
>      A *Mailbox* object has the following properties:
>   
>      o  *id*: "Id" (immutable; server-set) The id of the mailbox.
>   
>      o  *name*: "String" User-visible name for the mailbox, e.g.  "Inbox".
>         This may be any Net-Unicode string ([RFC5198]) of at least 1

MAY?

S 2.
>   
>      o  *role*: "String|null" (default: null) Identifies mailboxes that
>         have a particular common purpose (e.g. the "inbox"), regardless of
>         the _name_ (which may be localised).  This value is shared with
>         IMAP (exposed in IMAP via the [RFC6154] SPECIAL-USE extension).
>         However, unlike in IMAP, a mailbox may only have a single role,

MUST only

S 2.
>      o  *totalEmails*: "PositiveInt" (server-set) The number of emails in
>         this mailbox.
>   
>      o  *unreadEmails*: "PositiveInt" (server-set) The number of emails in
>         this mailbox that have neither the "$seen" keyword nor the
>         "$draft" keyword.

This is the first appearance of "keyword" in this document. Please
provide some explanation of what that means before this.

S 2.
>      o  *totalThreads*: "PositiveInt" (server-set) The number of threads
>         where at least one email in the thread is in this mailbox.
>   
>      o  *unreadThreads*: "PositiveInt" (server-set) An indication of the
>         number of "unread" threads in the mailbox.  This may be presented
>         by the client as a badge or marker associated with the mailbox.

I know we're in 8174-land, but it seems to me that this may is a bit
confusing in any case. The client can do anything it wants with this
at all, right?

S 4.1.
>         list rather than a single part as messages may have headers and/or
>         footers appended/prepended as separate parts as they are
>         transmitted, and some clients send text and images intended to be
>         displayed inline in the body (or even videos and sound clips) as
>         multiple parts rather than a single HTML part with referenced
>         images.

Does the server have to supply this? It seems like it's a bit of a
pain if you don't otherwise have a bunch of MIME-parsing machiner.

S 4.1.1.
>      properties is specified over the following three sub-sections.
>   
>   4.1.1.  Metadata
>   
>      These properties represent metadata about the [RFC5322] message, and
>      are not derived from parsing the message itself.

As above, is the server required to supply all of these?

S 4.1.2.3.
>   
>      o  Resent-Cc
>   
>      o  Resent-Bcc
>   
>      o  Any header not defined in [RFC5322] or [RFC2369]

Can this list be updated in the future? If so, how?

S 9.2.
>   
>      Subtle differences in parsing of HTML can introduce security flaws:
>      to filter with 100% accuracy you need to use the same parser when
>      sanitizing that the HTML rendering engine will use.
>   
>      o  Encapsulating the message in an "<iframe sandbox>" can help

You need a reference to this.

---

From: "Neil Jenkins" <[email-masked]>
Date: Tue, 19 Feb 2019 01:29:19 -0500

> server required to provide?

I'm sorry, I don't understand exactly what you're asking here. Which bit of the spec are you referring to? As a general thing, the client can request parsed representations of the RFC5322 message, and the server needs to be able to parse the message and return what was requested.

> This doesn't have any impact on Email/get? that seems odd.

This was kind-of implied. I realise it should be more explicit. I have added:

*Email objects may be fetched if they are in at least one mailbox with this permission.*

I have also added a section to the security considerations:

**Partial account access***
*
*
*
*A user may only have permission to access a subset of the data that exists in an account. To avoid leaking any unauthorised information, in such a situation the server MUST treat any data the user does not have permission to access the same as if it did not exist.
*
*
*
*For example, suppose user A has an account with two mailboxes, Inbox and Sent, but only shares the Inbox with user B. In this case, when user B fetches mailboxes for this account, the server MUST behave as though the Sent mailbox did not exist. Similarly when querying or fetching Email objects, it MUST treat any messages that just belong to the Sent mailbox as though they did not exist. Fetching Thread objects MUST only return ids for Email objects the user has permission to access; if none, the Thread again MUST be treated the same as if it did not exist.*

Actually this is a MUST. Will change to clarify.

> > However, unlike in IMAP, a mailbox may only have a single role,

Yes, will fix.

> provide some explanation of what that means before this.

Looking at this, I think a brief overview of the data model is useful. I have added the following to the introduction:

*This specification defines a data model for accessing a mail store over JMAP, allowing you to query, read, organise and submit mail for sending.
*
*
*
*The data model is designed to allow a server to provide consistent access to the same data via IMAP (RFC3501) as well as JMAP. As in IMAP, a message must belong to a mailbox, however in JMAP its id does not change if you move it between mailboxes, and the server may allow it to belong to multiple mailboxes simultaneously (often exposed in a user agent as labels rather than folders).
*
*
*
*As in IMAP, emails may also be assigned zero or more keywords: short arbitrary strings. These are primarily intended to store metadata to inform client display, such as unread status or whether a message has been replied to. An IANA registry allows common semantics to be shared between clients and extended easily in the future.
*
*
*
*A message and its replies are linked on the server by a common thread id. Clients may fetch the list of messages with a particular thread id to more easily present a threaded or conversational interface.
*
*
*
*Permissions for message access happen on a per-mailbox basis. Servers may give the user restricted permissions for certain mailboxes, for example if another user's inbox has been shared read-only with them.*

> confusing in any case. The client can do anything it wants with this

Yes. The expected common use case is as described, however I'm happy to just cut the sentence.

> pain if you don't otherwise have a bunch of MIME-parsing machiner.

Well, yes. The server MUST be able to do MIME parsing to implement the whole Email object, so I'm not quite sure what to make of this comment. There is a suggested algorithm lower down in 4.1 for taking the `bodyParts` property (the parsed MIME tree) and decomposing it into the appropriate `textBody`/`htmlBody` arrays.

> As above, is the server required to supply all of these?

Yes, if the client requests them.

> Can this list be updated in the future? If so, how?

This is a blacklist to prevent obviously nonsensical behaviour with common headers available today. Any future header not listed here may be used with the form. The blacklist is primarily to help client authors realise more quickly if they are doing something silly, but there is no danger if you were to use an inappropriate form with a header; it's just not very useful. A future RFC could update the list of course, but I don't think it's likely to be necessary.

> You need a reference to this.

Right, I have added a reference to the appropriate part of the HTML spec.

Cheers,
Neil.

---

From: Alexey Melnikov <[email-masked]>
Date: Tue, 19 Feb 2019 08:25:58 +0000

Hi,

> I'm sorry, I don't understand exactly what you're asking here. Which bit of the spec are you referring to? As a general thing, the client can request parsed representations of the RFC5322 message,…

I didn’t understand Ekr’s question either. I think the answer is that JMAP server needs to parse MIME structure of messages and be able to return any header field or attachment. (This is the same as IMAP model.)

Best Regards,
Alexey

---

From: Eric Rescorla <[email-masked]>
Date: Tue, 19 Feb 2019 05:48:45 -0800

On Mon, Feb 18, 2019 at 10:29 PM Neil Jenkins <[email-masked]>
wrote:

> I'm sorry, I don't understand exactly what you're asking here. Which bit
> of the spec are you referring to? As a general thing, the client can
> request parsed representations of the RFC5322 message, and the server needs
> to be able to parse the message and return what was requested.
>

Well it's not just parsed representations. For instance, see S 4.1.1, which
lists a bunch of metadata properties. Must the server include all of them
or can I omit (say) |size|? It seems you think the answer is that they all
must be present but the document doesn't seem to say that anywhere, and
absent that, it doesn't seem clear how to write an interoperable server.

DETAIL
> *Partial account access*
> *A user may only have permission to access a subset of the data that
> exists in an account. To avoid leaking any unauthorised information, in
> such a situation the server MUST treat any data the user does not have
> permission to access the same as if it did not exist.*
>
> *For example, suppose user A has an account with two mailboxes, Inbox and
> Sent, but only shares the Inbox with user B. In this case, when user B
> fetches mailboxes for this account, the server MUST behave as though the
> Sent mailbox did not exist. Similarly when querying or fetching Email
> objects, it MUST treat any messages that just belong to the Sent mailbox as
> though they did not exist. Fetching Thread objects MUST only return ids for
> Email objects the user has permission to access; if none, the Thread again
> MUST be treated the same as if it did not exist.*
>

Thanks.

S 2.
> Looking at this, I think a brief overview of the data model is useful. I
> have added the following to the introduction:
>
>
> *This specification defines a data model for accessing a mail store over
> JMAP, allowing you to query, read, organise and submit mail for sending.*
>
>
> *The data model is designed to allow a server to provide consistent access
> to the same data via IMAP (RFC3501) as well as JMAP. As in IMAP, a message
> must belong to a mailbox, however in JMAP its id does not change if you
> move it between mailboxes, and the server may allow it to belong to
> multiple mailboxes simultaneously (often exposed in a user agent as labels
> rather than folders).*
>
>
> *As in IMAP, emails may also be assigned zero or more keywords: short
> arbitrary strings. These are primarily intended to store metadata to inform
> client display, such as unread status or whether a message has been replied
> to. An IANA registry allows common semantics to be shared between clients
> and extended easily in the future.*
>
>
> *A message and its replies are linked on the server by a common thread id.
> Clients may fetch the list of messages with a particular thread id to more
> easily present a threaded or conversational interface.*
>
> *Permissions for message access happen on a per-mailbox basis. Servers may
> give the user restricted permissions for certain mailboxes, for example if
> another user's inbox has been shared read-only with them.*
>

Thanks. This is helpful.

> Yes. The expected common use case is as described, however I'm happy to

It's not a big deal, but I would.

> Well, yes. The server MUST be able to do MIME parsing to implement the
> whole Email object, so I'm not quite sure what to make of this comment.
> There is a suggested algorithm lower down in 4.1 for taking the bodyParts
> property (the parsed MIME tree) and decomposing it into the appropriate
> textBody/htmlBody arrays.
>

See my comment above. You don't actually say that I have to do this as
opposed to (say) rejecting the request with an error.

> This is a blacklist to prevent obviously nonsensical behaviour with common
> headers available today. Any future header not listed here may be used with
> the form. The blacklist is primarily to help client authors realise more
> quickly if they are doing something silly, but there is no danger if you
> were to use an inappropriate form with a header; it's just not very useful.
> A future RFC could update the list of course, but I don't think it's likely
> to be necessary.
>

OK.

-Ekr

---

From: "Neil Jenkins" <[email-masked]>
Date: Tue, 19 Feb 2019 22:33:00 -0500

On Wed, 20 Feb 2019, at 00:49, Eric Rescorla wrote:
> Well it's not just parsed representations. For instance, see S 4.1.1, which lists a bunch of metadata properties. Must the server include all of them

Yes, of course the server must support all of them. This is a description of the properties that may be fetched for this datatype. 

>  It seems you think the answer is that they all must be present but the document doesn't seem to say that anywhere

I am not really sure what to say here. If you don't implement the properties of a datatype defined in the spec, then you don't have a compliant implementation. It seems superfluous to start sprinkling more MUSTs around. There is no decision here, you are either implementing the spec or not.

Neil.

---

From: Eric Rescorla <[email-masked]>
Date: Tue, 19 Feb 2019 20:40:46 -0800

On Tue, Feb 19, 2019 at 7:33 PM Neil Jenkins <[email-masked]> wrote:

> Well it's not just parsed representations. For instance, see S 4.1.1,
> which lists a bunch of metadata properties. Must the server include all of
> them
>
>
> Yes, of course the server must support all of them. This is a description
> of the properties that may be fetched for this datatype.
>
> It seems you think the answer is that they all must be present but the
> document doesn't seem to say that anywhere
>
>
> I am not really sure what to say here. If you don't implement the
> properties of a datatype defined in the spec, then you don't have a
> compliant implementation. It seems superfluous to start sprinkling more
> MUSTs around. There is no decision here, you are either implementing the
> spec or not.
>

That's not generically true of IETF specifications. We routinely define
functionality that people aren't required to support or can return errors
for.

If it's a conformance property to support these properties, then you need
to actually say so. It's not superfluous at all.

-Ekr

---

From: "Neil Jenkins" <[email-masked]>
Date: Wed, 20 Feb 2019 21:39:43 -0500

On Wed, 20 Feb 2019, at 15:41, Eric Rescorla wrote:
> That's not generically true of IETF specifications. We routinely define functionality that people aren't required to support or can return errors for.

I'm not disagreeing with that, but it is normally clearly marked if something is optional. Regardless, would you find it sufficient if I add the following to the notational conventions section in the introduction?

*Servers MUST support all properties specified for the new data types defined in this document.*

Neil.

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 20 Feb 2019 20:36:40 -0800

On Wed, Feb 20, 2019 at 6:39 PM Neil Jenkins <[email-masked]> wrote:

> I'm not disagreeing with that, but it is normally clearly marked if
> something is optional.
>

This may be a matter of convention. That's not my experience with protocol
specifications.

Regardless, would you find it sufficient if I add the following to the
> notational conventions section in the introduction?
>

Yes, that's fine.

-Ekr