Subject: [lemonade] Strawman for Submit based message composition

From: Chris Newman <[email-masked]>
Date: Tue, 10 Jun 2003 17:07:43 -0700

I'll start by defining message composition as the process of assembling
multiple byte-counted objects from potentially different locations into a
single byte stream.  IMAP has no such facility.

SMTP/Submit, however, already has 90% of this facility with the CHUNKING
extension.  In particular, it assembles a series of byte-counted objects into a
single byte stream.

Suppose we create a new Submit extension called "BURL" which can be interleaved
with "BDAT" commands.  So you can now compose a message as follows:

BDAT ....               headers and first part
BURL 8bit <imap-url>    forwarded 8bit content from IMAP
BDAT ....               MIME delimiters or whatever
BURL base64 <http-url>  get http content, base64 encode it and include it
BDAT ....               final data

This can all be pipelined into one round-trip.

The BURL extension should advertise what types of URLs it supports:

250-BURL http imap://trusted-host.example.com/

This extension alone would solve the forward-without-download extension for the
subset of the problem space where the submit server has pre-existing trust
relationship with the IMAP server.

Indeed since 99% of the world uses plain text passwords (with or without TLS)
for authentication.  Authenticating to the submit server will grant sufficient
rights for the user to access and forward their own mail from IMAP anyway.  So
why not use it when appropriate?

If we wanted to go for the additional case where there isn't a trust
relationship between the submit server and IMAP server, then we simply add two
extensions to IMAP:

1. a "GENURL" command which turns a message or body-part reference within the
selected mailbox into an extended IMAP URL with an authorization token.  This
could either have a time-limit or a number of use limit.  The authorization
token would likely be an HMAC-MD5 covering the URL, a nonce, a time-limit and
using a per-user random secret known only to the IMAP server.

2. a "GETURL" command which is accessible by any IMAP user (including
anonymous) which returns a literal of the the relevant message/body part if the
authorization token is valid.

This just doesn't seem complicated to me.  Indeed it seems much simpler than
any approach to extending IMAP with a redundant submission and composition
facility in addition to the one already present in CHUNKING Submit.

                - Chris

_______________________________________________
lemonade mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/lemonade

---

From: Doug Royer <[email-masked]>
Date: Tue, 10 Jun 2003 18:43:03 -0600

Interesting idea.

I think the GENRUL command could be eliminated:

  As the client is going to be sent something that tells it that
  a non-downloaded body part exits, why not make that something
  compatible with the 'BURL'? Now that round trip is eliminated.
  Generating some authentication token / UID or CID/MID / whatever
  can not be that complex for each non-downloaded body part or
  message.

-- 

  Doug Royer                     |   http://INET-Consulting.com
  -------------------------------|-----------------------------
  [email-masked]                 | Office: (208)612-INET
  http://Royer.com/People/Doug   |    Fax: (866)594-8574
                                 |   Cell: (208)520-4044

                 We Do Standards - You Need Standards

---

From: Mark Crispin <[email-masked]>
Date: Tue, 10 Jun 2003 18:03:04 -0700 (Pacific Daylight Time)

> Interesting idea.

Golly gee.  Chris' strawman is essentially what I've been talking about.
The main new thing is the use of SMTP CHUNKING.  But I deliberately left
the mechanics of submission unspecified.

>   compatible with the 'BURL'?

Huh?  That's what GENURL creates.

>   Now that round trip is eliminated.

If you're talking about the client generating the URL, then perhaps we can
have something returned at select time that allows the client to build
URLs for any data in the mailbox via a well-known algorithm.

All that's needed is something that the IMAP server, upon receiving the
URL, can validate that an authorized client generated it.  So it's
important that a bad guy can't use an valid URL to create one of his own.

Chris can probably come up with a way to do that.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.
_______________________________________________
lemonade mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/lemonade

---

From: Lawrence Greenfield <[email-masked]>
Date: Tue, 10 Jun 2003 23:10:40 -0400

Date: Tue, 10 Jun 2003 17:07:43 -0700
   From: Chris Newman <[email-masked]>

   This extension alone would solve the forward-without-download
   extension for the subset of the problem space where the submit
   server has pre-existing trust relationship with the IMAP server.

   Indeed since 99% of the world uses plain text passwords (with or
   without TLS) for authentication.  Authenticating to the submit
   server will grant sufficient rights for the user to access and
   forward their own mail from IMAP anyway.  So why not use it when
   appropriate?

I'd rather not see these sort of trust relationships
supported. Implementations relying on receiving sufficient credentials
for further authentication just puts yet another burden on places that
want to avoid plaintext passwords. I'd rather see everyone use
something like GENURL/GETURL; if the "plaintext trust" mechanism is
supported, that's what everyone will test and the GENURL/GETURL won't
work with a substantial number of clients and servers.

Note that Chris's strawman does not address a couple of somewhat
interesting things:

. the setup costs of an SMTP session 

  I'm sure some people would get grumpy if the clients just had
  long-running SMTP connections to amortize the cost.

. a carbon copies folder

  One possible solution for carbon copies would be to standardize an
  IMAP annotation for "give me an e-mail address that will post
  directly to this mailbox". (This is also desirable for shared
  mailboxes.)

. synchronity of the submission server

  Does the submission server have to fetch external parts
  synchronously before returning success or failure to the client?

Larry

---

From: Mark Crispin <[email-masked]>
Date: Tue, 10 Jun 2003 20:20:35 -0700 (PDT)

>   directly to this mailbox". (This is also desirable for shared

I agree; we really need something like this.

However, it's a non-issue with the proxy server approach.

-- Mark --

---

From: Rick Block <[email-masked]>
Date: Wed, 11 Jun 2003 07:59:47 -0600

Mark Crispin wrote:
>>  directly to this mailbox".
> 
>

Non-issue meaning both the (SMTP-ish) submit and the (IMAP-ish)
append to "sent" folder would be handled by the proxy which is
in a position to expand any external references for each of
these operations.  On the other hand, if bandwidth is so precious
and latency so high perhaps the proxy server should be doing
the copy to sent folder operation itself rather than the
ultimate client, which makes the submit operation a little less
SMTP-ish.

Rick Block
Avaya Inc.

---

From: Mark Crispin <[email-masked]>
Date: Wed, 11 Jun 2003 07:35:16 -0700 (PDT)

On Wed, 11 Jun 2003, Rick Block wrote:
> these operations.

Correct.

> On the other hand, if bandwidth is so precious

Agreed.  The issue is the client<->proxy server path.  Presumably the
proxy server is on the low-latency/high-bandwidth side of things and is
freer to act.

I would guess that there would be an "fcc" field in the request that the
client makes to the proxy server, along with all the other receipient
information, and the proxy server translates that to whatever commands.

-- Mark --

---

From: Doug Royer <[email-masked]>
Date: Wed, 11 Jun 2003 14:27:53 -0600

Mark Crispin wrote:
 >
 >(your insulting comments removed from this reply)
>...
> 
> 

So why do the round trip?

> If you're talking about the client generating the URL,

No. I was talking about modifying the IMAP server so that we can avoid
the round trip in the first place.

 > then perhaps we can

Or that would work. Just eliminate that added round trip.

> important that a bad guy can't use an valid URL to create one of his own.

I still think that the best approach is to have the IMAP server do the
submit :-)
--

---

From: "Alan K. Stebbens" <[email-masked]>
Date: Wed, 11 Jun 2003 22:38:56 -0700

Hello Doug,

DR> I still think that the best approach is to have the IMAP server do the
DR> submit :-)

Here is an alternative submission protocol for mobile clients, supporting
forwarding without downloading, using standard IMAP4 features and standard
SMTP features.

I'll use a picture (because I like them :^)

            Please view in a fixed-width font such as Courier.

+--------+           +--------+           +----------+    5         +-----+
| Mobile |           |  IMAP  |---RECENT->|  Outbox  |--MAIL From-->| MTA |
| Client |--APPEND-->| Server |     2     |  Server  |  RCPT To     |     |
|        |    1      |        |<--SEARCH--| ( IMAP ) |  DATA        |     |
+--------+           |        |   RECENT  | (Client) |  ...         |     |
                     |        |     3     |          |  .           |     |
                     |        |---FETCH-->|          |              |     |
                     |        |     4     |          |              |     |
                     |"Outbox"|           |          |              |     |
                     +--------+           +----------+              +-----+

The "envelope" information that is needed for the SMTP transactions could be
carried in the message that is APPENDed to the IMAP4 "Outbox".  Example:

        To: [email-masked]
        From: [email-masked]
        Subject: a mail
        X-Env-From: [email-masked]
        X-Env-To: [email-masked]

        blah blah

The "X-Env-From:" and "X-Env-To:" headers would carry the MAIL and RCPT
information, and would be removed from the message transmitted over the SMTP
connection.

When a message is appended to a given user's Outbox, a later connection by
another, server-based IMAP client, called the "Outbox Server", would be used
to search for RECENT messages in the "Outbox".  For concurrent sessions,
asynchronous, untagged RECENT responses would be sent as a notification to
the Outbox Server, which would then do a SEARCH RECENT to get the recently
arrived messages in the Outbox.

After FETCHing the recent messages in the Outbox, the Outbox Server would
dereference any MIME parts consisting of IMAP URLs, the UIDs of which would
be relative to the same mailbox containing the Outbox.

After FETCHing any IMAP URLs to referenced MIME parts (or entire messages),
and replacing the MIME part of the outgoing message with the FETCHed
contents, the new outgoing message would be submitted to a configured MTA
using the envelope information defined by the "X-Env" headers.

This mechanism does not require any protocol changes to IMAP or to SMTP.

By virtue being part of the operator's infrastructure, the Outbox Server can
obtain access to each user's IMAP Outbox folder and limit itself to further
access solely within that same mailbox, including the IMAP URLs to messages
in the INBOX.

The sole liability to this mechanism is the current lack of a scalable
notification mechanism of newly APPENDed messages across all Outbox folders
for a given IMAP server.  It is not practical for an Outbox Server to
maintain sessions for *all* Outbox folders in order to receive asynchronous
untagged RECENT responses, and it is not practical for an Outbox Server to
repeatedly poll all the Outbox folders on a large scale IMAP server.

Instead, the occurrence of newly APPENDed messages into an Outbox folder
should trigger a notification mechanism to the Outbox Server.  How can this
be accomplished and by what mechanism should be an interesting discussion.

-- 
Best regards,
 Alan K. Stebbens <[email-masked]>
 Principal Product Technologist, Messaging
 Openwave Systems, Inc.
 Cell: +1.805.886.8886  Voice/Fax: +1.866.579.0801
 Desk: +1.805.884.3162
 YIM/MSN: alan_stebbens  ICQ: 51361674  AIM: akstebbens

---

From: Lyndon Nerenberg <[email-masked]>
Date: Wed, 11 Jun 2003 23:54:05 -0600

> information, and would be removed from the message transmitted over 

The problem here is you will always have a namespace collision (however 
unlikely).

One way around this would be to submit the entire thing as an 
application/batch-smtp.

--lyndon

---

From: Doug Royer <[email-masked]>
Date: Fri, 13 Jun 2003 07:47:34 -0600

> One way around this would be to submit the entire thing as an 

Or to register some pre-defined headers that do not start with 'x-'.

--