Subject: RE: [lemonade] IMAP push/pull plan

From: Mark Crispin <[email-masked]>
Date: Thu, 1 Jan 2004 15:37:27 -0800 (PST)

On Wed, 31 Dec 2003, Ted Hardie wrote:
> Deciding that the entire burden goes onto the IMAP server is contrary
> to existing practice, in that folks in many environments use external
> pointers by including URLs in their messages.

I don't think that you understand what I'm getting at.

People compose messages, using multiple sources, on a regular basis.
Sometimes, they send a link in the form of a public-access URL.

Other times, they include or attach data from a storage mechanism that
they already have.  This includes such things data from messages in other
mailboxes (perhaps on other servers), data from files, perhaps even data
from a private-access web page.  Word documents, spreadsheets, PDFs, all
sorts of things get sent.

CATENATE as it stands is a very inadequate mechanism for the second case.
However, with a very simple extension (ignoring a "thou shalt not"
paragraph) CATENATE can be made to do everything that's needed.

This presents an extremely attractive nuisance.

You may think that this "isn't the common case."  Well, maybe it's not for
people who use cell phones for their email.  But they aren't the only case
of email users.  For many email users (and client developers) there is a
very attractive prospect of being able to have the server snarf the PDF
file, instead of having to download it over a dialup to the client only to
push it out again.

Pull will do this.  So will CATENATE with one little extension.

> And while all of this is a very interesting, I think we have to go back
> to the common use case, and ask ourselves whether "forward without
> download", which is one of the immediately motivating cases here,
> would require anything more than what an IMAP server can do
> using the local mailboxes.

Indeed.  The advocates of pull have seen this coming from the onset.

> And perhaps we ough to even ask the larger question "are we
> optimizing for the right use case", since the client always has the
> option of accessing and including content.

Exactly.  I believe that if you do so, fairly, you will conclude that pull
is the superior solution.

Another observation is that CATENATE is not draft-friendly.  It is a
one-shot assembly.  CATENATE+SUBMIT is essentially doing in two operations
on the IMAP server compared to one operation (e.g. BURL) on the submit
server.  Drafts have to be on the client (or stored in a way that the
client can pick it apart to know how to CATENATE).

A better solution that is draft-friendly would be to have some sort of
reference to the text to be included at a particular point (possibly some
MIME parameter could be used).  Then have an ASSEMBLE command that knows
how to do the insertions.  That still is two commands, but doesn't have
the draft-unfriendliness that CATENATE has.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

_______________________________________________
lemonade mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/lemonade

---

From: Pete Resnick <[email-masked]>
Date: Thu, 1 Jan 2004 21:04:02 -0600

On 1/1/04 at 3:37 PM -0800, Mark Crispin wrote:

>Other times, they include or attach data from a storage mechanism 
>that they already have.  This includes such things data from 
>messages in other mailboxes (perhaps on other servers), data from 
>files, perhaps even data from a private-access web page.  Word 
>documents, spreadsheets, PDFs, all sorts of things get sent.

With an eye to the discussion below about "optimizing for the right 
use case", I think the vast majority of things in e-mail that come 
from different sources are:

1. Publicly-accessible resources which can be accessed via a URL at 
the remote end.
2. Things in local-to-the-client sources (word processing documents, 
spreadsheets, PDFs, that are on a device directly accessible by the 
client).
3. Things that are in my message store (which is either 
local-to-the-client or on a single IMAP server).

and that only a small minority of things that are:

4. Things from other-than-a-single-IMAP-server privately-accessible 
sources that are "closer" (in terms of bandwidth, compute power, etc) 

>CATENATE as it stands is a very inadequate mechanism for the second case.

This strikes me as inaccurate. Your second case is what I've labelled 
2, 3, and 4, and CATENATE (as it stands) works perfectly well to 
collect 2 and 3, and is optimized for 3. It is (as you say) 
inadequate for 4, but I do think 4 makes up a tiny piece of what gets 
included in e-mail.

>Exactly.  I believe that if you do so, fairly, you will conclude 
>that pull is the superior solution.

I think this is the crux of the debate for me. If you think that what 
you should be optimizing for is "local storage" and "what's on my 
IMAP server", I think that CATENATE+push is the superior solution. If 
you think that you should be optimizing for "content that is 'closer' 
to the submit server than to the local client" (which includes, but 
is not primarily, items on my single IMAP server), then pull is the 
superior solution.

My leaning is that the former is the right use case. There are other 
things I like about CATENATE; see below:

>one-shot assembly. CATENATE+SUBMIT is essentially doing in two 
>operations on the IMAP server compared to one operation (e.g. BURL) 
>on the submit server.  Drafts have to be on the client (or stored in 
>a way that the client can pick it apart to know how to CATENATE).

I think CATENATE is sufficiently draft-friendly. I see two choices 
for storing things in such a way that the client can retrieve a draft 
without having to download a part originally included from the IMAP 
store:

1. When CATENATE-ing, put a Content-Location on the part so that the 
client can notice from the BODYSTRUCTURE that the part was included 
from the local IMAP store. When the client CATENATE-s the new draft, 
you include what was in the Content-Location. The downside to this 
approach is that Content-Location would then be sent with the 
message, potentially introducing a security problem.

2. When CATENATE-ing, put a Content-ID on the part that has some 
local part that means to the client "I included this part". When the 
client looks at the draft, it sees this in the BODYSTRUCTURE. Then, 
when it CATENATE-s the new draft, it simply includes the part from 
the old draft (instead of from the original message). This is my 
preferred method.

>A better solution that is draft-friendly would be to have some sort 
>of reference to the text to be included at a particular point 
>(possibly some MIME parameter could be used).  Then have an ASSEMBLE 
>command that knows how to do the insertions.  That still is two 
>commands, but doesn't have the draft-unfriendliness that CATENATE 
>has.

We discussed something like this before (see 21 October discussion), 
where I said:

At 1:49 PM -0500 10/21/03, Pete Resnick wrote:
>The downside to expand is that you then have a message with 
>"expandable parts". That means that I can send you an "unexpanded 
>message" and perhaps invoke some unexpected shenanigans in your 
>server, perhaps getting it to expand things upon forwarding my 
>message that you didn't expect.

I think the Content-ID trick makes CATENATE draft-friendly. And of 
course BURL is *completely* draft-unfriendly: With BURL, the draft 
*must* be stored on the local client. That's another reason I still 
like CATENATE even if we go for pull instead of push.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
QUALCOMM Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

---

From: Mark Crispin <[email-masked]>
Date: Thu, 1 Jan 2004 20:42:22 -0800 (PST)

On Thu, 1 Jan 2004, Pete Resnick wrote:
> And of course BURL
> is *completely* draft-unfriendly: With BURL, the draft *must* be stored on the
> local client.

No.  There is no more draft unfriendliness in BURL than there is in
CATENATE/SUBMIT.

If the draft message does not have any external parts, then:
 . BURL: an IMAP URL to the draft is submitted to the submit server.
 . CATENATE/SUBMIT: a SUBMIT command is done.

If the draft message has external parts whose pointers are not in the
draft but are known to the client, then:
 . BURL: an IMAP URL to the draft, along with URLs to the other parts, are
    submitted to the SUBMIT server.
 . CATENATE/SUBMIT: a CATENATE command is done, then a SUBMIT command.

If the draft message has external parts that have pointers in the draft,
then:
 . BURL: the BODYSTRUCTURE of the draft is fetched, the pieces determined,
    and then a set of URLs (including IMAP URLs to the texts in the draft)
    are submitted to the SUBMIT server
 . CATENATE/SUBMIT: the BODYSTRUCTURE of the draft is fetched, the pieces
    determined, the various pieces are put together via CATENATE, and then
    a SUBMIT command is done.

None of the above is truly "draft-friendly".

What would be draft-friendly is if the submit mechanism could recognize
the pointers and do the assembly itself.  CATENATE is necessary in the
push model to avoid this (without CATENATE you'd have no choice), but is
draft-unfriendly.

In the pull model, CATENATE is a frill; it's not necessary but it may have
some use, particularly if it it is not restricted to URLs that reference
the currently selected mailbox.  BURL has no concept of "currently
selected mailbox" and thus such a restriction is meaningless.

Note that the CATENATE/SUBMIT case *does* involve an extra client command
step (separate CATENATE and SUBMIT commands), along with an implicit
assumption that the client wants an fcc on the IMAP server.  If the user
does not want an fcc, then the user must also do a delete and expunge step
on the IMAP server for a total of four commands.

The question of a mandatory fcc in the push case raises another thorny
issue.  Users who are near the limit of their disk quota may not be able
to send a message because they do not have the space to do the CATENATE.
This is not a problem with BURL.  Consider a person with a 10MB quota who
has a 6MB video clip in a message that he wants to forward to some other
place where he has more space.  He's out of luck in the push case; only
pull works.

Now, far be it for me to argue the benefits of quotas!  I'm in the camp of
"give the system manager a nickel so he can afford to double my disk
quota."  But a lot of sites still think that quotas are necessary.

-- Mark --

---

From: Pete Resnick <[email-masked]>
Date: Thu, 1 Jan 2004 23:24:06 -0600

>  . CATENATE/SUBMIT: a SUBMIT command is done.

No problem. That seems perfectly draft-friendly.

>draft but are known to the client, then:

I'm not sure I understand what you mean by this. Do you mean, "the 
draft message is incomplete at the moment and therefore doesn't have 
anything in it to be included from the server yet, but the client 

>  . CATENATE/SUBMIT: a CATENATE command is done, then a SUBMIT command.

then that's right.

>If the draft message has external parts that have pointers in the draft,

On this, I'm completely lost on what you mean. In the CATENATE case, 
if I am creating a draft which is to contain items that are external, 
there are no "pointers" in the draft; the external parts *are* in the 

>     a SUBMIT command is done.

No, in the CATENATE/SUBMIT case, a SUBMIT command is done, full stop. 
It only has to do the BODYSTRUCTURE business if it is re-editing a 
draft and re-CATENATE-ing it to the server.

I think you've assumed a "pointer then assembly" model, which 
instantly makes all cases at least two steps. But that's not 
required. If I make my drafts with an "always assemble the real thing 
now" model, then I can SUBMIT in one step if I need no other changes 
to my draft.

>push model to avoid this (without CATENATE you'd have no choice), but is

As I said in my previous message, it doesn't have to be 
draft-unfriendly so long as there is a Content-ID which indicates you 
don't have to download this part to re-edit the other parts of the 
message.

>In the pull model, CATENATE is a frill;

Agreed, but in the pull model using IMAP as a draft store without 
CATENATE (I assume by instead using APPEND and including the external 
parts as message/external-body or the like), the client is required 
to re-download the BODYSTRUCTURE of the draft and then submit the 
URLs to the SUBMIT server.

>assumption that the client wants an fcc on the IMAP server.

Yes, CATENATE/SUBMIT always requires a draft, where BURL does allow 
for not having a draft at all.

>place where he has more space.  He's out of luck in the push case; only

It depends on the server:

1. If the server implements CATENATE with references instead of 
copies, it might take up no more disk quota.

2. The server could allow a different temp quota for the drafts 
mailbox so that assembly could be done without blowing the normal 
quota. Things in the drafts mailbox might be auto-purged every so 
often if the admin is really a hard-ass.

pr

---

From: Mark Crispin <[email-masked]>
Date: Thu, 1 Jan 2004 22:09:43 -0800 (PST)

On Thu, 1 Jan 2004, Pete Resnick wrote:
> I'm not sure I understand what you mean by this. Do you mean, "the draft
> message is incomplete at the moment and therefore doesn't have anything in it
> to be included from the server yet, but the client wants to add those things"?

No, I mean "the draft has things to be included from external places, but
the pointers to those things are stored separately from the draft."

> On this, I'm completely lost on what you mean. In the CATENATE case, if I am
> creating a draft which is to contain items that are external, there are no
> "pointers" in the draft; the external parts *are* in the draft.

That's the problem with using CATENATE for a draft.  I create a draft that
will include a 6MB attachment of some form.  I decide to work on it later.
Now, I retrieve the draft to work on it...and I've downloaded that entire
6MB attachment.

I don't think that people are going to do drafts that way.  To be
explicit, I do not think that people will use CATENATE to create drafts.
They will use APPEND for drafts instead.

> I think you've assumed a "pointer then assembly" model, which instantly
> makes all cases at least two steps.

Indeed I do.  If forward-without-download is a requirement, then it should
be possible to re-edit a draft without downloading the forwarded content.
After all, it's a draft.  Drafts tend to be edited!

> But that's not required. If I make my drafts with an "always assemble
> the real thing now" model, then I can SUBMIT in one step if I need no
> other changes to my draft.

CATENATE naturally points to "always assemble the real thing now", so I'm
not surprised that this is how you suggest it should be used.  Hence my
observation about CATENATE being draft-unfriendly.

"Always assemble the real thing now" also presumes that, when editing, the
user has enough storage to store at least two copies of the draft with all
its attachments

> As I said in my previous message, it doesn't have to be draft-unfriendly so
> long as there is a Content-ID which indicates you don't have to download this
> part to re-edit the other parts of the message.

So, in order to maintain the forward-without-download requirement, instead
of just downloading BODY[] of the draft for editing, the client has to
download ENVELOPE and BODYSTRUCTURE first and then download individual
body parts.

This creates a substantial amount of new complexity.  Not only is message
submission a two-step process (CATENATE+SUBMIT), but now there's a
multi-step process to edit drafts.

There's also a problem with using Content-ID for anything.  Outlook has a
nasty bug in reading messages which use Content-ID.  Some other way has to
be used.

> Agreed, but in the pull model using IMAP as a draft store without CATENATE (I
> assume by instead using APPEND and including the external parts as
> message/external-body or the like), the client is required to re-download the
> BODYSTRUCTURE of the draft and then submit the URLs to the SUBMIT server.

This is less of a burden than the steps listed above to re-edit the draft,
which require much more than the BODYSTRUCTURE.

The only savings that CATENATE+SUBMIT would have is if a draft is saved on
the server, *not* sent, and then later, *after* the client has forgotten
about the draft, the user submits the draft with *no* editing.

> 1. If the server implements CATENATE with references instead of copies, it
> might take up no more disk quota.

This is akin to saying "choose push because the server will do pull
internally."

> 2. The server could allow a different temp quota for the drafts mailbox so
> that assembly could be done without blowing the normal quota.

I am not aware of any server which actually works this way.

This is yet another new implementation requirement upon the IMAP server
which will need to be documented someplace if the push model is
considered.

I can count at least three IMAP server implementors who oppose the push
model because of all these new mandates, although there's less opposition
to CATENATE.  Are there any IMAP server implementors who think that the
push model is a good idea?

-- Mark --

---

From: Pete Resnick <[email-masked]>
Date: Fri, 2 Jan 2004 01:17:40 -0600

>Now, I retrieve the draft to work on it...and I've downloaded that entire

Again, no, they don't have to re-download the attachment. That's a 
straw man and was never intended by anyone for the push model as far 
as I know. I've said this twice now.

>>draft-unfriendly so long as there is a Content-ID which indicates 
>>you don't have to download this part to re-edit the other parts of 
>>the message.
>

BODYSTRUCTURE, yes. I don't see why they would need to get the ENVELOPE.

>multi-step process to edit drafts.

I'm sorry, I don't see the substantial new complexity. In the case of 
BURL with a draft, you download the BODY, and then you must parse 
that message to figure out where the attachments were in order to 
edit it. Using the proposed method for CATENATE, you get the 
pre-parsed BODYSTRUCTURE, and then you download the body parts you 
need to edit the message.

Let's break down the actual steps involved here:

- Message submission with an IMAP draft using BURL but not CATENATE 
is a 3 step process: Create the draft using APPEND, download the 
BODYSTRUCTURE, submit the pieces using BURL.

- Message submission with an IMAP draft using CATENATE is a 2 step 
process: Create the draft using CATENATE, submit the entire message 
with SUBMIT.

- Re-edit of a draft in the BURL/no-CATENATE case means downloading 
the message, parsing it, editing it, and uploading the new message, 
deleting the old draft, and purging.

- Re-edit of a draft in the CATENATE base means downloading the 
BODYSTRUCTURE, downloading the parts of the message you need to edit, 
editing it, CATENATE-ing the new message , deleting the old draft, 
and purging.

I don't see this big difference.

Let's also remember that if you want to save a copy of the sent 
message, CATENATE gets you that for free. In the BURL/no-CATENATE 
model, you either are stuck with a template message or you have to 
get the submit server to give you a copy of the real message back.

>nasty bug in reading messages which use Content-ID.  Some other way has to

Content-Location is a possibility. We can think about others. It just 
needs to be a way to mark particular parts as catenated that can be 
retrieved with BODYSTRUCTURE.

>CATENATE (I

I don't see it. Please explain.

>This is akin to saying "choose push because the server will do pull

I am saying, "If your only concern with push is about quotas, it's 
not a problem if your server uses references instead of copies." But 
it's not a reason to choose push.

>This is yet another new implementation requirement upon the IMAP server

No, it's a suggested way to deal with a particular concern with push. 
There are other ways to deal with that concern that don't involve any 
new implementation requirements.

>I can count at least three IMAP server implementors who oppose the 
>push model because of all these new mandates

1. Don't speak for other people in order to make your own argument. 
Perhaps the other two (as yet unnamed) server implementors will be 
completely persuaded by the current debate and no longer oppose the 
push model. Perhaps they will think it all the worse. They can speak 
for themselves. It adds nothing to your argument to say "other people 
agree with me".

2. Saying "all these new mandates" is just hyperbole. Sure, there are 
several parts to doing push on an IMAP server that need to be 
implemented. The number of "new mandates" for a submit server in the 
pull model are pretty impressive to me. And there are things to do on 
the client side for each model. I'm not prepared at the moment to 
starting making guess-timates on the number of programmer hours to be 
spent on each model.

>there's less opposition to CATENATE.

As I've said before, I think CATENATE is a win even for the pull 
model, which is why I think there's less opposition to it.

>Are there any IMAP server implementors who think that the push model 
>is a good idea?

How about let's change the question to something that people can 
answer that might bring about consensus: What do other IMAP server 
implementors think are the good and bad points of the push model in 
light of what we've been discussing? If you happen to also be a 
submit server implementor, can you characterize what you see as 
implementation difficulties of push versus pull?

pr

---

From: Rob Siemborski <[email-masked]>
Date: Fri, 2 Jan 2004 13:07:55 -0500 (EST)

> get the submit server to give you a copy of the real message back.

CATENATE seems to be the sanest way to solve the "I need an FCC" problem
when using BURL to submit the message.

Of course, if you are pulling from non-IMAP sources (e.g. a web page) then
maybe this is bad because now *both* the IMAP and SUBMIT/SMTP server needs
to be able to talk to the source of the object.  But maybe that doesn't
matter since the client can just issue a BURL to the draft on the IMAP
server.

> model, which is why I think there's less opposition to it.

Yeah, I'd agree with this.

> light of what we've been discussing?

Personally, I don't really like the PUSH model in that it adds a new
submission mechanism, it assumes "everyone uses IMAP" and implies that if
we ever find a new mail retrieval protocol that everyone wants to switch
to after IMAP, we'll have to reinvent a submission mechanism for this
future protocol as well.

Up until this thread, the main drawback that I saw with the BURL model was
that getting a FCC back onto the IMAP server was hard.  But I think that
the use of CATENATE solves that.

I'm currently favoring a solution that used CATENATE+BURL+URLAUTH.
Sadly, I think this solution involves a lot of work on all sides
(SUBMIT/SMTP, IMAP, and client).  However, I also feel that it gives us
all of the features that LEMONADE needs (forward without download, along
with draft messages and fcc capability), and is more future proof than the
CATENATE/SUBMIT model.

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski | Andrew Systems Group * Research Systems Programmer
PGP:0x5CE32FCC | Cyert Hall 207 * [email-masked] * 412.268.7456
-----BEGIN GEEK CODE BLOCK----
Version: 3.12
GCS/IT/CM/PA d- s+: a-- C++++$ ULS++++$ P+++$ L+++ E W+ N(-) o? K- w-- O-
M-- V-- PS+ PE+ Y+ PGP+ t+@ 5+++ X- R@ tv-- b+ DI+++ D++ G e++ h+ r- y?

---

From: Mark Crispin <[email-masked]>
Date: Fri, 2 Jan 2004 10:50:24 -0800 (Pacific Standard Time)

> when using BURL to submit the message.

It is certainly the sanest way that has been offered so far.

That is why I have taken the trouble to try to get CATENATE be more
draft-friendly and work with non-IMAP sources, instead of flatly opposing
CATENATE.

> matter since the client can just issue a BURL to the draft on the IMAP

Perhaps the draft question provides some impetus for additional work in
this area.  I think that we need both "pre-assembled" (with some form of
pointer to the data) and "assembled" forms of CATENATE.

I can see some fcc users wanting to store in "pre-assembled" form in order
to save space.

> Yeah, I'd agree with this.

I agree as well, but weakly.  I'd feel better about CATENATE if its
deficiencies were fixed.

> to after IMAP, we'll have to reinvent a submission mechanism for this

I agree with all of the above.

> the use of CATENATE solves that.

Yes, CATENATE is a solution to the fcc problem, and I don't see much point
in coming up with an alternative to CATENATE.  CATENATE is close enough
now and its deficiences are fixable.

> I'm currently favoring a solution that used CATENATE+BURL+URLAUTH.

As am I.

> (SUBMIT/SMTP, IMAP, and client).

Yes, it does.  There are no "easy" ways out.

However, the benefit of rolling up our sleeves and doing the right thing
is that the resulting technology can be leveraged for other purposes.

-- Mark --

---

From: Rob Siemborski <[email-masked]>
Date: Fri, 2 Jan 2004 14:00:20 -0500 (EST)

> now and its deficiences are fixable.

Yeah, all of my points were assuming "CATENATE takes URLs" (which I
believe has been fixed) and "CATENATE doesn't have an explicit restriction
against being used for other purposes" (which I suspect people would be
inclined to ignore anyway if it became useful).

-Rob

-----BEGIN GEEK CODE BLOCK----
Version: 3.12

---

From: Mark Crispin <[email-masked]>
Date: Fri, 2 Jan 2004 11:04:52 -0800 (Pacific Standard Time)

> inclined to ignore anyway if it became useful).

I believe that the current CATENATE document only permits IMAP URLs that
refer to the selected mailbox.  That still needs to be fixed.

-- Mark --

---

From: Pete Resnick <[email-masked]>
Date: Fri, 2 Jan 2004 13:47:49 -0600

On 1/2/04 at 11:04 AM -0800, Mark Crispin wrote:

>I believe that the current CATENATE document only permits IMAP URLs 
>that refer to the selected mailbox.  That still needs to be fixed.

The only reason I put in that limitation, and the reason I chose the 
-00 syntax, was because I heard objections (specifically from Chris 
Newman) to opening up the command to other than the selected mailbox. 
I certainly have no objection to changing that (as a matter of fact, 
it was my desire at the get-go), but I'd like to hear from other 
folks that this is OK. (Chris, are you out there?)

Should I still go under the assumption that the URL must only refer 
to the current server and that other servers or other URL schemes are 
outside of scope and require extension?

pr

---

From: Ted Hardie <[email-masked]>
Date: Mon, 5 Jan 2004 16:18:55 -0800

> > I'm currently favoring a solution that used CATENATE+BURL+URLAUTH.

Are you still planning on writing a draft which documents the PULL usage,
and, if so, by what date do you have it out?   (You'd previously argued
that we did have enough data to decide between push and pull, and
I understood you to say you would right such a draft)
				regards,
					Ted Hardie

---

From: Ted Hardie <[email-masked]>
Date: Mon, 5 Jan 2004 16:24:17 -0800

At 1:47 PM -0600 01/02/2004, Pete Resnick wrote:
>
>The only reason I put in that limitation, and the reason I chose the -00 syntax, was because I heard objections (specifically from Chris Newman) to opening up the command to other than the selected mailbox. I certainly have no objection to changing that (as a matter of fact, it was my desire at the get-go), but I'd like to hear from other folks that this is OK. (Chris, are you out there?)
>
>Should I still go under the assumption that the URL must only refer to the current server and that other servers or other URL schemes are outside of scope and require extension?

In the discussion to date, there has been a back and forth about the security
assumptions around non-IMAP URL schemes.   As I understand it, at least
some folks believe that the non-IMAP URLs would be accessing data using
that was non publicly available.  The security model for the restricted CATENATE
is pretty easy, since it is providing an IMAP command that uses the same basic
permissions for a resource as would a download of that resource. 
In my personal opinion, the draft needs a very different security analysis if
the presumption is going to be that the facility can securely retrieve external
non-public resources. It also needs to carefully lay out which schemes can be
used (not all uri schemes have the presumption of data retrieval, for example,
and the use of those in this context would not make much sense).
			regards,
				Ted Hardie

---

From: Mark Crispin <[email-masked]>
Date: Mon, 5 Jan 2004 16:54:42 -0800 (Pacific Standard Time)

> I understood you to say you would right such a draft)

Yes, I am planning to do so.  I was not planning to do so until this week
(I made a half-hearted attempt to be on *vacation* until today), so I
haven't started it yet.  I hope to get on it in the next day or so,
assuming that the threatened major snowstorm tonight doesn't cut off
electrical power...

-- Mark --

---

From: Ted Hardie <[email-masked]>
Date: Mon, 5 Jan 2004 17:02:12 -0800

>Indeed.  The advocates of pull have seen this coming from the onset.

I'm afraid I don't follow you here.  Do you mean that the advocates
of pull have seen from the outset that "forward without download" requires
more than what an IMAP server can do using the local mailboxes?

There is a very critical scope question here, from my point of view.
If the working group sees this task as developing what we might call
"non-client compose", in which the client can direct some server
(be it Submit or IMAP) to assemble a message from multiple
data sources without downloading the parts, then the "compose
from a single, local data source" would be a degenerate case of the
"non-client compose".    Using the same mechanism for the
degenerate case as the full functionality makes lots of sense.

But I think this working group has a much more focused deliverable:

>2. Enhance the existing IMAP4 message retrieval and/or message
>submission (RFC 2476) protocols to satisfy the requirements for
>forwarding a message and/or its attachments without downloading
>the message to the client and subsequently uploading the message to a
>server.

"a message and/or its attachments" does look to me like
a message and its parts taken from a single data source
were/are the key motivating use case for the words
in the charter.  Charters are not unchangeable holy writ, of course,
and there is some room for disagreement about the limitations
inherent even in this statement.  But I personally believe that this
use case gives a very focused view of what needs to be done
and that getting it done quickly and right would be a very
valuable piece of work.  A quick round of the working group
to be sure which problem we're tackling seems to be in order,
as the engineering trade-offs are surely different with
the different perspectives. 

Again speaking personally, if the consensus of the working group
is that the task is "non-client compose", rather than "forward without
download", then I am not really sure that either a Submit server or
an IMAP server would be the right place to do the work.  Revisiting
the use of a multi-function proxy might be appropriate in that
case, because the core functionality may be beyond what are
safe and sane ways to extend IMAP or Submit servers.  I am nagged,
though, by the idea that we will end up re-inventing the remote shell,
as that is what I and many others use to handle this problem
(I sometimes use ssh to reach a remote server with better connectivity
when I need to handle email over a very poor link; sometimes, though,
I take it as a sign to take a day or two away from email.  This is the
source of the delay in my reply to this message.  Apologies for any
inconvenience).
			regards,
				Ted Hardie