Subject: RE: [lemonade] Mobile messaging clients barriers - Protocls vs. S erver Implementation

From: Mark Crispin <[email-masked]>
Date: Sun, 15 Jun 2003 08:17:11 -0700 (PDT)

On Sun, 15 Jun 2003, Erev Ari wrote:
> Arn't we mixing PROTOCOL with SERVER IMPLEMENTATION?
> In other words, if the submit-client talks to the proxy using "IMAP" or
> "IMAP with an additional SUBMIT/APPEND/SEND command", isn't this still the
> domain of the IMAP PROTOCOL (i.e, to define the extension commands, their
> semantics etc.)?

Not at all.

The proxy would be on a different port, and leaves open the possibility in
the future for extensibility to provide other non-IMAP services.

It may be convenient for you to think of it is "an extension of IMAP"; and
if it serves that purpose in a useful way for you, that would be a good
outcome.

> And if, the submit-client talks to the proxy using "SMTP" or "SMTP with
> compose using some extenral link semantics", don't we still have the same
> security and authentication/authorization issues we would have if the
> extended functionality would have been added to an SMTP (submit) server (for
> the Proxy to authenticate, get message data, etc. from the IMAP server)?

No, because those issues are already being addressed elsewhere.  The proxy
method means that the mobile device doesn't have to deal with it.

> And if we invent a new protocol for the submit-client to talk to the proxy,
> other than SMTP or IMAP,  doesn't it bring back the "Submit Client needs to
> support two submit protocols" problem?

What happens if your customers have a requirement to work in an existing
enterprise infrastructure?

You can't escape the "support two submit protocols" problem without a
proxy; and if your client won't work with a non-extended server then your
client's success depends upon the whim of the server managers.

The beauty of the proxy method is that it works with all existing servers.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.
_______________________________________________
lemonade mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/lemonade

---

From: Erev Ari <[email-masked]>
Date: Sun, 15 Jun 2003 17:26:09 +0300

Hi,

Regarding the Proxy suggestion, I have the following question:

(BTW, if an IMAP server implementation, selects to implement the "Proxy" as
part of the IMAP server itself, would it be a conforming implementation? (I
assume so). If yes, then why do we have to define the proxy as a separate
entity and not only the semantics of the various commands?)

So, the bottom line, is that I need some help to better understand what the
proxy idea buys us (for the "Forward without Download" requirement).

Ari

> -----Original Message-----
> From: Mark Crispin [mailto:[email-masked]]
> Sent: Saturday, June 14, 2003 7:01 PM
> To: Cyrus Daboo
> Cc: Mendelovich Meir; IETF LEMONADE (E-mail)
> Subject: Re: [lemonade] Mobile messaging clients barriers
> 
> 
> On Sat, 14 Jun 2003, Cyrus Daboo wrote:
> > A good solution to this is to have a proxy which is in 
> effect an imap
> > client on one end (that maintains the server connection and 
> caches mailbox
> > data), and a server (of some sort) on the other. One could 
> then access this
> > proxy by regular IMAP, mIMAP, HTTP, or any other mechanism 
> that would make
> > sense. The proxy could then handle composition/submission 
> etc as already
> > discussed. I think there is at least one simple connection 
> caching proxy
> > that's being used with webmail solutions right now.
> 
> This sounds like a concensus is forming around developing 
> this proxy --
> which will provide multiple necessary services for mobile clients
> including submit -- and NOT to hammer all this into IMAP.
> 
> -- Mark --
>

---

From: [email-masked]
Date: Sun, 15 Jun 2003 13:30:28 -0700

Folks,
	I believe that Erev is making a very important point that
we should not pass by so quickly.  The group's charter has as deliverable 0:

0. An informational RFC or RFCs will be produced on LEMONADE
architecture and the issues it seeks to address.

	It may be that the term "architecture" in deliverable 0 has
caused folks to consider server architectures and message stores
as the key initial consideration.  That's now how I understand them,
though, and I suspect that the reading Erev gives--that the group
is meant to be discussing protocol architecture--is closer to the
mark.  Re-reading all of the charter text (something I highly recommend
to anyone), the very first sentences are:

Lemonade is tasked to provide a set of enhancements and profiles of
Internet email submission, transport, and retrieval protocols to
facilitate operation on platforms with constrained resources, or
communications links with high latency or limited bandwidth. A primary
goal of this work is to ensure that those profiles and enhancements
continue to interoperate with the existing Internet email protocols in
use on the Internet, so that these environments and more traditional
Internet users have access to a seamless service.

	Much of the discussion of the past few days makes me
very uneasy about the group's focus on the primary goal of interoperability.
The proxy notions in Cyrus's recent email (using http, mIMAP, and IMAP
to access the proxy, for example) don't look terribly interoperable, and
statements like "The beauty of  the proxy method is that it works
with all existing servers." seem to indicate that interopability will be
achieved by introducing an entirely new network element for messaging,
with an entirely new protocol in support of client-server communication
with that element, plus a set of protocol enhancements to support
the interaction of that new element and the existing elements.
	That sounds a bit fragile as a way forward, to put it mildly.
It also sounds like developing a proxy for interaction with any
potential source of data to be included in a message is outside
of the current charter.
	I encourage those who have put forward straw proposals
to the group to get them into I-D form, however rough they may
be.  I also encourage those taking on that work to re-read the
charter as they do that writing.
			regards,
				Ted Hardie

At 8:17 AM -0700 6/15/03, Mark Crispin wrote:
>>  compose using some extenral link semantics", don't we ...
>...snip...
>>... would have been added to an SMTP (submit) server (for

---

From: Doug Royer <[email-masked]>
Date: Sun, 15 Jun 2003 14:36:29 -0600

> the future for extensibility to provide other non-IMAP services.

Or it could be on the same port with the real IMAP server hidden
from the client. As long as it passes the IMAP data through, no
existing client would know the difference. And having it on
the same port and just advertising a new capability would
make the same server (hardware) available to old and new clients.

When my Unix application connects to 993 it would not care
about the wire protocol extension. When my TCP aware cell phone
connects to exactly the same system and port it uses the
new extension with the 'real' IMAP server never knowing.

-- 

  Doug Royer                     |   http://INET-Consulting.com
  -------------------------------|-----------------------------
  [email-masked]                 | Office: (208)612-INET
  http://Royer.com/People/Doug   |    Fax: (866)594-8574
                                 |   Cell: (208)520-4044

                 We Do Standards - You Need Standards

---

From: Doug Royer <[email-masked]>
Date: Sun, 15 Jun 2003 15:37:12 -0600

Erev Ari wrote:

> compose using some extenral link semantics", don't we still have the 
> same security and authentication/authorization issues we would have if 
> the extended functionality would have been added to an SMTP (submit) 
> server (for the Proxy to authenticate, get message data, etc. from the 
> IMAP server)?

No, because the proxy could use its credentials when connecting
to the SMTP server. And that SMTP server could allow relaying
to that authenticated proxy server.

The SMTP server would not have to 'get' a message from the IMAP server
because:

  The proxy server would do the 'get' from its own IMAP server.
  (This WG seem to be in favor of all forwards and body parts
   coming from the currently authenticated IMAP users mailbox anyway.)

  The proxy itself would do the inline expansion of the included
  body part.

  The proxy server could setup the email headers to show the
  correct user, site, ... to match the currently IMAP authenticated
  user.

The proxy server could then just do standard [E]SMTP to the SMTP server.

> And if we invent a new protocol for the submit-client to talk to the 
> proxy, other than SMTP or IMAP,  doesn't it bring back the "Submit 
> Client needs to support two submit protocols" problem?

My initial guess is that it could look like the 'append' command
except be called 'submit'. The new IMAP clients would only
have to format the message and do a 'submit' command.

> So, the bottom line, is that I need some help to better understand what 
> the proxy idea buys us (for the "Forward without Download" requirement).

--

---

From: Mark Crispin <[email-masked]>
Date: Sun, 15 Jun 2003 16:24:34 -0700 (PDT)

On Sun, 15 Jun 2003, Doug Royer wrote:
> existing client would know the difference.

Using the same port has the effect of constraining the mobile client to
accept IMAP in all its chatty glory; rumors to the contrary
notwithstanding an IMAP client does *not* control what an IMAP server can
wreak.  Unpleasant surprises await client implementors who do not realize
the full implications of the unsolicited data model

Have you considered the impact of RFC 2193 referrals on submit-in-IMAP?
If the server returns a referral, the mobile client must establish a new
IMAP connection to the referred-to server and reissue the submit to that
server.  It is also necessary to constrain submit-in-IMAP to reference
only one mailbox, to prevent unresolvable submissions.

Neither of these are issues with a proxy server on a different port.  The
proxy server can offer guarantees to the mobile client that no unwanted
chat from the IMAP server will reach it; in turn the mobile client can be
programmed with that assumption.  The proxy can also ensure that RFC 2193
referrals do not happen as far as the mobile client is concerned; this
also removes the restriction of only one mailbox reference in a submit.

A mobile client can't count upon any of this if it talks to a raw IMAP
server instead of a proxy.  If you use the same port, only a correct DNS
name stands between the mobile client and that fate.

-- Mark --

---

From: Doug Royer <[email-masked]>
Date: Sun, 15 Jun 2003 20:10:11 -0600

> the full implications of the unsolicited data model

The proposal was not to create an new protocol anyway. The e-mail from
Ned suggests exactly that. So - not an issue as we were not proposing
a new wire protocol, just an extension intercepted by a proxy.

> only one mailbox, to prevent unresolvable submissions.

Currently ALL imap clients that connect to servers that return
'MAILBOX-REFERRALS' or any other extension would have this problem.
Nothing new.

> Neither of these are issues with a proxy server on a different port. 

The port number has nothing to do with it. It has to do with
submit and forward without download. The proposal was to
transparently forward IMAP commands to and from a real IMAP
server intercepting the SUBMIT command and sending the data
to an SMTP server.

> The

This could be done on the same port number by the proxy anyway,
or not done if the client understands MAILBOX-REFERRALS or any
other extension or IMAP feature.

The problem being solved is not how to make the mobile client use
some completely new protocol.

And we are not solving the problem of IMAP clients that do not want
to understand IMAP.

The problem we are solving is how to forward without download.

> name stands between the mobile client and that fate.

I have not seen any proposal that prevents a mobile client from talking IMAP
Nor have I seen a proposal for any other IMAP-like protocol.

The proposal is for a proxy that understands how to include messages
for the client and submit while making minimum changes to the wire
protocol which happens to be IMAP.

--

---

From: Mark Crispin <[email-masked]>
Date: Mon, 16 Jun 2003 08:34:31 -0700 (PDT)

> a new wire protocol, just an extension intercepted by a proxy.

Did you miss the part where Ned said that a charter revision for this
purpose would not be difficult?

> 'MAILBOX-REFERRALS' or any other extension would have this problem.

All good IMAP clients implement RFC 2193 and thus have no problem.  The
problem will be if mobile clients are mis-designed in such a way that they
can not work in an RFC 2193 environment.

> server intercepting the SUBMIT command and sending the data

By using the same port, the implication is that this is a service that can
(and should) be expected from all IMAP servers.  However, a server which
offers this service could not do RFC 2193.  It could do one or the other
but not both.

> other extension or IMAP feature.

If you require that the client understand referrals in submit, then the
entire "efficiency" argument for submit-in-IMAP is in tatters as the
client will open a new connection.

Furthermore, the restriction of "only one mailbox in a submit" remains,
and that restriction must be documented in the submit-in-IMAP
specification.

> And we are not solving the problem of IMAP clients that do not want

So IMAP being chatty is no longer a problem?

> Nor have I seen a proposal for any other IMAP-like protocol.

I propose both as possible alternatives that should be considered.

> protocol which happens to be IMAP.

Are you saying that all mechanisms which do not lead to submit-in-IMAP are
out of order, and this WG has no purpose other than to rubber-stamp?

If that is not what you are saying, then you should not attempt to
restrict the discussion of possible alternatives.

-- Mark --

---

From: Rob Siemborski <[email-masked]>
Date: Tue, 17 Jun 2003 09:29:33 -0400 (EDT)

On Mon, 16 Jun 2003, Mark Crispin wrote:

> > Currently ALL IMAP clients that connect to servers that return
> All good IMAP clients implement RFC 2193 and thus have no problem.

Could you give examples?  Our experience with a RFC2193 server has been
that very few clients (in fact, only pine and other c-client based
clients) support RFC2193.  For this reason (and the fact that the server
has no way of knowing other than if the client issues an RLIST command
that the client supports 2193), I don't think we'll see a large number of
servers issuing mailbox referrals unexpectedly in the foreseeable future.

> The problem will be if mobile clients are mis-designed in such a way
> that they can not work in an RFC 2193 environment.

Yes, if 2193 was ubiquitous, then this would be an issue.

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski | Andrew Systems Group * Research Systems Programmer
PGP:0x5CE32FCC | Cyert Hall 207 * [email-masked] * 412.268.7456
-----BEGIN GEEK CODE BLOCK----
Version: 3.12
GCS/IT/CM/PA d- s+: a-- C++++$ ULS++++$ P+++$ L+++(++++) E W+ N o? K-
w O- M-- V-- PS+ PE++ Y+ PGP+ t+@ 5+++ R@ tv-@ b+ DI+++ G e h r- y?

---

From: Mark Crispin <[email-masked]>
Date: Tue, 17 Jun 2003 06:51:10 -0700 (PDT)

On Tue, 17 Jun 2003, Rob Siemborski wrote:
> clients) support RFC2193.

I said "all good IMAP clients", didn't I?  :-)

> For this reason (and the fact that the server

Building an enterprise facility which requires 2193 is an excellent way of
getting rid of the pretty-picture virus-spreaders.  It's a standard trick
of IS managers to preclude the use of unauthorized software.

> Yes, if 2193 was ubiquitous, then this would be an issue.

Which leads to the obvious question: is LEMONADE willing to put itself in
the position of building a submit facility that breaks with 2193, on the
grounds that LEMONADE believes that nobody uses 2193?

If so, sooner or later there will have to be a showdown in the
standardization process.

-- Mark --

---

From: Rick Block <[email-masked]>
Date: Tue, 17 Jun 2003 08:50:59 -0600

> charter as they do that writing.

OK.  Not in the form of an I-D, but here's a detailed "proxy" straw
proposal for a "new" protocol (oops, I mean an extension of IMAP4)
intended for use by mobile clients which (for now) let's call imap++.
I don't think there's a large semantic difference between this
being considered a new protocol to be offered on some well known
port other than 143 versus being considered an IMAP4 extension
advertised via a CAPABILITY response.

1) imap++ is a strict superset of IMAP4 as defined in RFC3501

2) in the selected state, imap++ also supports "mail from",
"rcpt to", "rset", and "data" commands as defined for SMTP in RFC2821.
Selecting another mailbox will reset any SMTP-related state (i.e.
effectively perform an SMTP "rset").  To better fit the IMAP4
command structure, these "SMTP" commands will be preceded by
an IMAP4 tag.  [other imap4 parsing accommodations to be specified]

3) The MIME content transmitted following the "data" command can
include a syntax [to be specified] indicating inclusion of body
parts of messages in the currently selected mailbox.

4) the CAPABILITY response may include both extensions defined for
IMAP4 as well as extensions defined for SMTP [I haven't checked,
but hopefully existing extension names don't intersect]

Rick Block
Avaya Inc.

---

From: Doug Royer <[email-masked]>
Date: Tue, 17 Jun 2003 13:23:02 -0600

> If so, sooner or later there will have to be a showdown in the

As the current focus is IMAP aware cell phones, this is not
an issue as they ether will or will not be compliant. If they
are compliant, not an issue.

Additionally the PROXY could perform the redirection for the cell
phone and the cell phone would never need to know. And as (per
previous emails) the IP address changes, this may be the preferred
way to solve the problem. That is the proxy connects to the correct
system via some redirect and the cell phone never cares - it just
connects to any IMAP server it knows about and the proxy ether
gets a redirect or not, and the proxy returns to the cell phone
the correct data with the cell phone never caring that the proxy
was redirected.

And a proxy submit will not break 2193 as the proxy is deciding
which SMTP server to connect to without the IMAP clients input.
If the client connects to an IMAP server that can not submit,
that is no change from now. If it can do a submit, it works.

So 2192 may be the salvation and not the nemesis when the
IMAP client uses a proxy.

--

---

From: Mark Crispin <[email-masked]>
Date: Tue, 17 Jun 2003 13:39:34 -0700 (Pacific Daylight Time)

On Tue, 17 Jun 2003, Doug Royer wrote:
> an issue as they ether will or will not be compliant.

It is very much an issue if a reference to more than one mailbox is
permitted in the submit request.  The result is an unresolvable submission
request.

If you propose to solve this by requiring a proxy (which you suggest would
resolve referrals so the cell phone never sees it), then that begs the
question of how the user of the cell phone is to understand that detail.

Specifically; the user knows from his desktop client that his IMAP server
is imap.foo.com.  So he configures his cell phone to use imap.foo.com,
which subsequently doesn't work because the port 143 server on
imap.foo.com is not a proxy server.  Since you insist upon using port
143 for the proxy, a proxy server can't co-exist on imap.foo.com.

So the user has to know, specially for his cell phone, that he must use
proxy-imap.foo.com.  And foo.com has to set up this separate IP address
specifically for cell phones instead of just installing a new server on a
different port on the existing IP address.

All of this just to shoehorn submit-in-IMAP into the port 143 IMAP
service.  Why?

Is it to create an attractive nuisance to desktop clients, in an attempt
to pressure all IMAP server vendors to implement it?

If it's a separate port, then other desirable things specifically tailored
for cell phones can be done.  Think mIMAP.  Think NNTP-style overviews
instead of IMAP envelopes.

Why tie up the IMAP session while a submit is going on?  A separate IMAP
session from a submit session could be downloading new mail while the
message being submitted is being uploaded.

-- Mark --

---

From: Doug Royer <[email-masked]>
Date: Tue, 17 Jun 2003 15:10:09 -0600

> permitted in the submit request.  The result is an unresolvable submission

By mailbox do you mean MX host, or do you mean that you expect
the IMAP server to APPEND to the IMAP mailboxes - That was Alan's
proposal - not mine.

By having the PROXY do the SMTP submission, the IMAP client
would never need to know if the RCPT mailboxes were local,
remote, on one MX host, or several MX hosts. They just
submit the mime-blob.

> question of how the user of the cell phone is to understand that detail.

They would never know or care. Just like an IMAP client now never
need to tell the user that the IMAP server redirected them to an alternate
IMAP server.

> 143 for the proxy, a proxy server can't co-exist on imap.foo.com.

So we agree. If you point your client to a server that does not
understand submit, then they can not submit. :-)

It can co-exist as to non IMAP submit clients - it is a transparent proxy
so existing clients can use the proxy and they never need to know they
are using a proxy.

> different port on the existing IP address.

No. the ISP tells the user that they will be using imap.foo.com and they
will never need to know that it is a submit server. Their normal IMAP
client will work, and so will their cell phone. If their vendor
updates their IMAP client for submit, then it will also work for submit.

> All of this just to shoehorn submit-in-IMAP into the port 143 IMAP

Because it would be transparent to all existing IMAP clients, 143 and 993.

> to pressure all IMAP server vendors to implement it?

No.

> instead of IMAP envelopes.

If it is a separate port then all existing IMAP clients would have
to be re-configured to use the new port.

> message being submitted is being uploaded.

Tie up? Most IMAP sessions are persistent anyway. It should not
take any longer than an APPEND command.

--

---

From: Mark Crispin <[email-masked]>
Date: Tue, 17 Jun 2003 14:31:49 -0700 (Pacific Daylight Time)

> the IMAP server to APPEND to the IMAP mailboxes - That was Alan's

The term "mailbox" is defined in the IMAP base specification, RFC 3501.
You lectured me in public about how you are a great expect on all matters
related to IMAP; you should not need to ask me what a mailbox is.

The purpose of the exercise is to provide an efficient means of
forwarding/resending messages stored on an IMAP server.  This necessitates
a reference to a mailbox in the submit request.  That reference is either
implicit (selected mailbox) or explicit (by name).  If the latter, the
specification for the SUBMIT command has to stipulate one of three things:
 1) only one mailbox can be used in a submit request
 2) the server can not support RFC 2193 referrals
 3) the server must proxy RFC 2193 referrals (it can not return NO with a
     referral).

> remote, on one MX host, or several MX hosts. They just

What does an "MX host", which has to do with MTA operations, have to do
with RFC 2193 referrals or the submit operation?

If your answer is "proxy", then how is the client to know that it is
talking to a proxy as opposed to a non-proxy, since you insist that it be
on the same port as IMAP?

What you are creating is a separate fork in the IMAP protocol, sharing the
same fork with IMAP, which adds one capability (SUBMIT) and removes at
least one (referrals).  Is this something that the IETF wants to have?
Ned???

> need to tell the user that the IMAP server redirected them to an alternate

If the client does not know or care, how does it send a message?

> understand submit, then they can not submit. :-)

Are you admitting that "mobile clients" is just a backdoor means of
sneaking submit into IMAP for desktop clients?

> so existing clients can use the proxy and they never need to know they

As you are using the term "proxy", you are referring to an internal
implementation detail and not a protocol element.  It therefore does not
belong in an IETF protocol specification, and all references to the term
"proxy" should be removed.

> client will work, and so will their cell phone.

Just how will the cell phone "work" with an existing IMAP server that does
not have submit?

You are making a completely dishonest argument.

> If their vendor

Oh, so now desktop IMAP clients are expected to use this feature?

This is outside of the scope of LEMONADE.  It belongs in IMAPEXT.

> Because it would be transparent to all existing IMAP clients, 143 and 993.

No, it's a fork within IMAP, and an attempt to make one of the two forks
mandatory by making it impossible to send mail with some clients.

> to be re-configured to use the new port.

There is no reason for existing IMAP clients to use the new port.

The sole purpose of this WG is mobile clients.  If the scope is to expand
to other clients, the WG needs to be rechartered.

> take any longer than an APPEND command.

The APPEND command does not have to validate recipient addresses, while
your document explicitly shows that a submit would do that.

-- Mark --

---

From: Doug Royer <[email-masked]>
Date: Tue, 17 Jun 2003 15:58:12 -0600

> related to IMAP; you should not need to ask me what a mailbox is.

So could you answer the question? Were you refering to my proposal
or Alan's? My draft did not include the word 'mailbox'. So
I assume you are refering to the APPEND to OUTBOX and some
background process does the work?

--

---

From: Mark Crispin <[email-masked]>
Date: Tue, 17 Jun 2003 15:10:40 -0700 (Pacific Daylight Time)

On Tue, 17 Jun 2003, Doug Royer wrote:
> or Alan's?

The answer is "yes".

Any submit-in-IMAP proposal has to deal with the issue of how RFC 2193
affects a mailbox reference in the submit request.  There are three
possible ways of doing this, which I listed in my previous message.

> So

No, I am not referring to a mechanism which does submit via APPEND to a
special name such as OUTBOX.

I am referring to the entire reason why this exercise is taking place: an
efficient means of forward/resend, and a restriction which is unique to
the submit-in-IMAP mechanism as opposed to other mechanisms.

-- Mark --

---

From: Doug Royer <[email-masked]>
Date: Tue, 17 Jun 2003 16:32:46 -0600

>>So could you answer the question? Were you refering to my proposal

Thanks - I'll let Alan answer.

--