Subject: [lemonade] Server implementors for Pull

From: Chris Newman <[email-masked]>
Date: Tue, 10 Feb 2004 16:38:30 -0800

I have been asked by several people to weigh in on the mailing list about push
vs. pull.

I strongly favor the pull model.  It is a more scalable and better
architecture, will be much simpler to implement and deploy, and is a much safer
for the future of the email infrastructure.

1. Simpler

We have multiple access protocols (IMAP, POP, NNTP, etc) and multiple transport
protocols (SMTP, XMPP, SIMPLE, etc) for messages.  To do forward without
download, we can take all the complexity of every transport protocol and shove
it into every access protocol -- that's a complexity increase of N x M.  Or we
can do something elegant: change every access protocol so there is a simple and
secure way to extract content, and change every submission protocol so these
simple profiles can be used to assemble the message when appropriate.  The
result is significantly less complexity because we optimize both ends of the
protocol for integration.

2. Faster to Implement

We have separate teams of people working on IMAP and SMTP and I'm sure most
implementers do (sometimes they're even separate companies).  The "pull" model
splits the work so a modest parallel investment across the teams finishes the
work.  In fact, BURL and URLAUTH are bite-sized projects that get something
useful released and CATENATE can be done in a later release when the
"copy-to-sent-mail-and-forward-without-download" feature is needed.  On that
basis, I would recommend to management that we implement "pull" before we're
asked.

The push model requires all the work be done in the IMAP server.  The IMAP
server is more security sensitive, the protocol is much more complex already,
and we already have real reluctance by developers to use the protocol because
it is too bloated.  I will recommend to management that we do not implement the
"push" model unless it appears in RFPs from customers with lots of money.  The
cost is too concentrated on one team and better to risk being a bit behind the
times than adding unnecessary complexity to a sensitive system.

3. Scalable Architecture

A scalable IMAP server needs a very high caliber disk array and I/O subsystem
and tends to be the choke-point in the scalability of the messaging system.  We
have recently achieved surprising performance improvements in the messaging
system by moving complexity off the IMAP server and into the MTA (by using LMTP
instead of SMTP for delivery to the message store machine from the MTA
machine).  The last thing we want to do is put more load on the IMAP servers
again.  The submit servers, on the other hand, often have CPU, disk and network
I/O to burn in a messaging deployment.

4. Optimizing for Client Convenience can be a _Big Mistake_.

I used to think it was a good idea and proceeded to produce ACAP.  I learned my
lesson the hard way.  Good protocol design must balance the complexity of the
client against the complexity of the server.  Push ignores server and system
architecture in favor of client convenience.  You will notice most of the
server implementers on this list are crying foul.  We could end up with a
proposal that deploys just as fast as ACAP for the same reasons.  Go read item
2 again if you don't believe this is a risk.

5. Functional Separation

Functional separation is a very valuable property of the IETF's present
IMAP/SMTP messaging architecture.  It's better for deployment.  It's better for
support.  It's better for management.  It's better for scalability.  It's great
when the SMTP server can be upgraded separately from the IMAP server and
vice-versa.  You can get best-of-breed IMAP servers and best-of-breed SMTP
servers and they work together (presuming a common SMTP -> IMAP delivery
mechanism like LMTP).

The loss of functional separation can be _very_ expensive.  Even a loss as mild
as "POP before SMTP" results in a significant performance loss, requirements to
upgrade the systems in lockstep, buy them from the same vendor, and make
migrations to a different vendor more difficult.

I find the "authrole=submit" in URLAUTH to be worrisome for functional
separation (albeit unavoidable for the best security), but the "push" model is
a full-scale disaster for functional separation.  This will hurt a lot.  I wish
I could predict how, but I've learned over time that it's impossible to predict
exactly when and how architecture mistakes will bite.

6. IMAP bloat

Full-function access protocols for complex data structures are necessarily
complex and as a result, IMAP is complex.  Because IMAP is a bit different from
other protocols it is perceived to be more complex than it actually is.  And
because of the behavior variations permitted by the spec it is also complex in
practice.  This complexity is a deterrent to implementing IMAP.  Now IMAP must
grow some complex extensions as the access needs for a mailstore become better
understood and more sophisticated (e.g. i18n SORT/THREAD).  But the last thing
we should do is shove a boatload of functionality into IMAP that's unrelated to
accessing a mailstore.

GETURL is all about accessing a mailstore and even without the URLAUTH
facilities it enhances the protocol by providing a long desired shortcut for
simple access transactions.  But IMAP Submit is orders of magnitude more
complex and unrelated to the primary function of IMAP.

And yes, I've written both an SMTP client and an SMTP proxy so I'm well aware
of the complexity needed to implement submit correctly (since it's a blend of
the two).

7. Security

Putting my "deployable security" hat on, I find both proposals to be about
equally worrisome in different ways.  IMAP Submit creates a new channel for
message data that needs to be audited in addition to the "access over IMAP"
channel.  It also requires configuring a full trust proxy authentication
relationship between the IMAP server and the first MTA.  IMAP Submit is also
worrisome from a code-review perspective since it adds lots of complex code to
the more security-sensitive part of the system.

URLAUTH creates a new authentication mechanism with tremendous abuse potential
if the sysadmin misconfigures it.  The "authrole=submit" also requires setting
up a trust relationship between submit and IMAP, but it's really just a
different kind of administrative user so may be simpler than the IMAP pull
issue.  There's also the risk that the GETURL code to validate the URLAUTH has
bugs, although that's much less code than the submit code.

8. Bolt Sendmail onto UW imapd

This is how the push model will be implemented in most early deployments and it
will become the de-facto "reference implementation" for the feature.  I've
mentioned this to folks at both UW and sendmail and they all shudder at the
idea.  This is your future if you choose "push".  Trust me, you don't want this
to happen.

I'm sure I'm missing some, but that is enough of a tome for now.

        - Chris

_______________________________________________
lemonade mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/lemonade

---

From: Ted Hardie <[email-masked]>
Date: Tue, 10 Feb 2004 18:13:04 -0800

Hi Chris,
	Thanks for weighing in, some comments and responses below.

>protocol for integration.

Again, this working group is not chartered to do a cross product of 
every transport
and access protocol.  Nor is it chartered to change every access protocol
so there is "a simple and secure way to extract content" and "change every
submission protocol so these simple profiles can be used to assemble
messages".  It's chartered to achieve forward without download.  Push or
Pull, if it sets out to achieve the latter, it will fail.  I don't 
think it has the
expertise for that, and it certainly doesn't have the time.

Creating a reasonable "pawn ticket" model of authorization could be a
very useful piece of work, but assuming that we could get from there
to extraction from every access protocol is a big leap. 

>server is more security sensitive,

This is a critical point for me.  If you are starting from the more security
sensitive point and moving forward (which PUSH does), I believe it will
be easier to achieve confidence that you have the security done to a
degree adequate to the task.  Putting that same sensitivity in another
server seems like it creates a barrier to deployment.

>I/O to burn in a messaging deployment.

This is very useful data; thank you.  If the IMAP server moves the data quickly
from submission to an internal MTA (so that the IMAP server is not intending
to do final delivery), how much of this advantage can be retained? 

>2 again if you don't believe this is a risk.

Optimizing without regard for  client convenience is also a mistake.  It's
a balance.  It's particularly critical that we understand what the 
client thinks
the functionality in question is:  I believe it is "access to a message store
to get something done" in both cases.  I've argued that elsewhere, but it is
also worth pointing out that the client is already taking the brunt 
of the complexity
in the IMAP world today.  Putting this there has to be balanced not just
for this extension, but in general.

>exactly when and how architecture mistakes will bite.

I agree that functional separation is a desirable element of the current
architecture, and one of the reasons I have pushed for limiting the
scope is that it keeps the IMAP store out of the business of assembling
parts using other protocols.  The difference I see here is that the submit
server's functional separation is also lost, in my view, and I think it is
worse because of the risk that it will start working outside the set of
mail protocols entirely.  That's a "scrying game" charge, though, so I
should say simply that I agree with this point, but feel it applies equally
to both methods.

Some of my own points in favor of PUSH answer Chris's other points,
so I won't repeat them again.

Thanks again for your contribution,
				regards,
					Ted Hardie

---

From: Chris Newman <[email-masked]>
Date: Wed, 11 Feb 2004 15:36:45 -0800

Ted Hardie wrote on 2/10/04 18:13 -0800:
> Again, this working group is not chartered to do a cross product of every
> transport
> Pull, if it sets out to achieve the latter, it will fail.  I don't think it
> has the

Our charter clearly permits us to change both IMAP and SMTP so both push and
pull are clearly in scope for the charter.  When evaluating the quality of the
proposals, the impact on the entire suite of Internet protocols MUST be
considered.  So not only is this design issue in-scope, it would be
irresponsible to ignore it.

> server seems like it creates a barrier to deployment.

When it comes to actual implementation, the security sensitive code should be
as small and isolated as possible.  But an IMAP server generally has access to
an entire message store, so less code in the IMAP server is better for
security.  Push puts _all_ the new complexity in the already complex IMAP
server.

>From the scalability standpoint, if the data touches the disk on the IMAP
server, then push loses big time on scalability.  If the data doesn't touch the
disk, then the IMAP server becomes an SMTP proxy.  About 90% of the protocol
interoperability problems in SMTP are caused by poorly implemented SMTP
proxies.  I can tell you from first hand experience that writing a good SMTP
proxy is very hard.

And this brings up another security problem with the push model.  It is
critical to track the IP address and timestamp of the client.  So IMAP submit
MUST add a received header.  That makes the proxy approach even more
complicated.

I would estimate the amount of server code to implement the push model properly
is 3-4 times the amount of code necessary to implement the pull model.  The
client complexity for pull is at most twice as much as push.  I prefer the
simpler overall solution.

> a balance.

Actually one of the reasons I like GETURL so much is that it reduces the
complexity for many IMAP clients (especially for mini clients).  So the URLAUTH
approach is a rather balanced proposal.  The balance also must be skewed by the
motivation of the implementers.  Forward-without-download is a feature driven
by client requirements.  The clients need it and the server vendors don't.  So
you've got to balance the complexity in favor of the server vendors if you want
it to deploy.

> It's particularly critical that we understand what the client
> thinks
> to get something done" in both cases.

The functional separation between a "message store" and a "submit server" is
critically important to a well designed system.  But I will admit that the
widespread ignorance in the client implementer community of the value this
separation provides is probably the strongest argument in favor of the push
model.  Sometimes it is true that good architecture has to be compromised to
accommodate ignorance.  But this is not one of those cases.  If we standardize
the pull model, it looks like we have strong buy-in from server implementers.
Once the servers do it, the client vendors will follow because they need the
feature.

> should say simply that I agree with this point, but feel it applies equally

The difference is the level of entanglement.  In the push model, the IMAP
server gets all the functionality and responsibility of a submit server in
addition to its own functionality.  That's a complete loss of function
separation.  In the pull model, both protocols retain their core functions and
do not inherit any functionality from the other.  The submit server simply has
the additional ability to act as a very slim IMAP client (LOGIN + STARTTLS +
GETURL), but the functional separation is retained.

                - Chris

---

From: Ted Hardie <[email-masked]>
Date: Wed, 11 Feb 2004 16:17:58 -0800

At 3:36 PM -0800 02/11/2004, Chris Newman wrote:
>pull are clearly in scope for the charter.

Yes.

>When evaluating the quality of the

This is a big leap, in my opinion.  Going from saying that we're chartered
to consider changes to both SMTP and IMAP to saying we MUST
consider changes to every transport and access protocol because
there might be some reuse doesn't fly.

>security.  Push puts _all_ the new complexity in the already complex IMAP

But from a "roles" perspective, the IMAP server already has the role
"has access to the message store".  You're assigning a new role to
the SUBMIT serve in the PULL proposal.  Whether it is a small amount
of code or not, the security model is very different.

>disk, then the IMAP server becomes an SMTP proxy.

I'm missing something.  For the case that is core to this, the IMAP server
already has the data to be forwarded on disk.  Are you comparing this
to the case where the IMAP serve would assemble pieces that were not
already at the server or provided by the client?  If so, I agree that it
would not be a good idea, and I've argued against its inclusion in PUSH
for exactly this reason.

>simpler overall solution.

I'd really like a breakdown of this.  Are you counting just the code on
the IMAP server?  It seems to me adding the functionality of the pawn ticket
code and the IMAP client code, plus the new routines needed to handle
the error conditions is going to bloat SUBMIT server code pretty severely.
Then we need to consider what the changes would be on the clients.  I
think client handling of submit+pawn ticket is a major piece of new work,
since it will have to have the pawn ticket handling code built from scratch.

>you've got to balance the complexity in favor of the server vendors 

The *users* need it.  They need forward without download to work so their
mail works in conditions of low bandwidth.  They need both clients and servers
to cooperate to get that done.

And again, I see a real risk here that operators that pull will move to
submit+http (or webdav), with a complete loss of interoperability
with IMAP.  If server vendors are sanguine about that possibility, I
can't see why.

>Once the servers do it, the client vendors will follow because they need the

See above.  To quote an old song "it ain't necessarily so".  If you do
it with PULL, you have both the risk that operators will want to
use document store rather than message store access protocols and
the risk that they will not deploy something with an untested security
model.

>GETURL), but the functional separation is retained.

I really don't see how you can see it becoming a pull based access client,
with the consequent storage and assembly problems, as not entanglement.
It sure looks like it to me.
			regards,
				Ted Hardie

---

From: Randall Gellens <[email-masked]>
Date: Thu, 12 Feb 2004 19:09:14 -0800

>  there might be some reuse doesn't fly.

I agree that we absolutely must limit the scope of the work we take 
on to the immediate problem at hand.  I'd object very much if we 
tried to do a fully generalized split composition protocol that 
handled access to arbitrary data.

However, I think there are two different uses of "consider" in the 
above quotes, and I think that may be causing some of the confusion. 
I agree our charter allows us to "consider" (that is, to decide to do 
or not to do) changes in both IMAP and SMTP/Submit.  In making 
decisions, I agree that we need to "consider" (that is, think about 
the implications) of our work on other protocols and on future 
extensions.  I'd agree that we do not want to "consider" (that, 
seriously consider doing) making changes to other protocols.
-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
When we remember we are all mad, the mysteries disappear and
life stands explained.                          --Mark Twain