Subject: Feasibility research request: POP3 and IMAP4 extension to send mail

From: Ravi shankar <[email-masked]>
Date: Mon, 17 Aug 2009 16:15:32 +0530

Okay Al,

Let me try to explain the concept, and see if i can do it better this time.

My proposal has 2 parts:

1. Designating protocols for "Client to server" and "Server to server"
communication
2. Blocking port 25. (this is being adopted by ISP, but apparently not all
as evident from the spam figures gotten from botnets)

Now the first point.

Designating protocol as Client to server and Server to server (atleast in
email realm), gives greater control in implementing changes to it (making
changes to MUAs is much easier compared to MTAs.)
Also allows to vigourously control the ports used by home PCs (which
accounts for roughly70% of botnet spam)

The second point.

This method is "fairly" in use, but may be ISPs are waiting for this to be
made as a industry standard to implement it throughtout the world.

Problem part:
--------------------

Now the classification i proposed in point 1 requires a protocol to submit
messages, in addition to POP3/IMAP4, whcih can retrieve message. Considering
SMTP is very rich in feature and extensions,
if we designate it as "Server to server" protocol,  POP3 and IMAP4, can be
extended to submit messages by themselves by using the same ports 110/143.
This can work securely on top of SSL,
which most companies have implemented for POP3 and IMAP usage.

Depending on the architecture of the POP3/IMAP in a messaging server, the
difficulty level will vary, but this will give greater control over the
protocols for future changes and certainly bring down the botnet spam.

Regards,
Ravi

Again I'm not trying to propose this as a solution against account theft and
it is out of scope of this work as well.

---

From: Dave Cridland <[email-masked]>
Date: Mon, 17 Aug 2009 12:06:51 +0100

> Let me try to explain the concept, and see if i can do it better  
Let me try to explain the response again, and see if I can do it  
better this time.

> 1. Designating protocols for "Client to server" and "Server to  

This is already the case.

SMTP and Submission are two closely related protocols, but they are  
different, and run on different ports. Submission *cannot* be used  
for botnet "shotgun" spam, since it requires authentication, even for  
local deliveries.

> as evident from the spam figures gotten from botnets)

Which is entirely possible already.

It should be noted that blocking port 25 prevents people from running  
their own mailserver locally, which is a problem for many (including  
myself) who might wish to do so for security reasons, amongst others.

Still, however, this leaves port 587 open, allowing the  
client-to-server protocol - Submission - to operate unhindered.

> changes to MUAs is much easier compared to MTAs.)

Yes, but given that all existing MUAs already support SMTP, making  
them support Submission can be as simple as allowing them to have the  
port number changed. So if you're arguing on the basis of ease of  
implementation, I think you're arguing against yourself.

> made as a industry standard to implement it throughtout the world.

I'm sure there's many ISPs who'd be happy to make email suddenly  
entirely within their control.

> messages, in addition to POP3/IMAP4, whcih can retrieve message.

Like Submission.

>  Considering

The two halves of your argument here are not simply orthogonal, but  
actually your premise argues against your conclusion, and in many  
cases continues to do so throughout this section.

Perhaps you meant:

Considering SMTP is very rich in feature and extensions, we could  
reuse the protocol on a distinct port, and repurpose it specifically  
as a client-to-server protocol only - indeed, we already have, and  
called it "Submission".

> which most companies have implemented for POP3 and IMAP usage.

This can work securely on top of TLS, which most companies have  
implemented for SMTP usage.

> protocols for future changes and certainly bring down the botnet  

Independent of the architecture of the POP3/IMAP in a messaging  
server, the difficulty level will be low, immediately leveraging  
current and future changes, and should certainly bring down the  
botnet spam.

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

---

From: Ravi shankar <[email-masked]>
Date: Mon, 17 Aug 2009 17:00:51 +0530

The whole point of my discussion is to seperate SMTP from normal client
computers, so that botnets wont use SMTP to disguise a homepc as a
registered server and send spam impersonating some domain.

Ravi

On Mon, Aug 17, 2009 at 4:36 PM, Dave Cridland <[email-masked]> wrote:

>> Let me try to explain the response again, and see if I can do it better
> this time.
>
> My proposal has 2 parts:
>>
> different, and run on different ports. Submission *cannot* be used for
> botnet "shotgun" spam, since it requires authentication, even for local
> deliveries.
>
>
> It should be noted that blocking port 25 prevents people from running their
> own mailserver locally, which is a problem for many (including myself) who
> might wish to do so for security reasons, amongst others.
>
> Still, however, this leaves port 587 open, allowing the client-to-server
> protocol - Submission - to operate unhindered.
>
> Yes, but given that all existing MUAs already support SMTP, making them
> support Submission can be as simple as allowing them to have the port number
> changed. So if you're arguing on the basis of ease of implementation, I
> think you're arguing against yourself.
>
> The second point.
>>
> I'm sure there's many ISPs who'd be happy to make email suddenly entirely
> within their control.
>
> actually your premise argues against your conclusion, and in many cases
> continues to do so throughout this section.
>
> Perhaps you meant:
>
> Considering SMTP is very rich in feature and extensions, we could reuse the
> protocol on a distinct port, and repurpose it specifically as a
> client-to-server protocol only - indeed, we already have, and called it
> "Submission".
>
> This can work securely on top of TLS, which most companies have implemented
> for SMTP usage.
>
> Independent of the architecture of the POP3/IMAP in a messaging server, the
> difficulty level will be low, immediately leveraging current and future
> changes, and should certainly bring down the botnet spam.
>
> Dave.
> --
> Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]<[email-masked]>

---

From: Dave Cridland <[email-masked]>
Date: Mon, 17 Aug 2009 12:33:06 +0100

> registered server and send spam impersonating some domain.

And the whole point of my response is that this is already the case,  
and is better achieved with Submission, not an entirely new protocol.

Dave.

---

From: Tony Hansen <[email-masked]>
Date: Mon, 17 Aug 2009 07:38:50 -0400

Submission already *is* a separate protocol from SMTP, running on a 
separate port.

	Tony Hansen
	[email-masked]

Ravi shankar wrote:
> On Mon, Aug 17, 2009 at 4:36 PM, Dave Cridland <[email-masked] 
> <mailto:[email-masked]>> wrote:
> 
>     for botnet "shotgun" spam, since it requires authentication, even
>     for local deliveries.
> 
> 
> 
>     It should be noted that blocking port 25 prevents people from
>     running their own mailserver locally, which is a problem for many
>     (including myself) who might wish to do so for security reasons,
>     amongst others.
> 
>     them support Submission can be as simple as allowing them to have
>     the port number changed. So if you're arguing on the basis of ease
>     of implementation, I think you're arguing against yourself.
> 
> 
>         The second point.
> 
>     Dave Cridland - mailto:[email-masked] <mailto:[email-masked]>
>     - xmpp:[email-masked] <mailto:[email-masked]>
>     <http://acap.dave.cridland.net/byowner/user/dwd/bookmarks/>
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Apps-Discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/apps-discuss

---

From: Ravi shankar <[email-masked]>
Date: Mon, 17 Aug 2009 17:19:13 +0530

Dave,

I dont think this is turning out to be a healthy discussion, as your
responses, instead of re-inforcing your point, is like mocking me.
If you read my original post, I'm not proposing a new protocol altogather
but a extension to the already existing ones to give greater control over
how they are used.

And  i do know the fact that botnets do not send spam using submission
protocol, as it requires authentication.
Regards,
Ravi
On Mon, Aug 17, 2009 at 5:03 PM, Dave Cridland <[email-masked]> wrote:

> And the whole point of my response is that this is already the case, and is
> better achieved with Submission, not an entirely new protocol.
>
>
> Dave.
> --

---

From: Dave Cridland <[email-masked]>
Date: Mon, 17 Aug 2009 12:57:06 +0100

> responses, instead of re-inforcing your point, is like mocking me.

Then please read the responses you're getting, because this is the  
first indication I've seen that you're reading any of them.

> but a extension to the already existing ones to give greater  

I see that, but what nearly everyone is suggesting is that instead of  
doing anything new, simply using what we have already achieves your  
stated goals.

If your goal is actually that you don't think that the functions of  
mailstore and submission should use distinct protocols, then this is  
an entirely different argument - I'd note that whilst X.400 combines  
these two functions, internet mail has historically not done so, and  
the changes required both to client and server would be impressively  
large - much larger than simply using port 587, which is largely all  
that's required for typical simple cases of using Submission.

> protocol, as it requires authentication.

So let me turn the question around.

In what way are the problems you're trying to solve not solved by use  
of Submission?

Dave.

---

From: Ravi shankar <[email-masked]>
Date: Mon, 17 Aug 2009 23:43:10 +0530

Thank you all for replying to assist me. I much appreciate it.

It would be interesting to see, how much submission is used for this purpose
in commercial mailing servers.

Regards,
Ravi
On Mon, Aug 17, 2009 at 9:49 PM, Nicolas Williams
<[email-masked]>wrote:

> On Mon, Aug 17, 2009 at 05:19:13PM +0530, Ravi shankar wrote:
> Losing an argument may well not feel like having a healthy discussion,
> but if so that's because your expectations were wrong (i.e., you
> expected that people would agree with the particulars of your idea).
>
> A healthy discussion includes the possibility of losing an argument.
> Someone who rejects that possibility cannot engage in a healthy
> discussion whenever they are on the losing end.
>
> It's still something new, and since there's already something that
> accomplishes what you propose (Submission), you've lost the argument.
>
> Then that's all we need: Submission.  You yourself admit it.  But you
> *really* want an IMAP4 and a POP3 extension, and you're not explaining
> why, and even if you did, it's not likely that you'd win the argument.
>
> But I'll do you a favor and produce an argument for an IMAP4/POP3
> extension for e-mail submission: IMAP4/POP3 connections tend to stay
> alive for long periods of time[*], whereas SUBMIT connections tend to
> stay alive just long enough to empty an Outgoing folder, so you could
> reduce the number of TLS connections needed if you just extend
> IMAP4/POP3.  But I don't think that argument is very strong: TLS
> overhead is not a problem here.
>
> [*] At least when I use mutt.
>
> Nico
> --
>

---

From: Tony Hansen <[email-masked]>
Date: Mon, 17 Aug 2009 14:43:34 -0400

Ravi shankar wrote:
 > ...
 > If you read my original post, I'm not proposing a new protocol
 > altogather but a extension to the already existing ones to give
 > greater control over how they are used.
 >
 > Dave Cridland <[email-masked] wrote:
 >
 >         The whole point of my discussion is to seperate SMTP from
 >         normal client computers, so that botnets wont use SMTP to
 >         disguise a homepc as a registered server and send spam
 >         impersonating some domain.
 >
 >     And the whole point of my response is that this is already the
 >     case, and is better achieved with Submission, not an entirely new
 >     protocol.

Ravi, what you're asking to be done was already done over a decade ago, 
but the protocol that was modified/extended was SMTP itself. Out of SMTP 
came the Submission protocol. The Submission protocol is NOT SMTP, but 
looks a LOT like SMTP, normally runs on port 587, and does things like 
requiring the AUTH command to be used, as well as some other client-only 
types of actions.

In one of your email messages, you stated a false premise:

	In the present setup, SMTP can be used by any computer connected
	to Internet to send mails directly to the server, resulting in
	spam which by normal means in indistinguishable from legitimate
	mails.

Many ISPs turned off port 25 years ago and require their email user 
clients to use the Submission protocol to send email. They reserve SMTP 
for purely server level communication, and totally eliminate it from 
end-user communication.

I would like to know what it is you're expecting to gain from teaching 
IMAP or POP to do email submission that is not already achieved by the 
ISPs blocking user-level SMTP and requiring the Submission protocol to 
be used?

For your research, you should also read some of the white papers that 
the folks at MAAWG have written on the topic, including how and why port 
25 should be blocked and how use of the Submission protocol is to be 
used in its place.

A different question that I think WOULD make an interesting research 
topic is how much harm is happening due to the few ISPs out there that 
don't *yet* require the Submission protocol to be used?

	Tony Hansen
	[email-masked]

---

From: Tony Finch <[email-masked]>
Date: Sun, 23 Aug 2009 00:58:27 +0100

On Mon, 17 Aug 2009, Tony Hansen wrote:
>
> Ravi, what you're asking to be done was already done over a decade ago, but
> the protocol that was modified/extended was SMTP itself. Out of SMTP came the
> Submission protocol. The Submission protocol is NOT SMTP, but looks a LOT like
> SMTP, normally runs on port 587, and does things like requiring the AUTH
> command to be used, as well as some other client-only types of actions.

I should add that there's an architectural reason for submission to be an
SMTP variant rather than an extension to a mail store access protocol. Any
message submission mechanism needs a way for the MUA to specify the SMTP
envelope, that is the MAIL and RCPT commands *including* any extension
keywords.

Any mail transmission feature that must work between separate sites must
be specified as an SMTP extension, because that's the inter-domain mail
protocol. In many cases (e.g. DSN/NOTARY) the submitting MUA needs to
invoke the feature (or parts of it). So if the submission protocol is not
an SMTP variant then new mail transmission features will have to be
specified twice.

There aren't many difficulties caused by loose coupling between message
submission and mail store access. Saving a copy of sent mail is not very
efficient, nor is forwarding mail, but BURL can improve this. The latency
of sending mail is higher that it could be, but this need not slow down
the user if the MUA sends asynchronously in the background.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
GERMAN BIGHT HUMBER: SOUTHWEST 5 TO 7. MODERATE OR ROUGH. SQUALLY SHOWERS.
MODERATE OR GOOD.

---

From: Nicolas Williams <[email-masked]>
Date: Mon, 24 Aug 2009 00:54:53 -0500

> envelope, that is the MAIL and RCPT commands *including* any extension

Why would you assume that an extension to IMAP for this wouldn't provide
for dealing with the envelope?  Heck, one could tunnel SUBMIT over IMAP.

There is a good architectural reason for wanting a separate mail
submission protocol: not everyone uses IMAP or POP3.  Think of mailbox
over NFS, for example.  Also, one might want to reply to an e-mail
forwarded from a different account than the one where one is reading the
e-mail, and that might not work so well if one must use the mailstore
access protocol for submission.

There's also a good reason for wanting to have the mail submission to
happen over IMAP: to defeat traffic analysis.  I don't think that's
important enough to warrant such an extension.

Nico
--

---

From: Eric Burger <[email-masked]>
Date: Mon, 24 Aug 2009 07:36:26 -0400

All this does is drive deep packet inspection, so you do not even get  
to avoid traffic analysis.

We are in agreement: there is absolutely no value to tunneling SUBMIT  
over IMAP or POP: you get less security with less functionality,  
neither of which is a desirable result.

---

From: Nicolas Williams <[email-masked]>
Date: Mon, 24 Aug 2009 07:07:17 -0500

> to avoid traffic analysis.

Well, presumably you'd be using TLS.

> over IMAP or POP: ...

Yes.