Subject: [Endymail] spam versus cleartext

From: Eliot Lear <[email-masked]>
Date: Sat, 06 Sep 2014 08:38:48 +0200

Hi everyone,

In the early days fo perpass we had lengthy discussions about the
tension between privacy and ability of security systems to reduce spam. 
Below is an article from a gentleman who used to work at Google, which I
thought this group might find interesting.

https://moderncrypto.org/mail-archive/messaging/2014/000780.html

Eliot

---

From: Stephen Farrell <[email-masked]>
Date: Sat, 06 Sep 2014 13:34:23 +0100

Hiya,

On 06/09/14 07:38, Eliot Lear wrote:
> tension between privacy and ability of security systems to reduce
> spam. Below is an article from a gentleman who used to work at
> Google, which I thought this group might find interesting.
> 

That's a good posting all right. And I agree that anyone who's
trying to offer a solution in this space has to meet the really
hard challenge of producing a system that doesn't allow spam to
take over.

But recall that this list is not here to design or pick a solution,
but rather to see what bits of IETF work might enable some
solutions to evolve successfully. So it is valid to e.g. say that
some key management foo or message format bar work would be
needed and valuable to standardise even while we don't have a
solution available for the spam problem.

Cheers,
S.

> 
> Eliot
> 
> 
> 
> _______________________________________________ Endymail mailing
> list [email-masked] 
> https://www.ietf.org/mailman/listinfo/endymail
>

---

From: Eliot Lear <[email-masked]>
Date: Sat, 06 Sep 2014 15:16:01 +0200

Hi Stephen,

While I think it would be fun to talk with the gentleman about his
bitcoin thinking, the key part that I intended for this group was the
situational analysis involving spam and how bad guys behave.

Eliot

On 9/6/14, 2:34 PM, Stephen Farrell wrote:
> _______________________________________________
> Endymail mailing list
> [email-masked]

---

From: Viktor Dukhovni <[email-masked]>
Date: Sat, 6 Sep 2014 14:54:09 +0000

> situational analysis involving spam and how bad guys behave.

For many users there are parties to their email service that want
to apply additional content security policies beyond the immediate
personal security interests of the user.  Sometimes it is a service
to user (less spam), other times it is corporate security policy
(block malware, detect data leakage, comply with regulatory email
archiving requirements, ...).

This is why I generally think of protecting email as two separate
problems:

	* data in motion
	* data at rest

for data in motion, I am working on more flexibility and security
with STARTTS.  For data at rest, I'd like to see LMTP servers that
support S/MIME encryption at time of final delivery, which still
allows various processing of email before it is deposited into the
mailbox.  This extends PFS to email already delivered before any
warrants are served to intercept content.  Of course it does not
protect email received while under surveillance.

While truly end-to-end email is used already, and may be used more
widely in the future, I don't expect mass adoption, there are many
obstacles beyond just the key management.

-- 
	Viktor.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Sat, 6 Sep 2014 12:23:01 -0400

On Sat, Sep 6, 2014 at 9:16 AM, Eliot Lear <[email-masked]> wrote:
> Hi Stephen,
>
> While I think it would be fun to talk with the gentleman about his bitcoin
> thinking, the key part that I intended for this group was the situational
> analysis involving spam and how bad guys behave.

Well spam prevention is where a lot of the proof of work stuff got its start.

I don't have an implementation for this yet but this is my strategy

* An endy-key is a key that is held by an end user at their device.

* All mail is encrypted at the message layer when possible but not all
mail is encrypted under an endy-key.

* Domains may publish keys for use where an endy-key is not available
or use is not authorized.

* Use of an endy-key has to be specifically authorized. This is
because (1) I have to read my mail on every device I use and that
isn't practical till I can install decryption keys easily and (2) to
enable abuse filtering.

* The only normal user interaction here would be identifying mail as spam.

* Users with really extreme privacy concerns like Greenwald/Snowden
might exchange per user endykeys but this would be the exception, not
the rule.

---

From: "John Levine" <[email-masked]>
Date: 6 Sep 2014 19:33:13 -0000

>situational analysis involving spam and how bad guys behave.

The historical discussion is pretty good, give or take some minor
sequencing errors.  (Web mail is a lot older than he makes it out to
be, for example.)

The bit at the end with bitcoins and reputation bonds is just
nonsense.  (It's longstanding nonsense -- back in 2009, one of
Satoshi's first messages to the cryptography list suggested using them
for epostage, and I responded saying that wasn't likely to be a good
idea, thereby making me a bitcoin pioneer in the eyes of the Wall
Street Journal.)

I will spare you the long rant about why it won't work, but the rant
touches on the facts that bad guys have access to more computing power
than good guys, that bitcoin transactions are not fast and not
inherently cheap, and bad guys are just as creative as good guys and
are good at subverting systems intended to keep them out.

What I think will actually happen is twofold.  Many mail operators
will tell their users that if they let the mail system look at their
mail under tightly controlled conditions (for some definitions of
tight and control) their spam filtering will be a lot better, and
about 95% of the users will say yes.  For free webmail systems, it'll
be a condition of service, since the tight control will include using
the mail contents to target ads.

For the other 5%, strong crypto will presumably include a hard to
forge signature from the sender, so they can whitelist senders with
some painful way to get new senders into your local whitelist.

I think it's a very open question whether E2E crypto on mail will turn
out to be so painful that people won't bother, outside of small
clusters of people who already know each other and so can exchange
keys in ways that are formally hopeless but in practice work OK.

R's,
John

---

From: Pete Resnick <[email-masked]>
Date: Sun, 7 Sep 2014 10:21:37 -0300

> https://moderncrypto.org/mail-archive/messaging/2014/000780.html

Along similar lines to what John Levine said,: Obviously doing e2e 
crypto gets you signatures. Since we are blue-skying here, I think it is 
perfectly plausible to say, "If you want to send me e2e encrypted 
messages, you also have to send me signed messages, and you don't or 
your signature is not in my contacts list already, your encrypted mail 
is going to bounce." I think it's possible that in the fullness of time, 
many users go to a contact-list model of email (a la IM) where the mail 
simply bounces unless it has a signature that is already in the contacts 
list.

pr

-- 
Pete Resnick<http://www.qualcomm.com/~presnick/>
Qualcomm Technologies, Inc. - +1 (858)651-4478

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Sun, 7 Sep 2014 10:09:41 -0400

On Sun, Sep 7, 2014 at 9:21 AM, Pete Resnick <[email-masked]> wrote:
> Along similar lines to what John Levine said,: Obviously doing e2e crypto
> gets you signatures. Since we are blue-skying here, I think it is perfectly
> plausible to say, "If you want to send me e2e encrypted messages, you also
> have to send me signed messages, and you don't or your signature is not in
> my contacts list already, your encrypted mail is going to bounce." I think
> it's possible that in the fullness of time, many users go to a contact-list
> model of email (a la IM) where the mail simply bounces unless it has a
> signature that is already in the contacts list.

I think that is right, but not the whole picture.

I see endymail as a subset of mail. All mail should be encrypted at
the message layer but only whitelisted mail would be e2e encrypted.

This can be done automatically as follows:

A) Some sort of discovery infrastructure maps email addresses to key hashes.
B) Some sort of discovery infrastructure maps key hashes to keys.
C) Inbound mail server has an encryption key
D) User has an endymail encryption key.

1) Alice sends a message to Bob <[email-masked]>, she doesn't know him yet.

Alice's email client uses infrastructure A and B to pull the best
public data for bob. This turns out to be a domain level record with
the mail server key (C).

Mail is opportunistically encrypted under the mail server key (C).
Mail server decrypts then (optional) re-encrypts message for delivery
to Bob.

The sent mail includes a header with Alice's encryption key
fingerprint. It is signed using either DKIM or S/MIME depending on the
settings specified in the key record.

2) Bob receives message

2a) Bob reads message hits the 'spam' key
2b) Bob reads message, does nothing
2c) Bob replies to message

Only 2c is going to result in Bob's client whitelisting Alice. His
response contains the key fingerprint that Bob needs to perform
retrieval of the key using infrastructure B.

3) Alice sends another message to Bob after his reply

Client notes that it is whitelisted and pulls the key from infrastructure B.

Note:

1) This whole process is completely frictionless. Neither Alice nor
Bob has to do anything differently.

2) This is only one point on the security spectrum. In other
applications we might allow easier access to the endy key or might not
allow access at all.

3) The reason for decoupling the key from the system via a key hash is
that it enables key rollover.

One question left open in the above is how we prevent the use of
infrastructure A as a means to obtain email addresses.

I am currently working on efficient ways to do that.

---

From: Dave Crocker <[email-masked]>
Date: Sun, 07 Sep 2014 07:09:53 -0700

On 9/7/2014 6:21 AM, Pete Resnick wrote:
>   Obviously doing e2e
> crypto gets you signatures.

No it doesn't.  As a matter of practice, it probably will, but the
technology does not require it.  Sigs are an entirely independent action
when doing object encryption.

> Since we are blue-skying here, I think it is
> messages, you also have to send me signed messages, 

So you want to eliminate anonymous communications?  Anonymity has
historical importance for some kinds of communication.

> and you don't or

The Procrustean bed always makes things simpler, and with only a few,
uhhh... shortcomings.

My point is not that signing is bad or checking against address books is
bad, but that mandating such things constrains legitimate communication
in important ways.

d/

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Viktor Dukhovni <[email-masked]>
Date: Sun, 7 Sep 2014 14:34:10 +0000

> historical importance for some kinds of communication.

Signatures can be pseudonymous.  In the scheme Phillip proposed,
where whitelisting for encryption is an action akin to adding to
the contact list or replying with an attached key, ...  There is
nothing that requires Alice's signature to assert her "true"
identity.

Since email already carries identifying information in the form of
the reply mailbox address (also pseudonymous).  The signature does
not add new constraints.  Thus to send mail that is encrypted all
the way to the user, not just the gateway, the sender needs a
pseudonymous mailbox with an associated signature plus a willingness
by the recipient to whitelist an initial communication that is not
e2e.

-- 
	Viktor.

---

From: Pete Resnick <[email-masked]>
Date: Sun, 7 Sep 2014 11:54:26 -0300

> when doing object encryption.

Signatures, just like encryption, are part of cryptography. If you are 
doing cryptography (in the way we normally do so for e2e encryption), 
you can do signatures too. That's all I meant.

> historical importance for some kinds of communication.

Pseudonymity (i.e., a signature that is not attached to a particular 
human identity) may be sufficient for most cases. Doing so would still 
require a prior-to-real-communication step of me allowing that signature 
into my whitelist/contact list/whatever. For my personal email, I am 
perfectly willing to say, "You get two choices: (1) You set up a prior 
relationship with me with your signature, and only then do you get to 
encrypt e2e; or (2) you only get to encrypt as far as my spam scanning 
service."

Now, to take a recent example, the only way for Snowden to contact me 
encrypted, unbrokered, and anonymously would involve a rather 
interesting maneuver to get into my whitelist. But I think I can live 
with that.

> The Procrustean bed always makes things simpler, and with only a few,

Indeed. And that is true of both this future environment where I would 
bounce mail without a required signature, and my current environment 
that requires me (or my agent) to accept, scan, review, and otherwise 
deal with anonymous mail. Each has....shortcomings.

> bad, but that mandating such things constrains legitimate communication

Let's not miss the point that we are *currently* constraining legitimate 
communication in important ways, as my weekly hunt through my spam 
folder and my occasional out-of-band, "Why did my mail bounce?" 
complaint amply demonstrate. I choose my tradeoffs, I get the advantages 
and disadvantages of those tradeoffs.

pr

---

From: Eliot Lear <[email-masked]>
Date: Sun, 07 Sep 2014 17:02:49 +0200

Let's talk constraints for a moment.  Does the problem get easier if we
say, “let's not even attempt to address transactional email”, and focus
exclusively on h2h?  Also, is it a goal to completely do away with
spam?  Is that a non-goal?

Eliot

---

From: Kathleen Moriarty <[email-masked]>
Date: Sun, 7 Sep 2014 11:10:25 -0400

Sent from my iPhone

> On Sep 7, 2014, at 10:09 AM, Phillip Hallam-Baker <[email-masked]> wrote:
> 
It seems the point of the original posting may have been lost.  Folks are jumping right back to solution design instead of thinking through operational costs of various approaches.  How does handing not only spam, but other attack like phishing and spear phishing evolve when e2e messaging is the norm?  We may have increased privacy, but have we helped the attacker?  End points/users can't address these and other attacks on their own.  We need to think through operational practices and try to understand how changes will effect these practices to then influence what design choices make sense.

> public data for bob. This turns out to be a domain level record with

If anyone can search for a key, spammers and spear phishing attackers have this capability too.

> settings specified in the key record.

What happens when Alice's email is compromised?  This is a common tactic today and may only increase with e2e.

I'm not arguing against e2e, but rather that we think through the effect changes will have understanding today's operational and security/incident response practices.

Best regards,
Kathleen

---

From: Dave Crocker <[email-masked]>
Date: Sun, 07 Sep 2014 08:17:16 -0700

> doing cryptography (in the way we normally do so for e2e encryption),

At the level of "if you can get keys you can do either", sure.

Arguably the nature of the trust assessment issues is different for the
two, but that could get esoteric.

> human identity) may be sufficient for most cases.

Might.  Might not.

We have little operational experience with some of these constructs in
the practical world.  I'm pushing back about all this because we need
much more cautious language about the efficacy and risks of these
approaches.

In effect, I suggest approaches be characterized as (potentially) useful
options, rather than likely or certain "solutions".  Given the way these
topics tend to be discussed, that distinction is fundamental.

> Doing so would still
> perfectly willing to say,

You (and I and everyone else on this list) are not representative users.

Most of the human factors experience in this realm is that average users
don't appreciate the extra hassle and don't perform well with the
additional tasks.  So if you want these mechanisms to scale, they
require thinking very differently about end-user load.

> and disadvantages of those tradeoffs.

You haven't heard me suggest maintaining the technical or operational
status quo and ignoring the problem, nevermind the amount of time I
spend in the world of m3aawg.org and more recently Levison's effort.

Again, my concern is ensuring adequate caution about unintended (as well
as intended) consequences.  The very consistent tendency of folk making
proposals in this space is to be quite cavalier about the human
communication downsides from imposing excessive constraints.

d/

---

From: Dave Crocker <[email-masked]>
Date: Sun, 07 Sep 2014 08:27:31 -0700

On 9/7/2014 8:10 AM, Kathleen Moriarty wrote:
> How does handing not only spam, but other attack like phishing and spear phishing evolve when e2e messaging is the norm?

Spam and other abuse continue to occupy 90-98% of the email traffic
across the net.  Life is tolerable only because the receiving operators
have gotten quite good at keeping these barbarians outside of the gate.
 Note that a change of only a few percent in filtering efficacy will
likely double the amount of spam/abuse the receivers sees.  And double
is a best case scenario.

Modern filtering engines use an amazing array of information to assess
incoming mail.  IP Address, message meta data, content, traffic
analysis, etc.  Some of the filtering does not require looking at any
content (envelope, header, body).  Some does.

To the extent that particular content is hidden from the filtering
engine, that portion of the engine is useless.  (This observation is in
the realm of "duh", but it's needed for the sequence here.)

If that efficacy is to be retained/recovered, we need to find a way to
give the filtering engine access to that data, but without compromising
the confidentiality model.

As this has been discussed in other conversations, the only way I see
that happening is to move the relevant portions of the engine into the
recipient's MUA, and then have that sub-engine consult with the main
engine.  ("Consult" is a code word for needing an open protocol between
the MUA and the filtering engine.)

This will let more bad mail get to the inbox, but would still limit how
much actually burdens the recipient.

d/

---

From: Stephen Farrell <[email-masked]>
Date: Sun, 07 Sep 2014 16:44:50 +0100

> the MUA and the filtering engine.)

I think Dave's is a fine example of what we might get from this
list - identifying work that the IETF can do that'd potentially
help a number of different e2e email solutions.

Cheers,
S.

---

From: Dave Crocker <[email-masked]>
Date: Sun, 07 Sep 2014 09:06:03 -0700

> spam?  Is that a non-goal?

Eliot,

I've no idea what characteristics of 'transactional mail' -- as compared
with... personal mail, or ? -- worth distinguishing.  So while it's a
category that is often interesting to distinguish in email security and
abuse discussions, what do you have in mind here, exactly?

MTA-to-MTA (or, rather, Boundary MTA to Boundary MTA) is almost
certainly an interesting distinction from author to recipient. For
example, that's why DKIM has succeeded at Internet scale, where PGP and
S/MIME have not.

But we need to be clear about what benefits it gets us and what it doesn't.

If, for example, one is worried about their email operator being
compelled to produce keys for decrypting user mail...

d/

---

From: "John Levine" <[email-masked]>
Date: 7 Sep 2014 16:40:25 -0000

>with... personal mail, or ? -- worth distinguishing.

It only goes in one direction.  My bank writes to me all the time, but
I never write back.  This rules out any approach that uses an exchange
of messages to establish mutual trust.  

An obvious kludge would be some way for the bank to send you their key
out of band, analogous to the notes on websites now that say please
add our address [email-masked] to your address book.

R's,
John

---

From: Watson Ladd <[email-masked]>
Date: Sun, 7 Sep 2014 09:47:31 -0700

> the MUA and the filtering engine.)

To connect to server side filtering, the filtering engine on the
server just needs to put probabilities it thinks that the message is
spam in the headers, as well as have a standardized means for the
client to report spam or ham. This doesn't seem that complicated: just
a double and some sort of forwarding info to get the backchannel.
(This assumes naive Bayes as a filter design)

> much actually burdens the recipient.

True: how much does DKIM+sender based blacklists do vs. filtering
based on content? For mobile someone raised the issue of excessive
notifications and battery life, so we do need to worry a little about
server-side. But I think it's clear we can engineer a solution to spam
that doesn't look much different than today.

Sincerely,
Watson Ladd

-- 
"Those who would give up Essential Liberty to purchase a little
Temporary Safety deserve neither  Liberty nor Safety."
-- Benjamin Franklin

---

From: "John Levine" <[email-masked]>
Date: 7 Sep 2014 17:02:07 -0000

>(This assumes naive Bayes as a filter design)

Keeping in mind that upwards of 90% of mail is spam, you're going to
be downloading an order of magnitude mail if you do the filtering on
the end device.  On a desktop with a cable connection that's probably
OK.  On my phone, it's not.

>based on content?

In terms of volume, IP blacklists are still by far the most effective,
since they knock out most botnet spam.  Other than DMARC, which is a
separate can of worms, I don't know of anyone who does message
rejection based on DKIM signatures.  There's a whole lot of body
filtering going on.

R's,
John

---

From: Eliot Lear <[email-masked]>
Date: Sun, 07 Sep 2014 19:03:45 +0200

Hi,

> abuse discussions, what do you have in mind here, exactly?

human to human versus other.

> example, that's why DKIM has succeeded at Internet scale, where PGP and

How blue sky do we want to be in this discussion?  If the answer is
"very", then we should even leave the above terminology aside – for the
moment.  But now that I see a large object headed straight toward me...
> But we need to be clear about what benefits it gets us and what it doesn't.

Yes.

> compelled to produce keys for decrypting user mail...

As a very important example. 

Eliot

---

From: Cyrus Daboo <[email-masked]>
Date: Sun, 07 Sep 2014 13:07:56 -0400

Hi John,

> OK.  On my phone, it's not.

True - which likely means you are going to want your desktop client to 
always be on and actively filtering your email so the mobile device is not 
forced to download something that will eventually get removed. That 
basically implies you are running a server (though you could call it an 
"agent") on your desktop and would want guarantees of uptime, etc. That is 
basically only a small step away from running your own mail server...

-- 
Cyrus Daboo

---

From: Kathleen Moriarty <[email-masked]>
Date: Sun, 7 Sep 2014 13:19:10 -0400

> OK.  On my phone, it's not.
I think it would be worthwhile to detail out the operational practices used
today for phishing and spear phishing as well.  The APWG does a lot of good
work with their members to help combat this problem.  Their members include
financial institution (affected by these attacks), vendors (help with
take-down services in combination with law enforcement and service
providers of mail servers, malware distributions points linked in phishing
emails, etc.) and others venders/service providers assist with maintaining
and distributing up-to-date block lists - not just for email, but also for
the malware distribution servers linked in email through the help of
browser vendors.

Understanding what they need to get their jobs done today and trying to
figure out what changes make sense could be very useful.  E2e may just
change their approach, and that may be fine, but I do think it's important
to understand the current environment and side impacts (good & bad) as we
move forward.

Maybe someone involved int he APWG can help here in a way similar to the
email that started this thread?

Thanks,
Kathleen

> Same thing for malware distribution points in phishing attacks, blacklists
provided through web browsers is very effective.  It uses very few analytic
(human) resources and impacts every browser user (enterprise or home).

The changes in operational handling of phishing may not be as bad if end
users are relied upon most anyway to report, but it would be good to
understand how we might be changing things.

Thanks,
Kathleen

-- 

Best regards,
Kathleen

---

From: "John Levine" <[email-masked]>
Date: 7 Sep 2014 17:54:24 -0000

>basically only a small step away from running your own mail server...

This sounds very unlike a design that has any chance of being
implemented for millions of civilians.

My daughter has a laptop that she carries between her apartment where
she uses it on the building's shared broadband, and her school where
she uses it on the school's network.  She uses a Gmail account, which
she also checks from her phone, particularly when she's in transit and
the laptop is turned off.  At a company where I consult, there's a
similar setup, they give everyone a laptop which they use on their
desk, typically with an external screen, mouse, and keyboard, and can
and do take home or on trips.  So there isn't anything on which to
usefully run a mail server.

It seems to me that for any sort of E2E encryption, each user is going
to have some computer somewhere that they have to trust to hold the
keys and do the decryption.  Different people will make different
decisions about which computer that is, and it's not always going to
one they can physically touch.  A lot of people are likely to decide
that their mail provider is reliable enough to use as their end.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Sun, 7 Sep 2014 14:10:02 -0400

> spam?  Is that a non-goal?

Transactional mail is the easiest to do and has the biggest payoff. So
no, not doing it does not help in the slightest.

I don't think we are going to be using this scheme to complete
transactions. But it has to be possible to use it for applications
such as:

* Correspondence between lawyers and clients.
* Sending statements for bank and brokerage accounts.
* Sending invoices.

We need those applications because they are applications that will
enable a lot of trade and drive a lot of commercial transactions. And
if you want to deploy an infrastructure that is going to cost $10
million+++ / yr to run you better have a story that you can deliver
benefits in the $ billions.

I don't think we need to solve the spam problem. But we can't open up
a hole that the spammers can drive a truck through.

---

From: Watson Ladd <[email-masked]>
Date: Sun, 7 Sep 2014 11:23:11 -0700

On Sun, Sep 7, 2014 at 11:07 AM, John R Levine <[email-masked]> wrote:
>> So what's wrong with the phone not seeing all mail, because it might
>> be spam? 90% of the emails I get come from few sending sites, who I
>> know don't spam.
>
>
> I see you're a gmail user.  You have no idea how much spam Gmail is
> rejecting or discarding before it ever gets anywhere near your inbox.

It doesn't matter how much spam is rejected overall: what matters is
how much ham is rejected, and how early the spam can be rejected, and,
in the case of the phone, how much more tightly we can draw the
filtering compared to a desktop machine. Much of the spam in the spam
folder doesn't have SPF or DKIM indications, or comes from such
venerable institutions as winning.net. Even if the content was
encrypted, it would be possible to filter a significant amount of spam
through these indications.

This can be easily quantified in the case of naive Bayesian
classifiers, by looking at the entropy gain of each signal, and doing
the usual sort of threshold picking analysis.

>
>> Furthermore, emails are small
>
>
> Yours may be.  I see megabyte spams all the time in my spam folder.

And I don't see megabyte emails in my inbox that I want to read. I
guess my phone can ignore all of those then.

>
>> Furthermore, a laptop absolutely can grab a bunch of messages and
>> filter them: the fact that it isn't on all the time doesn't make it
>> unable to do this.
>
>
> The whole point of having a mail client on my phone is that I can check my
> mail when my laptop is turned off.

Do you care if you receive a message from someone who may be a spammer
a bit late, compared to a message from someone you email back and
forth regularly? Email is not reliable.

>
> As a general rule, whatever your or my mail flow may be, it's not typical of
> everyone's, and a design that appears to work for one of us is likely not to
> work in general.

Sure, but the question is "how much won't it work"?

>
> Regards,
> John Levine, [email-masked], Taughannock Networks, Trumansburg NY
> Please consider the environment before reading this e-mail.

---

From: "John R Levine" <[email-masked]>
Date: 7 Sep 2014 14:27:30 -0400

> filtering compared to a desktop machine. ...

Really, you cannot assume that the mail you get at your gmail account is 
typical of anything.  You also have no idea how much spam Gmail doesn't 
even put in your spam folder, based on content analysis they didn't tell 
you about.

> the usual sort of threshold picking analysis.

Um, have you ever talked to people who run large mail systems about the 
way their spam filtering really works? Many of us here have done so, 
and it's a lot more complicated than it might seem.

R's,
John

---

From: Pete Resnick <[email-masked]>
Date: Sun, 7 Sep 2014 18:12:30 -0300

On 9/7/14 11:09 AM, Phillip Hallam-Baker wrote:
> On Sun, Sep 7, 2014 at 9:21 AM, Pete Resnick<[email-masked]>  wrote:
>
>    

A tangential up-level: I haven't gotten through all of the mail on the 
list yet (travel and other things have slowed me down), but I do notice 
that there has been quite a bit of "whole picture" discussion. I think 
that's fine, as blue-skying does involve thinking about how all of the 
pieces fit together. But as Stephen and I said in the first message, the 
thing we're looking for on this list is to "identify some bit(s) of work 
that the IETF could credibly do that'd improve the real-world end-to-end 
security and privacy of email. And note that 'credible' there requires 
stuff to be both technically sane and to have a sufficient set of 
capable folks interested and willing to do work." So while it's 
*possible* that a forklift replacement of email as we know it might be 
one of those "bits of work", separating out some smaller work items that 
could eventually be fit together into a shiny new system are probably 
the more interesting ideas. :-)

> B) Some sort of discovery infrastructure maps key hashes to keys.

I've been wondering about this. When I think about using crypto (whether 
encryption or signatures), it seems like requiring a discovery mechanism 
was increasing the burden. For many of my correspondents, with whom I'm 
currently communicating in the clear, a TOFU key exchange in those 
emails (authenticated out-of-band) might be a plausible mechanism.

When we think about this, do we really need to assume that we either use 
the old or the new, and never the twain shall meet?

pr

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Sun, 7 Sep 2014 17:44:16 -0400

On Sun, Sep 7, 2014 at 5:12 PM, Pete Resnick <[email-masked]> wrote:
> A tangential up-level: I haven't gotten through all of the mail on the list
> yet (travel and other things have slowed me down), but I do notice that
> there has been quite a bit of "whole picture" discussion. I think that's
> fine, as blue-skying does involve thinking about how all of the pieces fit
> together. But as Stephen and I said in the first message, the thing we're
> looking for on this list is to "identify some bit(s) of work that the IETF
> could credibly do that'd improve the real-world end-to-end security and
> privacy of email. And note that 'credible' there requires stuff to be both
> technically sane and to have a sufficient set of capable folks interested
> and willing to do work." So while it's *possible* that a forklift
> replacement of email as we know it might be one of those "bits of work",
> separating out some smaller work items that could eventually be fit together
> into a shiny new system are probably the more interesting ideas. :-)

I don't think we can get rid of SMTP right now.

But any system that allows a sender to decide between sending
encrypted and sending in plaintext can easily contain a slot that
makes it *really easy* to swap out SMTP if desired.

The bit that I really don't want to redo is the S/MIME messaging
layer.  That works fine for packaging up the bytes. It works with
attachments and all the stuff we expect from modern mail.

Any new mail security standard has to be able to serve as a
replacement for both S/MIME and PGP. Otherwise we will just continue
the standards stalemate. It is easier to graft PGP functions onto the
fully developed S/MIME message format than to try to get PGP/MIME up
to a similar level of support.

But we do have to make sure PGP users don't get left behind and they
have to be able to achieve the same level of security they can achieve
today without being required to use a CA.

> encryption or signatures), it seems like requiring a discovery mechanism was
> increasing the burden. For many of my correspondents, with whom I'm
> currently communicating in the clear, a TOFU key exchange in those emails
> (authenticated out-of-band) might be a plausible mechanism.
>
> When we think about this, do we really need to assume that we either use the
> old or the new, and never the twain shall meet?

Well the discovery mechanism could be as minimal as 'take a domain
name and a hash of the key, pull the key from
http://<domain>/.well-known/phb/<hash>

I don't want to put anything more complicated in the client though.
Because as you point out, there are likely to be lots of ideas. And in
particular there will be ideas based on TRANS like infrastructures.

>From a trust point of view, the expiry of the Harber-Stornetta patents
is a very big deal. Putting keys and/or certs into notary logs gives a
huge amount of leverage. It also means that you will want some sort of
infrastructure fishing keys out of the logs.

But just don't weld any of that into clients yet...

---

From: "John R Levine" <[email-masked]>
Date: 7 Sep 2014 19:06:54 -0400

>emails (authenticated out-of-band) might be a plausible mechanism.

Take current implementations of S/MIME and adjust them to allow
self-signed certificates in addition to (or instead of) ones signed by
a list of CAs configured into the MUA.

All done.

In my experience, the main problems with S/MIME are key distribution
and key discovery.  For key distribution, you need to go to someplace
like Comodo or Startcom to get a signed cert, which goes into your
browser, and then you need to do some grotty software specific thing
to export it from the browser and import it into the MUA.

For key discovery, in practice everyone populates their keystores with
certs from incoming signed mail, which is supposed to be safe because
it only accepts keys that are signed.  It is supposed to be possible
to get keys via LDAP from a key server, but people don't do that.

A system with key discovery, so you can send all mail encrypted to
someone, including the first one, seems more useful than one that
requires an insecure handshake first.  Key distribtion via DANE could
be a reasonable approach.

R's,
John

---

From: Viktor Dukhovni <[email-masked]>
Date: Mon, 8 Sep 2014 03:09:41 +0000

> a list of CAs configured into the MUA.

Apple's desktop Mail.app allows one (via Keychain) to bind a
particular (self-signed or not) key to a particular sender.  With
Outlook the only choice is to trust the issuing CA for all addresses.

As for what the IETF can do, maybe address sign-then-encrypt vs.
encrypt-then-sign in S/MIME unless I missed the memo and this is
no longer an issue.

The SMIMEA draft is not yet published, I don't recall seeing a
mechanism there to explicitly publish domain (gateway) keys, in
addition to or instead of keys for each user.  Perhaps wildcard
DNS entries can simulate gateway keys.

When SMIMEA records supply only the hash of a trust anchor, or of
a user certificate, rather than the key, then the relevant public
keys would have to be conveyed in band.  For example via a reply
from the user as suggested by Phillip Hallam-Baker.  Thus SMIMEA
is compatible with various more detailed models of key distribution.

SMIMEA might not directly expose the actual public keys of individual
users or even their hashes, rather it could simply publish a fixed
certification authority that issues X.509 email certificates to
the users of a domain.  This can make the initial contact leaf of
faith more secure.

It might be useful to define message headers that an MSA or border
outbound MTA should use to infer e2e encryption policy, for example
by sending mail only when a suitable encryption key is available
for a recipient.  Gateways could send signed, but unencrypted
contact requests to such users requesting a signed response (which
could be verified via SMIMEA).  Receiving user agents could respond
directly with a signed message, or insert similar headers to have
the recipient's gateway provide the requisite signature and public
key certs.

So mostly what's missing is new MUA code to support new key management
models.  Implementations may unearth the need for new headers or
other relevant standards that enable the new feature set, but my
guess is that for now what is missing is new code more than new
standards.  Perhaps this list will produce something unexpected
that will lead the way to new implementations, but my guess is 
that it will be other way around.

-- 
	Viktor.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Mon, 8 Sep 2014 09:53:34 -0400

> DNS entries can simulate gateway keys.

http://tools.ietf.org/html/draft-hoffman-dane-smime-01

> is compatible with various more detailed models of key distribution.

My Omnibroker architecture could pull key information from the DNS.
But I don't see that as a very useful approach for anything finer than
domain level granularity.

The problem is that DNS is a host and service management
infrastructure and email accounts are a user infrastructure. Most
enterprises don't want to mix the two and with good reason. The EU
'privacy' issues are remarkably similar to user concerns about spam.

If you don't want or need finer granularity than the domain, then
putting the records in the DNS is a fine approach - see DKIM. But for
the same reason that the DANE folk ignored DKIM and did their own
thing, I can't see DANE as a very useful starting point. If we were
going to go anywhere then surely going back to DKIM for another email
infrastructure would make more sense.

There is also a difference between use of keys for signature and use
of keys for encryption. With signature verification you already start
from a signed document and that can carry the key. With encryption you
start from the address of the person you want to contact and have to
assemble all the rest of the info somehow.

> the users of a domain.  This can make the initial contact leaf of

That is certainly one viable approach. And it is probably a good one
for some applications. But it only really works for the existing
enterprise use case of S/MIME.

If we have [email-masked] then she is a google employee and if I am
talking to her on Google company business then it makes all good sense
to use the Google CA. One of the weaknesses of the PGP model was that
the design ignored the fact that in many circumstances we are in
hierarchical organization structures that the CA model matches very
well.

If we have [email-masked], which could be the same Alice as before
but in her personal capacity then it might make really good sense for
Google to run up a CA for gmail (or contract out to a CA) and issue a
certificate for her validated key. But the certificate issued is only
authenticating [email-masked], it isn't authenticating Alice.

The distinction here is critical if Alice changes email providers or
wants to use her certificate to exchange mail with her bank or a whole
host of similar activities.

[If is also quite possible that gmail.com has separate non-endy keys
which can be used for message level security.]

> that it will be other way around.

No, what is missing is new Web Service code to implement the prototype
of those systems.

Changing MUAs is hard. It is hard for developers and hard for users.
Which is why I have architected my testbed so that it has an open
source proxy for outbound mail which applies message encryption under
the direction of a trusted Web service.

People use a large number of mail clients - Outlook, Windows Live
Mail, Apple Mail, iOS Mail, Thunderbird, MUTT etc. etc. The only way
to work out if a trust infrastructure is viable is to try it out on a
large scale. Do you really want to have to update seven mail clients
each time we try a test?

Splitting out the trust decisions into a Web Service mean that you can
develop your idea of a trust infrastructure and I can develop mine and
we can make both available to people who use the same code.

I would very much prefer the code that is currently in the proxy to be
in the MUA but right now I want to decouple MUA development and trust
architecture development as far as we can. We can always migrate
functions into the MUA once we understand what they should be.

What I would like to get fixed in MUAs right now is the part that is
handled abysmally on every client I have tried so far: Key management.

---

From: Werner Koch <[email-masked]>
Date: Mon, 08 Sep 2014 20:41:59 +0200

> hierarchical organization structures that the CA model matches very

Which was fixed 16 years ago with OpenPGP (RFC-2440).  OpenPGP actually
provide a superset of the features you require to implement the X.509
model.  It does not demand its use as it also does not demand the use of
the WoT or any other key validation model - this is all left to the
implementation.  Both major implementations support the hierarchical
model.

Shalom-Salam,

   Werner

-- 
Die Gedanken sind frei.  Ausnahmen regelt ein Bundesgesetz.

---

From: Brandon Long <[email-masked]>
Date: Mon, 8 Sep 2014 14:13:56 -0700

On Sun, Sep 7, 2014 at 11:10 AM, Phillip Hallam-Baker <[email-masked]

> * Sending statements for bank and brokerage accounts.

In particular, it would be great if we could "solve" the problem such that
entities are willing to send things via email that they currently won't.
 For example, my bank won't send me a copy of my monthly statement to my
email address, only a notification to look at it on their site.  This
"hole" seems to be the largest in the email as the solution to paperless
postal mail.

Brandon

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Mon, 8 Sep 2014 17:34:59 -0400

TLS has enabled e-commerce which is generally held responsible for adding an extra trillion dollars a year to global GDP.

Email security is not going to be so dramatic but it allows a lot of the holes in the e-commerce interface to be filled in. And that could be worth a few percent extra growth of e-commerce.

Lets say it is 1 percent, or rather, lets say that we can make people think it is 1 percent. That is ten billion dollars a year.

1% of that is a hundred million and we can buy a lot of servers and other infrastructure with that.

If we get the security in the specs we can easily make transactions machine readable.

Sent from my iPad

> On Sep 8, 2014, at 5:13 PM, Brandon Long <[email-masked]> wrote:
> 
> 
> 
>> On Sun, Sep 7, 2014 at 11:10 AM, Phillip Hallam-Baker <[email-masked]> wrote:
> In particular, it would be great if we could "solve" the problem such that entities are willing to send things via email that they currently won't.  For example, my bank won't send me a copy of my monthly statement to my email address, only a notification to look at it on their site.  This "hole" seems to be the largest in the email as the solution to paperless postal mail.
> 
> Brandon

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Mon, 8 Sep 2014 17:40:30 -0400

yes and no.

yes you can do that sort of thing with PGP, but being able to to it and having one way to do it that everyone follows are very different things.

It is entirely possible to configure pgp to give online/offline key separation as well. But the amount of effort required is non-trivial and the value is negligible if there is no code that would make use of it.

Support for the model requires all the issue infrastructure support for the hierechical issuers.

Sent from my iPad

> On Sep 8, 2014, at 2:41 PM, Werner Koch <[email-masked]> wrote:
> 
> implementation.  Both major implementations support the hierarchicalthere
> model.
> 
> 
> Shalom-Salam,
> 
>   Werner
> 
> --

---

From: Leo Vegoda <[email-masked]>
Date: Tue, 9 Sep 2014 16:55:41 +0100

On Mon, Sep 08, 2014 at 09:53:34AM -0400, Phillip Hallam-Baker wrote:

[...]

> But the certificate issued is only

That's quite a subtle distinction. Experience shows that most people
do not understand the difference between a web browser and a search
engine[1]. How likely do you think it is that people will understand
the difference between the authentication of an e-mail address and
the person controlling that address?

Leo

[1] https://www.youtube.com/watch?v=o4MwTvtyrUQ

---

From: Viktor Dukhovni <[email-masked]>
Date: Tue, 9 Sep 2014 16:10:38 +0000

> the person controlling that address?

And if I want to send an email with sensitive business materials
to Alice's work email address, I don't expect to securely deliver
it to "Alice", rather it is intended for Alice's "at work" mailbox.

Which is not to say that it might not be interesting to have some
types of keys that are bounnd to a particular person, and allow
that person to establish related identities hosted by various email
providers.

But even then Alice might prefer certain types of messages to be
delivered to some addresses and not to others (Alice's fetish emails
should perhaps not be sent to the office).

So the picture is rather complex, ... Neither a pure "person"
identity nor a pure "role" identity is right for all cases.

-- 
	Viktor.

---

From: Cyrus Daboo <[email-masked]>
Date: Tue, 09 Sep 2014 12:24:42 -0400

Hi Viktor,

> identity nor a pure "role" identity is right for all cases.

Yes, endymail has to cope with typical "corporate" work practices. In 
particular, when Alice goes on vacation, her boss, co-worker, assistant etc 
should be able to read her email. The legal department probably needs 
access to everyone's email. Do we want endymail to be compatible with such 
requirements?

-- 
Cyrus Daboo

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Tue, 9 Sep 2014 12:36:11 -0400

Yes

Endymail has to be compatible. But mail with a hole in it won't be Endymail.

The important thing is users can tell the difference iff it matters

Smime with a gap is still far better than what we have now. It just
can't be the only option

Sent from my difference engine

> On Sep 9, 2014, at 12:25 PM, Cyrus Daboo <[email-masked]> wrote:
>
> Hi Viktor,
>
> --On September 9, 2014 at 4:10:38 PM +0000 Viktor Dukhovni <[email-masked]> wrote:
>
> Yes, endymail has to cope with typical "corporate" work practices. In particular, when Alice goes on vacation, her boss, co-worker, assistant etc should be able to read her email. The legal department probably needs access to everyone's email. Do we want endymail to be compatible with such requirements?
>
> --
> Cyrus Daboo
>

---

From: Dave Crocker <[email-masked]>
Date: Tue, 09 Sep 2014 10:33:38 -0700

On 9/7/2014 10:02 AM, John Levine wrote:
> I don't know of anyone who does message
> rejection based on DKIM signatures

Google and Yahoo say that they use DKIM signatures as part of reputation
assessment.  That's distinct from any use of DMARC.

d/

---

From: "John R Levine" <[email-masked]>
Date: 9 Sep 2014 13:53:17 -0400

> assessment.  That's distinct from any use of DMARC.

Oh, sure, but now that's just part of content based analysis: look at the 
message text, add special sauce, and decide whether to deliver to the 
inbox or the spam folder.

I suppose we can put DKIM in the very small category of content analysis 
that could still be useful with encrypted mail bodies, along with some 
checks for header defects typical of spambots.

R's,
John

---

From: Dave Crocker <[email-masked]>
Date: Tue, 09 Sep 2014 11:01:07 -0700

On 9/9/2014 10:53 AM, John R Levine wrote:
> Oh, sure, but now that's just part of content based analysis:

You made a simple, flat assertion.

Since the context of all email filtering these days is within an
elaborate engine, a purely isolated, literal and simplistic
interpretation of your statement wouldn't make much sense.

I queried about the claim and got back counter data.

> I suppose we can put DKIM in the very small category of content analysis

Reputation assessment is a complex game, involving many factors.  But
that's not the question at hand.

The question here is whether such assessments are made against
DKIM-validated names.  The answer is yes.

d/

---

From: "John R Levine" <[email-masked]>
Date: 9 Sep 2014 14:19:22 -0400

> You made a simple, flat assertion.

Yup, about SMTP time rejection based just on DKIM failure.  Other than 
DMARC, I don't know anyone who does that.

Regards,

---

From: Dave Crocker <[email-masked]>
Date: Tue, 09 Sep 2014 11:19:16 -0700

On 9/9/2014 11:19 AM, John R Levine wrote:
> Yup, about SMTP time rejection based just on DKIM failure. 

Oh?  Well, that would have been nice to include with the assertion.

And no, it isn't obvious from either the context of your message or the
rest of the content of that message.

Given the nature of DKIM, SMTP-time use should be expected to be
minimal, at best.

d/