Subject: Re: [dmarc-ietf] DKIM through mailing lists (rebutting MLs won't change)

From: Elizabeth Zwicky <[email-masked]>
Date: Mon, 2 Jun 2014 17:08:54 +0000

It is true that when attackers can't use our From: lines, they
try other things. 

Empirically, it's also clear that the attackers do not like

From: "Victim Name" <[email-masked]>

as much as they like

From: "Victim Name" <[email-masked]>

based on lowered volume when the second form is unavailable. Given that,
I see no reason to believe they won't go back to the second form if it
becomes available. And, taking into account Murphy's law, previous
tactics of these attackers, and the reasonable expectations of the public,
what will happen then is that they go very aggressive
at 2:00 in the morning my time, followed by a great deal of bad press.

At this point, I do not see going to p=quarantine in the hope
that attackers won't exploit data they already have exactly the same
way they did before, even though nothing is stopping them, as a reasonable
approach. 

	Elizabeth
	[email-masked]

On 5/31/14, 7:37 AM, "Stephen J. Turnbull" <[email-masked]> wrote:

>Elizabeth Zwicky writes:
>
> > So changes that maintain effective protection for users who are
> > being targeted by attackers with addressbook information, with less
> > disruption to email that people want, are of great interest to us.
>
>How about trying "p=quarantine" with a real short TTL just in case?
>After a while you crank it back up to the current level (which is only
>1800 in any case).
>
>The argument is that, seriously, since the attackers have addressbook
>information, you're done for anyway.  They're already hard at work on
>Plan B, using "I'm writing this from my friend's account" with self in
>Sender: (should work well on Outlook users despite having on-behalf-of
>point the wrong direction), and ...  Heck, I've already thought of a
>dozen of these dodges and my name isn't even Laurence Canter.
>
>I think it's worth a check to see if the miscreants will bother to
>come back at you with the previous style of spam shot even though they
>should expect that the vast majority of their spam will get rejected
>anyway (messages apparently from a "p=quarantine" domain should be
>given a rough time as encouraged by the DMARC protocol), and the rest
>will end up in spam folders.  I would think trying to avoid DMARC
>entirely would now be the best use of their resources, so maintaining
>quarantine should be enough.  There may be some directly relevant
>recent evidence on this, since GMail is evidently promoting mailing
>list traffic from "p=reject" domains to "quarantine".  If the spammers
>know this, they could continue targeting GMail customers in their
>stolen addressbook database.  Dunno if GMail will tell Yahoo!, but you
>could ask.
>
>BTW I hope you guys are basing "p=reject" (vs. "p=quarantine") on real
>data on how often fraudulent mail that ends up in spam folders
>actually succeeds in harming the targeted victim.
>

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Mon, 02 Jun 2014 16:28:21 +0900

Tony Hansen writes:

 > I would love to see that list of multiple mitigations shared with the 
 > broader community. That would be useful information for people in the 
 > IETF,

I've shared it here in various levels of detail more than once, and
sooner or later it will be in the Mailman FAQ as I'm sure that the
brief descriptions in the admin UI will need amplification.
Basically, Mailman provides two orthogonal features:

1.  Whether to mitigate, and when to mitigate, that is the question

    a.  Don't.
    b.  Always.
    c.  Only for posters from DMARC "p=reject" domains.

2.  How to mitigate:

    a.  Replace poster with list in From:, and diddle Reply-To so that
        reply-to-poster is as convenient as possible.
    b.  Wrap the message in a MIME message/rfc822 body.

Thanks to John Levine, we'll eventually have a 2c option: append
.INVALID to the poster's mailbox in From:.

 > as well as other MLM teams not involved wherever those discussions 
 > occurred.

AFAIK there is no particular place where MLM teams meet up; there are
very few interoperation considerations.  I would think other MLM teams
would be here now if they cared (cf John Levine's comment on your post).

 > Perhaps there is one or more of those solutions that we SHOULD be 
 > recommending.

The only solution currently available I can see recommending is
banning domains that DoS third parties.  However, that isn't going to
fly on many, probably the great majority, of Mailman and ListServ
lists.  (Majordomo, smartlist, and whatever is used with qmail may
have a rather different user population, as is suggested by John's
observation.)

All the others have defects that some people consider severe, so will
have to be chosen by the list's administration.

 > And perhaps broader discussions will provide an AHA moment where we see 
 > a way to solve BOTH MLM's problems AND DMARC's problems with MLMs in the 
 > face of a p=reject policies.

My personal belief (as previously mentioned) is that that is a logical
impossibility.  But we can hope I'm wrong!

 > Are the current p=reject semantics totally correct? As has been pointed 
 > out by others, even with mail sent out from banks, there are legitimate 
 > uses of mailing lists or things like mailing lists where you want copies 
 > to be received by multiple people.

The important thing is that the banks' problems with "p=reject" can be
solved (worked around, if you prefer) by the banks, without
cooperation of third parties, and without (much) harm to third parties
(it's unlikely that there are Mailman lists where 314 bank reps will
post to the same list in a week and so knock all subscribers into
disabled state).

While it would be as nice as getting a pony for the banks' tech staff
to be able to post to this list from the well-known domain, I don't
think that's very high on their list of tasks for their techies.  Just
registering a "somebank-inc.com" domain is satisfactory, I suppose.

 > Or alternatively, perhaps p=reject needs to be redefined slightly to 
 > take advantage of specific recommended practices for mailing lists.

Perhaps.  I'm a fan of "simple protocols used judiciously"
vs. "complex protocols that try to forestall all problems" myself.

If complexity would make it possible for Yahoo! to use "p=reject"
without DoS'ing mailing list subscribers and their own posters, it
would be worth it, I suppose.

 > > SPF and DKIM are now ancient history, at least as far as Mailman
 > > development goes.  We've already done what's technically
 > > possible.
 > 
 > Since I'm not privy yet to what all GNU Mailman has chosen to do in
 > the face of SPF and DKIM, so I don't know how to evaluate that
 > statement. Sorry.
 > 
 > If you're saying that you've thrown out all use of DKIM, I consider that 
 > a sorry state of affairs.

No.  What we've done is

(1) Put up a FAQ encouraging re-signing by MTAs hosting lists.
(2) Added an option to strip the (presumably broken, we don't check
    for it) original DKIM signature.

There's no real point in MLMs doing more than that as it requires
cooperation from the domain's DNS.  OAR (which I don't think is worth
it) could be done, but I think this probably is better done by the MTA
(local MUAs and admins might wish to refer to it).

---

From: Hector Santos <[email-masked]>
Date: Mon, 02 Jun 2014 14:34:28 -0400

I'm a list software product developer. I believe our list package was 
among the
first to implement domains controls with restrictive domains. In our 
case, we used the then IETF Proposed Standard ADSP.  It followed the 
guidelines written in the 2006 DSAP (DKIM Signature Authorization 
Protocol) I-D section 3.3:

    3.3.  Mailing List Servers

    Mailing List Servers (MLS) applications who are compliant with DKIM
    and DSAP operations, SHOULD adhere to the following guidelines:

    Subscription Controls

       MLS subscription processes should perform a DSAP check to
       determine if a subscribing email domain DSAP policy is restrictive
       in regards to mail integrity changes or 3rd party signatures.  The
       MLS SHOULD only allow original domain policies who allow 3rd party
       signatures.

    Message Content Integrity Change

       List Servers which will alter the message content SHOULD only do
       so for original domains with optional DKIM signing practices and
       it should remove the original signature if present.  If the List
       Server is not going to alter the message, it SHOULD NOT remove the
       signature, if present.

The main point is that there should be no advocation or promoting what 
is effectively violating a security protocol, especially of that being 
advocated with a 5322.From rewrite.

I think its an unethical practice to be bypassing a security protocol, 
intentionally. Its bound to bite you.  And what about the domains that 
don't want you to do this?  Unless there is a permission based concept 
to do this, it is a terrible mistake to open up this door.   You won't 
be able to trust any domain and From rewrites would cause the lost of 
any protective value the message originally had.   What is to suggest 
that the receiving domains won't learn to adapt to detect these 
"rewrite/rewrap" loopholes and begin to give such senders bad 
reputation blocks?

Overall, the hurdle to is to get systems to do a LOOKUP in order to 
get permission to resign.  If you assume this is going to be the case, 
then we don't to be kludging radical methods simply to bypass a 
security the originating domain does not want you to do anyway.  Thats 
a pretty risky thing to do.  I certainly can't support these ideas to 
rewrite lacking permission to do considerations and against the wishes 
of the originating domain.

--
HLS

[1] 
http://isdg.net/public/ietf/drafts/draft-santos-dkim-dsap-01.html#anchor14

---

From: Franck Martin <[email-masked]>
Date: Mon, 2 Jun 2014 13:47:59 -0500 (CDT)

----- Original Message -----
> From: "Stephen J. Turnbull" <[email-masked]>
> To: "Tony Hansen" <[email-masked]>
> Cc: [email-masked]
> Sent: Monday, June 2, 2014 12:28:21 AM
> Subject: Re: [dmarc-ietf] DKIM through mailing lists (rebutting MLs won't change)
> 
> Tony Hansen writes:
> 

I'm not sure it is wise to encourage the use of "invalid" domains in any of the email headers. The tendency is to become more strict with malformed emails (for instance: reject if none or more than one From field), not more loose.

The use of the .INVALID is likely to create problems in the future, if not already with reputation systems.

---

From: Kurt Andersen <[email-masked]>
Date: Mon, 2 Jun 2014 19:55:39 +0000

>.INVALID to the poster's mailbox in From:.

And how long do you think it will be before some clever organization buys
the .INVALID TLD?

--Kurt Andersen

---

From: Franck Martin <[email-masked]>
Date: Mon, 2 Jun 2014 15:18:47 -0500 (CDT)

----- Original Message -----
> From: "Kurt Andersen" <[email-masked]>
> To: "Stephen J. Turnbull" <[email-masked]>, "Tony Hansen" <[email-masked]>
> Cc: [email-masked]
> Sent: Monday, June 2, 2014 12:55:39 PM

Invalid is a reserved TLD for “invalid” domains. 

http://en.wikipedia.org/wiki/Top-level_domain#Reserved_domains

But why would you accept emails from invalid domains in the first instance?

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Tue, 03 Jun 2014 20:26:44 +0900

Elizabeth Zwicky writes:

 > way

Has Yahoo! has already tried 'p=quarantine', or is that merely your
expert opinion?  (Nothing against expertise, but "experiment" beats
"expertise" 10 letters to 9 in my dictionary.)

Obviously, there is a risk to Yahoo! in experimenting.

Nevertheless, DMARC is a *private* agreement among parties using the
*public* Internet, and Yahoo!'s use of DMARC is *demonstrably* harming
Yahoo! users as well as 3rd parties in no way participating in DMARC.
Perhaps "corporate social responsibility" requires that Yahoo! accept
those risks as an experiment.

 > what will happen then is that they go very aggressive at 2:00 in
 > the morning my time,

Which is, I remind you, exactly what Yahoo! did to list subscribers
and operators.  The difference is that you will know to expect it, and
you can probably arrange for an automatic tripwire to return to
"p=reject" much more quickly than you responded to the last attack.

You can also make an additional, even more private, agreement with
AOL, Google, and Hotmail (which should cover the vast majority of
very-vulnerable-to-fraudulent-recommendations-from-yahoo-mailboxes
users out there) to give messages failing identity alignment with
yahoo.com a more-thorough-than-usual spam check.

 > followed by a great deal of bad press.

Yahoo's good press doesn't make our ugly mitigations any prettier, and
we're not brimming over with good will toward Yahoo! now -- i.e.,
we're not happy that Yahoo! gets less bad press at our subscribers'
expense.  Does that matter?  I don't know, but I can tell you that
there is a "friends don't let friends use Yahoo!" meme spreading --
not just from disgruntled MLM developers! -- and Yahoo! could probably
use more friends.

For example, the Japanese Ministry of Education immediately broadcast
warnings to dependent organizations that Yahoo! mailboxes will have
trouble with unreliable delivery and cause bounces that upset mailing
lists and sysadmins.  They explicitly recommended avoiding use of
Yahoo! accounts in connection with academic activity.

Ironically enough, this was pure FUD. :-(  yahoo.co.jp did not publish
a 'reject' (or even 'quarantine') policy (AFAIK they never have, and
as of this writing they don't).  I don't know what MEXT thought they
were doing (although I suspect what the US Trade Representative would
call "structural impediments").

Do I tell my colleagues that truth?  Oh, over beers (but I don't drink
with the departmental computer committee), of course, and my local LUG
has heard all about it and we all had a good laugh.  I am not going to
waste words on the bureaucracy, though.  For one, trying to teach our
Computer Center (forget the central bureaucrats) what they don't know
is worse than dealing with sophomores.  But this is also very useful
to me, as on my educational lists I had few Yahoo! users (all with
Asian domains), now I have none, and I just ban them going forward
-- no RFC-violating mitigation necessary.

I've told my subscribers, too, but they don't understand the
difference between MEXT FUD about Yahoo! and the draconian policies on
file-sharing software imposed by Japanese universities, so most had
already switched anyway, and none consider my ban an imposition.
So AFAICT, MEXT's FUD has been effective.

Ban unnecessary?  For now, but I don't feel like trying to figure out
how closely Yahoo! Japan is going to follow the lead of the parent
company, or when.  Your own words make me sure that Yahoo! cannot be
trusted to behave with consideration to 3rd parties or to give warning.

Furthermore, it establishes a precedent: "p=reject" is grounds for
banning a whole domain.  I'll never need to use ugly mitigations on my
university lists, even if this practice spreads beyond Yahoo! and AOL,
and I have MEXT authority for it.  So this misunderstanding suits me
perfectly.  I doubt I'm alone in that.

One more note: all but one of my former Yahoo! users are Chinese, and
they tell me the same meme is spreading there.  If you think Japanese
"structural impediments" are annoying, just wait until you have to
deal with the Chinese version.

Regards,

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Tue, 03 Jun 2014 23:16:04 +0900

> But why would you accept emails from invalid domains in the first

Because the email is legitimate, of course.  I've seen people use
"example.com" in their addresses on list posts to ensure they won't
get personal replies.  I've seen people use "[email-masked]" as
their valid return address.

That kind of thing may not happen on your lists, and you may not care
to play along with the game.  Fine.  But remember, the domain, or even
the whole mailbox, in From: is only one consideration in making that
judgment of legitimacy.

Steve

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Wed, 04 Jun 2014 00:14:37 +0900

Franck Martin writes:

 > I'm not sure it is wise to encourage the use of "invalid" domains
 > in any of the email headers.

Remember, we think *all* of the mitigations should be discouraged in
favor of not permitting posts from "p=reject" domains.  But we also
know that will be unacceptable to most of our list operators.

 > if not already with reputation systems.

We'll warn our users (list operators) of the troubles we know of.
Corrupting an existing mailbox in the header of a post in any way is
at the bottom of the list of the things we think a mailing list should
do, of course (unless it's part of the terms of service of the list --
eg, anonymous lists).

BTW, I'm not particularly worried about reputation systems for spam-
fighting.  I believe it's a fundamentally unsound idea, aiming at the
one resource that we *know* spammers put zero value on: reputation of
Internet addresses.  Both their own (because they keep those carefully
hidden), and the reputations of the hosts, individuals, and services
they parasitize.

Regards,

---

From: Elizabeth Zwicky <[email-masked]>
Date: Tue, 3 Jun 2014 17:51:27 +0000

>Obviously, there is a risk to Yahoo! in experimenting.

That risk is not one my management team is willing to accept.

But I do, in fact, have data, and that data tells me that the attackers
forging our users based on stolen addressbooks have never stopped; we are
still blocking them now. So I don't need to turn on p=quarantine to see
what will happen -- I know at least one thing that will happen. The only
question is how the volume will change.

By the way, yahoo.co.jp is an independent company with a separate mail
system and a separate account database.

	Elizabeth Zwicky
	[email-masked]

---

From: Brandon Long <[email-masked]>
Date: Tue, 3 Jun 2014 12:43:37 -0700

> know that will be unacceptable to most of our list operators.

You're being a bit free with the "we" there.

I don't think I wish to be included in the "just exclude a billion people
from mailing ilsts" category.

This thread has strayed far from utility.

Brandon

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 3 Jun 2014 14:55:13 -0700

> from mailing ilsts" category.

I think "we" there refers to Mailman users, given Stephen's context.  He's
free to correct me if I'm wrong.  I didn't get the impression he's speaking
for all operators everywhere.

I do agree that we may be straying a bit off topic.  I really think this
group needs to focus on where we go from here.  We are all aware of the
predicament in which we find ourselves and how we got here.  Re-analyzing
the past might be interesting or cathartic, but I don't think it will
identify a new path forward.

I'm excited that we have representatives from Mailman, Yahoo, Gmail, and
several other critical players engaged.  I'd rather we take advantage of
that opportunity than waste it.

-MSK

---

From: Hector Santos <[email-masked]>
Date: Tue, 03 Jun 2014 18:14:25 -0400

> Because the email is legitimate, of course.

Until it isn't.....

The purpose of the ".invalid" TLD was not for bypassing a proposed 
security protocol.  This is a malicious hack that will ultimately help 
break down one of the last remaining mail communications "inherent 
trust" headers we have left that is expected by ALL parties, all 
software, the world, in any language, society, etc, MUST NEVER be 
changed -- the 5322.From header.  You just don't do it and those that 
do, well, I consider them the Enemy of Mail.  It opens up loopholes.

In fact, I think I will add a DATA filter script to check for this TLS 
and immediately reject it.  A single line rule:

    if mail.from TLD is ".invalid" then REJECT "550 Invalid From Address"

This rule can be added to our existing incoming DATA 5322 Validation 
script that checks for such things as multiple 5322.From headers 
violation, something that DKIM needs done by all receivers to protect 
against fraudulent fake 5322.From headers.

I prefer to update my software with the above script for our MTA 
receiver rather to add logic to rewrite the 5322.From to bypass a 
security protocol in our list server.  I think I can sleep better with 
that.

-- 
HLS

---

From: Franck Martin <[email-masked]>
Date: Tue, 3 Jun 2014 19:29:47 -0500 (CDT)

----- Original Message -----
> From: "Stephen J. Turnbull" <[email-masked]>
> To: "Franck Martin" <[email-masked]>
> Cc: "Tony Hansen" <[email-masked]>, [email-masked], "Kurt Andersen" <[email-masked]>
> Sent: Tuesday, June 3, 2014 7:16:04 AM

Yes the email is legitimate, but how does the MTA knows it?

Well a bayesian filter has learned that this type of content is legitimate, and then one day a spammer uses the same content, but change one link...

Like Symantec has claimed "outrageously" that anti-viruses are dead, same for content based filtering. It means more rules have to be put in the authentication of the emails, so they can be linked to domain names that do exists, so you have some form of recourse...

I know that an email is legitimate when I see one, is not really a practical rule...

---

From: "John Levine" <[email-masked]>
Date: 4 Jun 2014 03:44:39 -0000

>Well a bayesian filter has learned that this type of content is legitimate, and then one day a spammer
>uses the same content, but change one link...

That could happen to any mail feature you care to name.

Big companies send buckets of mail with return addresses like
"donotrespond".  A non-deliverable or non-replyable From: line has
never had much connection to whether to deliver the mail.

R's,
John

---

From: Matt Simerson <[email-masked]>
Date: Tue, 3 Jun 2014 22:01:37 -0700

> never had much connection to whether to deliver the mail.

I agree that From addresses such as these will suffer no adverse deliverability issues:

[email-masked], [email-masked]

But that is not equivalent to putting non-resolvable gibberish on the right side of the @ sign. That's a reliable way of assuring that such messages do not get queued on my server. As a matter of practicality, I highly doubt that I'm unique in requiring that the sender domain (envelope and header) resolve.

Matt

---

From: Franck Martin <[email-masked]>
Date: Wed, 4 Jun 2014 01:45:37 -0500 (CDT)

----- Original Message -----
> From: "Matt Simerson" <[email-masked]>
> To: [email-masked]
> Sent: Tuesday, June 3, 2014 10:01:37 PM
> But that is not equivalent to putting non-resolvable gibberish on the right
> side of the @ sign. That's a reliable way of assuring that such messages do
> not get queued on my server. As a matter of practicality, I highly doubt
> that I'm unique in requiring that the sender domain (envelope and header)
> resolve.
> 
You are not.

I can't recall if it is in security considerations, but should likely go in the BCP, if you do DMARC you need to reduce some of the workarounds possible.

Multiple From: headers is one
Invalid or known bad domains in From: is the other one

Otherwise it is easy to send an email with a domain that contains an extra letter and bypass DMARC.

Granted, you can buy domains with a stolen credit card, or even better, just use a nearly untraceable prepaid credit card from your local store to buy domains, but this may require a bit more organization than a 15 year old may be willing to do...

With a legit domain, you have 2 investigative/complain paths: the sending IP and the From: domain.

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Wed, 04 Jun 2014 18:57:16 +0900

Elizabeth Zwicky writes:

 > still blocking them now.

Interesting.  No perceptible change at all?  I am going to have to dig
up that graph of the AOL attack.  *They* show the attack on their
users stopping dead in its tracks and returning to the pre-onslaught
level.

 > system and a separate account database.

Sure, Japanese law means it almost *has* to be that way.  But how am I
to know whether the "independent" Japanese company takes its lead from
the US parent/trademark licensor, or not?  And, as I say, the
bureaucrats (who should know that as well as you or I) chose to ignore
it.

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Wed, 04 Jun 2014 19:18:51 +0900

> You're being a bit free with the "we" there.

Sorry if you understood it that way.  I'm here as a delegate of the
Mailman project, and in this case I believe my views are
representative of a working consensus of the core developers and some
influential users.  So I wrote "we" rather than "I".

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Wed, 04 Jun 2014 20:28:45 +0900

> Yes the email is legitimate, but how does the MTA knows it?

Aha!  Precisely where this conversation should go.

The MTA *doesn't* know.  A mailing list knows more, though.  And an
MUA knows a lot more than that.  Or they could.

For bandwidth reasons, it's important (especially at Humongous ESP) to
catch abusive mail as much as you can at the boundary MTA.  But that
doesn't mean you should abdicate all responsibility to the MTA.

 > Well a bayesian filter has learned that this type of content is
 > legitimate, and then one day a spammer uses the same content, but
 > change one link...

Bad, bad, baaaaad Bayesian filter!  No dessert for you tonight!

But, while that kind of failure *does* happen, it should be rare.  In
a well-designed filter, you also need to use header features, like the
content of the From: field and DKIM signature.

The recipient is also an important feature, which should be part of
the filter.  I've heard of mailing lists where the topic word (eg, on
this list "DMARC") is used: if the word is not present, the message
goes straight to 4.5 on a "5 is spammy enough to quarantine" scale.
In the old days (2007 or so) it apparently worked quite well, dunno if
it still does.

A conventional MTA *can't* be expected to know that kind of thing
(Google and Yahoo! work differently, I'd guess).  But an MUA easily
could, and at least for PC users, there's enough storage and CPU power
to do the analysis.

 > I know that an email is legitimate when I see one, is not really a

Why not?  As long as it's the last rule in a long chain!  It worked
against spam for quite a few years, with no other help.  But more
seriously, personal filters that learn what you like and what you
think is spammy could probably do a lot more than they currently do.

The other thing that MUAs could do to help mailing lists out here is
allow signatures of MIME parts instead of the whole message.

Unfortunately that doesn't help "on behalf of" services, but I'm not
sure what to do for them.  Somebody suggested OAuth2, if that works
(for typical non-technical users) that would be very cool.

Regards,

---

From: "John Levine" <[email-masked]>
Date: 4 Jun 2014 14:39:28 -0000

>But that is not equivalent to putting non-resolvable gibberish on the right side of the @ sign. That's
>a reliable way of assuring that such messages do not get queued on my server. As a matter of
>practicality, I highly doubt that I'm unique in requiring that the sender domain (envelope and header)
>resolve.

I've been using the .invalid hack on my lists for the past month, and
I haven't seen any failure reports at all that seem to have anything
to do with it.  (Only AOL and Yahoo get the .invalid, so it's not hard
to tell if recipients are treating the mail differently.)

As I've said before, it's a user-hostile crock, but so is everything
else other than the various forms of DMARC-bypass whitelisting.

R's,
John

---

From: Hector Santos <[email-masked]>
Date: Wed, 4 Jun 2014 10:58:09 -0400

But it does know. It is not legitimate mail per the proposed security protocol. The problem and conversation should be focused on resolving the 9 years old mail integration dilemma -- the dearth of resigners not wanting to check for bad DKIM-secured transactions via a policy layer protocol.  Keep in mind that the suggested rewrite applicability for p=reject domains implies that a DNS lookup is presumed to be part of the DKIM framework.  If we can get to that level, we are home free. But unfortunately, the concept has been killed by the IETF when it decided to make ADSP historic.

--
HLS

> On Jun 4, 2014, at 7:28 AM, "Stephen J. Turnbull" <[email-masked]> wrote:
> 
> Franck Martin writes:
> 
> _______________________________________________
> dmarc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dmarc
>

---

From: "John Levine" <[email-masked]>
Date: 4 Jun 2014 15:13:43 -0000

> extra letter and bypass DMARC.

It is ALWAYS easy to send an email with a domain that contains an
extra letter and bypass DMARC.  Lookalike or "cousin" domains are
specifically not something it addresses.

Keep in mind that is not always a bug.  There are a lot of domains
that publish DMARC policies, and it is unlikely that there are no other
domains that are lexically similar and send their own legitimate mail.

R's,
John

---

From: Hector Santos <[email-masked]>
Date: Wed, 4 Jun 2014 12:17:53 -0400

John, I doubt these aol and yahoo users give a hoot of what u snuck into your small local site. The odds are high these kind of addresses were first used for junk, aliases, "throw away" addresses like most people did with these public email service bureaus. Sure, for many, these public addresses have become more of their mainstream contact address, i.e. I use my junk gmail account more for today's sign ups. But they are probably not even aware of what u did and I would lean towards the reasonable presumed fact that most people, including myself and maybe you, will not like the idea that your mail is being displayed with "invalid" indicators on a wide spread set of mail reading devices.

--
Hector Santos
http ://www.santronics.com

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Thu, 05 Jun 2014 02:43:16 +0900

Hector Santos writes:

 > [Mail From: a domain under .INVALID] is not legitimate mail per the
 > proposed security protocol.

Sorry, in this subthread, "legitimate", as used by Franck and myself,
means "delivery desired by the addressee".  If you want to insist on a
different definition, go ahead, but that's very rude to the previous
posters and confuses other readers.

Nor does DMARC say it's nonconforming; in fact, it automatically
passes identity alignment, because there's nobody who is allowed to
create domains under .invalid, so there can be no _dmarc.x.y.invalid.

I suppose it's nonconforming to RFC 5322, but I wouldn't reject mail
merely because From contains a mailbox at an invalid domain.  Of
course you can do what you want in your domain.

Like you, I think it violates the spirit of the security protocol --
but so do all mitigations that preserve the address in From in such a
way as to indicate that is the mailbox of the author, and so lend the
message whatever prestige the author may have with the recipient.
Such mitigations are clearly desired by the users of mailboxes at AOL
and Yahoo!, and some such mitigation recommended (and in the case of
Yahoo!, practiced) by the ESPs publishing "p=reject".

You can take a letter-of-the-law stance (I believe you do), and as far
as I can tell that means banning those domains from your lists (unless
the lists preserve signatures).  But that's not acceptable to our users.

 > The problem and conversation should be focused on resolving the 9
 > years old mail integration dilemma -- the dearth of resigners not
 > wanting to check for bad DKIM-secured transactions via a policy
 > layer protocol.  Keep in mind that the suggested rewrite
 > applicability for p=reject domains implies that a DNS lookup is
 > presumed to be part of the DKIM framework.  If we can get to that
 > level, we are home free. But unfortunately, the concept has been
 > killed by the IETF when it decided to make ADSP historic.

DMARC resurrected the concept of a DNS lookup to discover policy, no?

But I don't think it's as easy as you say.  It may be trivial to
publish records authorizing third parties to sign message with your
mailboxes in From:, but I doubt it will be done, at least not for the
thousands of tiny lists out there with no personal contact with the
big ESPs.

---

From: Franck Martin <[email-masked]>
Date: Wed, 4 Jun 2014 14:08:41 -0500 (CDT)

----- Original Message -----
> To: "Hector Santos" <[email-masked]>
> Cc: "Tony Hansen" <[email-masked]>, [email-masked], "Kurt Andersen" <[email-masked]>, "Franck Martin"
> <[email-masked]>
> Sent: Wednesday, June 4, 2014 10:43:16 AM

As we are in semantics... There are two versions of legitimate and people will choose one or the other depending on context. May be we need a glossary to separate the two...

May be there is legitimate in the sense of "follow the protocols and security" and legitimate in the sense "I want that email".

Anyhow... my point is that moving an human decision to the machine is difficult, this is why we rely on protocols and we need to refuse more and more emails that are not following the protocols. Once again see the paragraph about the John Postel principle in the malformed emails RFC. John said (maybe?): if the protocol is vague, be lenient, but if the protocol is clear then be strict. He did not say "accept anything, the user will sort it out"

>From the discussion I have this question: How an MTA knows the email is from a mailing list?

Also looking at bayesian implementations in say spamassassin/amavis (I looked quickly) it does not seem the bayesian rules are organised around authentication identifiers. The bayesian is left to figure what is relevant in an email. Will it pick the domain in the From for that? May be, may be not.... It seems going forward, there should be different baeysian learning, depending on the authenticated domain attached to the email. I don't think we are fully there yet.

For instance, the simplified rule should be if this is an email looking like made only by X, but not authenticated by X, junk it. I'm not sure there is any rule like this out there for lot of MTAs. My experience is that a phishing campaign using an email content of X will impact all emails that look the same regardless of source.

So to close this long tirade, I think we are moving towards a domain authenticated world of emails, using valid domains as identifiers of source.

And yes, MUA have their role to play, in filtering according to user preferences, but the tendency, is for the MUA to pass the user preferences to the MTA, so it can work on received emails even when the MUA is not connected. I don't want my mobile to do the filtering, but tell the MTA what emails I like/don't like.

---

From: "J. Gomez" <[email-masked]>
Date: Wed, 4 Jun 2014 21:16:15 +0200

On Wednesday, June 04, 2014 12:14 AM [GMT+1=CET], Hector Santos wrote:

> security protocol

Rewriting the Header-From field in mailing list traffic is not "bypassing a security protocol", because the rewritten Header-From itself still is subject to the security protocol (in this case, DMARC checking) at the Receiver's side, so the security protocol in question is not bypassed.

Regards,
J.Gomez

---

From: "J. Gomez" <[email-masked]>
Date: Wed, 4 Jun 2014 21:21:39 +0200

> never had much connection to whether to deliver the mail.

That is true, but it is not the same to obfuscate the local part in the ReturnAddress/FromHeader, than to obfuscate the domain.

Obfuscating the domain is quite suspicious because then, what entity is taking responsibility for that email? What abuse help-desk can the potential receiver recourse to?

Regards,
J.Gomez

---

From: "John R Levine" <[email-masked]>
Date: 4 Jun 2014 15:34:25 -0400

> Obfuscating the domain is quite suspicious because then, what entity is taking responsibility for that email? What abuse help-desk can the potential receiver recourse to?

The one whose DKIM signature is on the mail, of course.  Sigh.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail.

---

From: "J. Gomez" <[email-masked]>
Date: Wed, 4 Jun 2014 21:44:51 +0200

On Wednesday, June 04, 2014 9:34 PM [GMT+1=CET], John R Levine wrote:

> > That is true, but it is not the same to obfuscate the local part in
> > the ReturnAddress/FromHeader, than to obfuscate the domain. 
> > 
> > Obfuscating the domain is quite suspicious because then, what
> > entity is taking responsibility for that email? What abuse
> > help-desk can the potential receiver recourse to?  
> 

But DKIM signatures are not mandatory, not even to be able to get a pass in DMARC checking.

I see holes in that remedy.

Regards,
J.Gomez

---

From: Hector Santos <[email-masked]>
Date: Wed, 04 Jun 2014 16:30:29 -0400

> Rewriting the Header-From field in mailing list traffic is not "bypassing a security protocol", because the rewritten Header-From itself still is subject to the security protocol (in this case, DMA…

Hmmmm, thats a clear protocol security hole.

The problem has always been that the middleware was intentionally 
ignorant of the proposed author domain DKIM signing protocol in order 
to allow it to offer an open-ended 3rd party DKIM signing service.  It 
was not recognized, and the author of ADSP was not shy to advocate to 
everyone to not support it.

The impact of this ignorance was real, proven and demonstrated by me 
to highlight the fact that we needed 3rd party semantics and MLS support.

But it was yet to be felt and to close the deal, the ADSP proposal was 
make historic.   That is until YAHOO enabled a strong policy via the 
alternative DMARC and now you feel it.

Now, today, the list service is aware of the DMARC protocol but 
remains ignorant to follow it, honor it and knowing full well, 
DKIM+DMARC compliant downlink receivers will reject it, it violates 
the intent of RC2606 by using a ".invalid" TLD in order to bypass and 
preempt downlink rejections.  That was not the intent of RFC2606.

So rather than support DMARC fully, update it for 3rd party semantics 
to allow the resigner to exist in a DKIM+DMARC world, it proposes a 
hack to bypass the security.

This is a major security loophole so the good news is that it was 
highlighted and now mail receivers will be aware of needing to add new 
".invalid" check and rejection scripts.

-- 
HLS

---

From: Douglas Otis <[email-masked]>
Date: Wed, 4 Jun 2014 13:56:45 -0700

> Rewriting the Header-From field in mailing list traffic is not "bypassing a security protocol", because the rewritten Header-From itself still is subject to the security protocol (in this case, DMA…

Dear J Gomez,

Customers seeing From headers changed will not have a practical means to understand who sent the message.  Of course, this assumes who initiated content in an exchange is a critical aspect of any security related goal.  For mailing-lists, this understanding is critical to being able to review conversations as well.  For invoicing, it is the client of a service likely to be recognized by recipients.  In both of these cases, TPA-Labels offer a practical and safe solution.

If it is critical, have the TPA-Label require the OAR be checked.  This can be quickly added if that is the hold-up

Bayesian probabilities are not very reliable when dealing with clever malefactors since they can lead email into the proverbial ditch by offering irrelevant keying of transient elements. TPA-Labels help provide a clear trust relationships only author-domains are best able to determine.  To ensure accuracy, this domain related information needs to be conveyed to receivers. 

Regards,
Douglas Otis

---

From: "J. Gomez" <[email-masked]>
Date: Thu, 5 Jun 2014 00:07:32 +0200

On Wednesday, June 04, 2014 10:56 PM [GMT+1=CET], Douglas Otis wrote:

> > Rewriting the Header-From field in mailing list traffic is not
> > "bypassing a security protocol", because the rewritten Header-From
> > itself still is subject to the security protocol (in this case,
> > DMARC checking) at the Receiver's side, so the security protocol in
> > question is not bypassed.    
> 
> Customers seeing From headers changed will not have a practical means
> to understand who sent the message.  Of course, this assumes who
> initiated content in an exchange is a critical aspect of any security
> related goal.  For mailing-lists, this understanding is critical to
> being able to review conversations as well.
 
If the rewritten Header-From references the remote author Name in its description and carries the mailing list Mailbox address (i.e., the immediate author who invalidated the remote author's DKIM signature) in the Header-From, I posit customers will have a practical means to understand who sent the message through said mailing list. In fact, many MUAs only display de Header-From description/name and not the Header-From mailbox address to the user, so even less confusion would be possible in those cases.
 
Plus, a mailing list is an opt-in device. If the user chooses to be part of it, he accepts the mailing list terms of service, and if those terms involve rewriting Header-From in a DMARC-interoperable way, then that's what it is. The user is free to not use the mailing list service operated thusly if he objects to such a practice.

> If it is critical, have the TPA-Label require the OAR be checked. 
> This can be quickly added if that is the hold-up 

I, as a domain owner, do not want to go chasing my users around demanding them to declare to me what mailing lists they subscribe to, in order to publish them as whitelists/exceptions for my domain as a Sender. What if I have a user subscribed to a transgender/whatever mailing list? Do I want to know? Does he want to tell me? I as a domain owner either trust my user, or fire my user, but I don't want to have to preemptively monitor my user not do I want him to disclose to me the minutiae of his email usage. If TPA-label involves this, I see no future in it.[*]

[*] But it may well be that TPA-label works in a completely different way, I tried to read the draft but I could not understand much of it (therefore my Klingon joke, for which I apologize to you).

Regards,
J.Gomez

---

From: Brandon Long <[email-masked]>
Date: Wed, 4 Jun 2014 16:33:34 -0700

On Wed, Jun 4, 2014 at 3:07 PM, J. Gomez <[email-masked]> wrote:

> If the rewritten Header-From references the remote author Name in its
> description and carries the mailing list Mailbox address (i.e., the
> immediate author who invalidated the remote author's DKIM signature) in the
> Header-From, I posit customers will have a practical means to understand
> who sent the message through said mailing list. In fact, many MUAs only
> display de Header-From description/name and not the Header-From mailbox
> address to the user, so even less confusion would be possible in those
> cases.
>

An obvious example many users are now familiar with is Facebook
notifications, which all come from the same address and place the name of
the poster in the subject.  G+ notifications place the name in the from
header and the post subject in the subject (so they thread).  I've seen
notifications from various bulletin boards work the same way.  Y!Groups
seems to have changed to always placing the name/address in the
display-name portion of the from for all domains, not just p=REJECT ones,
and I just noticed while looking for examples.

Plus, a mailing list is an opt-in device. If the user chooses to be part of
> it, he accepts the mailing list terms of service, and if those terms
> involve rewriting Header-From in a DMARC-interoperable way, then that's
> what it is. The user is free to not use the mailing list service operated
> thusly if he objects to such a practice.
>

We were debating placing the poster's email address in the footer of the
message, so that its visible, since X-Original-From or whatever wouldn't
be, and we aren't placing the address in the From display-name (since in
our experiments, Y!Mail would reject those messages with the same DMARC
message, at least some times).

i'm beginning to lean towards liking the "embedded message" solution that
mailman has, though.  In my testing, it works mostly fine in Gmail, though
it assumes (and prefixes with) "forwarded message".  YMail's handling isn't
that pretty, however, as it ignores the headers of the embedded message
completely.

Brandon

---

From: "John Levine" <[email-masked]>
Date: 5 Jun 2014 00:03:59 -0000

>I'm beginning to lean towards liking the "embedded message" solution that

It gets delivered fine, but it's basically a MIME digest, and most
MUAs don't handle them very nicely.

R's,
John

---

From: "John Levine" <[email-masked]>
Date: 5 Jun 2014 01:39:02 -0000

>that pretty, however, as it ignores the headers of the embedded message

I meant to ask, what's your spear phish strategy here?  Do you expect
to see a valid DKIM signature on the internal message?  Or do you just
treat it as a digest and not look inside the wrapper?  Or something
else?

>From what I can see on the mailman lists, looking for the internal
DKIM signatures won't work too well, since mailman sometimes removes
parts from multipart/alternative to fix to some formatting issue.
Yahoo's mail is apparently particularly likely to need this.

R's,
John

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Thu, 05 Jun 2014 17:31:15 +0900

> But DKIM signatures are not mandatory, not even to be able to get a

If there's no DKIM signature, John's rule doesn't apply, and you fall
back to whatever your rule for unsigned mail is.

Seriously, you guys have to give up on the idea that the whole world
will follow arbitrarily strict protocols just because it makes spam
fighting simpler.  If a big ESP tries to impose them on receipt, and
the users don't receive their mail, the users will vote with their
feet and the ESP will cave.  Conversely, if you try to impose them and
a big ESP finds them inconvenient, the ESP will disobey (just as AOL
and Yahoo! have done, implicitly, with "p=reject").  The best you can
do is use protocols to gather information about the messages you
receive, and use that information appropriately (for example,
discounting the presence of a DKIM signature from a domain you have
independent reason to believe is hacked and under the control of the
Black Hats).

So, if there is a valid DKIM signature, then you know who to complain
to.  Don't you?  What's the problem?

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Thu, 05 Jun 2014 18:07:44 +0900

Franck Martin writes:

 > Anyhow... my point is that moving an human decision to the machine
 > is difficult, this is why we rely on protocols

Sure.  That's why the U.S. military relies on landmines, too.  "Smart
bombs" aren't perfect, obviously, but they're better than landmines!

DMARC "p=reject" isn't life-threatening when used by Yahoo!, of
course, but it has collateral damage in the same way.  And also in the
same way, eventually you get to a point where the filtering decisions
need to be made by a human.  Unconditional reliance on protocols is
irresponsible.

 > principle in the malformed emails RFC. John said (maybe?): if the
 > protocol is vague, be lenient, but if the protocol is clear then be
 > strict. He did not say "accept anything, the user will sort it out"

Indeed he did not say "accept anything".  But no, he did not refer to
the clarity of the protocol.  He said (paraphrased), if you're
producing output according to a protocol, conform strictly (implying:
even if it's stupid).  If you're accepting input according to a
protocol, use your judgment (implying: give the end user what they
want).  I think Google's treatment of DMARC "p=reject" is in exemplary
conformance to Postel's principle.

In some cases (often indicated by an obs- production in ABNF), we even
explicitly specify a certain degree of leniency as SHOULD.  But that's
not always possible; often it must be left up to implementations.

 > email is from a mailing list?

Because List-Post and/or List-Id is signed with a known key.

 > Also looking at bayesian implementations in say spamassassin/amavis
 > (I looked quickly) it does not seem the bayesian rules are
 > organised around authentication identifiers.

No, because the rules labeled "Bayesian" are only part of the system.
However, stuff like SPF and DKIM have add-on modules to generate auth
results and assign them to variables, and then you can create rules
which assign spam points according to the values of those variables,
and thus take advantage of those facilities.  Strictly speaking not an
application of Bayes rule, but a sort of hybrid classifier.

 > the From for that? May be, may be not.... It seems going forward,
 > there should be different baeysian learning, depending on the
 > authenticated domain attached to the email. I don't think we are
 > fully there yet. 

We never will be "there".  But I have to wonder if our time might not
be better used in building a better Bayes rather than trying to repair
the DMARC protocol (which for some use cases just ain't broke).

 > For instance, the simplified rule should be if this is an email
 > looking like made only by X, but not authenticated by X, junk
 > it. I'm not sure there is any rule like this out there for lot of
 > MTAs.

Of course there is.  I've never used it, but SpamAssassin has had an
SPF verifier for something like 10 years.  You can add a rule that
says if SPF fails, give it 100 spam points, and that will cause it to
be junked.

 > So to close this long tirade, I think we are moving towards a
 > domain authenticated world of emails, using valid domains as
 > identifiers of source.

If you mean using valid domains with valid signatures as the *only*
identifier, "I don't think I want to be part of that 'we'."

I think a lot of your feeling about this is that (1) you just don't
trust classifiers (and yes, there are plenty of real examples where
they've been fooled badly), and so (2) you're really not up on the
classifier technology.

 > And yes, MUA have their role to play, in filtering according to
 > user preferences, but the tendency, is for the MUA to pass the user
 > preferences to the MTA, so it can work on received emails even when
 > the MUA is not connected. I don't want my mobile to do the
 > filtering, but tell the MTA what emails I like/don't like.

I don't blame you.  I don't know any decent mobile MUAs.  But consider
the GMail app.  The MUA may actually be integrated with the MTA (any
Google Mail devs want to comment), I don't know, but in principle
GMail is an MTA.  The "advanced" version of the MTA part of GMail is a
*distributed system*: much of it is implemented in Javascript in your
browser, but the filter rules and so on are implemented in the Google
cloud.  I don't see any reason not to implement filtering in the
mobile MUA when mobile MUAs are actually such distributed systems.

Regards,

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Thu, 05 Jun 2014 18:30:25 +0900

John Levine writes:

 > DKIM signatures won't work too well, since mailman sometimes
 > removes parts from multipart/alternative to fix to some formatting
 > issue.

Mailman implements certain MIME structure manipulations as a per-list
policy setting.  Specifically, as a simple antivirus and resource
conservation measure, disallowed MIME types are removed.  I think this
is a deal-breaker for DKIM signatures, as MTAs aren't supposed to
break into MIME structure, and I don't think most sites want their
users doing signatures in the MUA.

But there is exactly one case where Mailman tries to fix up content
for formatting: at the list owner's option, if the only text part is
text/html, an external utility (usually lynx) may be used to convert
it to atext/plain.

There are third party patches that actually try to manipulate
formatted text.  A typical example is one to allow the footer to be
added *inside* the BODY element of a text/html part.  But AFAIK none
have made into the mainline.

---

From: Kurt Andersen <[email-masked]>
Date: Wed, 4 Jun 2014 10:48:50 -0700

> create domains under .invalid, so there can be no _dmarc.x.y.invalid.

Actually, it does not and can not pass alignment. The alignment status is
undefined which is neither a pass nor a fail.

--Kurt Andersen

---

From: Douglas Otis <[email-masked]>
Date: Thu, 5 Jun 2014 10:03:01 -0700

> Plus, a mailing list is an opt-in device. If the user chooses to be part of it, he accepts the mailing list terms of service, and if those terms involve rewriting Header-From in a DMARC-interoperab…

What tools will recipients have for relating a plethora of re-writing techniques to then review both current and prior conversations?  In addition, this may take once secure messages that may have had alignment with the Sender header field and damage a DKIM signature which MUST include the From header field.  Once the From header field is "rewritten" to comply with a "non-compliant DMARC scheme" unable to accommodate perfectly legitimate RFC conforming messages, the receiver and recipient alike are left guessing.  

IMHO, re-writing schemes to not represent practical solutions, just very poorly considered bandaids.

> I, as a domain owner, do not want to go chasing my users around demanding them to declare to me what mailing lists they subscribe to, in order to publish them as whitelists/exceptions for my domain…

As a domain owner, you would be doing your users a valuable service by making an effort to exclude rogue sources while avoiding what should be clearly evident normal use.  If you feel this opens the door too wide, a ORA condition in TPA-Label can be added for when such problems occurs.  One might assume there is a desire to ensure delivery of 100% legitimate AND desired messages and not NOT alter messages.

> [*] But it may well be that TPA-label works in a completely different way, I tried to read the draft but I could not understand much of it (therefore my Klingon joke, for which I apologize to you).

I have made a few changes to the draft hoping to make it easier to understand.  If you read it again and still have trouble, please indicate where and I'll make another pass.

The basic goal is to devise a means to establish information federations of domains with the goal of not altering messages. With this goas and therefore can be assumed to protect the underlying identities.  Once a message is re-written by an unknown third-party, nothing can be trusted.  Very soon, domains will be the only practical means to follow chains of trust.  Domains that are making use of DMARC are receiving valuable feedback that should help avoid any need to ask users about which services they are using, especially those reported as not passing an alignment test.  Before throwing the switch to p=reject, the provider should make their best effort to ensure no legitimate email is lost and that no message is modified just to suit a non-compliant DMARC scheme.  A fully transparent informal federation scheme will provide fCall this the emailmen's credo. :^)

Regards,
Douglas Otis