Subject: [dmarc-ietf] Indirect Mail Flows

From: "Silberman, Sam" <[email-masked]>
Date: Fri, 14 Nov 2014 17:50:02 +0000

In anticipation of today's DMARC WG meeting, I want to highlight one of
the many important use cases.  Specifically:
	
	Use of "unrelated" outbound SMTP servers
	Commercial email using free email address
	Newspaper Sites
		Reference wiki:  
https://tools.ietf.org/wg/dmarc/trac/wiki/MilestoneOneWiki

Past discussion around these use cases assume the email author has control
of the domain they send email from.
While it may be true for the world's digital elite (i.e. us),  it's not
true for everyone.
	There are billion(s) of people who use "free" or very low cost hosting
services to provide email.
	For many people, switching email providers is hard (or not an option).
	If their mail bounces,  they simply don't have the context to decipher
the difference between "mailbox non-existent" and "DMARC policy reject due
to non-alignment"

Most of the world does not have control over their sending domain.

Previous posts have suggested this is a small problem. I respectfully
disagree.  We need to be focusing on # of users impacted, not percentage
of mail bounced.

Take the following example:  My local PTO (Parent Teacher Organization).

	They are a volunteer non-profit.
	They have no time or expertise to deal with technical things like email
setup.
 	They can't ask the local school board for email mailbox (local policy,
bureaucracy , etc).

	They have no $$, so they use a free mail service (
[email-masked])

When their mail provider switched to p=reject
	They no longer can email some parents directly who used forwards
	They no longer can reliably send emails via a 3rd party service provider
(ESP)
	They no longer can reliably send update to community mailing list servers

IMO, this is not the 1% use case. It's bigger, much bigger.

I don't claim to have all the answers.
Telling user like this one to change mail providers solves nothing in the
long term. 
DKIM Forward signatures do have promise and solves some of the issues I've
listed above.

Ultimately, solving DMARC indirect flows for this user will get us very
close to solving indirect flows over the rest of the world.

-Sam

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Sat, 15 Nov 2014 15:55:49 +0900

Silberman, Sam writes:

 > Previous posts have suggested this is a small problem.

I'm not quite sure what "this" refers to, but I think that is unfair.
Several of us have repeatedly insisted on the importance of aspects of
the issue other than the ones that get the most discussion, despite
our personal concern with those aspects of the problem (eg, I'm an MLM
developer) and personal lack of experience with other 3rd-party email
services and backed-up-against-the-wall-by-mail-abuse large providers.

Beyond acknowledging the need, advocacy needs to be informed by
expertise in the particulars of these issues.  If you have that
expertise, I for one am all ears, and I suppose so is everyone else.
It's been sadly deficient so far.

 > We need to be focusing on # of users impacted, not percentage of
 > mail bounced.

 > [email-masked])

which is a specifically deprecated use-case in the DMARC document (and
at least some such services are well-aware that what they are doing is
a Bad Idea[tm]).  Should we spend effort specifically on remediating
foot-shooting behavior by mailbox-provider services?

 > Telling user like this one to change mail providers solves nothing
 > in the long term.

Of course it does make a difference, though.  If enough users switch,
that email/portal service will awaken to the need for solutions like
DKIM-Delegate, DKIM-Conditional, and TPA-Labels, or alternative
solutions to their specific spam problems.  Otherwise I imagine they
will be most comfortable continuing to throw their problems over the
fence into our backyards, as they have been doing.  Changing that
benefit-cost proposition is essential to getting implementation of
solutions effected.

So far these services have contributed nothing helpful to the
discussion of design of protocol improvements that I've seen; they
clearly don't see a profitable (for them) way forward from the status
quo.  And the most active contributor from the DMARC-using operator
group is an advocate of positions that I would summarize as "typical
MLMs and 3rd-party services are broken and need to fix themselves to
adapt to a DMARC 'p=reject' world".

 > Ultimately, solving DMARC indirect flows for this user will get us
 > very close to solving indirect flows over the rest of the world.

But *we* can't *solve* indirect flows.  All *we* can do is provide a
protocol that mitigates the problem in theory.

To have an effect, that protocol must be adopted by the same folks who
created the problem and are busily telling 3rd parties to fix their
service models, and thanking the 3rd parties for behavior clearly not
conformant to the most basic of RFCs.  I don't see why we can expect
them to stop doing these things -- DMARC p=reject been quite effective
in stopping some very dangerous spam/phishing, and blame the victim
has convinced many of their users that the problem is in the 3rd-party
service models, and those users turn around and complain to lists and
other indirect mail services.

Viz. the recent post to this list, requesting that list tags in
Subject and footers containing detailed contact information no longer
be added to list posts.  That the poster would take that position
doesn't surprise me: he's advocated that same measure on Mailman lists
as well.  But that other members of this WG would give even qualified
support shows a clear lack of confidence that a solution attractive to
the 'p=reject' freemail providers will be found *and* implemented.

---

From: Douglas Otis <[email-masked]>
Date: Mon, 17 Nov 2014 18:48:07 -0800

> foot-shooting behavior by mailbox-provider services?

Dear Sam and Stephen,

A draft may attempt to dissuade assertion of disruptive policies (i.e. p=quarantine, or p=reject) against typical email use. However, this effort could be seen as explaining to a 900lb gorilla where they should sleep.  Justifications heard from those at DMaRC meeting were:

-|- Disruptive policies protect their "brand" by disallowing other sources use of their domain in From header fields.

-|- Disruptive policies protect their customers from being "spoofed" by other domains.

-|- Authorization schemes can never work because malefactors quickly jump from domain to domain.

-|- Forwarded email going to a different domain, even if approved on a domain/user basis, is too difficult to manage.

In other words, say goodbye to email's agility as a result of restrictive policies placed on general use email accounts by AOL, Yahoo, and others as they employ policies originally intended to protect transactional email.  Those promoting DMaRC claim this scheme now protects 60% of all inboxes from being phished. 

As a result, we can expect those being disrupted will arrange the From header field information into the Reply-To header field and ignore 
http://tools.ietf.org/html/rfc5322#section-3.6.2 
which states: "In all cases, the "From:" field SHOULD NOT contain any mailbox that does not belong to the author(s) of the message."

DMaRC simply does not offer a practical scheme that permits "alignment" of the From with either DKIM or SPF for third-party services.

> benefit-cost proposition is essential to getting implementation of

Internet access often includes email accounts facilitating Internet services.  Most users are reliant on their Internet providers to restore services when problems are experienced.  As such, most are reluctant to modify services offered, even when there may be a means to substitute email-addresses from different providers. Typically no examples are given leaving most unaware of possible options.  This should be expected since such options would interfere with the marketing of ads.

In addition, not every email service relies on SPF or DKIM.  Both of these protocols offer weak protection which has led to a rapid uptake of DANE/SMTP.  In addition, without TPA-Labels, DMaRC is unable to align with most third-party services.  Mailing-lists that tag subject lines and flatten messages actually provide safer services by making messages less suitable for exploitation.  It is a sad outcome, since DMaRC offers feedback allowing various domains a means to monitor and identify compromised accounts. Use of the TPA-label scheme would allow an immediate method to omit abusive domains and alert users to a problem while selectively permitting improved authentication schemes.  Domains offering bundled email combined with free accounts remain abuse prone.  Any risk associated with authorizing a third-party domain is offset by improved feedback able to achieve an immediate remedy, when compared against abuse emitted directly from the DMaRC domain that can be replayed to an unlimited number of recipients for days at a time. 

> adapt to a DMARC 'p=reject' world".

Sadly, I agree.  We even offered to help manage the use of the TPA-label scheme.  There was a concern regarding the Forwarding of email from one domain to another which might require special handling. It seems this need to be explored further.

> the 'p=reject' freemail providers will be found *and* implemented.

Agreed.  Over time, domains too big to block causing work for others may soon find that status fading.  Only then, does it seem likely progress can be made.  I'll repeat an offer to help manage handling this very serious problem.   It seems that offering even weaker forms of DKIM is not something that would be safe.  An alternative solution might involve using the Outlook scheme of "on behalf of" or displaying the Sender header field (an available option with many MUAs).  A DMARC flag could then safely permit inclusion of Sender to satisfy alignments when being used for general purpose user email accounts.  This would give most third-parties a simpler solution.  TPA-Label could also allow selective deployment of the Sender option as well. 

Regards,
Douglas Otis

---

From: Elizabeth Zwicky <[email-masked]>
Date: Wed, 26 Nov 2014 18:08:16 +0000 (UTC)

By my calculation, purely indirect mail -- mail that never authenticated -- is a more frequent problem than mail that was broken along the way. 

If I take a day's worth of data for a couple of end-user domains at p=reject and average them together, and the same for p=none, I get this table:

        DMARC pass    DMARC fail
                      DKIM broken    DKIM absent
p=reject    95.32%        0.27%        4.41%
p=none      30.88%        1.02%        68.10%

Much of the absent DKIM, regardless of policy, is horrible mail you wouldn't want to deliver. But even so, there's more good mail in that pile than in the broken DKIM pile.

But you should notice that the ratios are also starkly different for p=reject and p=none; there's about 4 times as much broken DKIM mail at p=none as there is at p=reject. There's more than 15 times as much absent DKIM mail (and yes, the p=none domains I average do consistently sign their outbound). So, in practice, it looks like the absent-DKIM mail is faster to change than the broken-DKIM mail. 

I think both problems need attention. The broken-authentication problem is small, but there's no current solution to it that preserves desired semantics, and the places it happens are important. 

The never-authenticated problem is not going to be solved by the same solutions, it also affects good mail, and it also involves semantic changes or significant workarounds. In general, the workarounds are more available, because the good never-authenticated mail is strongly dominated by commercial entities; that opens a range of solutions that are less available for many broken-authentication cases. In practice, Intuit's purely third-party mail was not affected in the long term, but was interrupted for a few days. 

    Elizabeth Zwicky

On Friday, November 14, 2014 9:52 AM, "Silberman, Sam" <[email-masked]> wrote:

-Sam

_______________________________________________
dmarc mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/dmarc

---

From: Franck Martin <[email-masked]>
Date: Wed, 26 Nov 2014 15:23:15 -0600 (CST)

----- Original Message -----
> From: "Elizabeth Zwicky" <[email-masked]>
> To: "Sam Silberman" <[email-masked]>, [email-masked]
> Sent: Wednesday, November 26, 2014 10:08:16 AM
> Subject: Re: [dmarc-ietf] Indirect Mail Flows
> 
> 
> 
> By my calculation, purely indirect mail -- mail that never authenticated --
> is a more frequent problem than mail that was broken along the way.
> 
> If I take a day's worth of data for a couple of end-user domains at p=reject
> and average them together, and the same for p=none, I get this table:
> 
> 
>         DMARC pass    DMARC fail
>                       DKIM broken    DKIM absent
> Much of the absent DKIM, regardless of policy, is horrible mail you wouldn't
> want to deliver. But even so, there's more good mail in that pile than in
> the broken DKIM pile.
> 

My experience points to the difficulty of DKIM signing bounces. It is notoriously known that postfix cannot DKIM sign the messages it generates(MDN).

I wonder if you can pull data to show how many bounces are in the DKIM absent column, they are easy to identify: the Envelope From is empty

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 26 Nov 2014 14:07:40 -0800

On Wed, Nov 26, 2014 at 1:23 PM, Franck Martin <[email-masked]>
wrote:

> My experience points to the difficulty of DKIM signing bounces. It is
> notoriously known that postfix cannot DKIM sign the messages it
> generates(MDN).
>

sendmail also has this limitation.  The reason for both is that the plugin
interface they provide is typically how DKIM signing is accomplished, but
the interface is by definition only applied to incoming SMTP connections
(including from localhost, for outgoing mail), and MDNs are generated
internally, bypassing the SMTP server code.

-MSK

---

From: Franck Martin <[email-masked]>
Date: Wed, 26 Nov 2014 16:29:35 -0600 (CST)

----- Original Message -----

> From: "Murray S. Kucherawy" <[email-masked]>
> To: "Franck Martin" <[email-masked]>
> Cc: "Sam Silberman" <[email-masked]>, [email-masked],
> "Elizabeth Zwicky" <[email-masked]>
> Sent: Wednesday, November 26, 2014 2:07:40 PM

> On Wed, Nov 26, 2014 at 1:23 PM, Franck Martin < [email-masked] >
> wrote:

> internally, bypassing the SMTP server code.

I guess these two (sendmail and postfix) are the backbone of many appliances... 

Therefore it is important to read http://trac.tools.ietf.org/html/rfc7208#section-10.1.3 on how to setup SPF to work with bounces. 

I know (or have known) several large properties that did not have this setup. Unfortunately, bounces are not very visible and it is hard to stop sending emails to an invalid address if you cannot receive bounces due to policy.

---

From: José Ferreira <[email-masked]>
Date: Thu, 27 Nov 2014 00:35:35 +0000 (WET)

----- Original Message -----

> Sent: Wednesday, November 26, 2014 10:07:40 PM

> internally, bypassing the SMTP server code.

Funny you mention it 

Postfix doesn't have that issue. It can sign bounces, opendkim rejects the default From address provided by Postfix. 

opendkim[5120]: 3jnzZZ1h1Lz2pG1f: can't parse From: header value ' MAILER-DAEMON (Mail Delivery System)' 

With DefaultSender set you can overcome this. 

José Borges Ferreira 
AnubisNetworks

---

From: Franck Martin <[email-masked]>
Date: Thu, 27 Nov 2014 13:15:38 -0600 (CST)

----- Original Message -----

> From: "José Ferreira" <[email-masked]>
> To: [email-masked]
> Sent: Wednesday, November 26, 2014 4:35:35 PM

> Postfix doesn't have that issue. It can sign bounces, opendkim rejects the
> default From address provided by Postfix.

> With DefaultSender set you can overcome this.

see point 4 on http://www.postfix.org/MILTER_README.html#limitations 

We are not talking about bounces passing through postfix, but bounces generated by postfix (or sendmail)

---

From: José Ferreira <[email-masked]>
Date: Thu, 27 Nov 2014 19:46:13 +0000 (WET)

> From: "Franck Martin" <[email-masked]>
> To: "José Ferreira" <[email-masked]>
> Cc: [email-masked]
> Sent: Thursday, November 27, 2014 7:15:38 PM

> generated by postfix (or sendmail)

Nice catch, I'll post that bug on Postfix list . check: 

http://www.postfix.org/postconf.5.html#non_smtpd_milters 
http://www.postfix.org/postconf.5.html#internal_mail_filter_classes 

Here is the relevant parts in the postfix setup: 

internal_mail_filter_classes = bounce 

non_smtpd_milters = inet:127.0.0.1:8888 
bounce_size_limit = 1 

I use this dedicated openDKIM milter instance for this purpose to override all DSNs unconditionally. 
The From in the bounce message may be a problem for DMARC alignment and to use with SigningTable/KeyTable in openDKIM. 
To solve that you can copy /usr/share/doc/postfix-doc/examples/bounce.cf.default to /etc/postfix/bounce.cf , edit the file and change the From header. 
Then add to main.cf : 

bounce_template_file=/etc/postfix/bounce.cf 

For openDKIM i use this config. 

# /etc/opendkim-dsn.conf 

Syslog yes 
LogWhy yes 
UMask 002 

InternalHosts 127.0.0.1 
Domain example.net 
KeyFile /etc/postfix//dkim.key 
Selector dsnFsdk9 

Mode s 
On-BadSignature accept 
On-DNSError accept 
On-InternalError accept 
On-NoSignature accept 
On-Security accept 

Canonicalization relaxed/relaxed 
SignHeaders date,from,subject,to,auto-submitted,mime-version,content-type,message-id 

AutoRestart True 
AutoRestartRate 10/1m 

LogResults yes 

Hope it helps 

José Borges Ferreira 

Anubisnetworks