Subject: [dmarc-ietf] Solution for DMARC disruption of normal email use while still offering its normal protection

From: Douglas Otis <[email-masked]>
Date: Tue, 27 May 2014 19:49:26 -0700

Dear DMARC WG,

A draft has been submitted for review.  It covers past failures while also providing a path forward.

I have experience with similar systems operating at much higher scale without difficulty or using much in the way of resources.  Serving several very large ISPs worth of users making queries against every received message that then returned about 2 billion unique resource responses. Originally, the service was free.

In the wake of a massive compromise of accounts, some fairly large ISPs are doing perhaps the only thing that is not (yet) ignored, DMARC.

However, this new scheme only needs to sustain queries against already validated third-party domains, but that then fail DMARC alignment assertions. The number of resource records likely needed by large ISPs will be in the 10s of thousands.  For smaller domains, this will likely only be a hand-full.  Domains asserting DMARC alignment practices are receiving cooperative feedback from many receivers who are also acting on behalf of these domains to either reject or quarantine non-aligned messages.  Comparing this feedback against their own outbound logs should permit fairly automatic alignment exception list creation that can then be kindly offered to their cooperative receivers.  These record permit several mitigation strategies in the case of trouble. This scheme should reduce the amount of feedback collected or support required to deal with broken services.  This can be done by creating an informal federation of third-party providers.  Perhaps one of the requirements for being included in the federation would be to provide normal DMARC feedback. ;^)

http://tools.ietf.org/pdf/draft-otis-tpa-label-00.pdf

Regards,
Douglas Otis

---

From: Brandon Long <[email-masked]>
Date: Wed, 28 May 2014 00:07:49 -0700

So...

I think this buries the lede a bit more than the OAR suggestion does.  I
could imagine something like this for broadcasting who a service trusts the
OAR header from.

This basically would require someone like Yahoo/Gmail to host some 30k DNS
records granting any one of those third party services the ability to send
mail for yahoo.com/gmail.com.

For a service at this scale, you'd need to only do this for places where
you "trust" their Authentication-Results header.  There is a potential
issue of conflicting AR headers, which is one benefit of the OAR.

Its not clear to me that gmail.com needs to tell another service to trust
the OAR from a given third party, the choice to trust that service could
easily be up to the receiving service.

Finally, doesn't this imply a potentially large number of DNS queries?
 Given the various mechanisms for finding the domain, and given that you
have to look up the tpa record to know whether that mechanism is supported,
it would seem you would need to look up tpa records for up to 7 possible
domains, and potentially more with sub-domain checking.

Brandon

On Tue, May 27, 2014 at 7:49 PM, Douglas Otis <[email-masked]> wrote:

> I have experience with similar systems operating at much higher scale
> without difficulty or using much in the way of resources.  Serving several
> very large ISPs worth of users making queries against every received
> message that then returned about 2 billion unique resource responses.
> Originally, the service was free.
>
> In the wake of a massive compromise of accounts, some fairly large ISPs
> are doing perhaps the only thing that is not (yet) ignored, DMARC.
>
> However, this new scheme only needs to sustain queries against already
> validated third-party domains, but that then fail DMARC alignment
> assertions. The number of resource records likely needed by large ISPs will
> be in the 10s of thousands.  For smaller domains, this will likely only be
> a hand-full.  Domains asserting DMARC alignment practices are receiving
> cooperative feedback from many receivers who are also acting on behalf of
> these domains to either reject or quarantine non-aligned messages.
>  Comparing this feedback against their own outbound logs should permit
> fairly automatic alignment exception list creation that can then be kindly
> offered to their cooperative receivers.  These record permit several
> mitigation strategies in the case of trouble. This scheme should reduce the
> amount of feedback collected or support required to deal with broken
> services.  This can be done by creating an informal federation of
> third-party providers.  Perhaps one of the requirements for being included
> in the federation would be to provide normal DMARC feedback. ;^)
>
> _______________________________________________
> dmarc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dmarc
>
>

---

From: "John Levine" <[email-masked]>
Date: 28 May 2014 16:37:23 -0000

>easily be up to the receiving service.

Good point.  That's why I keep saying that one or a few shared
DMARC-bypass whitelists would work a lot better than anything
per-sender.  The set of senders where it makes sense to skip DMARC
checks for Yahoo or AOL or Gmail addresses are likely to be the same.

Also, considering the complete lack of interest that two large mail
providers have shown in mitigating the costs of their DMARC policy
decisions, it seems pretty unlikely that they'd implement anything
like this for their own domains.

>Finally, doesn't this imply a potentially large number of DNS queries?

For per-user lists, hard to say.  For a shared list, it should be
insignificant, no more than one per message.

R's,
John

---

From: Douglas Otis <[email-masked]>
Date: Wed, 28 May 2014 11:20:28 -0700

On May 28, 2014, at 12:07 AM, Brandon Long <[email-masked]> wrote:

> So... 
> 
> I think this buries the lede a bit more than the OAR suggestion does.  I could imagine something like this for broadcasting who a service trusts the OAR header from.
> 
> This basically would require someone like Yahoo/Gmail to host some 30k DNS records granting any one of those third party services the ability to send mail for yahoo.com/gmail.com.
> 
> For a service at this scale, you'd need to only do this for places where you "trust" their Authentication-Results header.  There is a potential issue of conflicting AR headers, which is one benefit of the OAR.
> 
> Its not clear to me that gmail.com needs to tell another service to trust the OAR from a given third party, the choice to trust that service could easily be up to the receiving service.
> 
> Finally, doesn't this imply a potentially large number of DNS queries?  Given the various mechanisms for finding the domain, and given that you have to look up the tpa record to know whether that mechanism is supported, it would seem you would need to look up tpa records for up to 7 possible domains, and potentially more with sub-domain checking.

Dear Brandon,

http://tools.ietf.org/html/draft-kucherawy-original-authres-00

You'll see statements in the TPA-Label draft about including authentication-results and indeed this new header should be referenced as well.  Sorry about the omission. When there is a problem, the DMARC domain is still unable to respond to what might cause expensive damage when misplaced trust leads to customer attrition whenever uncontrolled sources are allowed to flood inboxes spoofing their identity.  The TPA-Label draft permits an effective response in minutes.

TPA-Label does not imply a large number of DNS transactions, unlike SPF or even DKIM.  TPA-Label is a single transaction directed toward a DMARC domain. There should only be one DMARC domain recognized per message.  Of course, as with any DNS transaction, this can be redirected to other domains.  It is still a single transaction, unlike SPF which can result in more than 100 transactions with any number of additional redirections involving other domains completely unrelated to domains seen in the exchange.  The TPA-Label can assert which validation methods should be used to help further reduce the number of DNS transactions needed. 

A shared DMARC-bypass whitelists would remove control away from the DMARC domain that has an incentive to ensure protection.  Community white-lists would make responding to any abuse impractical, to say the least.  In today's environment, an ability to respond should not be overlooked.

A TPA-Label authorization would not be granting anyone the ability to send email for a domain.  This authorization simply permits alignment exceptions for specific third-party services that must still permit the validation of their own domain.  The authorization can also stipulate third-party domains must include their domain in a Sender or a List-ID.  If a problem is seen by any of their messages, the DMARC domain can retract authorization in minutes.  Those who apply DMARC alignment checks would be able to make exceptions for these third-party messages based on advice given directly from the DMARC domain.

TPA-Labels as a disruption remedy can be done fairly rapidly since this would involve only those implementing DMARC.  Changing 30K mailing lists where one might be running in the basement of a local church should be expected to take much longer.

Google already provides free recursive DNS for anyone to use.  It seems possible, Google could even get the ball rolling by offering a "_smtp._tpa.gmail.com" zone that others might wish to reference.  This would allow an easy and immediate solution for any DMARC domain willing to allow Google to manage their third-party services.  TPA-Labels will require the support of some large ISP before is likely to gain adoption.  I'll update the draft to include draft-kucherawy-original-authres-00.

Regards,
Douglas Otis

---

From: Tim Draegen <[email-masked]>
Date: Wed, 28 May 2014 15:51:14 -0400

> like this for their own domains.

We (the people behind the machines) need to stop the above misinformation.  Yahoo posted this:

  http://yahoomail.tumblr.com/post/82426900353/yahoo-dmarc-policy-change-what-should-senders-do

.. it includes an open invite to support developers.  The problem that Yahoo has now is that not enough people know about their efforts to fix things.

*OR*, people might not know how to fix things.  There's that, too.
=- Tim

---

From: Tim Draegen <[email-masked]>
Date: Wed, 28 May 2014 16:13:55 -0400

> checks for Yahoo or AOL or Gmail addresses are likely to be the same.

Doug,

I read through the spec, and it is clear a lot of work went into it.  I think I echo Brandon and John's above opinions.

From my PoV, there exists an immense pile of work to get through before the draft under discussion becomes a solution.  Aside from support, tooling, getting senders to deploy accurately and getting receivers to perform additional checks.. what is missing is the justification for the additional work.

DMARC is a tradeoff between keeping things as simple as possible (as unnecessary complexity acts as a giant barrier to adoption), building on existing technologies (as new code/libraries in the world of email means tacking on additional calendar years), and solving a problem that hurts enough to warrant doing anything at all.

I don't believe TPA-Label hits the mark between "solving a big hurt" and "simple".  IOW, it's too complicated for the amount of pain it would resolve.  Just my opinion, take care,
=- Tim

---

From: "John R Levine" <[email-masked]>
Date: 28 May 2014 16:33:36 -0400

> .. it includes an open invite to support developers.  The problem that Yahoo has now is that not enough people know about their efforts to fix things.

Hi, Tim.  That page is an excellent example of the large mail providers' 
complete lack of interest in mitigating the costs of their DMARC policy. 
That web page is full of suggestions about how other people can spend 
their time and money to make changes to their existing software that make 
the software work worse and break features that their users depend on, 
solely to work around DMARC.  If the providers were interested in 
mitigating the damage, that would involve spending their own time and 
money so the people who they've hurt don't have to.

> *OR*, people might not know how to fix things.  There's that, too.

I have been in extensive correspondence with the people who develop and 
maintain the Mailman discussion list software, and can only find that 
suggestion gratuitously insulting.  They know as much about e-mail as 
anyone, they've been tearing their hair out trying to figure out how to 
deal with the problems that the two mail providers have created, they've 
implemented and tested all sorts of workarounds including several not on 
that web page, and they agree that they all stink.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail.

---

From: Tim Draegen <[email-masked]>
Date: Wed, 28 May 2014 17:07:42 -0400

On May 28, 2014, at 4:33 PM, John R Levine <[email-masked]> wrote:
> Hi, Tim.  That page is an excellent example of the large mail providers' complete lack of interest in mitigating the costs of their DMARC policy. That web page is full of suggestions about how other people can spend their time and money to make changes to their existing software that make the software work worse and break features that their users depend on, solely to work around DMARC.  If the providers were interested in mitigating the damage, that would involve spending their own time and money so the people who they've hurt don't have to.

Hello John, what you're missing -- and its easy to miss -- is that Yahoo has an outstanding offer to help developers (this means $!) fix things.  This is not as clear as it should be, which is why I point it out to people as much as a I can.  To be clear, this is a counter point to the "complete lack of interest in mitigating the costs of their DMARC policy" theme.  It's just not true.

A sentiment like "Large mail providers aren't doing enough to mitigate the costs of their DMARC policy" is something I'd agree with.  That's an interesting discussion, which might even be fruitful.  Complaining about the inability for large mail providers to "fix email" isn't something I'm interested in -- decentralized email is worth keeping around, even if it means sometimes having to change stuff.

> I have been in extensive correspondence with the people who develop and maintain the Mailman discussion list software, and can only find that suggestion gratuitously insulting.

John, you are very difficult to communicate with, maybe because you're easily insulted, even when there is no insult.  I too have been in correspondence with mailing list developers, and also developers behind businesses that rely on email, and also a slew of decision makers... and they're all trying to understand how they can fix things without going backwards.  Figuring out how to make email slightly less capable of harming people is a big deal.

> They know as much about e-mail as anyone, they've been tearing their hair out trying to figure out how to deal with the problems that the two mail providers have created, they've implemented and tested all sorts of workarounds including several not on that web page, and they agree that they all stink.

Tell the people you're communicating with to post their finding to either this list or the dmarc-discuss list.  There is no reason why their findings should be frustrating -- others are meeting the challenge, and others still would likely benefit from their learnings.

=- Tim

---

From: Dave Crocker <[email-masked]>
Date: Wed, 28 May 2014 14:11:36 -0700

On 5/28/2014 2:07 PM, Tim Draegen wrote:
> John, you are very difficult to communicate with, maybe because you're easily insulted, even when there is no insult.  I too have been in correspondence with mailing list developers, and also developers behind businesses that rely on email, and also a slew of decision makers... and they're all trying to understand how they can fix things without going backwards. 

All these background technical discussions sound like they are fascinating.

Given that this list is supposed to be for technical discussion of
matters relating to DMARC -- with an eye towards possible
standardization -- and given that specification of methods for
interworking with mailing lists is such an extremely salient issue,
perhaps folks could consider moving presentation and review of design
choices into this forum?

d/

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Franck Martin <[email-masked]>
Date: Wed, 28 May 2014 16:31:55 -0500 (CDT)

----- Original Message -----
> From: "John R Levine" <[email-masked]>
> To: "Tim Draegen" <[email-masked]>
> Cc: [email-masked]
> Sent: Wednesday, May 28, 2014 1:33:36 PM
> Subject: Re: [dmarc-ietf] Solution for DMARC disruption of normal email use while still offering its normal
> protection
> 
> > .. it includes an open invite to support developers.  The problem that
> > Yahoo has now is that not enough people know about their efforts to fix
> > things.
> 
Pushing your version of things, does not make it more true...

The changes in mailman to handle DMARC came from people involved with DMARC.org. So we cared. I recall clearly, you did not want to see any change happening so you could reinforce your conspiracy theory.

While the mailman developers are not happy on the current solutions and are looking for better solutions, they are at the same time happy to had one, when yahoo did the DMARC policy change, and quickly extended on the possibilities based on the experience they had before Yahoo did the policy change.

Now, while mailman has some solutions, it is in the recent versions and many people are still running old versions of mailman and upgrading or patching for them is difficult. So saying people do not know how to fix things is perfectly valid too.

Time to stop these non-technical threads and focus on making email more secure by providing solutions for people to choose from.

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 28 May 2014 17:52:08 -0400

On Wednesday, May 28, 2014 14:11:36 Dave Crocker wrote:
> >> > suggestion gratuitously insulting.> 
> > John, you are very difficult to communicate with, maybe because you're
> > easily insulted, even when there is no insult.  I too have been in
> > correspondence with mailing list developers, and also developers behind
> > businesses that rely on email, and also a slew of decision makers... and
> > they're all trying to understand how they can fix things without going
> > backwards.

This may sound like a flip question, but it's a serious one:

Since, as I understand it, DMARC will not be an IETF specification, what is the 
IETF work here?  Is there a clear boundary between the independent 
submission's requirements and what the IETF should be doing?

It feels like the demarcation (pun intended) is DMARC is immovably what's 
currently deployed by a number of large providers that invested in it and the 
IETF work is to clean up the side effects.  Now that's not a very pretty way to 
put it, but I don't think it's completely wrong either.

How do we define the scope of work for this list?

Scott K

---

From: Dave Crocker <[email-masked]>
Date: Wed, 28 May 2014 14:54:59 -0700

> How do we define the scope of work for this list?

Merely as an example, one line of effort could be towards methods of
DKIM signature survival through mailing lists.

It's not a new topic and it's not an easy one, and it doesn't even have
the string D-M-A-R-C in it, but any real progress for this could be
quite helpful in mitigating collateral damage with especially aggressive
DMARC use.

d/

---

From: Jim Fenton <[email-masked]>
Date: Wed, 28 May 2014 15:18:04 -0700

> quite helpful in mitigating collateral damage with especially aggressive

Since you don't mention it, what about the "mail this article to a
friend" use case that has also been mentioned? Is that a problem that
should be addressed here?

I want to make sure that isn't forgotten just because it doesn't affect
IETF directly.

-Jim

---

From: Franck Martin <[email-masked]>
Date: Wed, 28 May 2014 17:28:02 -0500 (CDT)

----- Original Message -----
> From: "Jim Fenton" <[email-masked]>
> To: [email-masked]
> Sent: Wednesday, May 28, 2014 3:18:04 PM
It should not be forgotten. There are also all the use cases on how you get third parties sending emails on your behalf to become compliant. There are so many options available.

The question, I have, is if the format of an RFC is the proper format to give implementation advice to "webmasters". Will they read it? or should it be a nice glossy email marketer style blog/white paper?

---

From: Dave Crocker <[email-masked]>
Date: Wed, 28 May 2014 15:29:30 -0700

On 5/28/2014 3:28 PM, Franck Martin wrote:
> how you get third parties sending emails on your behalf to become compliant.

Please clarify what that means.

d/

---

From: Franck Martin <[email-masked]>
Date: Wed, 28 May 2014 17:45:44 -0500 (CDT)

----- Original Message -----
> From: "Dave Crocker" <[email-masked]>
> To: "Franck Martin" <[email-masked]>
> Cc: [email-masked]
> Sent: Wednesday, May 28, 2014 3:29:30 PM

Before one can move to p=reject, it needs to take control of all the email streams.

Here are a few use cases:
-sending marketing emails via an ESP
-sending surveys to employees and non-employees
-contracting a third party helpdesk
-contracting a third party Customer Relation Manager
-contracting a third party travel system
-Apps in the cloud and their notifications system
-mobile phones using the telecom provider email system
-service providers sending email to employees (HR, health, insurance, retirement,...) to their employee address or personal address
-sending emails to interns/contractors, who like to forward everything to their primary email account
-....

Some solutions: add to spf/give dkim key, delegate a subdomain, delegate records in a subdomain, use the third party domain.

---

From: Brandon Long <[email-masked]>
Date: Wed, 28 May 2014 15:46:57 -0700

On Wed, May 28, 2014 at 11:20 AM, Douglas Otis <[email-masked]>
wrote:

> You'll see statements in the TPA-Label draft about including
> authentication-results and indeed this new header should be referenced as
> well.  Sorry about the omission. When there is a problem, the DMARC domain
> is still unable to respond to what might cause expensive damage when
> misplaced trust leads to customer attrition whenever uncontrolled sources
> are allowed to flood inboxes spoofing their identity.  The TPA-Label draft
> permits an effective response in minutes.
>

I don't think it does.

Take another way, without AuthRes, you're exposing your entire domain to
phishing attacks if there is any open relay on the "whitelisted" domain.
 Even with AuthRes, you're exposing your users to any intrusion on a third
party, and with 30k such third parties, some of them are going to get
hacked some times.  We've already seen "open posting" mailing lists on
trusted third parties used for spear phishing attacks, and that's not
something that "response in minutes" would solve.

> TPA-Label does not imply a large number of DNS transactions, unlike SPF or
> even DKIM.  TPA-Label is a single transaction directed toward a DMARC
> domain. There should only be one DMARC domain recognized per message.  Of
> course, as with any DNS transaction, this can be redirected to other
> domains.  It is still a single transaction, unlike SPF which can result in
> more than 100 transactions with any number of additional redirections
> involving other domains completely unrelated to domains seen in the
> exchange.  The TPA-Label can assert which validation methods should be used
> to help further reduce the number of DNS transactions needed.
>

Unless I'm mis-understanding, there is a single DMARC domain, but there are
N third party domains allowed, and how each one is allowed is specified in
their specific record.

So, I look in list-id, and then do a lookup for <list-id domain
hash>._smtp._tpa.<dmarc domain> and see if list-id domain is valid for tpa.
 If not, I then have to do <sender domain> and then go through the list of
7 possible locations, each of which could be different.

And if sub-domains are allowed, then if the list-id is "
foo.groups.google.com", I have to check "groups.google.com" and "google.com
".

Also, if some of those domains are googlegroups.com or yahoogroups.com,
whitelisting the entire thing is impractical, its not a controlled
environment where you can garuntee zero abuse.

Now, with caching, especially negative caching, and "legitimate" email most
likely having the same domain in most of the seven locations, real world
may be only a single lookup per message, and spoofed/bad messages would
likely need all 7.

Given that a small number of users is subscribed to any given mailing list,
I think some sort of mail relay token or a new dkim canonicalization would
be more practical.  Then, you have "this list is allowed to relay this
message" or "this user is subscribed to this list" or something.  Even
then, you may have issues with spear phishing attacks.

Brandon

>
> A shared DMARC-bypass whitelists would remove control away from the DMARC
> domain that has an incentive to ensure protection.  Community white-lists
> would make responding to any abuse impractical, to say the least.  In
> today's environment, an ability to respond should not be overlooked.
>
> A TPA-Label authorization would not be granting anyone the ability to send
> email for a domain.  This authorization simply permits alignment exceptions
> for specific third-party services that must still permit the validation of
> their own domain.  The authorization can also stipulate third-party domains
> must include their domain in a Sender or a List-ID.  If a problem is seen
> by any of their messages, the DMARC domain can retract authorization in
> minutes.  Those who apply DMARC alignment checks would be able to make
> exceptions for these third-party messages based on advice given directly
> from the DMARC domain.
>
> TPA-Labels as a disruption remedy can be done fairly rapidly since this
> would involve only those implementing DMARC.  Changing 30K mailing lists
> where one might be running in the basement of a local church should be
> expected to take much longer.
>
> Google already provides free recursive DNS for anyone to use.  It seems
> possible, Google could even get the ball rolling by offering a "_smtp._
> tpa.gmail.com" zone that others might wish to reference.  This would
> allow an easy and immediate solution for any DMARC domain willing to allow
> Google to manage their third-party services.  TPA-Labels will require the
> support of some large ISP before is likely to gain adoption.  I'll update
> the draft to include draft-kucherawy-original-authres-00
> <http://tools.ietf.org/html/draft-kucherawy-original-authres-00>.
>
> Regards,
> Douglas Otis
>

---

From: Dave Crocker <[email-masked]>
Date: Wed, 28 May 2014 15:48:39 -0700

> Before one can move to p=reject, it needs to take control of all the email streams.

OK.  Thanks for the clarification.

The examples you provide are for third parties that have a business
relationship with the domain owner.

The interesting challenge is when a random user sends legitimate mail
from an un-associated third party.

d/

---

From: Brandon Long <[email-masked]>
Date: Wed, 28 May 2014 15:49:34 -0700

> I want to make sure that isn't forgotten just because it doesn't affect

Franky, that case has always been kind of ick, and is "easily" solved by
sending From the domain in question ([email-masked]).  Granted, I
don't know how many of those there are and fixing them all is certainly
annoying work that its bad to force on the world... also, arguably most of
those shares have been replaced by sharing to the social website of the
week.

Brandon

---

From: Kurt Andersen <[email-masked]>
Date: Wed, 28 May 2014 22:55:53 +0000

>from an un-associated third party.

"Taking control" means that there is no such legitimate usage.  Not that
there isn't usage, just that it is, by definition, no legitimate.

--Kurt

---

From: Franck Martin <[email-masked]>
Date: Wed, 28 May 2014 17:59:50 -0500 (CDT)

----- Original Message -----
> Sent: Wednesday, May 28, 2014 3:48:39 PM

You are welcome

> from an un-associated third party.

In many cases the domain owner IT department does not know an employee decided to use/contract someone to send emails... This case is not too far from the one Jim pointed to, and amount to detect and then inform the third party to pick from a set of possible solutions.

---

From: Douglas Otis <[email-masked]>
Date: Wed, 28 May 2014 17:48:55 -0700

> I don't believe TPA-Label hits the mark between "solving a big hurt" and "simple".  IOW, it's too complicated for the amount of pain it would resolve.  Just my opinion, take care,

Dear Tim,

All that is needed is a few bandaids?

The motivation behind TPA-Label was to ensure both quick and efficient methods to offer necessary feedback to receivers.  DMARC already expects receivers to offer failure feedback to DMARC domains.  Unfortunately, once the DMARC domain has decided which third-parties need to be granted exceptions, there is no way to do so.   It seems dangerous to suggest this can be hard-coded in the form of informal lists indicating which DMARC domains should be ignored.

In the case of Yahoo, there is a real issue they are attempting to mitigate.  It would be nice to have a solution able to deal with massively compromised user accounts, as ugly as that is.  This is an issue that is not going away any time soon.  The issue is much worse in China, for example.  Please don't decide being helpful in such situations is simply too hard.  Is it really better to create lists about which domains get ignored? It seems this quickly degrades DMARC's initial intent, which was to offer results receivers felt safe to act on.  Is this still a worthy goal?

Regards,
Douglas Otis

---

From: "John R Levine" <[email-masked]>
Date: 29 May 2014 00:05:08 -0700

> Hello John, what you're missing -- and its easy to miss -- is that Yahoo 
> has an outstanding offer to help developers (this means $!) fix things.

Really, that makes no difference.  I don't want Yahoo or anyone else to 
pay us to screw up our mail software to work around them -- I want them to 
spend their money to fix things so we don't have to.

Yahoo, in their own blog, estimates there are 30,000 mail systems that 
they have damaged by their DMARC actions.  I would be surprised if there 
were more than a few hundred mail systems acting on DMARC policies, 
although some of those are very, very large.  Is it that hard to 
understand why someone might think it was unreasonable to demand that the 
30,000 make changes of no benefit to themselves, rather than the few 
hundred fix their buggy fussp?

The 30K estimate is probably low, since there are likely many small mail 
systems they aren't aware of but that they are damaging.  For example, 
yesterday a middle school teacher who found my old Dummmies web site wrote 
to me out of the blue to say that his web form that lets students and 
parents send mail to him stopped working for AOL and Yahoo addresses, 
which just disappear.  It took about two seconds to figure out what was 
wrong when he told me that his script sends mail to his Gmail account.  I 
told him what was wrong, and he did a hack that sticks in a fake From: 
address, so the mail gets through but now his script works worse since he 
can't write back without extra effort.  If he hadn't written to me, he'd 
probably never have figured out what was wrong.  These are real people who 
are really hurt by the two providers' actions.

Regards,

PS: Here endeth the rant.

---

From: "John R Levine" <[email-masked]>
Date: 29 May 2014 00:06:29 -0700

By the way, to return to the original point, it still seems vanishingly 
unlikely to me that if we invented per-sender whitelists that the two mail 
providers would implement them.

Regards,

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 29 May 2014 00:09:44 -0700

> providers would implement them.

Has anyone tried asking them?

I'm not sure what value I should put in all this (ahem) third-party
speculation about their intentions or what they care about.

-MSK

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Thu, 29 May 2014 19:21:32 +0900

Hi, I've been invited by Murray Kucherawy and Franck Martin to
participate in these discussions, so let me introduce my affiliation
briefly.  I've been operating GNU Mailman lists since 1999, an
occasional contributor for about 10 years, and a GSoC Mentor for
Mailman since 2012.  I have somewhat ambiguous authorization to speak
for the developers (ie, a rolling consensus of the Mailman Steering
Committee[1]).

Tim Draegen writes:

 > John, you are very difficult to communicate with, maybe because
 > you're easily insulted, even when there is no insult.  I too have
 > been in correspondence with mailing list developers,

Which ones?  GNU Mailman is now here.  Besides me, John is a reliable
source of summaries of past Mailman list discussions in what I've read
of his posting here.

I'm sure in the following I'll be going over a lot of ground that's
been covered before, but since you think it's controversial enough to
address, I hope I'll be forgiven for a certain degree of redundancy.

 > and also developers behind businesses that rely on email,

That's insufficiently precise.  To my mind, there are two kinds (at
least) of businesses that rely on provision of mailboxes (other use
cases follow).

(1) Corporate use case: mailboxes are provided for the convenience of
    organizational representatives communicating directly with agents
    outside the organization.

(2) Mail service provider (ESP) use case: mailboxes are provided to
    customers for personal use of customers.

I have no problem with "p=reject" in the corporate use case because I
don't believe it causes significant burdens on users or unreliablity
of delivery.  I believe that John has the same assessment, and it is a
weak consensus in the Mailman community AFAICT.  Our issue is with
"p=reject" as used by large ESPs like Yahoo! and AOL, especially since
it seems to be associated with severe security problems, either at
those services or on the users' own machines.  (Evidently this is a
consensus here as well, I'm just making Mailman's understanding clear.)

Does DMARC actually impose significant burdens on the "corporate use
case", contrary to my belief?

Of course there are other use cases of "businesses that rely on
email".  I understand the "mailing list as user forum" case, and I
believe Mailman has already implemented a sufficiently broad set of
measures for business use in this use case.  The tradeoffs aren't
pleasant, but that's a cost of doing business with Yahoo! and AOL
users.  A business has the usual set of options (pay the costs, find
less costly customers, use alternative forum technology, etc).  I
don't see a real problem here.

There's also the "on behalf of" content provision use case.  Others
describe a good remedy in this thread, but I would like to point out
that such providers may publish "p=reject" to good effect, as an
instance of the "corporate use case".

But my knowledge of "business use" is quite limited.  Are there other
"business activities relying on email" such that DMARC imposes
burdens?  Are those burdens specific to "p=reject", or more general?

(If this is all well-known, please point me to a reference where I can
book up without disturbing the Councils of the Wise.)

 > and also a slew of decision makers... and they're all trying to
 > understand how they can fix things without going backwards.

IMO, they're wasting their time.

Email service *must* deteriorate in the presence of users who send
messages from "p=reject" domains, unless those domains are draconian
in enforcing direct transmission to final destinations that observe
"p=reject" (see Franck Martin's post later in the thread, and the
following subthread on "legitmate use of 'p=reject'" that follows).
Unfortunately, these domains are proposing that third parties adjust
to their (unilaterally imposed) policy, rather than modifying those
policies or restricting their users to safe email usage.  But the "let
third parties adjust" solution is a pretty dismal option.  There are
just too many MXes and MTAs and services that are DMARC-incompatible.
It's going to take years, maybe a decade, to modify both the software
and the installations.

We really can't expect help from Yahoo! and AOL.  You talk about
money, well, Mailman doesn't care about money, it's a volunteer
project.  The effort in development required is actually tiny, less
than 100 new/changed SLOC in Python for any given mitigation available
in Mailman (probably about 200 N/CSLOC for all of them together).
Maybe we'll take their money, but the effort to actually accept and
use the money in a basically pure volunteer organization may make it a
net zero.

The help we need is in explaining to our users that they cannot
maintain past configurations and allow posting from "p=reject" domains
at the same time.  For many of our users, they get a MLM as a part of
a hosting package and the only solution they can implement themselves
is to refuse posts from those domains.  Where's the money to
compensate those list operators for lost subscribers or the costs of
switching hosting services?  Where's the money to encourage hosting
providers to upgrade their MLM installations?  Where's the money to
compensate operators for the emotional and social damage done by
Yahoo! users who throw fits privately and publicly over lost posts?
Yahoo!'s offer of support for developers is a pure FUD PR stunt, as
far as GNU Mailman concerned.

The question is, to paraphrase John, how best to defend our users
(subscribers as well as list operators) from the unilateral behavior
of corporations desperate to mitigate their own problems.  The rest of
us *will* go backwards.  The question is how do we minimize the
retreat, and in which backward direction to go.  Mailman, at least,
offers list operators (precisely, those who can install the most
recent versions) several options, so they can choose the one they
dislike least.

 > Figuring out how to make email slightly less capable of harming
 > people is a big deal.

Yahoo!'s use of "p=reject" reminds us that there is little we can do
to make email as such less harmful.  The DMARC consortium can revise
their protocol, which I remind you was a purely private agreement at
the time of Yahoo!'s action, at any time.  For example, they could
decide that the presence of strings resembling their mailboxes
anywhere in a From: field is subject to DMARC identity alignment.
That would cause the same set of problems for several of the less
unpalatable mitigations developed so far.

The only way to completely prevent future disruptions is to ostracize
"p=reject" domains.  That's a technical assessment, not a
recommendation!  Obviously, businesses and most discussion list
operators will find that "solution" completely unacceptable.

 > > They know as much about e-mail as anyone, they've been tearing
 > > their hair out trying to figure out how to deal with the problems
 > > that the two mail providers have created, they've implemented and
 > > tested all sorts of workarounds including several not on that web
 > > page, and they agree that they all stink.

Slight correction: all are more or less aromatic in most Mailman use
cases, and for some use cases all stink like rotting fish.

One of the big problems for Mailman is that on many discussion lists
there is strong resistance to changing things like the Subject tag
(many people use it auto-folder filtering and the like) and the footer
(I believe for some businesses it's the best solution to requirements
to make it easy to unsubscribe which is more or less legally required
AFAIK).  Users often resist From: corrupting modifications that do not
put the Author Domain in the display name, but this of course goes a
long way to defeating identifier alignment requirements and reenabling
phishing and spam (not necessarily via mailing lists, but also via
crooks imitating the header format in direct mail shots).

I accept that this may not be a "big" problem from the point of view
of this group, that's just Mailman's point of view.

 > Tell the people you're communicating with to post their finding to
 > either this list or the dmarc-discuss list.  There is no reason why
 > their findings should be frustrating -- others are meeting the
 > challenge,

Nice pep talk, but please, save it for the board room.  If you want to
present specific solutions, I'd love to hear about them (even if they
don't apply directly to mailing list management).

 > and others still would likely benefit from their learnings.

Watch this space!

Footnotes: 
[1]  The ultimate authorities for GNU Mailman are Barry Warsaw
(project lead) and Mark Sapiro (release engineer and lead for the
stable version).  I've been pretty good at channeling them in the
past, but I have no formal authority.

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Thu, 29 May 2014 20:19:30 +0900

Franck Martin writes:

 > The changes in mailman to handle DMARC came from people involved
 > with DMARC.org.

Not all of them.  The "message-wrapping" suggestion was mine (at least
I invented it independently and was an early public proponent), and it
was implemented by Mark Sapiro AFAIK.  A From-corrupting patch was
provided by DMARC.org, but final integration and of course release
management was provided by Mark Sapiro.  Other From-corrupting patches
have been proposed, including one provided by John Levine (not yet
implemented in a released Mailman, but obviously Mark will be involved
in that).

None of these patches have been ported to Mailman 3 yet.  That sounds
like a possible use for Yahoo! and DMARC.org funds, although we don't
have organizational infrastructure to manage external funding now.

 > So we cared.

Nobody has denied that DMARC consortium members have put some effort
into developing mitigation strategies.  Nevertheless, I describe
reasons why that effort (and Yahoo!'s offer of financial support) may
be properly characterized as *negligible* in my response to Tim
Draegen.  (N.B. I don't claim the argument has broader applicability
than the GNU Mailman project.  For us, the contribution of DMARC
consortium members has been net negative.  Sorry, Franck.  Feel free
to poll the other Mailman developers if you disagree; I admit my
strong feelings on the matter may cause bias.)

 > I recall clearly, you did not want to see any change happening so
 > you could reinforce your conspiracy theory.

That happens not to be the case.  Conspiracy theories abound on the
Mailman lists (especially Mailman-Users), it's true, but John was not
the source.  John's account of decision-making at the freemail ESPs is
substantially the same as the rationale you have repeatedly presented
yourself (a "push the panic button" response to a concentrated spam
attack).

 > While the mailman developers are not happy on the current solutions
 > and are looking for better solutions, they are at the same time
 > happy to had one, when yahoo did the DMARC policy change, and
 > quickly extended on the possibilities based on the experience they
 > had before Yahoo did the policy change.

Indeed we are *not* happy, and yes we did respond as quickly as
possible to the *denial of service* experienced by many list
subscribers, not restricted to the denial of service imposed by Yahoo!
and AOL on their own subscribers.  However, my wrapping suggestion had
already been implemented, so the DMARC.org From-corrupting setting was
not the only mitigation available (both became available in version
2.1.16).

And we remain unhappy, not least because list operators are unhappy.
The configurations are somewhat tricky and not understood at all by
most list operators -- they follow recipes that are appropriate for
common cases, but may conflict with unusual settings.  The requirement
by some mitigations that Mailman access the DNS in order to apply them
only to "p=reject" posters (which is frequently mentioned by list
operators requesting help) introduces substantial additional
complexity.  This is going to be an ongoing burden for the project, I
fear.

 > Now, while mailman has some solutions, it is in the recent versions
 > and many people are still running old versions of mailman and
 > upgrading or patching for them is difficult. So saying people do
 > not know how to fix things is perfectly valid too.

That's misdirection.  We know one simple and guaranteed fix for the
problem, and it is trivial to implement: domains providing mailboxes
for personal use (including sending as a mailbox user of that domain
from a different one, transfers that modify Content-Transfer-Encoding,
mailing lists, use of on-behalf-of content distribution services, and
so on) should stop publishing DMARC policies with "p=reject".  There
appear to be two of them, they can do this in 5 minutes, and the
problem will be completely eliminated within a day or so.

That is not the only solution, but it really should be on the table,
despite the additional costs of spam and spam-fighting those ESPs may
suffer.  I don't expect them to follow it any time soon, but that's no
reason why we should sanction their current policy.  We can condemn
the policy and implement rational countermeasures at the same time.

I support adding your "conditions for legitimate use of 'p=reject'" to
the DMARC-base document, for example.  I have no hope that DMARC.org
will acquiesce, of course.

 > Time to stop these non-technical threads

These threads are input to the BCP process, at the very least.  In any
case, "p=reject" cannot be treated as a "technical" problem since the
damage it does affects third parties who are not party to the DMARC
protocol.

 > and focus on making email more secure by providing solutions for
 > people to choose from.

I've made the lists used by my students more secure (against denial of
service) by filtering "p=reject" domains.  As it happens, *all* of my
users agree that is the appropriate solution for us (the Yahoo! users
didn't even think about fighting before switching, they already had
GMail addresses).  Of course I hesitate to recommend that policy to
anybody else, but it *is* technically simple, and was the optimum
response in my case.

Regards,
Steve

---

From: Hector Santos <[email-masked]>
Date: Thu, 29 May 2014 07:47:33 -0400

On 5/29/2014 3:09 AM, Murray S. Kucherawy wrote:
> On Thu, May 29, 2014 at 12:06 AM, John R Levine <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     By the way, to return to the original point, it still seems
>     vanishingly unlikely to me that if we invented per-sender
>     whitelists that the two mail providers would implement them.
>

Good point.  Why don't you ask them?   We need positive endorsement 
for 3rd party semantics which we don't have.  I've implemented ATPS 
for ADSP and it works.  Update the extension for DMARC and I'm sure 
Yahoo can manage 30K records if they decide to use it.  That shouldn't 
be a problem and it will be a growth thing, not adding 30K records in 
one shot.  It will be more manageable. A major consideration is that 
not all domains are YAHOOs so it they won't need the same scale, not 
even close.

But you see, thats been the problem with all this all along -- picking 
who this or that protocol, DKIM itself, will be using them and 
leveraging its value via a policy layer.  We can't do that.  The 
protocol modeling must fit all.  That doesn't mean it works for all 
but from a mail integration and engineering standpoint, it has to 
apply to all -- small and large.  It has to make sense at all scales. 
The danger was to miss this with the large who will have a higher 
impact when assumed they won't be using it.

-- 
HLS

---

From: Dave Crocker <[email-masked]>
Date: Thu, 29 May 2014 07:17:03 -0700

> Has anyone tried asking them?

I'll suggest that it's premature to do such asking, just as it is
premature to say that they are uninterested or would reject the idea.

Absent a very concrete proposal -- along the lines of a specification --
the request would be generic.

Companies don't (and shouldn't) make meaningful commitments to vague
concepts.

d/

---

From: "John Levine" <[email-masked]>
Date: 29 May 2014 17:47:19 -0000

>> should be addressed here? ...

>those shares have been replaced by sharing to the social website of the

There seem to be rather a lot, since it's a feature on most magazine
and newspaper web sites.  Since you mentioned the WSJ, they use the
user's own address as the From: address (I just checked.)  Some do the
hack you mentioned, but a lot don't.  My college class has a mailing
list, and I sometimes send articles to it from my WSJ account, which
would stop working if they didn't let me put my own address on the
From: line.

Do you have any numbers on how much if any of a spam or phish problem
these things are?  They've never been on my radar, I think because they
do a lot of rate limiting and inbound filtering.  Some also limit it
to subscribers.

Also please keep in mind uses like the school teacher I mentioned
earlier today.  Again, useful, not abusive, and I think there are
likely to be a lot of little setups like his that are now mysteriously
failing.

R's,
John

---

From: Brandon Long <[email-masked]>
Date: Thu, 29 May 2014 10:58:35 -0700

> likely to be a lot of little setups like his that are now mysteriously

I'm sure most of the big ones are well protected with abuse limits, but I'm
sure there are plenty of other ones which are not.  I'm sure each one that
is abused is a drop in the bucket of the overall abuse.

That being said, with DMARC, I see no way forward for them.  Even if the
WSJ and other large publishers are whitelisted by one of the schemes we're
talking about, the forms like your teacher's are never going to get
whitelisted.  At best, a provider may provide a mechanism to whitelist
things for a specific user, but I haven't imagined anything like that which
would be particularly user friendly either.

Brandon

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 29 May 2014 11:05:45 -0700

On Thu, May 29, 2014 at 10:58 AM, Brandon Long <[email-masked]> wrote:

> I'm sure most of the big ones are well protected with abuse limits, but
> I'm sure there are plenty of other ones which are not.  I'm sure each one
> that is abused is a drop in the bucket of the overall abuse.
>

I'd love to see some usage statistics from one or more of them.  Lately it
seems to me a lot more of such link sharing happens over social media
(where this is all moot) rather than over email.

-MSK

---

From: "J. Gomez" <[email-masked]>
Date: Thu, 29 May 2014 22:28:29 +0200

On Wednesday, May 28, 2014 10:13 PM [GMT+1=CET], Tim Draegen wrote:

> I don't believe TPA-Label hits the mark between "solving a big hurt"
> and "simple".  IOW, it's too complicated for the amount of pain it
> would resolve.  Just my opinion, take care,  

I'm of the same opinion as above.

In my own words, I would say the TPA-label draft Otis posted reads like perfectly polished Klingon to me.

Id est, overly too complex for very little gain.

Regards,
J.Gomez

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 29 May 2014 14:23:13 -0700

> Id est, overly too complex for very little gain.

+1.

-MSK

---

From: "Popowycz, Alex" <[email-masked]>
Date: Thu, 29 May 2014 17:55:38 -0400

My Klingon may be rusty, but I think that makes it a + (apologies for those seeing this in text as I don’t have Klingon.ttf on my work laptop).

From: dmarc [mailto:[email-masked]] On Behalf Of Murray S. Kucherawy
Sent: Thursday, May 29, 2014 5:23 PM
To: J. Gomez
Cc: [email-masked]
Subject: Re: [dmarc-ietf] Solution for DMARC disruption of normal email use while still offering its normal protection

+1.
-MSK

---

From: "J. Gomez" <[email-masked]>
Date: Fri, 30 May 2014 00:56:36 +0200

> complexity.  This is going to be an ongoing burden for the project, I

Missuse of DMARC's p=reject by Senders is here to stay. It won't go away. It will only grow.[*]

On the other hand, when hard pressed, [I think] email users are going to choose to keep their mailbox/address over a mailing list subscription, therefore mailing list software will have to adapt to the new theater of operations -- provided DMARC gets deployed in the real world by some significant measure, of course.

In my opinion, the least disruptive adaptation which mailing list software can do is to take ownership --in a DMARC-compatible way-- of the Header-From, just like they already take ownership of the envelope's ReturnPath-From.

And, also in my opinion, that mailing list adaptation to DMARC should be the new default out-of-the-box behaviour of mailing list packages from now on, and the old behaviour should be regarded as legacy and deprecated. Why? Because we cannot count on the average mailing list operator to stop to ponder the fine points of "the DMARC issue" while he is setting up his VPS server in a haste.[**] Therefore [in my opinion], a sane, out-of-the-box DMARC-compatible behaviour should be the default for mailing list software, from now on.

[*][**] Because that's the way the world goes by.

> We know one simple and guaranteed fix for the

Yes, but it just won't happen. Once someone publishes p=reject, if he is "too-big-to-reject", then he is not going to go back to the previous situation. He expects the world to accomodate him, and this is EXACTLY what will happen if he just happens to be, by definition, "too-big-to-reject".

I think we should accept this as a fact, and move forward, and take the neccesary "countermeasures" as the new by-default situation.

> anybody else, but it *is* technically simple, and was the optimum

That solution is good, but can only be expected from savvy mailing list operators who fully understand the fine details of "the DMARC issue" -- I think we cannot expect that to be the case for the vast majority of mailing list operators world-wide. Also, the support costs of such a solution are high (disgruntled users calling the help desk), and may even be unbearable if the mailing list operator happens not to have a more or less "captive" user base.

Regards,
J.Gomez

---

From: Dave Crocker <[email-masked]>
Date: Thu, 29 May 2014 16:08:20 -0700

On 5/29/2014 4:19 AM, Stephen J. Turnbull wrote:
> These threads are input to the BCP process, at the very least. 

+1

d/

---

From: Hector Santos <[email-masked]>
Date: Thu, 29 May 2014 19:26:49 -0400

> Id est, overly too complex for very little gain.

Mr. Gomez,

TPA has a wider scope and an overly described functional specification.

But the main idea should not be thrown away due to lack of 
fundamentally understanding the long time problem and proposed solutions.

In its simplest terms, the idea is to lookup a 3rd party domain for 
authorization to sign or resign originating author domain mail.

The earliest suggestions like in the 2006 DSAP proposal used an 
Allowed Signer List (ASL) "asl=" tag in the policy records. So 
applying this simpler idea to DMARC, an example policy for your 
domain, seryrich.com, might be

   _dmarc.seryrich.com  TXT ("v=dmarc1; p=reject; asl=ietf.org; ...")

This would expose to the world the policy that says:

       Only domains seryrich.com and ietf.org are
       authorized to sign mail for seryrich.com.
       If you see anything else, reject it.

Simple idea. No extra lookup beyond the DMARC lookup.  The problem 
with ASL is that it works for the shorter list domains.  It would not 
scale for the larger domains, i.e. can't fit Yahoo's 30K potential 
authorization list.  I think its an optimization idea for the majority 
of domains who are smaller than YAHOO.

This is where Doug's TPA (Third Party Authorization) and Murray's 
simpler version of TPA called ATPS (Authorized Third Party Signer, 
RFC6541) comes into play.

TPA and ATPS is basically the same when it comes to the lookup 
formula, which is to combine the signer domain as a subdomain zone of 
the author domain.   I have implemented ATPS but its basically the 
same as TPA with a query formula of:

    BASE64(SHA1(signer-domain))+"."+author-domain

So for you, your DMARC and ATPS subdomain records for seryrich.com 
zone would be:

_dmarc TXT ("v=dmarc1; p=reject; atps=y;")
PQ6XADOZSI47RLUIQ5YOHG2HY3MVJYOO._atps  TXT ("v=atps01; d=ietf.org;")

So the ideas has been worked out.  The problem has been to get the 
IETF List Operators and Administrators to support this.

So why not?  Note, we also develop list software, so I have no problem 
doing what is necessary to get it all to work right.

The main problem is that list operators prefer to use a TRUST model by 
looking up the signer domain. Not the author domain.

They currently do neither, but they prefer to use the signer domain 
and this was the DKIM standard suggestion. Use the SIGNER domain. 
Forget about the author domain.  Again, I am just looking for a total 
mail integrated solution between all parts. So lets go ahead and 
lookup the signer domain.

What's the problem then?

Well, what happens when any of these simple use cases occur:

   a) The signature does not exist in the message. Its missing.

   b) Signature exist, but it doesn't hash verify. Its broken.

   c) Signature exist and its valid, but its not your signature.
      Its some unknown signer that wants to tell the world its
      signing on your behalf.

So the basic argument against depending on a signer domain only is 
that may not exist nor be valid.  It can be completely fake.   Now, 
Levine's VBR was a step in the right direction with the "combo" author 
and signer lookup but again, what happens when the signature is 
missing or invalid so that there is no signer domain to lookup?  The 
trust advocates don't have an answer for these simple first level 
security issues that are easily detectable as failure.

In summary, this has been examine all ways and the best and original 
idea was looking up the author domain because its the single identity 
that MUST exist in the message.  It is the most important header of 
them all, the 5322.From header.  This is the reason it is also the 
only header that MUST be hash bound to the signature. No other header 
is required to be hashed to the signature.

But the resigners don't want to look it up. They want the freedom to 
resign mail. The policy advocates wants a more flexible 1st and 3rd 
party resigner framework, but one with options to lock down the 3rd 
party signers.

I believe the latter is the more protocol strong solution. Its require 
more wider support and it will clamp down on the laissez-faire 
resigners who want to resign and don't wish to even check to see if 
its valid.

Yes, there is the problem of the legacy users that has long used 
public, but also highly spam polluted big email brand domain.  The 
Yahoos are not the only one who domains are polluted and they wish to 
finally take control and increase its security quality value.

The IETF need to stop trying to make policy itself on what parts of 
the mail market it wishes to support.  Its not just about resigners.

Hope the above makes sense.

-- 
HLS

---

From: Kurt Andersen <[email-masked]>
Date: Fri, 30 May 2014 00:05:42 +0000

On 2014-05-29, 16:26 , "Hector Santos" <[email-masked]> wrote:

>. . .the idea is to lookup a 3rd party domain for
>. . .The problem . . . is that . . . [i]t would not
>scale for the larger domains

I have to confess that I have not (yet) waded through the details of TPA
or ASL or ATPS, but from a corporate perspective, it would be extremely
unworkable for any but the smallest company to manage DNS records to
whitelist every list server on the internet that my employees would happen
to use.

Even if I did, why would I want to give blanket permission for all of
those list services to sign on my behalf? I doubt that I would trust them
not to be exploited - even such a highly esteemed organization as IETF :-)

--Kurt Andersen

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 29 May 2014 20:20:29 -0400

On May 29, 2014 3:09:44 AM EDT, "Murray S. Kucherawy" <[email-masked]> wrote:
>vanishingly

I think Yahoo's communications have been very clear about what they care about. No speculation is needed. 

Scott K

---

From: Douglas Otis <[email-masked]>
Date: Thu, 29 May 2014 23:39:04 -0700

> not to be exploited - even such a highly esteemed organization as IETF :-)

Dear Kurt,

There seems to be some confusion about how TPA-Labels would operate.  It is an Authorization scheme.  At no time would any other domain be able to sign on your behalf.  It simply permits specific alignment exceptions regarding both domains and headers for those domains your company allows employees to use.  This could also be setup to automatically authorize mailing-lists used by your industry, and all outsourced services without exchanging any credentials or granting network access.  I doubt any company executive will today blithely exchange credentials to permit an HVAC firm to submit invoices.

By requiring a Sender header field, Outlook will modify what is seen on the From header to say Intuit.com on behalf of Linkin.com when sending out invoices on your behalf.  Recipients should not have a hard time understanding who both signed and sent the message.  Most MUAs will even allow you to normally display the Sender header when it is there.  Would this be a major issue for Linkedin?  At least there would be a far greater certainty rogue messages will continue to be rejected as desired.

Regards,
Douglas Otis

---

From: Tim Draegen <[email-masked]>
Date: Fri, 30 May 2014 04:25:08 -0400

> All that is needed is a few bandaids?

Hi Doug, I don't see the problem space as allowing bandaid approaches.  The widespread ability to build controls on top of stable domain level identifiers is relatively new.  In my view, people that want to use these controls are stuck in one of two ways:

1. They're trying to modify how their email is structured so that controls can safely be put into place, and there is some sort of relationship that allows change to happen.  Franck Martin wrote previously about this here:
	http://www.ietf.org/mail-archive/web/dmarc/current/msg00878.html

2. They're trying to address "legitimate-but-unauthorizable" email.  In this bucket I'd put everything that is mostly beyond the ability for individual domain owners to change: mailing lists, forward-to-friend, services that send on behalf of users.  Scott Kitterman wrote about this and asked about the scope of IETF work in this bucket:
	http://www.ietf.org/mail-archive/web/dmarc/current/msg00872.html

Pain due to inaccurate deployment of controls is the domain owner's problem (#1 above).  There (should be!) plenty of feedback available for domain owners to recognize their own problems.

Pain due to everything else (#2 above) is something domain owners can't expect to solve on their own.  It's in this area that "bandaids" might work in the short term (like publishing lists of exceptions), but long term I'd personally prefer not to see a codified model of exceptions.

I think it would be better to specify how impacted email in bucket #2 can be dealt with, but not with a goal of preserving all existing email use cases.  IMO a better goal would be to specify how specific categories of email should be presented.  For example, codify how mailing lists go about their business (maybe: adhere to relevant RFCs + DKIM sign with ML's domain + perform OAR-like checks).  A different example: analyze the use of "Reply-to:", determine what is preferable behavior, and codify what should be expected.  The resulting specifications would at least tell developers (across the spectrum from MTA through MUA) what they need to do to interoperate.

> The motivation behind TPA-Label was to ensure both quick and efficient methods to offer necessary feedback to receivers.  DMARC already expects receivers to offer failure feedback to DMARC domains.…

I think I understand the motivation.  I guess I'm in the horrible position of viewing TPA-Label as a bandaid, given my own view of the scope and amount of work involved in making email suck less.  I don't mean to disparage TPA-Label -- I just mean that given a finite amount of focus and fuel, I'd prefer to work out what I wrote about above: "specifying how impacted email in bucket #2 can be dealt with".

> In the case of Yahoo, there is a real issue they are attempting to mitigate.  It would be nice to have a solution able to deal with massively compromised user accounts, as ugly as that is.  This is…

One facet of the problem that Yahoo & AOL are addressing via DMARC-based controls goes beyond "compromised user accounts".  A malware-infected PC's local address book gets scraped, and the infected PC emits spam/malware that spoofs the owner of the PC's email address (because fraud appearing to be from a known friend/colleague is pretty effective fraud).  This fraud does not flow through Yahoo or AOL.

I hope the above shows that the size and scope of difficult problems is not something I'm looking to work around.  IMO bandaids are exception-lists and mechanisms like TPA-Label, and the difficult (but necessary) work remains in proposing, researching, specifying, implementing, and advocating for how well established mailing practices can interoperate in an email environment where stable domain-level identifiers are the norm.

-= Tim

---

From: Tim Draegen <[email-masked]>
Date: Fri, 30 May 2014 05:12:22 -0400

On May 29, 2014, at 3:05 AM, John R Levine <[email-masked]> wrote:
> Really, that makes no difference.  I don't want Yahoo or anyone else to pay us to screw up our mail software to work around them -- I want them to spend their money to fix things so we don't have to.

Yes, I get it, I guess in my own jaded way I don't think there is any amount of money that Yahoo and AOL can spend that will fix things (because email isn't owned by Yahoo or AOL).  BUT, if Yahoo or AOL is willing to experiment, let that experiment be me!

I replied to Doug earlier (not yet in archive), and hashed out my own thinking around how much domain owners can do vs. how to address "legitimate-but-unauthorizable" email.

TLDR summary: addressing "legitimate-but-unauthorizable" mail is my answer to Scott Kitterman's question: "How do we define the scope of work for this list?".

> 
> Yahoo, in their own blog, estimates there are 30,000 mail systems that they have damaged by their DMARC actions.  I would be surprised if there were more than a few hundred mail systems acting on DMARC policies, although some of those are very, very large. Is it that hard to understand why someone might think it was unreasonable to demand that the 30,000 make changes of no benefit to themselves, rather than the few hundred fix their buggy fussp?

I don't think there is/was a way for Yahoo to fix the estimated few hundred mail systems acting on DMARC policies, especially since most are not controlled by Yahoo.  Maybe they could have published a list of 30,000 mail systems that are white-listed, but wouldn't that just be a publication of 30,000 holes to exploit?

The absolute most work I could imagine Yahoo and AOL having done would have been to analyze and publish a series of articles/guidance on how impacted email can be fixed, complete with accessible patches to all known mailing systems.  THEN, give the entire internet enough time to apply said patches.  This is my unicorn.

For the next 10 years, I'm going to attempt to recreate this unicorn.

> 
> The 30K estimate is probably low, since there are likely many small mail systems they aren't aware of but that they are damaging. For example, yesterday a middle school teacher who found my old Dummmies web site wrote to me out of the blue to say that his web form that lets students and parents send mail to him stopped working for AOL and Yahoo addresses, which just disappear.  It took about two seconds to figure out what was wrong when he told me that his script sends mail to his Gmail account.  I told him what was wrong, and he did a hack that sticks in a fake From: address, so the mail gets through but now his script works worse since he can't write back without extra effort.  If he hadn't written to me, he'd probably never have figured out what was wrong.  These are real people who are really hurt by the two providers' actions.

In a similar vein, there are a fair number of businesses that do stuff like encapsulate their customer mail with bling (fancy headers, pictures, footers w/ disclaimers... "stationary"), and they're having to figure out how to maintain their service when sending on behalf of clients with Yahoo and AOL addresses.  

What is missing is "how am I supposed to do this right"?  I'm not being glib, there's a real vacuum due to email being what it is, and it's a vacuum that I personally don't think corporations can/should fill.

-= Tim

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Fri, 30 May 2014 18:50:18 +0900

Kurt Andersen writes:

 > I have to confess that I have not (yet) waded through the details
 > of TPA or ASL or ATPS, but from a corporate perspective, it would
 > be extremely unworkable for any but the smallest company to manage
 > DNS records to whitelist every list server on the internet that my
 > employees would happen to use.

I don't see that a small company, or any company, would want to.  Your
employees are representatives of the organization when using (most of)
your subdomains and your main domain, and "p=reject", with no
mitigation on your part, seems perfectly reasonable to me.[1]

Use a subdomain like just-for-fun.example.com or list-post.example.com
with "p=none" if you want to permit personal posts to 3rd-party
mailing lists.  If companies can agree on a single such subdomain (or
even 5 or 10 of them), the MUAs-for-people-prone-to-entering-passwords-
in-email-forms can recognize it and treat that subdomain as suspicious.
Eg, disabling all links in the email, or redirecting to a page which
explains why clicking on these links is dangerous.

If your employees find it tedious to switch return addresses, let them
use XEmacs![2]

The idea is that Yahoo! and AOL can use these protocols to mitigate
the damage they are doing, not just in actual DoS, but in causing
people to run around saying "The sky is falling!  The sky is falling! 
Let's change the semantics of RFC5322.From!".  I find the thought of
"p=reject" domains putting their own resources on the line appealing.

Footnotes: 
[1]  The exception would be the small set of vendors like QuickBooks
who provide you with value-added services, so that you have a business
reason for whitelisting them.

[2]  Yes, I know what happened to Marie Antoinette.  Just kidding.
However, Emacsen-based MUAs are proof of concept.  It is not that hard
to write an MUA smart enough to automatically switch personalities
when posting to a list or writing home to Mom.

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Fri, 30 May 2014 19:09:01 +0900

> away. It will only grow.[*] 

I'm not so sure.  Anyway, that doesn't mean we need to sanction it.

 > In my opinion, the least disruptive adaptation which mailing list
 > software can do is to take ownership --in a DMARC-compatible way--
 > of the Header-From,

I disagree.  The least disruptive adaption is whatever the users of
the mailing list think is the least disruptive adaptation.  That's why
Mailman provides multiple mitigation options, and will probably add
more as we think of them or they're contributed to us.

 > just like they already take ownership of the envelope's
 > ReturnPath-From.

Ah, but "just like" is a complete misstatement of the situation.
There's a big difference.  Users on a mailing list think of the
poster, not the mailing list, as responsible for the content.  So
according to RFC 5322, the poster's mailbox belongs in From:.
Remailed or not, the author belongs there.

According to the RFCs, Return-Path does not correspond to From, it's
more similar to the semantics of Sender (but not identical).  Whoever
is going to handle delivery issues belongs in Return-Path.  The
remailing agent belongs there, because the poster, in general, doesn't
know where the message is being sent, and can't help debug problems
between the remailer and the subscriber.

 > And, also in my opinion, that mailing list adaptation to DMARC
 > should be the new default out-of-the-box behaviour of mailing list
 > packages from now on, and the old behaviour should be regarded as
 > legacy and deprecated.

Actually, in GNU Mailman I suspect that going forward the default
behavior will be either no mitigation (as now, just by momentum), or
wrap-in-a-new-message-if-p=reject-detected, which is fully RFC
conformant, although not iDevice-Apple-Mail-friendly.  Some developers
are just prickly curmudgeons.[***]  cPanel and Plesk and RHEL are
welcome to patch their versions, of course.

[***] And that, too, is the way the world goes by.

Steve

---

From: Tim Draegen <[email-masked]>
Date: Fri, 30 May 2014 07:04:00 -0400

Stephen!  Welcome!  No one said making sausage was pretty.  :-(

On May 29, 2014, at 6:21 AM, Stephen J. Turnbull <[email-masked]> wrote:
> Which ones?

I don't think it would be appropriate to name people, but to a larger point, I have to apologize if I come across like a jerk.  There are a lot of people trying to deal with DMARC across many forums, and my attempt to imbue a sense of breadth and depth was ham fisted.  I live and learn. 

> least) of businesses that rely on provision of mailboxes (other use

I was thinking of businesses that rely on the perfectly acceptable (but now at loggerheads with DMARC-based controls) practice of sending on behalf of customers.  For example, "stationary" businesses (described towards the end of this mail:
	http://www.ietf.org/mail-archive/web/dmarc/current/msg00923.html).
There are also businesses that perform "SMTP Cleaning" ala anti-spam filtering, plus "SMTP backup" services that people find enough value in to pay for.

My ham-fisted point was that mailing lists are interesting for sure, but the scope of email (potentially) impacted by DMARC is bigger, and maybe we should think about the space a bit more generically.

> case", contrary to my belief?

I don't personally carve up the email space along corporate vs non-corporate lines.  People at corporations communicate via mailing lists.

Before Yahoo and AOL went to p=reject policies, the advice was to keep DMARC-reject policies to "transactional" domains. "Transactional" in this context mapped to "email that isn't likely to be impacted by things that break DMARC like mailing lists"....  stuff like account notifications, receipts, not people-to-people.

But it turns out that quite a few people create mini mailing lists to map a single recipient of a transactional piece of email to multiple recipients... like receipts going to [email-masked], where the parents and an archive are the final recipients.  So, dang.

> don't see a real problem here.

From your mouth to the Internet.God's ear!

> book up without disturbing the Councils of the Wise.)

One giant problem that ISPs face is that they have large numbers of customers that send email through their infrastructure but for domains that are not related to the ISP.  Imagine a residential network providers with lots of customers that configured their email clients 20 years ago.  It's largely worked up until now.  Is the ISP supposed to contact each customer and get them to use the correct outbound SMTP servers?  What is the right thing to communicate?

Council of the Wise?!  I always thought the IETF was a bunch of people hanging off the edge of the world, getting together every once in a while to maybe allow an accidental shared piece of understanding to be written down.

> It's going to take years, maybe a decade, to modify both the software

I think you're largely correct, but I don't agree with the "wasting their time" part.

Bringing domain-level identifiers to email took over a decade of largely haphazard work.  It probably *will* take another decade of work to modify both software and the installations.

Given that much time, it should be possible to structure email service so that service/features do not necessarily have to deteriorate.  Well, today the damage is being done, but is the response from the Council of the Wise to be "email is deteriorating, this is the new normal", or can we put together documents that describe how to interoperate -- not just for dealing with p=reject domains within the context of mailing lists, but interoperability between MLs and MUAs?

> is to refuse posts from those domains.

This is the stuff, right here.  Asking Yahoo and AOL for help in educating users is spot on.

> Where's the money to

A few hours ago I replied to John and expressed my view that Yahoo and AOL aren't in a position to fix the internet... as much I can, I tried not to be ham-fisted:

	http://www.ietf.org/mail-archive/web/dmarc/current/msg00923.html

There's wreckage, yes.  My aim, because I'm eternally an optimist, is that the wreckage doesn't end with yahoo and aol being given the middle finger.  Sure, do that, but there's more work to be done that doesn't belong to yahoo or aol.

> recent versions) several options, so they can choose the one they

I get it, but painting the scenario as this being only Yahoo and AOL's problem is a distraction.  There are plenty of small organizations and individuals that are using DMARC to curtail abuse.  I'd argue that the problem of email's trivial ability to be used for abuse is everyone's problem.  It's only Yahoo and AOL's size that brought this work to the forefront.
> to make email as such less harmful.

Email is the #1 online attack vector.  The "attack surface" that email presents is incredibly huge.  Attempting to change email so that defrauding millions of people using email becomes only incrementally more difficult is what is going on here.

The flip side is that the introduction of stable domain-level identifiers should make filtering a lot simpler.  How does the Internet take advantage of these identifiers in the context of mailing lists, forward-to-friend, send-on-behalf, and other contexts?

[my quarters are running out, so snip snip]

> of this group, that's just Mailman's point of view.

I don't think "big" is required to be important.  I for one am trying to understand as much as I can about DMARC's impact from all angles.

> don't apply directly to mailing list management).

I am not a mailing list developer, but I do try to get people involved in the same areas of work to at least get a chance to collaborate.  What you've communicated is far better than any filtered version.

These folks take a nice and deliberate approach:

  http://onlinegroups.net/blog/2014/05/01/dmarc-taking-responsibility-sending-group-email/#more-934

..different problem space, because of Mailman's huge installed & distributed base, but at least ideas are there.

-= Tim

---

From: John Sweet <[email-masked]>
Date: Fri, 30 May 2014 11:17:21 -0700

There's probably no point in coding a patch unless you feel the people
responsible for the codebase are likely to apply it. That's a lot of effort
down a rathole, especially since some number of the intended audience feel
that it's inappropriate to ask them to change anything in their software.

It's probably more constructive to offer dev time, since a lot of these
packages are volunteer projects. "Once you decide what you want to do, if
you need developers, we'll provide one half-time on your project for as
long as it takes." It might be even better to offer some QA/test help. A
lot of volunteer-only projects tend to lag on the unit/regression test
front.

J

On Fri, May 30, 2014 at 2:12 AM, Tim Draegen <[email-masked]> wrote:

> Yes, I get it, I guess in my own jaded way I don't think there is any
> amount of money that Yahoo and AOL can spend that will fix things (because
> email isn't owned by Yahoo or AOL).  BUT, if Yahoo or AOL is willing to
> experiment, let that experiment be me!
>
> I replied to Doug earlier (not yet in archive), and hashed out my own
> thinking around how much domain owners can do vs. how to address
> "legitimate-but-unauthorizable" email.
>
> TLDR summary: addressing "legitimate-but-unauthorizable" mail is my answer
> to Scott Kitterman's question: "How do we define the scope of work for this
> list?".
>
>
> >
> although some of those are very, very large. Is it that hard to understand
> why someone might think it was unreasonable to demand that the 30,000 make
> changes of no benefit to themselves, rather than the few hundred fix their
> buggy fussp?
>
> I don't think there is/was a way for Yahoo to fix the estimated few
> hundred mail systems acting on DMARC policies, especially since most are
> not controlled by Yahoo.  Maybe they could have published a list of 30,000
> mail systems that are white-listed, but wouldn't that just be a publication
> of 30,000 holes to exploit?
>
> The absolute most work I could imagine Yahoo and AOL having done would
> have been to analyze and publish a series of articles/guidance on how
> impacted email can be fixed, complete with accessible patches to all known
> mailing systems.  THEN, give the entire internet enough time to apply said
> patches.  This is my unicorn.
>
> parents send mail to him stopped working for AOL and Yahoo addresses, which
> just disappear.  It took about two seconds to figure out what was wrong
> when he told me that his script sends mail to his Gmail account.  I told
> him what was wrong, and he did a hack that sticks in a fake From: address,
> so the mail gets through but now his script works worse since he can't
> write back without extra effort.  If he hadn't written to me, he'd probably
> never have figured out what was wrong.  These are real people who are
> really hurt by the two providers' actions.
>
> In a similar vein, there are a fair number of businesses that do stuff
> like encapsulate their customer mail with bling (fancy headers, pictures,
> footers w/ disclaimers... "stationary"), and they're having to figure out
> how to maintain their service when sending on behalf of clients with Yahoo
> and AOL addresses.
>
> What is missing is "how am I supposed to do this right"?  I'm not being
> glib, there's a real vacuum due to email being what it is, and it's a
> vacuum that I personally don't think corporations can/should fill.
>
> -= Tim
>
> >
> > Regards,

-- 
"We've learned some important lessons about Democracy. After 100 years, you
have to let your slaves go. And after 150, you have to let your women
vote." (Kurt Vonnegut)

---

From: "J. Gomez" <[email-masked]>
Date: Fri, 30 May 2014 23:49:05 +0200

> I'm not so sure.  Anyway, that doesn't mean we need to sanction it.

To maneuver towards being interoperable with something (the misuse of DMARC's p=reject as a sad fact of life), does not equal sanctioning that something.

Furthermore, what is more important - to deserve or not to deserve the prize of being sanctioned as kosher, or keeping a world-wide system interoperable?

> more as we think of them or they're contributed to us.

Yes, that is true. But a default out-of-the-box always has to exist, and in my opinion that default should be the most interoperable which is possible --interoperable with the real world, not with how we would like the world to be--, while keeping as much valuable features as possible, and while keeping operator's duties as straight forward and simple as possible.

> Remailed or not, the author belongs there.

That is debatable. As long as the mailing list program tampers with the message's content, rendering its DKIM signature invalid, it can be argued that the mailing list program is rewriting the message's content, and therefore that the mailing list program now becomes "the system responsible for the writing of the message" (as per RFC 5322 parlance, section 3.6.2), and thus the mailing list address would earn its legitimate place in the Header-From field, making interoperability with rogue DMARC Senders a solved problem.

Regards,
J.Gomez

---

From: Hector Santos <[email-masked]>
Date: Fri, 30 May 2014 19:44:16 -0400

> That is debatable. As long as the mailing list program tampers with the message's content, rendering its DKIM signature invalid, it can be argued that the mailing list program is rewriting the mess…

In my book, this is mail tampering. It will be hard to justify adding 
support for this radical behavior in our mail list server product 
which puts customers at risk.  You are putting yourself at product 
liability risk but intentionally defying a security protocol against 
the wishes of the publishing restrictive domain.  Of course, its only 
becomes a problem when its used as a loophole to further spread 
harmful mail and someone actually gets harmed.  Thats all you have to 
prove in a courtroom.   If you had all the tools before you to tell 
you definitively, "This is unauthorized mail"  and you intentionally, 
deliberately and neglectfully ignore it, do nothing about it but 
further distribute to end points, well, who knows what a young punk, 
tech savvy, high tech, new age lawyer will think about suing you.  If 
you got deep pockets, well, don't think it can't happen.

It is this sort of mentality that completely makes this old game no 
more fun to work with. Seriously.

We can do it right.  All we have to do is LOOKUP the policy and follow 
it.  Give the YAHOOs the tools to authorize the resigners and you and 
I won't have these new ethical problems to content with.

-- 
HLS

---

From: "John R Levine" <[email-masked]>
Date: 30 May 2014 19:50:02 -0700

> TLDR summary: addressing "legitimate-but-unauthorizable" mail is my answer to Scott Kitterman's question: "How do we define the scope of work for this list?".

Yup.

> amount of money that Yahoo and AOL can spend that will fix things 
> (because email isn't owned by Yahoo or AOL).  BUT, if Yahoo or AOL is 
> willing to experiment, let that experiment be me! ...

Sounds good.

> not controlled by Yahoo.  Maybe they could have published a list of 
> 30,000 mail systems that are white-listed, but wouldn't that just be a 
> publication of 30,000 holes to exploit?

Um, wait. Are we doing experiments or not?

In answer to your second question, well, no.  There's no reason to think 
that "can't be described by DMARC" is related to "insecure".  A lot of 
them, like the WSJ and the schoolteacher who wrote to me, don't even have 
inbound MTAs to attack.

Regards,

---

From: "J. Gomez" <[email-masked]>
Date: Sat, 31 May 2014 10:08:30 +0200

On Saturday, May 31, 2014 1:44 AM [GMT+1=CET], Hector Santos wrote:

> > That is debatable. As long as the mailing list program tampers with
> > the message's content, rendering its DKIM signature invalid, it can
> > be argued that the mailing list program is rewriting the message's
> > content, and therefore that the mailing list program now becomes
> > "the system responsible for the writing of the message" (as per RFC
> > 5322 parlance, section 3.6.2), and thus the mailing list address
> > would earn its legitimate place in the Header-From field, making
> > interoperability with rogue DMARC Senders a solved problem.       
> 
> the wishes of the publishing restrictive domain.

I am not sure I am following you here.
 
When I say that mailing list programs should take ownership of the Header-From if they tamper with the original message content rendering its DKIM signature invalid, and that this behavior should happen irrespective of the original Sender's published DMARC policy, I am not saying that mailing list programs should relay to its subscribers fake or phishing email from unverified sources.
 
It would go something like this:

1.Sender -> 2.MTA at mailing list host -> 3.ML processes message -> 4.ML program relays message.

DMARC/DKIM/SPF checking should happen at stage 2, not at stage 3. Fake/phishing email should be detected and rejected at stage 2, and only clean legitimate email should arrive at stage 3, and at that point the mailing list program would "do its work" adding subject tags, body footers, AND taking ownership of the Header-From (hopefully indicating the original Sender in the description inside the Header-From), and then go to step 4 and relay to the list members that message. And "that message" would be a clean, wanted, legitimate email which ALSO would solve the DMARC issues with mailing lists.
 
The only check the mailing list program should do (step 3) is to ascertain whether the original Sender is or is not a subscribed address. Checking the validity/authentication of that address belongs to step 2 and to the MTA just before the mailing list program sees the message.
 
So I think that your worries of "putting customers at risk" are not warranted, unless I do not understand what you are saying.

> more fun to work with. Seriously.

Please, do not get frustrated. It is not helpful to the conversation.

Regards,
J.Gomez

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Sat, 31 May 2014 21:18:53 +0900

J. Gomez writes:

 > Furthermore, what is more important - to deserve or not to deserve
 > the prize of being sanctioned as kosher, or keeping a world-wide
 > system interoperable?

In the face of bullying by large operators counting on the fact that
ostracizing them would seriously annoy millions of *our* users and
customers, I think calling it "bullying" is one of the more important
steps we can take to keep that system interoperable.

 > Yes, that is true. But a default out-of-the-box always has to
 > exist, and in my opinion that default should be the most
 > interoperable which is possible --interoperable with the real
 > world, not with how we would like the world to be--, while keeping
 > as much valuable features as possible, and while keeping operator's
 > duties as straight forward and simple as possible.

There's interoperability and there's interoperability.  GNU Mailman
has a feature such that (1) a DNS query checks for "p=reject" and (2)
if found, the message is wrapped as a message/rfc822 part in a message
from the mailing list and containing the header (if any) and footer
(if any) as separate MIME parts.  This is fully RFC conformant (it's
basically a one-message digest) and almost all MUAs should be able to
handle it.  It is what I propose as default for Mailman.

Do you see any defects in that proposal?

 > content,

It can be and is argued, but the argument is clearly incorrect.  The
most recent RFCs state quite clearly that "same message" is in the eye
of the users, and I cannot imagine that addition of a "[list]" tag to
Subject or an unsubscribe footer to the message body would be
interpreted by anyone involved in the list as making the list the
author.

Steve

---

From: "J. Gomez" <[email-masked]>
Date: Sat, 31 May 2014 19:48:54 +0200

> steps we can take to keep that system interoperable.

I don't mind calling it bullying, but I would not focus on that, but on the interoperability side of the issue.

Users won't care about the politics of the email system, but about relevant and wanted email landing on their inbox -- hopefully in an easily readable manner.

> Do you see any defects in that proposal?

That proposal is fine from a technical point of view - it solves the problem and is interoperable. However, that proposal is not perfect from a usability point of view - how well will an attached message/rf822 part be legible when the final recipient is using webmail, or a mobile device with a simple MUA? - how much a non-technical user will freak out when he gets such a message from a mailing list? - will he call the help desk about it, incurring support costs and being a burden for that help desk? - will that proposal unnecessarily consume time in the Recipient side of the communication, basically off-loading into the final innocent Recipient the cost of solving "the DMARC issue"? - is that proposal convoluted for the final Recipient's comfort?
 
I guess that once the interoperability issue is agreed to be important, now the ergonomics of the proposed solution should get the focus. And that proposal is not very ergonomical, in my humble opinion.

> interpreted by anyone involved in the list as making the list the

If DKIM signatures were not involved, you would be right. Now that DKIM signatures are indeed involved, and considered useful in the email ecosystem, any tampering with a message's content means taking responsibility of the result of such tampering, therefore taking ownership of the Header-From.

Regards,
J.Gomez

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Sun, 01 Jun 2014 04:44:55 +0900

J. Gomez writes:

 > Users won't care about the politics of the email system, but about
 > relevant and wanted email landing on their inbox -- hopefully in an
 > easily readable manner.

If you have users like that, configure your lists accordingly.  The
options are available (in GNU Mailman, at least).

 > That proposal is fine from a technical point of view - it solves
 > the problem and is interoperable. However, that proposal is not
 > perfect from a usability point of view - how well will an attached
 > message/rf822 part be legible when the final recipient is using
 > webmail,

Works fine in GMail and SquirrelMail, IIRC (another developer did the
testing).

 > is that proposal convoluted for the final Recipient's comfort?

No.  The proposal is intended to ensure delivery of messages posted
from an ESP whose DMARC policy is in conflict with mailing list
policy, with the message as posted by the author and modified by the
mailing list, and comfortably readable in any MUA conforming to RFC
1341 (published in June 1992).  A more pleasant reading experience is
available in MUAs conforming to RFC 1806 (June 1995).

If the list operator knows that she has users with MUAs that don't
conform to 20-year-old standards, she can change the options easily at
any time.

 > I guess that once the interoperability issue is agreed to be
 > important, now the ergonomics of the proposed solution should get
 > the focus. And that proposal is not very ergonomical, in my humble
 > opinion.

So tell the MUA developers about it.  It takes a moderate amount of
code to DTRT with nested MIME parts and Content-Disposition: inline,
but it's not rocket science, not compared with supporting text/html.