Subject: [dmarc-ietf] Proposing an extension to DMARC to optionally inform receivers about mailing list activity

From: "J. Gomez" <[email-masked]>
Date: Fri, 12 Apr 2013 23:35:35 +0200

Hello all.

I would like to propose an extension to DMARC, in order to address its mailing list problem, as much as it can be addressed --which is not much-- considering current mailing list software behaviour of keeping the original RFC5322.From header (while altering either/both the Subject and the Body) when forwarding emails to its subscribers, therefore both (A) breaking the original DKIM signature and (B) relaying from a MTA which is not SPF-authorized by the Organizational Domain in the RFC5322.From header; all of which causes DMARC-failures for those posts being received from mailing lists.

I propose an additional tag for the DMARC resource record in DNS. This tag is to be optional. This optional tag is "l=" and it could have the values "yes" or "no" or "dunno". Default value would be "dunno" which would be the effective value if the "l=" tag was omitted. The meaning of "dunno" in the "l=" tag is explicitly 'undefined'. The meaning of "yes" and "no" in the "l=" tag are described in the next paragraph.

The "l=" tag is an informative declaration from the Sending Domain Owner to all Receivers about whether the sending Organizational Domain's users may participate or not in mailing lists. The "l=" tag does no convey policy from Sender to Receiver, and Receivers are not formally required nor recommended any course of action based on the content of the "l=" tag. Instead the "l=" tag only conveys dry, aseptic information from Sender to Receiver. Receivers are free to either disregard the contents of the "l=" tag altogether or apply local policies based on the information disclosed by the Sender by means of the "l=" tag.

This approach basically outsources to the Receivers the problem of dealing --if at all, at the Receiver's discretion-- with the by-design DMARC false positives of mailing lists posts. As the default value for the "l=" tag is "dunno", this 'outsourcing' is an opt-in tunable knob which the Sender can enable or disable at will, and also a dry, aseptic information exchange which the Receiver can choose to either ignore or apply special local policies/checks/verifications on it.

----

Counter-replies to possible objections:

1. (Operative problem) There is a considerable already functional base of pre-standard DMARC installations on the field and this proposition of a new tag in the DNS RR for DMARC makes their lives hard.

    --> This proposition just adds a new tag to the DMARC RR in DNS, and that same thing has already happened when the DMARC draft was moved from pre-IETF track to the IETF track less than a month ago, when the new tag "fo=" was added to the DMARC RR in DNS apparently to none's objection. ( See Section 6.2, "General Record Format", http://www.dmarc.org/Diff_draft-dmarc-base-00-02_IETF-00.html )

---

2. (Semantic problem) There is a considerable already functional base of pre-standard DMARC installations on the field and this proposition changes the semantics of DMARC and therefore makes their lives hard.

    --> The "l=" tag is optional, and its default value is explicitly 'undefined' (i.e., "dunno"). This means the semantics of a DMARC RR in DNS do not change unless the Sender explicitly deviates from default behaviour, and even then Receivers are free to disregard the additional information provided by the Sender by means of the "l=" tag.

---

3. (Complexity problem) This propositions makes DMARC more complex, and complexity tends to deter adoption.

    --> The "l=" tag is optional, and it's default value changes nothing from the current DMARC status quo. Most senders can ignore the "l=" tag when configuring their DMARC RR in DNS, and most Receivers will just do whatever their DMARC plugin/milter does by default regarding the "l=" tag without giving it a second thought unless they feel the itch of the mailing list problem.

---

4. (Uselessness problem) This proposition achieves nothing.

    --> True. It's not the intent of the "l=" tag to achieve anything nor to convey policy from Sender to Receiver. The "l=" tag merely conveys additional information from Sender to Receiver, and the Receiver is free to consume it as he wishes or to disregard it altogether.

---

5. (There ain't a problem in the first place) This proposition is not needed.

    --> Yeah, right.

----

Regards,

J. Gomez

---

From: Michael Adkins <[email-masked]>
Date: Fri, 12 Apr 2013 22:03:02 +0000

You forgot 
6. Layering Violation
DMARC doesn't have a problem with mailing lists.  DKIM and SPF have a
problem with mailing lists and trying to fix their problem in DMARC is the
wrong place.

And

7.  Operational Issue
This problem is better solved by making data about legitimate mailing
lists publicially available so everyone can apply local policy overrides
consistently.

On 4/12/13 2:35 PM, "J. Gomez" <[email-masked]> wrote:

>Hello all.
>
>I would like to propose an extension to DMARC, in order to address its
>mailing list problem, as much as it can be addressed --which is not
>much-- considering current mailing list software behaviour of keeping the
>original RFC5322.From header (while altering either/both the Subject and
>the Body) when forwarding emails to its subscribers, therefore both (A)
>breaking the original DKIM signature and (B) relaying from a MTA which is
>not SPF-authorized by the Organizational Domain in the RFC5322.From
>header; all of which causes DMARC-failures for those posts being received
>from mailing lists.
>
>I propose an additional tag for the DMARC resource record in DNS. This
>tag is to be optional. This optional tag is "l=" and it could have the
>values "yes" or "no" or "dunno". Default value would be "dunno" which
>would be the effective value if the "l=" tag was omitted. The meaning of
>"dunno" in the "l=" tag is explicitly 'undefined'. The meaning of "yes"
>and "no" in the "l=" tag are described in the next paragraph.
>
>The "l=" tag is an informative declaration from the Sending Domain Owner
>to all Receivers about whether the sending Organizational Domain's users
>may participate or not in mailing lists. The "l=" tag does no convey
>policy from Sender to Receiver, and Receivers are not formally required
>nor recommended any course of action based on the content of the "l="
>tag. Instead the "l=" tag only conveys dry, aseptic information from
>Sender to Receiver. Receivers are free to either disregard the contents
>of the "l=" tag altogether or apply local policies based on the
>information disclosed by the Sender by means of the "l=" tag.
>
>This approach basically outsources to the Receivers the problem of
>dealing --if at all, at the Receiver's discretion-- with the by-design
>DMARC false positives of mailing lists posts. As the default value for
>the "l=" tag is "dunno", this 'outsourcing' is an opt-in tunable knob
>which the Sender can enable or disable at will, and also a dry, aseptic
>information exchange which the Receiver can choose to either ignore or
>apply special local policies/checks/verifications on it.
>
>
>----
>
>
>1. (Operative problem) There is a considerable already functional base of
>pre-standard DMARC installations on the field and this proposition of a
>new tag in the DNS RR for DMARC makes their lives hard.
>
>    --> This proposition just adds a new tag to the DMARC RR in DNS, and
>that same thing has already happened when the DMARC draft was moved from
>pre-IETF track to the IETF track less than a month ago, when the new tag
>"fo=" was added to the DMARC RR in DNS apparently to none's objection. (
>See Section 6.2, "General Record Format",
>http://www.dmarc.org/Diff_draft-dmarc-base-00-02_IETF-00.html )
>
>---
>
>2. (Semantic problem) There is a considerable already functional base of
>pre-standard DMARC installations on the field and this proposition
>changes the semantics of DMARC and therefore makes their lives hard.
>
>    --> The "l=" tag is optional, and its default value is explicitly
>'undefined' (i.e., "dunno"). This means the semantics of a DMARC RR in
>DNS do not change unless the Sender explicitly deviates from default
>behaviour, and even then Receivers are free to disregard the additional
>information provided by the Sender by means of the "l=" tag.
>
>---
>
>    --> The "l=" tag is optional, and it's default value changes nothing
>from the current DMARC status quo. Most senders can ignore the "l=" tag
>when configuring their DMARC RR in DNS, and most Receivers will just do
>whatever their DMARC plugin/milter does by default regarding the "l=" tag
>without giving it a second thought unless they feel the itch of the
>mailing list problem.
>
>---
>
>    --> True. It's not the intent of the "l=" tag to achieve anything nor
>to convey policy from Sender to Receiver. The "l=" tag merely conveys
>additional information from Sender to Receiver, and the Receiver is free
>to consume it as he wishes or to disregard it altogether.
>
>---
>
>_______________________________________________
>dmarc mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/dmarc

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 12 Apr 2013 18:09:37 -0400

What problem does SPF have with mailing lists (hint: there aren't any)?

Scott K

---

From: "J. Gomez" <[email-masked]>
Date: Sat, 13 Apr 2013 00:20:26 +0200

On Saturday, April 13, 2013 12:03 AM [GMT+1=CET], Michael Adkins wrote:

> problem with mailing lists and trying to fix their problem in DMARC
> is the wrong place.

I'm sorry, but the posts I get from this "dmarc-ietf" mailing list are SPF-by-itself "pass" (and could be DKIM-by-itself "pass" if the list administrator wanted to DKIM-sign also). Only when DMARC enters the game is it that I get DMARC "fail", because DMARC mandates "RFC5322.From header alignment" which is not mandated in either SPF nor DKIM by themselves.

So SPF by itself has NO PROBLEM with mailing list. And DKIM by itself has NO PROBLEM with mailing list. It is DMARC which has problems with mailing lists.

Regards,

J. Gomez

---

From: Michael Adkins <[email-masked]>
Date: Fri, 12 Apr 2013 22:31:14 +0000

The original authentication mechanisms no longer pass after transiting a
mailing list.  If either of them did, then it wouldn't be a problem, would
it?

On 4/12/13 3:20 PM, "J. Gomez" <[email-masked]> wrote:

>I'm sorry, but the posts I get from this "dmarc-ietf" mailing list are
>SPF-by-itself "pass" (and could be DKIM-by-itself "pass" if the list
>administrator wanted to DKIM-sign also). Only when DMARC enters the game
>is it that I get DMARC "fail", because DMARC mandates "RFC5322.From
>header alignment" which is not mandated in either SPF nor DKIM by
>themselves.
>
>So SPF by itself has NO PROBLEM with mailing list. And DKIM by itself has
>NO PROBLEM with mailing list. It is DMARC which has problems with mailing
>lists.
>
>Regards,
>
>J. Gomez
>

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 12 Apr 2013 18:41:37 -0400

For email lists that retransmit mail using their own mail from (which is any 
list that is more than a simple expander) they will pass SPF just fine.  The 
issue from a DMARC perspective is that typically there's no identity alignment 
between the mailing list's Mail From and the original messages Body From.  
That's not an SPF issue.  SPF has nothing to do with what happens inside the 
body of the message.

Scott K

---

From: "John Levine" <[email-masked]>
Date: 12 Apr 2013 22:55:06 -0000

>What problem does SPF have with mailing lists (hint: there aren't any)?

If you consider a forwarder that forwards to multiple addresses a very
simple mailing list, you get the usual forwarder issues.

In case anyone (not Scott) was wondering, this is not DMARC's problem.

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 12 Apr 2013 19:26:48 -0400

> In case anyone (not Scott) was wondering, this is not DMARC's problem.

For the subset of lists that are essentially operating as forwarders, this is 
correct.  The good news is that the original DKIM signature generally has a 
high survival rate with this class of lists.

For transparent forwarding (including cases where it's expanded as above) SPF 
will tend to fail and DKIM will tend to succeed, so DMARC should mostly work 
OK.

This issue with mailing lists and DMARC is really confined to the case of lists 
that modify the message in transit (generally with a new Mail From) so that 
the original DKIM signature no longer validates and the SPF check will pass, 
but does not have the identifier alignment that DMARC wants.  Just as the well 
known SPF forwarding "problem" is not DMARC's problem (as John says), DMARC 
checks failing due to this lack of alignment is not SPF's problem either.

Scott K

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 13 Apr 2013 15:23:55 +0200

On Sat 13/Apr/2013 00:20:26 +0200 J. Gomez wrote:
> I'm sorry, but the posts I get from this "dmarc-ietf" mailing list
> are SPF-by-itself "pass" (and could be DKIM-by-itself "pass" if the
> list administrator wanted to DKIM-sign also). Only when DMARC
> enters the game is it that I get DMARC "fail", because DMARC
> mandates "RFC5322.From header alignment" which is not mandated in
> either SPF nor DKIM by themselves.

As a purely syntactical exercise, let me try and rewrite the last
paragraph using the syntax of the A-R header field:

  I'm sorry, but the posts I get from this "dmarc-ietf" mailing list
  have spf=pass (and could have dkim=pass if the list administrator
  wanted to DKIM-sign also). Only when DMARC enters the game is it
  that I get dmarc=fail, because DMARC mandates header.from alignment
  which is not mandated in either SPF nor DKIM by themselves.

Nice eh?

IMHO, a DKIM-Signature by the MLM would help when the post is
transparently forwarded to some account different from from the
subscribed address.  In that case one gets spf=fail.

---

From: Tim Draegen <[email-masked]>
Date: Sun, 14 Apr 2013 22:08:43 -0400

>    --> True. It's not the intent of the "l=" tag to achieve anything nor to convey policy from Sender to Receiver. The "l=" tag merely conveys additional information from Sender to Receiver, and th…

This here.  If the tag doesn't achieve anything and could very well be wrong ('cause there is no measurable impact to getting it right), it'll just add noise to a receiver's decision making, and therefore {all of your other counter-arguments}.

To game this out, *IF* this tag existed and was accurately used, as a receiver I'd still have to figure out what sources are mailing lists in order to make the extra "l={yes,no,dunno}" tag useful.  Let's say I have perfect mailing list knowledge, what then am I supposed to do with email coming through a mailing list from a domain with the "l=no" tag?  If the answer is "nothing", then why is this tag even there for me to look at?

I guess I'm left with "How does this make anything better?"

> lists publicially available so everyone can apply local policy overrides

I'd add an additional option: "patch mailing list software to play better with DMARC", but I'm only offering this up with the understanding that upgrading stuff that everyone uses all the time is a slow and painful process, and really doesn't have much to do with DMARC-the-protocol.

-= Tim

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 14 Apr 2013 20:50:57 -0700

On Sun, Apr 14, 2013 at 7:08 PM, Tim Draegen <[email-masked]> wrote:

> I'd add an additional option: "patch mailing list software to play better
> with DMARC", but I'm only offering this up with the understanding that
> upgrading stuff that everyone uses all the time is a slow and painful
> process, and really doesn't have much to do with DMARC-the-protocol.
>
>
+1 to all of that.

An option like this in DMARC could be set by anyone, including bad actors.
That means an operator needs to decide whether it believes the flag's
presence for a given domain, because the domain owner may or may not be
legit.  That means an operator needs intelligence about the domain to make
that decision.  That means an operator can gather that intelligence and
apply it irrespective of whether this flag is set for a given domain.  That
means we don't need the flag.

This feels like an instantiation of the evil bit.  (See RFC3514)

-MSK

---

From: "John Levine" <[email-masked]>
Date: 15 Apr 2013 11:17:22 -0000

>This feels like an instantiation of the evil bit.  (See RFC3514)

Indeed, and not the first time someone didn't understand that 3514 is
a joke.

---

From: "J. Gomez" <[email-masked]>
Date: Mon, 15 Apr 2013 22:56:59 +0200

On Monday, April 15, 2013 4:08 AM [GMT+1=CET], Tim Draegen wrote:

> >    --> True. It's not the intent of the "l=" tag to achieve
> > anything nor to convey policy from Sender to Receiver. The "l=" tag
> > merely conveys additional information from Sender to Receiver, and
> > the Receiver is free to consume it as he wishes or to disregard it
> > altogether.    
> 
> 
> This here.  If the tag doesn't achieve anything and could very well
> be wrong ('cause there is no measurable impact to getting it right),
> it'll just add noise to a receiver's decision making, and therefore
> {all of your other counter-arguments}.

It's not noise. It's additional information the domain owner makes publicly available about what can or cannot be expected from his mailing users. If the domain owner uses the "l=" tag, the information is there. If the receivers decides to not apply local policies based on it, that's fine and their choice. But I don't see that as noise.

If you see a ship at the high seas with a pirate flag, you can choose to ignore it because you think it's a joke, or you can choose to change your sailing route. But you have the option to choose based on the information you got from the other party. The information is there. You opt to not believe it? Fine. But don't pretend the information was not there, because it was.

The "l=" flag achieves nothing by itself, because it is not about declaring the policy of the Sender, but about declaring the practices of the Sender so that the Receiver can apply his local policies for that practices of the Sender. So it's the receiver who can opt to achieve something based on that information, or to ignore it.

> To game this out, *IF* this tag existed and was accurately used, as a
> receiver I'd still have to figure out what sources are mailing lists
> in order to make the extra "l={yes,no,dunno}" tag useful.  Let's say
> I have perfect mailing list knowledge, what then am I supposed to do
> with email coming through a mailing list from a domain with the
> "l=no" tag?  If the answer is "nothing", then why is this tag even
> there for me to look at?

The idea in that case is that you as a Receiver are supposed to do whatever your local policy is about accepting mail from mailing lists. I myself, when acting as a Receiver, if I got email which failed DMARC because of "p=reject" and also found "l=no" in the Sender's DMARC record, then I would irrevocably reject/dump that incoming message and avoid the processing time/effort of post-processing it after-DMARC against any additional local mailing-list detection heuristics/whitelists. I see value in having that option as a Receiver, makes life easier for the Receiver, don't you think?

Regards,

J. Gomez

---

From: "J. Gomez" <[email-masked]>
Date: Mon, 15 Apr 2013 23:18:12 +0200

Murray S. Kucherawy wrote:
> An option like this in DMARC could be set by anyone, including bad
> actors.  That means an operator needs to decide whether it believes
> the flag's presence for a given domain, because the domain owner may
> or may not be legit.  That means an operator needs intelligence about
> the domain to make that decision.

An operator also needs intelligence about a domain when he finds an incoming email that fails DMARC and the Sender has a DMARC policy of "p=quarantine" -- is that domain reputable?, is that domain frequently victim of phishing?, etc.

> That means an operator can gather
> that intelligence and apply it irrespective of whether this flag is
> set for a given domain. That means we don't need the flag.

Uh? Non sequitur. An operator also has to decide to believe or not to believe the "p=none" policy of any sender because it "can be set by anyone, including bad actors".

I guees then we also don't need "p=quarantine" nor "p=none", right?

> This feels like an instantiation of the evil bit.  (See RFC3514)

LOL

Regards,

J. Gomez

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 15 Apr 2013 15:12:02 -0700

On Mon, Apr 15, 2013 at 2:18 PM, J. Gomez <[email-masked]> wrote:

> An operator also needs intelligence about a domain when he finds an
> incoming email that fails DMARC and the Sender has a DMARC policy of
> "p=quarantine" -- is that domain reputable?, is that domain frequently
> victim of phishing?, etc.
>

A domain that publishes "p=quarantine" or "p=reject" does itself no favors
in terms of reaching the inbox when a failure occurs.  What incentive does
an attacker (whose goal is to reach the inbox) have to do that?

But more generally, yes, I imagine an operator could (and should, and does,
and will) detect an abuser that is saying "p=none" and act accordingly.

-MSK

---

From: Dave Crocker <[email-masked]>
Date: Sat, 20 Apr 2013 11:03:05 -0700

>> lists publicially available so everyone can apply local policy overrides

Anything that smacks of hints, heuristics, guessing and statistics 
invites more noise into the system.  As a rule, that's not good for 
standards-based activities and are best left to outside processes.

The underlying issue here seems to be a) message handling by 
non-transparent intermediaries, and b) their not necessarily playing in 
the dmarc sandbox.

I think there is nothing useful that can be done in terms of DMARC 
standards for the latter.  If they break stuff and won't help fix it, 
the topic is done.

For the former, it sounds like a case of wanting some kind of transitive 
trust, which moves us back to having the mediator do inbound dmarc 
validation, record it into the A-R header field(?) and then ensure SPF 
and DKIM authentication and maybe dmarc publishing, too.

It might also make sense to add some sort of "I am a mediator who 
processed this message and assert the following truths..." and then 
include that field in the dkim signature hash.  That is, to define some 
additional authentication semantics for mediators, if they wish to play 
in this sandbox.  The presence of this extra header field asserts a set 
of semantics and the dkim signature validates its presence (depending 
upon the reputation of the signing mediator's domain, of course.)

This provides a kind of authenticated audit trail, but of course still 
leaves open the question of how it would be handled by receivers.

d/

-- 
  Dave Crocker
  bbiw.net