Subject: [dmarc-ietf] Ideas for list handling

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 5 Apr 2015 18:33:52 -0700

Colleagues,

I've posted a new version of draft-kucherawy-dkim-list-canon, which had
expired.  It was one of several we were considering a while back as a way
of dealing with some indirect mail flows.

https://datatracker.ietf.org/doc/draft-kucherawy-dkim-list-canon/

Also, I've posted a new one that proposes a way to include in the signature
some information about message transformations that happened at a Mediator,
allowing the Verifier to undo said changes and re-try the author
signature.  Something else to consider:

https://datatracker.ietf.org/doc/draft-kucherawy-dkim-transform/

Comments on either or both are welcome.

-MSK

---

From: Michael Jack Assels <[email-masked]>
Date: Mon, 06 Apr 2015 11:25:17 -0400

On Sun, 05 Apr 2015 18:33:52 PDT, 

> Comments on either or both are welcome.

Thanks for both of these.  They significantly enhance the ability of
compliant list mail surviving DKIM verification.

Tiny spelling nits:

In draft-kucherawy-dkim-list-canon, Appendix A, just before figure 1:

   "descendents" should be "descendants"

(The former is becoming obsolete.)

In draft-kucherawy-dkim-transform, Section 6, first paragraph:

   "additonal" should be "additional".

A small nit:

In draft-kucherawy-dkim-list-canon, Section 4, item 3, asks for

   "An integer expression of the number of children at that node."

Just in case there are more than nine children, it might be helpful to
specify 

   "The decimal number of children at that node."

A slightly larger nit:

In draft-kucherawy-dkim-list-canon, Section 6.2, I can't make the last
sentence parse:

   Operators that might grant preferential handling based on valid DKIM
   signatures from favorable domains; assuming that appended content in
   the presence of such signatures does not mean the appended content
   is necessarily safe.

If I understand what you're trying to communicate, I'd suggest:

   Operators that might grant preferential handling based on valid DKIM
   signatures from favored domains should not assume that the added content
   is necessarily safe, despite the presence of a valid DKIM signature.

In both documents, there's a conspicuously missing item that would make
list subscribers -- and owners -- a lot happier:  A mechanism for changing
the RFC5322.Subject header.  Since most lists nowadays add something like
"[list-name] " immediately after "Subject: " in the originating Author's
message, they still won't pass DKIM validation even after complying
with the proposed body modification rules.  Would it not be fairly
easy to add an easily reversible "change-subject" transformation to
draft-kucherawy-dkim-transform document, along with a corresponding "cs"
DKIM-Signature tag?  Assuming that the mediated RFC5322.From header is
unchanged, this would make it fairly simple for the originating Author's
message to be reconstructed from the message delivered to list members.
But perhaps it might be better to put header transformations in a
separate draft.

Of course, none of this touches the issue of From-munging, but that's
a DMARC alignment problem, not a DKIM problem, so these drafts aren't
the place to address it.

Thanks again,

MJA

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 6 Apr 2015 10:06:44 -0700

> But perhaps it might be better to put header transformations in a

It's conspicuously missing mainly because of the thread from last week that
talks about why we avoided dealing with Subject: tagging during the
original development of DKIM.  However, if consensus is that this general
approach is viable, then yes, it would be possible to have a second set of
additional reversible mutations such as this.

The difference in this case is that the list is taking responsibility for
the mutations by signing the mutated form.

The advantage of this method over the CDKIM or "v=2" method is that it's
purely incremental to DKIM, so we don't have to touch RFC6376.

-MSK

---

From: Michael Jack Assels <[email-masked]>
Date: Tue, 07 Apr 2015 13:26:31 -0400

On Mon, 06 Apr 2015 10:06:44 PDT, 
"Murray S. Kucherawy" <[email-masked]> wrote:

> On Mon, Apr 6, 2015 at 8:25 AM, Michael Jack Assels
> <[email-masked]> wrote:
> 
> > the RFC5322.Subject header. [...]
> 
> original development of DKIM.

Sorry.  It wasn't my intention to raise a question that had already been
settled.  I did read last week's thread.  However, since your drafts appear
to have done so much to address other issues that bother mailing list
people, I thought it would be useful to handle subject tags in essentially
the same way, the only significant difference being that they appear in
headers rather than in the message body.

> However, if consensus is that this general approach is viable, then yes,
> it would be possible to have a second set of additional reversible
> mutations such as this.

The consensus is deafening. :-)

MJA

---

From: "John Levine" <[email-masked]>
Date: 8 Apr 2015 16:19:06 -0000

>Comments on either or both are welcome.

They both have the same problem: the list says:

 Here's what I did.  Whadda ya think?

Every recipient system has to come up with its own heuristics to
decide what combinations of changes are or are not acceptable, which
means that the exact same message will be accepted by one 100%
conformant implementation and rejected by another.  This does not
seem to me to be a major improvement over the current situation.

R's,
John

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 8 Apr 2015 10:26:30 -0700

> seem to me to be a major improvement over the current situation.

But I think that's true of every protocol we have now.  For example,
independent of DMARC, a valid DKIM-signed message might be rejected by "A"
and not by "B" because of its content based on local policy and filtering.
Local heuristics about acceptable content will always be there regardless
of what we do.

The goal here is not acceptance, but deterministic results from the
authentication layer.  Or, more generally, we need to be able to recover a
validated identifer that aligns in a way that doesn't degrade the integrity
of that validation.

Being able to have the author signature cover the original content and the
list signature cover any changes seems like a win to me.

-MSK

---

From: "John R Levine" <[email-masked]>
Date: 8 Apr 2015 14:06:54 -0400

> authentication layer.

Well, that's the problem.  The current spec has a well defined rule that a 
verifier uses on the headers and body and the key from the DNS.  Either 
the signature's valid or it isn't.  The recipient can certainly decide to 
do whatever it wants with that bit, but it's one well defined bit.

Unless I'm misreading these drafts, the signature now says "I took the 
message, and then I deleted this part, and then I added that part" or the 
like.  While it's likely possible for the recipient to say, yes, that's 
what you did, the recipient still has to make up its own rules about what 
transformations it likes, probably including body filtering of new parts.

That seems too squishy to have much hope of interoperating.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 8 Apr 2015 14:14:05 -0700

On Wed, Apr 8, 2015 at 11:06 AM, John R Levine <[email-masked]> wrote:

> verifier uses on the headers and body and the key from the DNS.  Either the
> signature's valid or it isn't.  The recipient can certainly decide to do
> whatever it wants with that bit, but it's one well defined bit.
>

Are these not well-defined rules, in the same vein that canonicalizations
are?  That's certainly the intent here.

The existing "relaxed" canonicalizations spell out a bunch of things you do
to the content before you compute the hashes.  That's all these do as
well.  It's more involved, to be sure, but in the end you're just trying to
figure out if the "d=" domain took responsibility for the content.

-MSK

---

From: "John R Levine" <[email-masked]>
Date: 8 Apr 2015 17:21:12 -0400

> are?  That's certainly the intent here.

They are, but they're not useful in the way that the existing ones are.

The existing ones are strict enough that everyone appears to agree that if 
the signature validates, it's the same message for a pretty strict version 
of "same".  These say that it's a message somewhat related to the 
original, but "somewhat related" isn't useful in the same binary way that 
"same" is.

Regards,

---

From: Douglas Otis <[email-masked]>
Date: Wed, 08 Apr 2015 15:04:09 -0700

> figure out if the "d=" domain took responsibility for the content.
Dear Murray,

You seem to presume DKIM redefinition is able to override
DMARC assumption based policies.

DMARC makes questionable assumptions of From header fields
playing the role of Sender.   An assumption especially
unlikely when there is a different signed Sender header
field which undermines an assertion of figuring out if the
"d=" domain took responsibility for the content.

DMARC was promoted by bulk senders who continue to ignore
broadband provider's (ab)use of DMARC now affecting millions
who must routinely examine their spam folder for errant
message placement.

DMARC also continues a mistake made by DKIM that
non-negotiated message structure is somehow assured
validated by a store-and-forward transport.  A mistake still
leveraged in phishing attacks; perhaps even fine tuned with
use of DMARC feedback, since Botnets have automated
DMARC deployment.

DMARC does indeed discourage phishing abuse of transactional
messages.  When (ab)used elsewhere, it exposes mailing-list
member privacy without those so exposed ever sending a
single message.  Malefactors can subscribe to mailing-lists
and subsequently toggle their DMARC assertions.

DMARC feedback given to these malefactors allows gleaning of
otherwise private subscription information.  While not all
feedback is returned, malefactors can easily differentiate
local policy results based on DMARC count offsets, offering
malefactors far better insight into their proficiencies.

The sheer might of broadband providers undermines Author
roles of the From header field when incorrectly assumed also
playing the role of Sender. The only sure escape for
non-transactional services is to abandon From header field's
assured role of Author and to seek headers so defined but
not (ab)used by transactional services.  Rather than using a
complex encapsulation scheme not needed when the
mailing-list is well managed, defining a different header
should offer a less complex and more immediate solution
without broadband provider cooperation.

DMARC with DKIM does not  determine whether a domain took
responsibility.  DMARC with DKIM forces an errant role of
Sender on the From header field while deprecating the role
of Author. 

Regards,
Douglas Otis

---

From: Michael Jack Assels <[email-masked]>
Date: Wed, 08 Apr 2015 19:08:27 -0400

On 08 Apr 2015 17:21:12 EDT, 

> original, but "somewhat related" isn't useful in the same binary way that 

These new well-defined rules provide a mechanical procedure for determining
whether the encapsulated message body is the "same", provided that the full
message is DKIM-verified.  It's arguably more the "same" than a normal
mailing list posting with a list trailer added, because the list trailer
under these rules will belong to the list's part of the composite message.
Only the headers are different, and they were going to be different anyway,
given DMARC's strict requirements on the From header.

MJA

---

From: "John R Levine" <[email-masked]>
Date: 8 Apr 2015 19:18:42 -0400

> message is DKIM-verified.

Yeah, I can add a giant new MIME part of arbitrary spamminess and it'll 
DKIM verify.  Can someone explain in detail how a verifier is supposed to 
use this new hack.  Consider these two messages:

a) has a one line trailer part saying
"for more information about foo list see http://foolist.org"

b) has a 50 line trailer explaining that my credit card has been 
cancelled and I need to click on this malware link immediately.

Both have a valid list-whatever signature.

Regards,

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 8 Apr 2015 16:35:44 -0700

On Wed, Apr 8, 2015 at 4:18 PM, John R Levine <[email-masked]> wrote:

> b) has a 50 line trailer explaining that my credit card has been cancelled
> and I need to click on this malware link immediately.
>

Aren't you going to run them through your spam filter regardless, so the
nasty stuff will get caught anyway?

Assuming the schemes in those drafts worked, both cases have a valid
list-whatever signature AND a valid author signature, AND you know the (a)
or (b) added bit is solely the responsibility of the list (and, conversely,
you also know where the original content starts and ends).  Nobody's saying
it's safe in any case, but you do know who did what, and that's more than
we know today.

-MSK

---

From: "John R Levine" <[email-masked]>
Date: 8 Apr 2015 20:12:46 -0400

> it's safe in any case, but you do know who did what, and that's more than

Indeed, but I don't see why it's useful.  If you're going to run stuff 
through content filters anyway, what's the point?  This sounds like it's 
going to reduce to mostly whitelisting well behaved remailers, which is an 
approach we know large systems aren't likely to use.

Regards,

---

From: Michael Jack Assels <[email-masked]>
Date: Wed, 08 Apr 2015 20:29:19 -0400

On 08 Apr 2015 20:12:46 EDT, 

> approach we know large systems aren't likely to use.

But surely you're going to run all DMARC-OKed mail through content filters,
too.  The person in charge of your domain's DNS servers can't really assure
me of anything except that the message I'm receiving came from your domain.
Almost all the phishing and spam I see comes from compromised, legitimate,
DMARC-certifiable sources.  Maybe I'm safe rejecting unmediated mail that
fails DMARC with p=reject, but everything else goes through content
filters.  So why is DMARC any more useful than these "hacks"?

MJA

---

From: "John R Levine" <[email-masked]>
Date: 8 Apr 2015 20:32:57 -0400

> So why is DMARC any more useful than these "hacks"?

Good question.  As originally intended, DMARC was for mail from sources 
where a failure reliably meant phish.  Then AOL and Yahoo repurposed it to 
push their support costs onto other people, and its value has been under 
debate ever since.

Regards,

---

From: Douglas Otis <[email-masked]>
Date: Wed, 08 Apr 2015 18:45:36 -0700

Dear Murray,

What will knowing more about mailing-lists hope to solve? 
Why not define minimum recommendations for third-party
services for avoiding mistaken assumptions made by DMARC
which may result in valid messages being disrupted by reject
or quarantine DMARC handling.  Frankly, redefining DKIM will
not solve DMARC's basic problem of confusing Author with
Sender roles. The only reasonable course of action is to
cede From Header field role redefinitions to DMARC based on
pragmatic principles and find another header field less
likely (ab)used by transactional messaging.

Regards,
Douglas Otis

---

From: Steve Atkins <[email-masked]>
Date: Wed, 8 Apr 2015 18:47:39 -0700

On Apr 8, 2015, at 5:32 PM, John R Levine <[email-masked]> wrote:

> Good question.  As originally intended, DMARC was for mail from sources where a failure reliably meant phish.  Then AOL and Yahoo repurposed it to push their support costs onto other people, and its value has been under debate ever since.

Also a major reason that people who were dubious about SPF policy and extremely dubious about ADSP supported DMARC was that it has reporting and dry run functionality. Run it in p=none mode; use the reports to make sure that nothing breaks; if nothing breaks switch to p=reject.

I didn't think that anyone significant would skip the testing, reporting and decision making steps and leap directly to intentionally breaking email for their users their users' correspondents.

Cheers,
  Steve

---

From: Hector Santos <[email-masked]>
Date: Wed, 08 Apr 2015 23:14:19 -0400

> I didn't think that anyone significant would skip the testing, reporting and decision making steps and leap directly to intentionally breaking email for their users their users' correspondents.

I don't think this is a fair assessment.  Obviously for many, it was 
expected for a DKIM+POLICY to prevail in the market place where 
publishers will expose strong rejection policies and receivers will 
begin to honor it.  DKIM+POLICY (ADSP) was a standard track IETF WG 
work item with many RFC documents produced.  Investments were done. 
APIs were written.  After all, Yahoo invented the idea with Domainkeys 
with its built-in "o=" policy tag, so we should not be surprise Yahoo 
was also the first to finally enable it abeit after 10 years evolution 
to DKIM+DMARC.    What ADSP did not have was a reporting feature. It 
might had changed its fate.  DMARC is fundamentally no different than 
ADSP otherwise.

DKIM+ATPS should not be a problem for most domains. Not everyone is a 
YAHOO and even then, I don't think they should have a problem managing 
"50,000" list domain records, if that is their maximum exposure to this.

If you look at the ideas, ATPS is the least expensive.  All the others 
have the same scale issue for the domain needing to authorize a larger 
list of domains.  And they also have a higher change requirement at 
three locations; signers, receivers and middle ware.  In Levine's 
idea, the middle ware MUST NOT remove the weak 1st party signature.

-- 
HLS

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 09 Apr 2015 16:44:49 -0400

On April 8, 2015 9:47:39 PM EDT, Steve Atkins <[email-masked]> wrote:
>> Good question.  As originally intended, DMARC was for mail from
>sources where a failure reliably meant phish.  Then AOL and Yahoo
>repurposed it to push their support costs onto other people, and its
>value has been under debate ever since.
>
>Also a major reason that people who were dubious about SPF policy and
>extremely dubious about ADSP supported DMARC was that it has reporting
>and dry run functionality. Run it in p=none mode; use the reports to
>make sure that nothing breaks; if nothing breaks switch to p=reject.
>
>I didn't think that anyone significant would skip the testing,
>reporting and decision making steps and leap directly to intentionally
>breaking email for their users their users' correspondents.

I don't think they were confused about the breakage (see Yahoo's original announcement of the change).  They knew what would happen and decided it was Okay given the perceived benefit. 

Scott K

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Fri, 10 Apr 2015 14:25:49 +0900

John R Levine writes:

 > use this new hack.

"Paranoid and easy" approach: Check From alignment of original message
parts.  If verified, strip all additions and deliver, else check DMARC
policy of author domain, and deliver stripped to appropriate folder
per policy (ie, /dev/null for reject).  This really is easy, the code
for stripping specified parts is available in Mailman and presumably a
ton of other email applications.

More sophisticated approaches could be imagined, especially for
mailbox providers who are also MUA providers.

---

From: "John R Levine" <[email-masked]>
Date: 10 Apr 2015 11:05:47 -0400

> parts.  If verified, strip all additions and deliver, ...

Gee, list mail will look just like malware phishes, "we removed dangerous 
content from this message, but for reasons we can't explain we delivered 
it to you anyway".

Regards,

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Sat, 11 Apr 2015 01:44:20 +0900

> content from this message, but for reasons we can't explain we delivered 

No, it will look exactly like what some folks think MLMs should be
doing anyway: passing those posts through verbatim with no value-added
doodads added.  Users will never notice, I bet, and if they do, you
explain and they go "oh, ok" 99.44%[1] of the time.

(Boy you're in a pessimistic mood.  Usually your criticisms are quite
constructive even when they're negative, but this series has been
quite, uh, "and your point might be?")

Footnotes: 
[1]  If you're not an ancient American, you might not recognize that
figure: it's the old Ivory Soap claim of 99.44% purity.  It's my way
of saying "I made the number up, but I believe it to be high".

---

From: "John R Levine" <[email-masked]>
Date: 10 Apr 2015 12:54:10 -0400

> explain and they go "oh, ok" 99.44%[1] of the time.

We have lists where we would be unhappy if the added stuff went away, 
e.g., a line at the front reminding people that the messages are private 
and not to be forwarded.  Even in the unlikely event that MUAs would do 
this, which I find vanishingly unlikely, it doesn't do what lists managers 
need.

I think it would be useful to limit ourselves to hacks that don't require 
changes to MUAs, minimal changes to list software and sending mail 
systems, and don't use external oracles.  Then they might have some 
chance of adoption.

Regards,

[1] This proposal doesn't float.

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Sat, 11 Apr 2015 05:02:18 +0900

John R Levine writes:

 > and not to be forwarded.

OK, so that's new harm done by this behavior.  On some such lists it
might be mitigated by the fact that the mail actually gets through
now.  (But unambiguously in other cases the mail *was* getting through
unscathed but now the recipient MTA decides to strip.)

 > Even in the unlikely event that MUAs would do this,

MUAs, unlikely, I suppose.  MTAs could do it too, and I think that's
more likely.

 > which I find vanishingly unlikely, it doesn't do what lists
 > managers need.

Stop exaggerating.  It is not the case that anywhere near 100% of list
managers actually *need* doodads, although a few (covering less than
1% of recipients, I bet) do need them, and many find them attractive.

 > changes to MUAs,

draft-kucherawy-dkim-list-canon requires no changes to MUAs, although
mailbox + MUA providers *might* want to do it on the MUA side rather
than the MTA side.

 > minimal changes to list software

draft-kucherawy-dkim-list-canon requires no changes to list software,
and it's purpose is to avoid changes to list configurations (or enable
changes that currently must be avoided because of DMARC).

 > and sending mail systems,

I don't see how draft-kucherawy-dkim-list-canon requires more changes
of the sending *system* than draft-levine-dkim-conditional or
draft-kucherawy-dkim-delegate does.  True, the software would be more
complex, but presumably it would be developed by Postfix, Exim,
Sendmail, and Microsoft, plus maybe an independent milter project or
two, and the "paranoid and easy" configuration is dead simple and
possibly suitable as a default (although somewhat dangerous, see
"unambiguously" above).

 > and don't use external oracles.

I see no external oracles required.  Oracles are optional and
desirable, yes, but that's true of everything in mail filtering.

 > Then they might have some chance of adoption.

---

From: "John R Levine" <[email-masked]>
Date: 10 Apr 2015 16:42:38 -0400

> 1% of recipients, I bet) do need them, and many find them attractive.

Well, now we're back to arbitrarily close to 100% of list managers can 
live with rewriting the From: line, too.

> > I think it would be useful to limit ourselves to hacks that don't require

Well, OK, don't depends on recipients rewriting incoming messages.

> of the sending *system*

That one doesn't.  Other proposals involve the sending system publishing 
list of authorized forwarders and the like, which seems like a 
non-starter.

> I see no external oracles required.

That would include global lists of known mailing lists.  Again, Murray and 
I don't need them but other proposals do.

Regards,

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Sat, 11 Apr 2015 06:40:30 +0900

John R Levine writes:

 > Again, Murray and I don't need them but other proposals do.

OK, FWIW, I'm only interested in your proposals at the moment. :-)

Thanks for the courteous response; I didn't entirely deserve it.

---

From: Franck Martin <[email-masked]>
Date: Tue, 28 Apr 2015 12:36:34 -0500 (CDT)

----- Original Message -----

> From: "Murray S. Kucherawy" <[email-masked]>
> To: [email-masked]
> Sent: Sunday, April 5, 2015 6:33:52 PM
> Subject: [dmarc-ietf] Ideas for list handling

> allowing the Verifier to undo said changes and re-try the author signature.
> Something else to consider:

> https://datatracker.ietf.org/doc/draft-kucherawy-dkim-transform/

This one was not in the dmarc-interoperability draft, added.