Subject: [dmarc-ietf] Indirect mail flows

From: "Kelley, John" <[email-masked]>
Date: Mon, 8 Sep 2014 19:05:29 +0000

Hi.  

I'm not sure if it is too soon to start the discussion on indirect mail
flows, but theses are the chief problems we (AOL) are seeing with indirect
mail.

1. Auto Forwards, principally where the email is munged in some way
causing DKIM to fail.
2. Mailing lists; although the big ones seem to be rewriting the From
(thanks).
3. Groups (these might be considered a subset of mailing lists, but folks
seem to think of them differently)

John Kelley

---

From: Elizabeth Zwicky <[email-masked]>
Date: Mon, 8 Sep 2014 20:57:35 +0000

Also these get chained together -- think of the person who subscribes to a mailing list from a university email address that then ends up being forwarded.
Another big, difficult case is people sending mail via an ISP; you have cable Internet at home, you want to send mail via SMTP, you don't want to use "[email-masked]", but you are forced through the ISPs SMTP server. 
We are seeing less "mail to a friend" (where the mail is generated by an app or website) but still some.
Also down but still existent is commercial mail sending for free email addresses -- somebody uses business services to send mail but the business has an email address in somebody else's domain (think "happy birthday" from your dentist, for instance).
Elizabeth Zwicky
      From: "Kelley, John" <[email-masked]>
 To: "[email-masked]" <[email-masked]> 
 Sent: 
 Subject: [dmarc-ietf] Indirect mail flows
   
Hi.  

John Kelley

_______________________________________________
dmarc mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/dmarc

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Tue, 09 Sep 2014 17:39:33 +0900

>From what I've seen on Mailman Project lists[1], your users may not feel
the same way, though.  There have been complaints to list owners that
they're getting singled out (a popular Mailman feature, for example,
checks DMARC policy and munges From or wraps the message only if it's
p=reject).

Have you had any such feedback?

 > 3. Groups (these might be considered a subset of mailing lists, but
 >    folks seem to think of them differently)

What's a "group"?  Specifically, how do messages get submitted to
groups and then injected into the mail system?  If there are multiple
ways of submitting messages, does it matter which one is used?

Footnotes: 
[1]  Can't speak from personal experience, all my "p=reject"
subscribers said "thanks for the heads-up" and changed posting
address.

---

From: "Kelley, John" <[email-masked]>
Date: Tue, 9 Sep 2014 17:08:02 +0000

>Have you had any such feedback?

I have no such feedback, but it might not come to me.

>ways of submitting messages, does it matter which one is used?

We had a pow-wow a few days ago and this came up.  I believe that for
DMARC purposes, we can refer to it as a mailing-list, but it seems as if
people think of them differently.  I believe that folks use the term group
to identify things they "belong" to (neighborhood association, swimming
pool) and the term list to identify things they subscribe to.  I broke it
apart simply because that is the way it was brought up to me.  The inner
workings are, I am sure, very similar to a mailing list.  Perhaps a
moderated mailing list.

John Kelley

---

From: Steve Atkins <[email-masked]>
Date: Tue, 9 Sep 2014 10:50:58 -0700

> checks DMARC policy and munges From or wraps the message only if it's

Some of the changes that have been made to mailing lists to work
around DMARC have made them significantly less useful.

It unavoidably breaks the ability to search for emails or filter inbound
mail by author email address.

It also makes it hard / impossible to reply to the
author, or see what domain the author is associated with - though
that may be because the changes made by the mailing list operators
aren't as well done as they could be rather than something
unavoidable.

Cheers,
  Steve

---

From: Hector Santos <[email-masked]>
Date: Tue, 09 Sep 2014 14:24:39 -0400

That is should be expected when people monkey around with long time 
mail infrastructure. Its a bad idea and sets a terrible precedent by 
alluding to the idea "its normal."  No its not normal. It will be 
exploited and probably its too late to put this one back if a few 
mailing list packages are already doing it.   If isolated, fine, but 
it can't be IETF endorsed in any way.

YAHOO does not want people to use their domain without permission. 
Period and the big gorilla has set the action in motion for many 
others to follow suit with strong policies that with DMARC will bring 
things because it lacks the definition for wider use cases. So we need 
the fastest way to get this resolution out there -- a POLICY LOOKUP 
METHOD with expanding use cases definitions that was long envisioned 
since the git-go.

We been battling this for a long time and its really about what domain 
(1st or 3rd party) do we want to receivers to lookup?

Well, one side wants receivers to look up some 3rd party trust entity 
(ala SSL) and the other side says, fine, but we also want the 
self-signing authority lookup as well - the ownership domain.  There 
are conflicts between the two because the 3rd party guy does not want 
the 1st party guy to have any say over any its mail changes or 
tampering and/or forwarding.  This "corollary" was unfortunately 
written into RFC5016.

    10. SSP MUST NOT provide a mechanism that impugns the existence of
        non-first party signatures in a message.  A corollary of this
        requirement is that the protocol MUST NOT link practices of first
        party signers with the practices of third party signers.

          INFORMATIVE NOTE: the main thrust of this requirement is that
          practices should only be published for that which the publisher
          has control, and should not meddle in what is ultimately the
          local policy of the receiver.

          Refs: Deployment Consideration, Section 4.3.

Unfortunately, this is still a strong desire. Which is fine, but they 
don't want to even bother to ask the 1st party domain what does the 
correct or bad transaction is expected to LOOK like.  It is this 
separation that the DKIM wants to maintain that has long held back the 
resolution to this problem.

Keep it simple.

--
HLS

-- 
HLS

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Wed, 10 Sep 2014 03:40:28 +0900

> It unavoidably breaks the ability to search for emails or filter
 > inbound mail by author email address.

It's not unavoidable, but it's certainly beyond the capability of the
vast majority of users to change their MUAs to deal with it well.  Eg,
in my MUA it's possible to treat any field as unstructured, so you can
recognize addresses from the display name.  You can also filter on
both From and Reply-To at the same time.

In the most recent releases of Mailman, it's possible to wrap a
DMARC-triggering message in a message/rfc822 part, having the outer
>From being the list, and the inner From (and the rest of the inner
message) being as you like it.  For the oddball MUA (mine) it's
possible to program it to go into such messages (this feature is also
useful for certain kinds of forwards, bounces, and digests) and find
the inner from for search or filtering purposes.  Again, I don't
expect The Average Everyday MUA to support that soon.

 > It also makes it hard / impossible to reply to the author,

Mailman automatically adds the author to Reply-To.  If this results in
a header like

    From: [email-masked]
    To: [email-masked]
    Reply-To: [email-masked]

the right thing happens.  However, there are some cases, such as if
the list was in the habit of munging Reply-To to the list, where you
have a choice between too many addresses from a plain "reply" (if you
set munging to *both* the author and the list) or the inability to
automatically reply to author (if you munge to list only).

I should update the wiki.

 > or see what domain the author is associated with

That's the whole point of DMARC, to obfuscate the author unless they
are using the domain's infrastructure to transmit the mail.  So that's
going to be hard to get around.

The display name hack is available, though:

    From: "Alfred E. Neuman ([email-masked]) via Some List" <[email-masked]>

 > - though that may be because the changes made by the mailing list
 > operators aren't as well done as they could be rather than
 > something unavoidable.

Some mailing list management packages already do a substantially
better job, so I would expect the problems to reduce to (a) systems
that won't upgrade their software and (b) the genuinely hard or
impossible cases within a few months.

Steve

---

From: Derek Diget <[email-masked]>
Date: Tue, 09 Sep 2014 14:40:30 -0400 (EDT)

On Sep 9, 2014 at 17:08 -0000, Kelley, John wrote:
=>On 9/9/14 4:39 AM, "Stephen J. Turnbull" <[email-masked]> wrote:
=>>Kelley, John writes:
=>>
=>> > 1. Auto Forwards, principally where the email is munged in some way
=>> > causing DKIM to fail.
=>> > 2. Mailing lists; although the big ones seem to be rewriting the From
=>> > (thanks).
=>>
=>>From what I've seen on Mailman Project lists[1], your users may not feel
=>>the same way, though.  There have been complaints to list owners that
=>>they're getting singled out (a popular Mailman feature, for example,
=>>checks DMARC policy and munges From or wraps the message only if it's
=>>p=reject).
=>>
=>>Have you had any such feedback?
=>
=>I have no such feedback, but it might not come to me.

How are such modifications RFC5321 compliant?  See section 3.9.

...the message header section (RFC5322 [4]) MUST be left unchanged; in 
particular, the "From" field of the header section is unaffected.

=>> > 3. Groups (these might be considered a subset of mailing lists, but
=>> >    folks seem to think of them differently)
=>>
=>>What's a "group"?  Specifically, how do messages get submitted to
=>>groups and then injected into the mail system?  If there are multiple
=>>ways of submitting messages, does it matter which one is used?
=>
=>
=>We had a pow-wow a few days ago and this came up.  I believe that for
=>DMARC purposes, we can refer to it as a mailing-list, but it seems as if
=>people think of them differently.  I believe that folks use the term group
=>to identify things they "belong" to (neighborhood association, swimming
=>pool) and the term list to identify things they subscribe to.  I broke it
=>apart simply because that is the way it was brought up to me.  The inner
=>workings are, I am sure, very similar to a mailing list.  Perhaps a
=>moderated mailing list.

I thought a different RFC gives more detail, but see section 3.9.1 and 
3.9.2 of RFC 5321.

IMO both "lists" and "groups" are a sub-set of "exploders".  The way I 
(and the MTA we run) differentiate the two is that "lists" change the 
RFC5321.MailFrom address for bounce handling (see RFC5321 section 3.9.2) 
while "groups" ("alias" in RFC5321 section 3.9.1) don't.

-- 
***********************************************************************
Derek Diget                            Office of Information Technology
Western Michigan University - Kalamazoo  Michigan  USA - www.wmich.edu/
***********************************************************************

---

From: "John Levine" <[email-masked]>
Date: 9 Sep 2014 18:42:58 -0000

>2. Mailing lists; although the big ones seem to be rewriting the From

Just for the record, the mailing lists I know that are rewriting the
From: line are not doing so because the change is in the interest of
their users, but because of the enormous market power of the mail
systems that will no longer deliver list mail in the form that the
users are accustomed to and prefer.

R's,
John

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Wed, 10 Sep 2014 12:44:10 +0900

> particular, the "From" field of the header section is unaffected.

RFC 5321 is irrelevant in the case of mailing list management software
(MLM) like GNU Mailman, because the changes don't happen in the MTA.
Mail is received, processed outside of the purview of RFC 5321
(including the usual alterations made by the MLM), and then reinjected
into the mail transport system.  Note the last paragraph from 3.9.2:

    There exist mailing lists that perform additional, sometimes
    extensive, modifications to a message and its envelope.  Such
    mailing lists need to be viewed as full MUAs, which accept a
    delivery and post a new message.

where "new message" clearly should *not* be construed to necessarily
mean "new" in the sense of changing RFC5322.Message-Id.  It means
wrapping some version of the message in a fresh "envelope".

OTOH, RFC 5322 specifies the content of the From field to be the
address(es) of the author(s), and that it is an originator field.  I
don't have a channel to the original intent of the RFC authors, but I
personally interpret "originator field" to mean that From-munging is
violation of that RFC, as per policy of most mailing lists (in the
sense of "GNU Mailman", not RFC 5321.  Like John Levine, I don't like
From-munging, and consider it a serious imposition.

 > I thought a different RFC gives more detail, but see section 3.9.1 and 

I interpreted John Kelley to be describing difficulties users report.

 > IMO both "lists" and "groups" are a sub-set of "exploders".  The
 > way I (and the MTA we run) differentiate the two is that "lists"
 > change the RFC5321.MailFrom address for bounce handling (see
 > RFC5321 section 3.9.2) while "groups" ("alias" in RFC5321 section
 > 3.9.1) don't.

That's a useful distinction, although we should probably be careful
about terminology here.  I don't think RFC 5321 terminology is a good
default, since the applications that are hurting are generally not
bound at all by RFC 5321, being MUAs in the sense of that RFC, as
described by the paragraph I quoted above.

But it's fine to deliberately borrow from RFC 5321.  For the
distinction Derek describes above, I like the term "mailing list
manager" (MLM) for exploders that set RFC5321.MailFrom to themselves
and "alias" for exploders that don't.  I think that even if the WG
members all agree on "group" for the latter, some of the people whose
use cases we'll discuss are going to be using "group" for something
else.  I'd rather not have to deal with that.  I can live with "list"
instead of "MLM", though.

---

From: Hector Santos <[email-masked]>
Date: Thu, 11 Sep 2014 07:41:20 -0400

For multiple decades, starting with mail systems predating RFC822, with everyone one of them there was a common mail engineering taboo, "thou shall not tamper with mail" and one of the primary anchoring fields, the author of the message was a principle field you didn't screw around with.   You either get it or you don't.  Mostly those at the application level didn't and it's understandable. But at the system level, it will cause problems.  Even if there was a legitimate transformation, it needs to be reversible and the security remained intact.  In this rewrite, it is neither. It was done prematurely without even trying to fix DMARC first and this irresponsible rewrite action will cause problems for everyone.  The door was cracks opened and even if there is a new solution to address it, you can't put this one away.  The bad guys now know how to get around DMARC security.  This is not good mail engineering.   I make no apology for having strong moral ethical feelings about it.  

--
Hector Santos
http://www.santronics.com

> On Sep 9, 2014, at 11:44 PM, "Stephen J. Turnbull" <[email-masked]> wrote:
> 
> Derek Diget writes:
>

---

From: Roland Turner <[email-masked]>
Date: Thu, 18 Sep 2014 21:31:08 +0800

On 09/11/2014 07:41 PM, Hector Santos wrote:

> For multiple decades, starting with mail systems predating RFC822, with everyone one of them there was a common mail engineering taboo, "thou shall not tamper with mail" and one of the primary anchoring fields, the author of the message was a principle field you didn't screw around with.   You either get it or you don't.

I would suggest that accepting Subject: header tagging, footer addition, 
attachment stripping, etc. but then objecting to From: header rewrites 
to identify the [second] author who made all these changes is not merely 
straining at a gnat having swallowed an entire herd of camels, it's 
missing the point:

  * The "forward it clean" crowd would have messages unmodified right
    down to byte-stream level, but for the pre-pending of trace headers.
    Messages handled this way have various desirable properties,
    particularly including passing DKIM effortlessly, and therefore
    DMARC likewise.
  * Other forwarders change stuff. DKIM makes heroic efforts to
    compensate - and it would be sensible to specify further mitigations
    if there remain workable ones to specify - but the fact remains that
    once you start changing stuff - particularly big things like
    stripping attachments, adding advertisements, etc. - you're no
    longer in the "forward it clean" category; it's no longer even a
    relevant question.

- Roland