Subject: [dmarc-ietf] Additional List-foo Header Field To Help Mitigate Mailing List Damage

From: Scott Kitterman <[email-masked]>
Date: Sat, 13 Sep 2014 12:43:45 -0400

My MUA of choice (Kmail for what it's worth) has a very nice feature called 
"Reply to Mailing List"  It uses the List-Post field to determine the address 
to set the reply to.  I find this very useful, since the many mailing lists I 
am subscribed to are set up differently, so sometimes "Reply" goes to the list 
and sometime it goes to the original sender.

This got me thinking that it might be useful to standardize a way to 
communicate who the original author was in cases where the mailing list is 
configured to rewrite From to avoid DMARC issues.  Google is already doing this 
in a non-standard way in Google Groups.  You'll see something like:

X-Original-From: <[email-masked]>

If this were standardized, then MUA authors could use it to add a new reply 
option, something like "Reply to Mailing List Author". 

This would mitigate some of the damage caused by From rewriting as a DMARC 
damage avoidance technique.  One can argue all day if From rewriting should be 
done or not, but it is and it's not going to do away.

Here's the current set of List-* header fields added by this list:

List-Id: "Domain-based Message Authentication, Reporting,
 and Compliance \(DMARC\)" <dmarc.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/dmarc>,
 <mailto:[email-masked]?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/dmarc/>
List-Post: <mailto:[email-masked]>
List-Help: <mailto:[email-masked]?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/dmarc>,
 <mailto:[email-masked]?subject=subscribe>

Perhaps the new one could be:

List-Author: <[email-masked]>

This falls within the working group charter for "Addressing the issues with 
indirect mail flows", specifically, "Message modification by an intermediary, to 
avoid authentication failures, such as by using specified conventions for 
changing the aligned identity."

In terms of work items, I think it is definitely within the scope of "Draft 
description of interoperability issues for indirect mail flows and plausible 
methods for reducing them."  I doubt, however, that standardization of a new 
List-foo field should be buried in a single large document that describes the 
entire issue/mitigations situation for DMARC.

If this seems reasonable to pursue, I think it should be it's own draft that 
the result of this work item can then reference.  If we do this, I can 
guarantee at least one MUA implementation.

Scott K

---

From: Steve Atkins <[email-masked]>
Date: Sat, 13 Sep 2014 10:08:11 -0700

> entire issue/mitigations situation for DMARC.

It's a redefinition of the current meaning of the From: field, and the addition of another
header to take the place of the existing From: field. That's bigger than DMARC, and it
seems like the 5322 people should be involved in that discussion.

Cheers,
  Steve

---

From: Dave Crocker <[email-masked]>
Date: Sat, 13 Sep 2014 10:18:12 -0700

> option, something like "Reply to Mailing List Author". 

This is worth exploring.  In an abstract sense, this is "merely" another
approach to encapsulation.  It uses additional header views, rather than
MIME wrapping, but I think it achieves the same end.

On 9/13/2014 10:08 AM, Steve Atkins wrote:
> It's a redefinition of the current meaning of the From: field, and 
> the addition of anotherheader to take the place of the existing From:
> field. That's bigger than DMARC, and itseems like the 5322 people
> should be involved in that discussion.

Steve is of course correct.

But I think that no matter what gravitate towards, we are talking about
a change to the 'model' of signaling the distinction between the
'author' and whoever is an intermediary.

I am pretty sure that my earlier observation will prove true, no matter
what we choose.  That is, if it works for scenarios with good actors, it
will also be useful for bad actors.

d/

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 13 Sep 2014 13:21:36 -0400

On Saturday, September 13, 2014 10:08:11 Steve Atkins wrote:
> > configured to rewrite From to avoid DMARC issues.  Google is already doing
> > this in a non-standard way in Google Groups.  You'll see something like:
> > 
> > description of interoperability issues for indirect mail flows and
> > plausible methods for reducing them."  I doubt, however, that
> > standardization of a new List-foo field should be buried in a single
> > large document that describes the entire issue/mitigations situation for
> > DMARC.
> 
> It's a redefinition of the current meaning of the From: field, and the
> addition of another header to take the place of the existing From: field.
> That's bigger than DMARC, and it seems like the 5322 people should be
> involved in that discussion.

Rewriting From or adding the new List-foo redefines it?

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 13 Sep 2014 16:38:21 -0400

> MIME wrapping, but I think it achieves the same end.

While the MIME wrapping idea "works", it seemed overly complicated to deal 
with U/I wise and it makes messages "Look funny".  The new header field should 
be easier to deal with in both the MLM and the MUA.  Also, since I already see 
X-Original-From in use, it builds on existing practice.

Given X-Original-From is being used, I think it'd make sense to standardize a 
field name for this sooner rather than later before the non-standard one 
becomes too firmly entrenched.

> will also be useful for bad actors.

For bad actors, I don't think it gives them anything more than they already 
have with reply-to.  For good actors, there are advantages.  So I agree in 
theory, but in practice, I don't think it'll make a difference.

Scott K

---

From: Dave Crocker <[email-masked]>
Date: Sat, 13 Sep 2014 14:03:25 -0700

> While the MIME wrapping idea "works", it seemed overly complicated to deal 

I wasn't arguing preference or superiority of either approach.  I merely
wanted to note that they are in the same conceptual bin.

I should also note that as soon as we go down this path of design
thinking (independent of which might be preferred) we should carefully
consider costs of making changes.

For example, changing all the MUAs in the world, versus changing all the
mailing list software...

> theory, but in practice, I don't think it'll make a difference.

Actually, it does.  Reply-to does not make assertions of authorship.
The type of change being contemplated does.

d/

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 13 Sep 2014 17:28:56 -0400

> For example, changing all the MUAs in the world, versus changing all the

I'm confident we're going to end up with a portfolio of mitigations, similar to 
what's in the Appendices of RFC 7208.  I'm equally confident that despite the 
distaste many of us feel when we consider it, rewriting From in the MLM is 
going to be on the list of mitigations.  It seems to be the most popular in 
use.  It's not going away.

Given that, I'm not sure where your proposed cost comparison could make a 
great difference.  To mitigate the impacts of rewriting from (in this case 
losing information on the original author of the message and having an easy 
way to send a reply to them rather than the list), it's going to take both MLM 
and MUA changes to make it work well.

Eventually, MUAs adapt to provide popular features.  If this idea proves 
popular to their target audience, they'll adopt it, if it doesn't, they won't.

> The type of change being contemplated does.

How much difference does that make to end users?  I mash a button than get a 
message going back to an address.  They both could do that.

Scott K

---

From: Hector Santos <[email-masked]>
Date: Sat, 13 Sep 2014 17:39:14 -0400

This is a GATEWAY concept.  Transformations and its need to be 
reversible with NO LOST OF SECURITY.  Gateways help with heterogeneous 
mail networking and that is what we are basically creating here.

So one way to solve this is for a MEMBER of a list to an 
outbound/inbound translation alias, where the LIST would replace the 
incoming 5322.From Address with a LIST-DOMAIN-OWNED 5322.From 
Membership address and it is this address used with the resigned mail.

A a new list header can be used to indicate that this transformation 
has taken place.

BUT.... WE STILL NEED THE ORIGINAL DOMAIN TO AUTHORIZE THIS 
TRANSFORMATION -- A Lookup Policy protocol.  Even with an in-band 
method, it needs to be verifiable.

This is what everyone keeps forgetting. We came up with all sorts of 
ideas to make it all work over the years, but it required the original 
domain authorization and/or allowance for a secured list 
transformation to take place.

-- 
HLS

On 9/13/2014 12:43 PM, Scott Kitterman wrote:
> _______________________________________________
> dmarc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dmarc
>
>

---

From: Hector Santos <[email-masked]>
Date: Sat, 13 Sep 2014 17:42:29 -0400

> use.  It's not going away.

I hope not. No doubt if its not secured as I mentioned in my last 
post, its going to get filtered just like we added the 5322 Double 
 From FILTER test as well before DKIM even sees it.  This rewrite will 
get exploited and it is irresponsible to promote it.  I will not add 
it to our Wildcat! LIST Server Product line and will use the idea that 
others do as marketing leverage against them. How about that? <g>

-- 
HLS

---

From: Steve Atkins <[email-masked]>
Date: Sat, 13 Sep 2014 14:59:56 -0700

> Rewriting From or adding the new List-foo redefines it?

Yes.

From 5322:

   The "From:" field specifies the author(s) of the message,
   that is, the mailbox(es) of the person(s) or system(s) responsible
   for the writing of the message.  The "Sender:" field specifies the
   mailbox of the agent responsible for the actual transmission of the
   message."

The proposal here is to use the From: field to specify the mailbox
of the agent responsible for the actual transmission of the message
(the mailing list manager); the value that 5322 specifies as the Sender.
That's a significant change to 5322.

Also, to add a new field that will contain the mailbox of the person
responsible for writing the message; what 5322 defines as the From:
field. While adding a new header is fine, adding one that is defined to
contain the value that 5322 says is the value of the From: header is
stepping on some toes.

It's a significant change to RFC 5322, so any discussion of it
needs to take that into account (perhaps by having the appropriate
IETF WG involved in the discussion earlier rather than later).

Cheers,
  Steve

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 13 Sep 2014 18:35:06 -0400

> IETF WG involved in the discussion earlier rather than later).

OK.  What WG?

With respect to rewriting from, I think the horse has already left the barn.  
People can not like it all they want, I don't see it stopping any more than I 
see basic DMARC design changing to not break non-DKIM preserving mailing 
lists.

Would it make sense to discuss a draft here and then once it's beat into shape 
a bit send it $PLACES for appropriate review?

Scott K

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Sun, 14 Sep 2014 17:25:38 +0900

Scott Kitterman writes:

 > > Actually, it does.  Reply-to does not make assertions of
 > > authorship.  The type of change being contemplated does.
 > 
 > How much difference does that make to end users?  I mash a button
 > than get a message going back to an address.  They both could do
 > that.

That's the intended use.  The problem is that any "Really-Truly-From"
field is likely to be used by MUAs as a From substitute.  That is,
instead of using the GNU Mailman logo as an "avatar" in the GUI when
presenting a post on the "Mailman Developers" list, it will grab
"[email-masked]" out of the "Really-Truly-From" field, and present
*my* avatar to the user.  Who will then believe that I sent them the
link to the phishing site, click on it, and provide their credit card
credentials.  (And if you have a convincing argument that nobody would
be that stupid, please tell the AOL and Yahoo! PHBs so that they'll
stop their p=reject folly, and the Bad Guys, so they'll stop hammering
AOL and Yahoo! MXes.)

Really, the only general solution is author-signed mail.  For mailing
lists and the like, the MUA can help (for pholks who actually use
their domain's web MUA).  I would guess that for "on behalf of",
domains would need to provide an OAuth solution or, for trusted ESPs,
third party authorization.  (But I hasten to add that TPA is hardly a
generic solution.  The logistics of authorization are prohibitive.)

Regards,

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Sun, 14 Sep 2014 17:45:52 +0900

> With respect to rewriting from, I think the horse has already left

True.  The parts of RFC 5322 describing the Reply-To field are quite
analogous, though.  Reply-To munging by misguided mailing lists has
been happening since RFC 733 was current as far as I can tell, but the
obsoleting RFCs have consistently refused to sanction it.

I suppose that From munging will get the same treatment, for the same
reasons.

 > People can not like it all they want, I don't see it stopping any
 > more than I see basic DMARC design changing to not break non-DKIM
 > preserving mailing lists.

Sure, but Steve is quite right.  The From field is the RFC-defined way
to indicate the author of a message.  That is its *precise* semantics,
no more and no less.  Any proposed standard that presumes otherwise
(including that the From field might not contain the author) is going
to need to revise the definition of the From field, as well as propose
the alternative field for "really truly from".

---

From: Hector Santos <[email-masked]>
Date: Sun, 14 Sep 2014 09:54:09 -0400

> On Sep 14, 2014, at 4:45 AM, "Stephen J. Turnbull" <[email-masked]> wrote:
> 
> Scott Kitterman writes:
> 

I don't think so. It will be isolated. Don't assume all systems are going to honor it. They won't.   It's a hack, a kludge and it's irresponsibly ugly.  However, it will be exploited by the bad guys to send "on behalf" phishing DKIM exploits and that is when you will see filters to trap for these exploits.  

At the end of the day, all the restrictive domains usages will be pruned out, filtered away. Just like all our restrictive domains, like yahoo.com users subscribed to our support lists have shown.  This will most likely occur at the vast amount of smaller systems which is by far the majority as a  total group. But the same effect will occur at the more isolated larger site too. 

By the  time the WG will figure out any complex solution to get around this, I suspect the depleting targets user base will have moved on.  Why would you keep using a domain that you can no longer use in public like in the past?  A user might use it at a site because it is rewriting but he won't be able to use it at other sites and in not in general.   

--
Hector Santos
http://www.santronics.com

---

From: Scott Kitterman <[email-masked]>
Date: Sun, 14 Sep 2014 10:53:57 -0400

>the alternative field for "really truly from".

And by the time that's done, how many non-standard variants will be in use and "too hard to change"?  Whatever IETF documents say, it's already been redefined in the world of running code.

I'd propose something along the lines of "we don't endorse From rewriting by mailing lists, but if you're going to do it anyway, do it this way".  Then this can be decoupled from the larger semantic issue since we know there's a loss of information problem to be solved regardless of anyone's opinion about From rewriting. 

Scott K

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Mon, 15 Sep 2014 01:20:13 +0900

Scott Kitterman writes:

 > And by the time that's done, how many non-standard variants will be
 > in use and "too hard to change"?

Why should anyone care?  That's an honest question, because I think
the chances are quite good that Hector is right.  Yahoo! and AOL have
demonstrated that they care less about reliable delivery than other
providers do, and they will lose mailbox users because of that, and
the rate of loss will increase over time as people make other
arrangements or store up annoyance.  It won't kill either of them, but
it will hurt, and they will find ways to do without p=reject.  Is it
really worth trying to patch up the nonconforming implementations if
the need will automatically go away?  (At least for GNU Mailman, where
no p=reject => no From munging and no message wrapping).

 > Whatever IETF documents say, it's already been redefined in the
 > world of running code.

My point is that so has Reply-To, and the IETF has *pointedly* ignored
that fact for decades.

 > I'd propose something along the lines of "we don't endorse From
 > rewriting by mailing lists,

This WG is not just about mailing lists.  Have you thought about other
indirect mail use cases?

 > but if you're going to do it anyway, do it this way".  Then this
 > can be decoupled from the larger semantic issue since we know
 > there's a loss of information problem to be solved regardless of
 > anyone's opinion about From rewriting.

Good luck, because I expect the IETF to prefer pointedly ignoring
nonconforming implementations in its documents, or perhaps outright
condemnation, to anything like what you're proposing.

Regards,
Steve

---

From: "A. Schulze" <[email-masked]>
Date: Sun, 14 Sep 2014 20:49:31 +0200

> For example, changing all the MUAs in the world, versus changing all the

good point.

currently it looks to me like list managers are the biggest point of pain.
The list managers policy (adding subject tag + footer) stay against the
DKIM signature to prove the owner ship. The current solution to mitigate
DMARC damage is to takeover the ownership by rewriting RFC822.From

What about simply honer the senders policy and avoid message changes?
No need to change any trillions of MUA in the univers but only 100k  
list managers.

Andreas

---

From: Scott Kitterman <[email-masked]>
Date: Sun, 14 Sep 2014 15:34:11 -0400

>condemnation, to anything like what you're proposing.

Witness the excellent success of the head in the sand approach to stopping use of NAT.

People are already routing around the damage. The mailing list issue is particularly problematic because it's affecting users of domains that don't have reject policies. 

Google didn't invest the engineering resources in selective From rewriting for Google Groups to solve a non-problem.

I can (an plan to) write code that leverages their X-Original-To.  I'd rather have something standardized, but it's not essential for me to solve the problem I'm having. For the broader internet, I'm not so sure. 

Scott K

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Mon, 15 Sep 2014 09:53:37 +0900

> What about simply honer the senders policy and avoid message changes?

What about it?  That suggestion was first made years ago and rejected
by the users, even after the April Fools DMARC debacle.

History from my point of view: Mailman devs tried for months to come
up with a way to avoid implementing From munging, but like Reply
munging, list admins insist on this because users like some of the
message-modifying features (subject tags and serial numbers, access to
settings page in the footer) and others are more or less legally
required for some lists in some jurisdictions (visible simple way to
unsubscribe).  Note that removing subject tags may be an imposition on
hundreds of millions of users, who don't know any other way to filter
on mailing list (even if those other ways are obvious to us).

Regards,

---

From: Brandon Long <[email-masked]>
Date: Mon, 15 Sep 2014 12:39:53 -0700

On Sun, Sep 14, 2014 at 12:34 PM, Scott Kitterman <[email-masked]>
wrote:

> People are already routing around the damage. The mailing list issue is
> particularly problematic because it's affecting users of domains that don't
> have reject policies.
>
> Google didn't invest the engineering resources in selective From rewriting
> for Google Groups to solve a non-problem.
>

Honestly, I've spent way more engineering resources debating it than it
took to actually do it.  That's part of the lure, of course, is that its so
damn easy to do.

And, yes, we had to do something to fix things for our subscribers, since
despite what others may want to believe or witness as anecdote, people
aren't flocking away from yahoo/aol addresses due to their now being
p=REJECT domains.

Brandon

---

From: Brandon Long <[email-masked]>
Date: Mon, 15 Sep 2014 12:46:02 -0700

On Sun, Sep 14, 2014 at 12:34 PM, Scott Kitterman <[email-masked]>
wrote:

>
> I can (an plan to) write code that leverages their X-Original-To.  I'd
> rather have something standardized, but it's not essential for me to solve
> the problem I'm having. For the broader internet, I'm not so sure.

(assuming you meant X-Original-From):

Theoretically, this seems fine... as long as you don't display the results
directly in the UI, since that would defeat the whole purpose of DMARC and
re-writing the From in the first place.

As for why we did that, its kind of a our default to add an X-Original or
X-Google-Original prefix when rewriting headers.  We're still debating
whether or not we should do anything more with it, or adjust how we
generate reply-to headers in these cases... or even add the author to the
list footer, but there's been no burning need so far.

Brandon

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 15 Sep 2014 15:56:22 -0400

> list footer, but there's been no burning need so far.

I do (mean that one).  I agree that displaying it would be wrong.  My thought 
is to, as a companion to the current Reply to Mailing List reply option, add a 
Reply to Mailing List Author option that would direct the reply to X-Original-
>From (and/or whatever equivalent might get standardized) or the actual From if 
none of those are present.

Thanks,

Scott K

---

From: Roland Turner <[email-masked]>
Date: Tue, 16 Sep 2014 11:04:44 +0800

> That's a significant change to 5322.

This is actually pretty easy to fix. As the concern is about authoring 
lists (those which modify messages rather than merely forward them) it 
is both technically accurate and real-world-sensible for the From: 
header to reflect the list as the author of the message.

To remove the ambiguity created by the original proposal, the header 
identifying the poster to the authoring list might instead be called 
List-Poster: which both correctly expresses the relationship and avoids 
any confusion about authorship.

- Roland

---

From: Steve Atkins <[email-masked]>
Date: Tue, 16 Sep 2014 08:09:56 -0700

On Sep 15, 2014, at 8:04 PM, Roland Turner <[email-masked]> wrote:

> This is actually pretty easy to fix. As the concern is about authoring lists (those which modify messages rather than merely forward them) it is both technically accurate and real-world-sensible for the From: header to reflect the list as the author of the message.

If you wouldn't suggest such a thing unless you were trying to mitigate DMARC damage then it's probably neither technically accurate nor real-world-sensible. (And it's not something that I recall anyone suggesting before April).

> To remove the ambiguity created by the original proposal, the header identifying the poster to the authoring list might instead be called List-Poster: which both correctly expresses the relationship and avoids any confusion about authorship.

Not an entirely unreasonable line of argument - but it's a significant change to 5322, so the right forum to propose it is likely somewhere else. I'd guess it's ietf-smtp, but I'm not entirely sure. Dave? Murray?

Cheers,
  Steve

---

From: Roland Turner <[email-masked]>
Date: Wed, 17 Sep 2014 05:55:17 +0800

> If you wouldn't suggest such a thing unless you were trying to mitigate DMARC damage then it's probably neither technically accurate nor real-world-sensible. (And it's not something that I recall a…

I would certainly suggest a thing independently of DMARC (thus "both 
technically accurate and real-world-sensible") and, indeed, can't even 
claim credit for it: this has come up before in both the DKIM/ADSP 
discussion and in the origin of the term "authoring list". I am 
surprised to learn that you were not around for those discussions.

> Not an entirely unreasonable line of argument - but it's a significant change to 5322, so the right forum to propose it is likely somewhere else. I'd guess it's ietf-smtp, but I'm not entirely sure…

No, it specifically _*preserves*_ RFC 5322 and can therefore be 
specified independently.

- Roland

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Wed, 17 Sep 2014 09:55:57 +0900

Roland Turner writes:

 > discussion and in the origin of the term "authoring list".

I believe that "Mediator" is the currently standardized generic term
(RFC 5598), so "mediating list" would be the best such term.  For the
present discussion, although Mediators are indeed Authors (as defined
by RFC 5598), the important specialization is that a Mediator
"attempts to preserve the original Author's information in the message
it reformulates".

Earlier on this list the terms "alias" (non-mediating) and "list"
(mediating) were suggested, as well as a few variations on these
themes.  However, I think those terms are likely to engender confusion
in new participants (as would "authoring list"), while I suspect most
new participants will immediately "guess" a fairly accurate
interpretation of "mediating list".

---

From: "John Levine" <[email-masked]>
Date: 17 Sep 2014 09:10:33 -0000

>surprised to learn that you were not around for those discussions.

I was.  Lists that rewrite the From: line were a bad idea then, and
are an equally bad idea now, for reasons that have been discussed to
death.

If we're just going to beat dead horses here, we can save a lot of
time and shut down the WG now.

R's,
John

---

From: Tim Draegen <[email-masked]>
Date: Wed, 17 Sep 2014 09:02:20 -0400

> time and shut down the WG now.

This is a friendly reminder to keep the discussions civil and respectful.

Everyone is free to summarize previous discussions and contrast them with today's context for the benefit of the WG's chartered work.  If prior discussions are brought up, please take the time to summarize, contrast with today's context, and make relevant to the WG's work.

Last thing - the WG is chartered to perform specific work:

  https://datatracker.ietf.org/wg/dmarc/charter/

..and the WG will continue until this work is done.  Participation is entirely voluntary.  I'm asking everyone to please treat the volunteers (including me!) with civility and respect.

=- Tim
1 of 2 Chairs of this WG

---

From: Roland Turner <[email-masked]>
Date: Thu, 18 Sep 2014 21:08:23 +0800

> "attempts to preserve the original Author's information in the message

Sounds good.

I'd note that the Mediator's Authorship trumps the poster's in any 
situation in which they clash (poster wants to include an attachment, 
Mediator removes attachments -> the forwarded messages will have no 
attachments) but yes, can see the sense in describing delivered list 
posts as having joint authorship.

That said, the specifying of a List-Poster: header doesn't require 
determining which author is _*the*_ author for DMARC and 5322 purposes, 
I'd suggest that it's simply about describing an appropriate mechanism 
to use to indicate the author of the message prior to its passing 
through the Mediator for those operators who have decided to change the 
From: header to refer to themselves. It is hardly realistic to assume 
that no Mediators will make this choice, nor that all will. It would 
seem desirable to have a standardised method of specifying the poster 
for those who do.

- Roland

---

From: Roland Turner <[email-masked]>
Date: Thu, 18 Sep 2014 21:16:58 +0800

On 09/17/2014 05:10 PM, John Levine wrote:

> Lists that rewrite the From: line were a bad idea then, and are an 
> equally bad idea now, for reasons that have been discussed to death. 

Your position on that debate is well known. I'd suggest that arguing 
over whether or not anyone should rewrite the From: header or not would 
be rather wasteful at this point as it is now abundantly clear that will 
be significant fractions of list-forwarding conducted both with and 
without From: header rewrites for the foreseeable future.

The opportunity for this WG would appear to be to spell out sensible 
practices for use in the two situations, and perhaps to spell out the 
trade-offs for deciding between the two, assuming that we are able to do 
so without devolving into further equine abuse. I'd suggest that 
specifying a List-Poster: header for use in the rewrite case is an 
appropriate example of the former.

- Roland

---

From: "John Levine" <[email-masked]>
Date: 18 Sep 2014 13:44:16 -0000

>appropriate example of the former.

While I realize that the enormous market power of the DMARC group has
given mailing lists no choice but to use various workarounds, I don't
think you'll find any consensus in this WG that From: rewriting or
anything similar is a fait accompli.  I also haven't seen anyone
seriously address the concern that List-Poster: and its ilk are a
short term workaround with the long term effect of training users to
be phished, even more than they are already.

I would much rather look at technical approaches that allow mailing
lists and other forwarding agents to continue operating as they have
for the past 30 years, in as secure a way as possible, without
inventing new giant holes for the use of the criminals that DMARC is
supposed to deter.

That's what my two-level DMARC forwarding signature proposal is
supposed to do.  I don't claim it's perfect, but I think it's within
tweaking distance of being workable.  I'm not wedded to it, there are
other approaches that might work, too.

It also has the advantage that the implementation effort is primarily
by large users of DMARC, who I would expect are aware of this WG and
might actually do it, rather than makers of MUAs, who aren't, and
won't.

R's,
John

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Thu, 18 Sep 2014 23:44:06 +0900

Roland Turner writes:

 > It is hardly realistic to assume that no Mediators will make this
 > choice [to munge From], nor that all will.

Agreed.

 > It would seem desirable to have a standardised method of specifying
 > the poster for those who do.

Not at all clear to me.  The From field currently is the standardized
method of specifying the poster, and I'm not sure it's a good idea to
have another.

Especially since it doesn't solve many other indirect mail issues, and
would clearly be inappropriate to deal with, say, the "virus checker
breaks DKIM signatures."

Don't get me wrong -- an answer to Life, the Universe, and Everything
is *not* a necessary condition.  But I find the idea of a new standard
Really-Truly-From-for-Indirect-Channel-X for each channel X quite
distressing, and I'd rather not start down that path if there are less
ugly alternatives available.

Regards,

---

From: Roland Turner <[email-masked]>
Date: Fri, 19 Sep 2014 12:06:40 +0800

On 09/18/2014 09:44 PM, John Levine wrote:

> given mailing lists no choice but to use various workarounds,

You've already pointed out the inappropriateness of rehashing this 
argument. The appropriate response here would appear to be to agree to 
disagree and, therefore, to avoid attempts to pick a winner.

Using pejorative language to describe the actions of legitimate 
participants in the email system would appear to be out of scope.

> I don't
> anything similar is a fait accompli.

I did point this out. The WG need not involve itself in trying to pick a 
winner.

> I also haven't seen anyone

I have yet to see a credible threat model (plenty of hand-waving, to be 
sure).

> inventing new giant holes for the use of the criminals that DMARC is

Your personal preferences are [very] well understood, and aren't 
appropriate here. As you have already pointed out, rehashing these 
arguments is not appropriate.[1]

> other approaches that might work, too.

I'd be happy to review it; got a URL?

- Roland

1: I accept Tim's point about in-context summaries in relevant contexts.

---

From: Roland Turner <[email-masked]>
Date: Fri, 19 Sep 2014 12:28:18 +0800

> method of specifying the poster, and I'm not sure it's a good idea to

Not quite: it's the standardised method of specifying the author(s) (RFC 
5322 3.6.2), however specifying both authors in the From: header in 
Mediated mail is problematic because (a) DMARC recommends rejecting 
messages listing multiple authors (so far, this recommendation is not 
contentious) and (b) the listing of multiple authors in the From: field 
is vanishingly rare in real-world email.

For mailing lists, the use of From: to specify the poster in preference 
to the Mediator is specifically not standardised. It has of course been 
customary for a long time, however a substantial volume of Mediated 
email now identifies the Mediator in the From: field instead of the 
poster. It is correctly observed that much of this shift is in response 
to DMARC, but that does not make it any less real.

> ugly alternatives available.

This is unduly burdening the proposal. If mailing lists were some 
obscure corner case then, sure, but (a) they're not (you may have 
noticed their being mentioned occasionally in discussions around SPF, 
DKIM, ADSP, ...) and (b) they're an important enough special case that 
two draft standards already exist to specify additional List-* headers 
to identify list traffic (RFC 2919) and to state list commands (RFC 2369).

If some more general mechanism comes up in this WG that deals with the 
mailing list case as well then, certainly, let's consider adopting it. 
Until or unless that occurs, the fact that it doesn't solve all of the 
known cases (let alone as-yet-undiscovered cases) is not an argument 
against it.

- Roland

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Fri, 19 Sep 2014 19:10:02 +0900

Roland Turner writes:

 > I have yet to see a credible threat model (plenty of hand-waving,
 > to be sure).

"Credible" is obviously subjective and inappropriate in describing a
threat *model*.  You're certainly welcome to provide evidence that
it's more or less likely to be realized, of course.

The threat model is simple: to be useful to mail readers, the
List-Poster field must be displayed in addition to (and I would say,
*instead* of) the From field.  For maximal effect from the *reader*'s
perspective, not only "instead of", but it should also be given the
same treatment by the MUA as the From field, starting with automatic
addressing in replies, and proceeding to display of preferred full
name and "avatars" from contact lists, etc.

Of course, to attack users of such MUAs, all a spammer or phisher
needs to do to subvert DMARC is to use a bad-guy-controlled domain in
the From field, DKIM sign per DMARC rules, and use the stolen contact
information in the List-Poster field.  If necessary, add other RFC
2369 headers to further emulate the form of list traffic.

There is no way to prevent or reduce the effectiveness of such attacks
that doesn't simultaneously have the same effects on the usefulness of
List-Poster.  Note that it is exactly this form of attack that induced
Yahoo! and AOL to turn to p=reject.

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 19 Sep 2014 06:27:26 -0400

>name and "avatars" from contact lists, etc.

Not at all. You are now objecting to something other than what I originally proposed. It would be the opposite for the reasons you say and more. 

The point is to provide a mechanism to facilitate a reply to the original author when desired (reply-to is problematic since it's rather unconditional).  There is no need to display it.  To do so would be contradictory and confusing.

Scott K

---

From: [email-masked]
Date: Fri, 19 Sep 2014 20:12:56 +0900

Scott Kitterman writes:

 > The point [of X-Original-From/List-Poster] is to provide a
 > mechanism to facilitate a reply to the original author when desired
 > (reply-to is problematic since it's rather unconditional).  There
 > is no need to display it.

Need, no.  But we are having this conversation precisely because
people use header fields (not to forget the DNS!) in ways not intended
by standards authors.  Why would List-Poster be exempt?

 > To do so would be contradictory and confusing.

Can you expand on this confusion?  I can't imagine why displaying
List-Poster as the address of the author of the post would be
confusing, especially in cases where the post did not provide a
display name in the From field.

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 19 Sep 2014 07:27:58 -0400

> by standards authors.  Why would List-Poster be exempt?

It wouldn't be exempt, but display isn't necessary to achieve the intended 
objective.  While I can't exclude the possibility that someone will write an 
MUA that exposes this, the MUAs that most people use (either via a traditional 
application or a web U/I) are written by the companies that have developed 
DMARC.  I expect them to resist the temptation.

> display name in the From field.

First, I expect display names to be less commonly displayed in the future.  
They make it much too easy to trick people into being confused about who sent 
a message.  I don't think that no display name if From will drive anything.

Second, you snipped the key part I was replying to:

> >*instead* of) the From field.

I was replying to your primary point of "in addition to".  Having two entities 
the messages is "from" is inherently confusing.

Scott K

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 20 Sep 2014 13:41:36 +0200

[I refrain from swapping Subject: and display-phrase in an attempt to
evoke some more brainstorming, because of the mess that possible
replies would make if I did.]

On Fri 19/Sep/2014 12:10:02 +0200 Stephen J. Turnbull wrote: 
> threat *model*.
> [...Description of credible threat model elided...]
> 

Of course we wouldn't have been having this discussion if DMARC had
taken care of mailing lists needs.  Now we have to specify a possible
MLM behavior, such that list participants are not affected by their
domains' DMARC policy decisions.  If the model is phish-proof, all the
better.

IMHO, we should specify a credible MLM model, even if that can be
slightly off topic for the WG, in order to maximize its probability of
being adopted.  The rest of this message has some notes to this end.

*Field Name*

List-Poster: is too similar to List-Post:, especially for foreigners;
List-Author: would be clearer.  X-Original-From: conveys a precise
meaning --if we specify that-- and pairs nicely with
X-Original-Authentication-Results:.

*Rewritten Display Phrase*

It seems to be going without saying that the display phrase of the
rewritten From: is taken from the original.  That allows the omission
to display the latter, while still keeping the least-surprise
principle when users click reply-to-poster.

*Rewritten Domain*

The address domain shall be that of the MLM.  That's what rewriting
From: is all about.  Reasons why that is semantically correct have
been discussed already.

*Rewritten Local Part*

It is appropriate to specify the role of the local part too, for our
spec to make sense.  I'd suggest some sort of thread-id, so that the
"normal" reply buttons works as expected.  Subscribers can choose
whether they want private notifications on thread replies, much like
they do for blog comments.  Currently, this choice is made by replying
authors on a per-post basis.

Note that this obsoletes reply-to-list buttons.  List-Post: is only
useful to start new threads, in this model.  Although the poster
intention can be inferred from other message details, it may be
convenient to replace that field with List-Post-New:.  Besides
disabling obsolete buttons, the mere presence of the latter field may
hint to MUAs not to store the From: address upon replying.

*Conclusion*

I outlined an idea for a new model of mailing list where From: is
rewritten smoothly for all posters, irrespectively of their domains'
DMARC settings.  There are certainly zillions alternatives.

Differently from Netnews, MLM behavior was never specified.  Had it
been, maybe DMARC could have taken it into account.

Ale

---

From: Roland Turner <[email-masked]>
Date: Mon, 22 Sep 2014 12:51:39 +0800

> it's more or less likely to be realized, of course.

"Credible" is not subjective, but to be more precise: sufficiently 
concrete to assess, and can be assessed as a realistic threat.

> List-Poster field must be displayed

It is not a requirement that adding a List-Poster: field be useful to 
mail readers. The existing use of X-Original-From is an existence proof 
of [perceived] utility, despite the absence of widespread (any?) MUA 
display of this data.

It is certainly interesting to pursue what happens if MUA 
developers/vendors see utility in displaying this information.

> in addition to (and I would say,
> same treatment by the MUA as the From field,

Setting aside John Levine's argument that MUA developers won't implement 
this stuff at all, your model is only realistic if the MUA developer is 
seeking to maximally confuse the reader, or more charitably: is somehow 
utterly ignorant of DMARC's existence and impact. It would also only be 
sensible to display the List-Poster: (or in any other sense make use of 
it) in the context of being via the list (which is more-or-less the 
rationale behind changing the From: display-name to  "Poster via List").

As Scott points out, the overlap between developers/vendors of MUAs in 
widespread use 
<https://www.campaignmonitor.com/resources/will-it-work/email-clients/> 
and the creators of DMARC <http://dmarc.org/about.html> is significant. 
In concrete terms, as at September 2012 by user count:

  * DMARC creators: ~50%
  * Apple: ~40%
  * Other: ~10%

Given Apple's recent ...involuntary progress on the security front, I'd 
suggest that the developers of the MUAs in use by ~90% of people are 
strongly motivated to act in a way which does not cause the problem that 
you're describing. The rest are likely to be motivated to adjust their 
behaviour when encouraged and/or can be disregarded as being too small 
to present a worthwhile attack surface.

(To be clear: I'm not suggesting that it is reasonable to stomp on the 
little guy, only that ignorance at such a small scale is not a 
significant concern. Also: yes, the data is two years old, however if 
the "other" category has moved at all, it hasn't grown enough to alter 
the argument.)

...

> 2369 headers to further emulate the form of list traffic.

A List-Poster: header should not be displayed at all unless there is 
some reason to trust the MLM identified in the From: header (and 
authenticated) and, as above, should be displayed alongside the list 
identifier. There is precedent for this, e.g. only honouring 
List-Unsubscribe: for authenticated senders who have a solid track 
record of reliable behaviour. As John Levine has pointed out repeatedly, 
the trustworthy set of list operators is small and slow changing; 
keeping track of this is not a serious problem.

For the reasons described above, it is not necessary to worry about MUA 
vendors getting this wrong and/or failing to correct it once it's 
pointed out.

> Yahoo! and AOL to turn to p=reject.

That was purely about spoofing From: headers for direct delivery to 
victims, there was no involvement of known-to-be-well-behaved lists.

- Roland

---

From: "John Levine" <[email-masked]>
Date: 22 Sep 2014 21:15:30 -0000

>of [perceived] utility, despite the absence of widespread (any?) MUA 

Who uses X-Original-From ?  This is a real question, I'm not aware of
anyone who does.

R's,
John

>In concrete terms, as at September 2012 by user count:

Yes, we're aware of the enormous market power of the DMARC group.

---

From: "Carl S. Gutekunst" <[email-masked]>
Date: Mon, 22 Sep 2014 14:55:53 -0700

> Who uses X-Original-From ?  This is a real question, I'm not aware of

In my experience, the utility of X-Original-From has been for humans 
diagnosing mistakes in the mail router's rewriting rules. Since this was 
implemented independently by many vendors, the contents and semantics of 
the field have varied considerably, ranging from the original 
821.MailFrom, to the original 822.From, to the sending user's login 
credentials.

Don't get me started on the X- prefix.

<csg>

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 22 Sep 2014 18:00:55 -0400

> Who uses X-Original-From ?  This is a real question, I'm not aware of

Google (at least Groups) does when it rewrites the From address for messages 
from a DMARC  p=reject domain.  I have mail that has X-Original-From: 
[email-masked] in it.  It also has X-Original-Sender: [email-masked], which 
the mail from non-rewritten domains also has.  The non-rewritten domains don't 
have X-Original-From.

This is what got me started on "let's pick a standard name".

Scott K

---

From: Josh Aberant <[email-masked]>
Date: Mon, 22 Sep 2014 15:17:31 -0700

> Who uses X-Original-From ?  This is a real question, I'm not aware of

At Twitter we use the X-Original-From to route our security tickets with
users within our ticketing system Salesforce. Users can open and reply to
tickets by emailing what is a Google Groups alias. Google Groups (takes
ownership of the From per our #RejectPolicy) and then forwards those emails
to Salesforce where they are automatically turned into support tix where we
key off of the user's email address in the X-Original-From header.

Josh

---

From: Steven M Jones <[email-masked]>
Date: Mon, 22 Sep 2014 15:56:27 -0700

On 09/22/2014 03:17 PM, Josh Aberant wrote:
> At Twitter we use the X-Original-From to route our security tickets
> with users within our ticketing system Salesforce. Users can open and
> reply to tickets by emailing what is a Google Groups alias. Google
> Groups (takes ownership of the From per our #RejectPolicy) and then
> forwards those emails to Salesforce where they are automatically
> turned into support tix where we key off of the user's email address
> in the X-Original-From header.

Interesting use case. Presumably the messages are flowing from Google
Groups to Salesforce over the Internet. Certainly an example of how
Google Groups can be configured to manipulate headers.

However I would consider everything that happens after Salesforce
receives the messages as an application workflow. Messages the customer
sees would be in-scope; I'm not so sure about things happening between
Google Groups and Salesforce, or within Salesforce.

If we consider all the ways email messages are spindled, folded and
mutilated when they become part of an application workflow, I expect
we'd see an absolute explosion of additional weirdness...

--Steve.

---

From: Brandon Long <[email-masked]>
Date: Mon, 22 Sep 2014 16:43:06 -0700

On Thu, Sep 18, 2014 at 9:28 PM, Roland Turner <[email-masked]>
wrote:

> contentious) and (b) the listing of multiple authors in the From: field is
> vanishingly rare in real-world email.
>
> For mailing lists, the use of From: to specify the poster in preference to
> the Mediator is specifically not standardised. It has of course been
> customary for a long time, however a substantial volume of Mediated email
> now identifies the Mediator in the From: field instead of the poster. It is
> correctly observed that much of this shift is in response to DMARC, but
> that does not make it any less real.

But note that "similar" tools, such as bulletin-boards, have almost always
used the mediator as the From header on notification messages.  Mailing
lists which provide an "anonymous" function also use the list/mediator
address as the From in order to hide the original poster's email address.

Brandon

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Wed, 24 Sep 2014 02:26:38 +0900

> > > Who uses X-Original-From ?  This is a real question, I'm not
 > > > aware of anyone who does.
 > >
 > > At Twitter we use the X-Original-From to route our security
 > > tickets with users within our ticketing system Salesforce. Users
 > > can open and reply to tickets by emailing what is a Google Groups
 > > alias. Google Groups (takes ownership of the From per our
 > > #RejectPolicy) and then

I don't understand "our #RejectPolicy".  What are you talking about?
Surely not DMARC?  That reject policy would be related to the From
Domain of the user, not anything about Twitter.  Or do users use an
@twitter address for email they send?

 > > turned into support tix where we key off of the user's email
 > > address in the X-Original-From header.
 > 
 > Groups to Salesforce over the Internet.

So what we have is

                  SMTP                         SMTP
Customer (Author) ---> Google Group (Mediator) ---> Twitter (Destination)

right?

 > receives the messages as an application workflow.

Indeed.  But as long as these message don't leave the Twitter
workflow, I'd consider all of this a private protocol that isn't
really any of the IETF's business.

 > Messages the customer sees would be in-scope;

Not if they don't go back through the Google Group.  Nothing so far
suggests that they do, and the word "security" suggests that they do
not, to me anyway.

 > I'm not so sure about things happening between Google Groups and
 > Salesforce,

In scope.

 > or within Salesforce.

Not.

 > mutilated when they become part of an application workflow, I
 > expect we'd see an absolute explosion of additional weirdness...

Not our business, though.  Some of the specific mutilations may be,
but the whole collection surely is not.  We would consider specific
instances case-by-case.

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Wed, 24 Sep 2014 02:33:00 +0900

Brandon Long writes:

 > But note that "similar" tools, such as bulletin-boards, have almost
 > always used the mediator as the From header on notification
 > messages.

I'm not sure what you mean by "notification".  I would take that to be
things like a regular FAQ, Hold notification from moderation, DSNs for
delivery issues, scheduled downtime, posting policy, and so on.  These
actually are authored by the BBS (or MLM).

I don't understand what is interesting about "From is Author" in this
context.  Subscribers might confuse user posts with "official"
notifications?

 > Mailing lists which provide an "anonymous" function also use the
 > list/mediator address as the From in order to hide the original
 > poster's email address.

This is a private agreement between the MLM and the poster; the From
munging is done on the authority of the poster.  But this has never
caused confusion that I know of.

---

From: Steven M Jones <[email-masked]>
Date: Tue, 23 Sep 2014 10:47:24 -0700

On 09/23/2014 10:26 AM, Stephen J. Turnbull wrote:
> Domain of the user, not anything about Twitter.

My reading was that this was some kind of configuration item within
Google Groups. Therefore it's interesting if it is a normally available
feature that any Google Group might use to alter the RFC5322.From or add
an X-Original-From: header.

Josh, anybody, feel free to correct/amplify...

--S.

---

From: Brandon Long <[email-masked]>
Date: Tue, 23 Sep 2014 14:44:06 -0700

> context.  Subscribers might confuse user posts with "official"

Sorry, they often term the messages as "notifications" instead of posts,
when they actually contain
the full post and author information/etc.  Others only say "there has been
a post" which is more
a notification.

I guess my overall point here is that some mailing lists evolved into BBS
style hybrids (Y!Groups/GGroups),
while some BBS's have evolved more mailing list type functionality "receive
posts via email, respond to posts via email".  The ones that started as MLM
tend to use "from is author" and the ones that started as a BBS tend to use
"from is mediator".

Brandon

---

From: Brandon Long <[email-masked]>
Date: Tue, 23 Sep 2014 14:58:38 -0700

> an X-Original-From: header.

Google Apps currently implements "aliases" as Google Groups (this has been
true for a number of years now, prior to that there were separate aliases
and groups).  Because of this, a "[email-masked]" address that
redirects to internal users or an external CRM tool (salesforce) would be
getting a groups rewritten message.  These messages will not pass DKIM due
to the rewriting, and so if they're from a DMARC p=REJECT/QUARANTINE domain
such as yahoo.com, the from header will be rewritten to be the group name (
[email-masked]) and the x-original-from will be the original sender.

Its technically possible with Google Apps domain routing rules to reroute
[email-masked] to another address without passing through Groups and
therefore DKIM would still pass... obviously SPF wouldn't, however.  Doing
this for a number of addresses would probably be more complicated than
using Groups as the alias.

Currently, Groups rewrites messages even if the group has no footer or
subject prefix.  We're investigating having an "alias" like mode for Groups
which would allow DKIM messages to pass unscathed.

In any case, support folks, especially when dealing with paying customers,
tend to want to get all of the email, they don't want their nicely paying
customers to not get support just because of spam false positives or the
mail routing breaking dkim.

I'm not sure if any of this is particularly relevant except to say that if
one does from header rewriting, it is generally useful for there to be a
machine readable and potentially standard place to find the original
sender.  That said, I also agree that displaying that address directly to
the end user, especially without verification through some mechanism like
OAR, would defeat the purpose of DMARC in the first place.  If we knew how
to display from addresses to users without making them phishing targets, we
wouldn't have needed DMARC.

Brandon

---

From: Douglas Otis <[email-masked]>
Date: Tue, 23 Sep 2014 16:31:13 -0700

Dear Brandon,

The goal of DMARC is to ensure MTA rejection allows recipients of messages containing a particular From header better retains their trust of not being a spoof.

Any effort to constrain From header fields handled by filtering rules from other sources requires an authentication process that specifically authorizes these various sources.  Without an additional extension, retaining a recognizable and trusted From header field without devolving back into unbounded spoofing is not possible.  Having all possible sources modify From header fields in conflict with simplistic DMARC assertions creates unproductive work without direct benefit likely seen as being analogous to rearranging deck chairs.

A few decades ago we managed all of AOL's inbound email along with several other large ISPs.  In comparison, minor resources would be required to handle exceptions needed for AOL or other ISPs wishing to implement DMARC while still permitting messages from various sources such as IETF.org mailing-lists or Intuit.com invoices.  This approach ensures recognition of their From is not tainted by abuse beyond their control.

http://tools.ietf.org/html/draft-otis-tpa-label-04

We would be happy to work with any interested ISPs.  Only From domain owners receive DMARC feedback and know through examining domains which third-party services are employed by their users.  This would offer a much safer scheme than that provided by munging the From header field which creates greater recipient confusion while lessening email's general utility.

Regards,
Douglas Otis

On Sep 23, 2014, at 2:58 PM, Brandon Long <[email-masked]> wrote:

> Google Apps currently implements "aliases" as Google Groups (this has been true for a number of years now, prior to that there were separate aliases and groups).  Because of this, a "[email-masked]" address that redirects to internal users or an external CRM tool (salesforce) would be getting a groups rewritten message.  These messages will not pass DKIM due to the rewriting, and so if they're from a DMARC p=REJECT/QUARANTINE domain such as yahoo.com, the from header will be rewritten to be the group name ([email-masked]) and the x-original-from will be the original sender.
> 
> Its technically possible with Google Apps domain routing rules to reroute [email-masked] to another address without passing through Groups and therefore DKIM would still pass... obviously SPF wouldn't, however.  Doing this for a number of addresses would probably be more complicated than using Groups as the alias.
> 
> Currently, Groups rewrites messages even if the group has no footer or subject prefix.  We're investigating having an "alias" like mode for Groups which would allow DKIM messages to pass unscathed.
> 
> In any case, support folks, especially when dealing with paying customers, tend to want to get all of the email, they don't want their nicely paying customers to not get support just because of spam false positives or the mail routing breaking dkim.
> 
> I'm not sure if any of this is particularly relevant except to say that if one does from header rewriting, it is generally useful for there to be a machine readable and potentially standard place to find the original sender.  That said, I also agree that displaying that address directly to the end user, especially without verification through some mechanism like OAR, would defeat the purpose of DMARC in the first place.  If we knew how to display from addresses to users without making them phishing targets, we wouldn't have needed DMARC.
> 
> Brandon

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Wed, 24 Sep 2014 09:22:41 +0900

Brandon Long writes:

 > Sorry, they often term the messages as "notifications" instead of
 > posts, when they actually contain the full post and author
 > information/etc.

I guess I'd need to see one to decide how I would classify them and
how they fit into this WG as use cases.  Examples?

The only thing I can think of that fits that description is a bug
tracker, and there the message is usually only one component of the
information conveyed, and often is missing.  In that case I would
consider the actual message to be encapsulated in the outer message as
delivered by the service, similar to the way I quoted you above --
this message is *from* me, *quoting* you.  (Note that some
authors/MUAs using a quoting style that looks much like RFC 822
headers, increasing the similarity.)

I would consider that mode of operation to be deficient in a
discussion list.  I don't recall <commercial service> Groups using
that mode, though -- they always have the author in From IIRC.

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Wed, 24 Sep 2014 09:35:31 +0900

Brandon Long writes:

 > In any case, support folks, especially when dealing with paying
 > customers, tend to want to get all of the email, they don't want
 > their nicely paying customers to not get support just because of
 > spam false positives or the mail routing breaking dkim.

Sure, but this isn't a problem if the support site ignores p=reject or
treats it as p=quarantine.  I think this is a twitter-side problem,
not a Google-side problem or IETF problem.

The "recipient MUST reject" aspect of DMARC is broken as far as I'm
concerned.  I personally consider "p=reject" to be informational, with
preferred recipient semantics of "as far as we're concerned you can
black-hole this message, and for your own good you probably should if
you don't have *very* good reason to do otherwise *and* your own
strong defenses against inauthentic mail."

Support staff also can be trained not to get phished (is that really
even necessary?)

Presumably the remaining problem here would be the amount of
inauthentic mail that gets through.  Yahoo! admins speak of millions
of messages per minute, which I find entirely plausible, and it
certainly would knock down not just my tiny host, but probably my
employer's whole network.

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 23 Sep 2014 21:16:42 -0400

>that mode, though -- they always have the author in From IIRC.

Except at the very least Google Groups.

Scott K

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Wed, 24 Sep 2014 14:09:39 +0900

> >I don't recall <commercial service> Groups using that mode, though
 > >-- they always have the author in From IIRC.
 > 

If you are talking about their workaround for DMARC p=reject From
Domains, that's irrelevant to my point.  This subthread was discussing
how modern lists/bbses/forums have evolved over time.  What I am
saying is that in the absence of DMARC p=reject at public mailbox
providers, as far as I know Google Groups and Yahoo Groups normally
put the poster's name and address in the From field.  This is 100%
confirmed by a sample of nearly 100 of my personal archives of GG and
YG mailing lists.[1]

If you have a counterexample, I'd love to hear about it, in enough
detail to determine whether the From-munging was done with at least
implicit authorization by the posters.

Eg, subscribing to an anonymous mailing list described as such on its
information page constitutes implicit authorization.  Posting to a
list which changed to from-munging as a DMARC mitigation, without an
explicit announcement of this ToS change, is not.

Footnotes: 
[1]  As it turns out, my GG lists have *zero* Yahoo and AOL posters,
and my YG lists most recent post was 2008/8/12, so mostly irrelevant
to recent policy.

---

From: Brandon Long <[email-masked]>
Date: Tue, 23 Sep 2014 22:09:57 -0700

> >how they fit into this WG as use cases.  Examples?

I'll see if I can find some, its been years since I used any of them.

> >headers, increasing the similarity.)

Sure, a bunch of bug / case / support type systems work like that.

Notifications from FB, G+ and Google document comments also work like that.

> Except at the very least Google Groups.

Google Groups has had "private" groups where email addresses are private,
so messages always came "from" the group.  The product forums in our help
center work in this mode, for example.  Yahoo Groups has a similar
feature.  I'm not sure if you can actually post to the groups via email in
that mode, however.

Obviously, more recently, both Google and Yahoo have started re-writing the
>From header for messages posted from DMARC domains.  I think Y!Groups is
actually doing this for all messages now, not just DMARC affected
domains... and it also uses an X-Original-From header.

Brandon

---

From: "John Levine" <[email-masked]>
Date: 25 Sep 2014 02:25:10 -0000

>tend to use "from is author" and the ones that started as a BBS tend to use

Having been there at the time, I'm fairly confident that the BBS mail
gateways that put the list name on the From: line did so out of
expedience on tiny computers with terrible programming environments,
rather than any metaphysical belief that the list was somehow Frommier
than the author.

We've been using mailing lists on the Internet and other systems (uucp
and such) for close to 40 years.  We wrote our own software, and at
any point we could easily have put the list name on the From: line, if
we thought it was a good idea.  It wasn't in the 1970s, 1980s, 1990s,
and 2000s, and it still isn't now.  I don't see any utility in beating
this particular equine corpse any more.  It's possible that it may
turn out to be the least bad of a set of unpleasant options, but that
still doesn't make it a good idea.

R's,
John

---

From: Douglas Otis <[email-masked]>
Date: Fri, 26 Sep 2014 18:12:05 -0700

On Sep 23, 2014, at 10:09 PM, Brandon Long <[email-masked]> wrote:
> Obviously, more recently, both Google and Yahoo have started re-writing the From header for messages posted from DMARC domains.  I think Y!Groups is actually doing this for all messages now, not just DMARC affected domains... and it also uses an X-Original-From header.

Dear Brandon,

X-Original-From is not standard and is misleading.  Perhaps we should attempt to standardize this header.  Doing so could provide people a better idea where this change is headed.

For example, instead of X-Original-From: this could be Submitter:  Rather than defining this header field per RFC4405, simply state it should represent the original author and ignore PRA concepts based on users re-introducing messages where tracking its path proved ineffective. 

It seems the proper approach for making such a change would have been to expect those benefiting to carry the resulting burden and suffer through the transition.  For example, instead of restricting the From header field and changing its role, a new header could have been added such as Submitting-Domain: restricted by DMARC policy.  Of course many would argue the phishing problem must be addressed by restricting a normally visible and trusted header-field; Hence the use of From.

Defining a Submitter header field would give mailing lists, and invoice and notification vendors a clear and concise indication what change is needed.  It will take several years before this new header is likely visible and for that header field to then become a phishing target.  Just perhaps filtering rules for mailing lists will tend to thwart its abuse.  A means to informally federate third-party domains might also be effective at defending this new header if needed. 

Regards,
Douglas Otis

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Sat, 27 Sep 2014 17:15:21 +0900

Douglas Otis writes:

 > X-Original-From is not standard and is misleading.

Aside from BCP 178, I don't see what is unclear about it.  It quite
obviously indicates the originator's From field has been altered, and
the [X-]Original-From field contains what the originator put in the
>From field.  Later alterations would leave [X-]Original-From alone, I
suppose, unless they actually want to claim authorship, in which case
they would delete.

If you want a history in the case of not claiming authorship, use a
References-like mechanism and call the field "From-History".  But that
seems pointless to me.

 > For example, instead of X-Original-From: this could be Submitter:

I thought we already had Submitter semantics, but in RFC 5322 the
field is spelled "Sender".  In any case, that confusion seems far more
problematic to me than any that could arise from Original-From.

 > Rather than defining this header field per RFC4405,

Now you're seriously confusing me.  RFC 4405 is an SMTP extension and
defines no header fields at all.  The concept of "responsible
submitter" defined there does not correspond accurately to an RFC 5322
originator field (specifically in the case where RFC4405.SUBMITTER is
not the RFC5322.Sender, and does not wish to represent itself as
author of the message).

 > Defining a Submitter header field would give mailing lists, and
 > invoice and notification vendors a clear and concise indication
 > what change is needed.

No change is needed.  This is useless to indirect mailers until MUAs
display it.  Scott assures us they never will, and because of lags in
adoption by MUA maintainers and then further lags in upgrading by
users, I would expect a large minority of users to be unable to see
this field after a decade even if his prediction is falsified.

 > A means to informally federate third-party domains might also be
 > effective at defending this new header if needed.

If we had "federation" we'd just "defend" the From field.

---

From: Douglas Otis <[email-masked]>
Date: Sat, 27 Sep 2014 14:42:24 -0700

> References-like mechanism and call the field "From-History".  But that

Dear Stephen,

Thank you for your thoughtful response, however there are a few problematic points.

Use of DMARC on non-transactional email, without acceptable alternatives to ensure delivery, requires the From originator field definition to represent that of a third-party without clarity where or when this occurred, which might happen more than once within a delivery path.  With From now muddled, From-History, Original-From, Resent-From or any term suggesting a version of a From field carried forward can not offer role clarity.

> problematic to me than any that could arise from Original-From.

True, but DMARC specifically does not use Sender.  It only uses From with the intent of Sender being ignored. Nor does Original-From shed any light on this header-field's role once From has been muddled.

> not the RFC5322.Sender, and does not wish to represent itself as

Should have said please completely ignore RFC4405. Call the header field Originator or On-Behalf-Of, if you don't like Submitter or you find it too confusing. The basic point is there should not be a rote carrying forward of a potentially muddled From header field into a newly defined header field attempting to preserve originator roles played by its contained identifiers.  DMARC does not use this new field, so it can safely carry forward this critical role.  In other words, carry forward the Originator header if present rather than a now questionable From. 

> this field after a decade even if his prediction is falsified.

Unfortunately DMARC when not restricted to transactional messaging, requires some kind of change, but X-Original-From represents a poorly considered choice. You really mean From has become useless for indirect mailers.   Sender does not handle what might be contained within the From header field nor is this header field always seen.  Originator can act as a safer alternative where Sender is unable to convey the appropriate role anyway. 

> If we had "federation" we'd just "defend" the From field.

Agreed. For any large ISP interested, we would be happy to setup the needed infrastructure for them to make this happen.  Alas, these large vendor's dominance can force change on others who will suffer through the transition rather than on those instigating the change.  The justification might be market dominance makes right. :^(

For OS X Mail, Choose Mail > Preferences, click Viewing, then choose All from the “Show header detail” pop-up menu.
To customize header details, choose Custom from the pop-up menu, click Add (+), enter the field name, then click OK.

For Outlook, iOS Mail or others there are fewer options without changing email clients. It seems this will make someone some money.  Some clients can only show all headers which would be unfriendly for those wanting to know who authored the message after circumnavigating all the DMARC foo aimed at keeping them safe with their high value transactional messages.

Regards,
Douglas Otis

---

From: Douglas Otis <[email-masked]>
Date: Mon, 29 Sep 2014 11:22:51 -0700

On Sep 22, 2014, at 3:17 PM, Josh Aberant <[email-masked]> wrote:

> At Twitter we use the X-Original-From to route our security tickets with users within our ticketing system Salesforce. Users can open and reply to tickets by emailing what is a Google Groups alias. Google Groups (takes ownership of the From per our #RejectPolicy) and then forwards those emails to Salesforce where they are automatically turned into support tix where we key off of the user's email address in the X-Original-From header.

Dear Josh,

From (normally contains a mailbox representing the author) might be substituted for a proxy domain to satisfy DMARC acceptance requirements typically needed for non-transactional email.  As DMARC affects normal use of From, greater care is required to retain the original role defined for From.  In other words, the replacement header field should not be seen to represent a Resent-From or a trace header structure, since this should occur only once within the path of a message since the identifier for the original role will not have changed.

This is a working group, so we should be thinking in terms of doing what is proper and right.  As has been pointed out by BCP178 or RFC6648, X-Original-From creates other problems specifically due to a lack of defined roles and how or when this new header is to be added and used.  Any new header field should be properly defined.  Should this header be introduced every time a change is made to satisfy DMARC acceptance requirements?  What are the security threats related to its use?  Alternatively, should DMARC define a re-writing scheme such as dmarc-noncompliant: where perhaps the From is presented by RFC6854 defined group syntax?  It might be nice to have DMARC embracing use of a special group syntax, since it seems recognizable special group would be fairly unlikely to be useful spoofing users.
From: Dmarc-Noncompliant:[email-masked];
Regards,
Douglas Otis

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Tue, 30 Sep 2014 15:58:43 +0900

Douglas Otis writes:

 > >From (normally contains a mailbox representing the author) might
 > >be substituted for a proxy domain to satisfy DMARC acceptance
 > >requirements typically needed for non-transactional email.  As
 > >DMARC affects normal use of From,

This is rather inaccurate description of the situation.  First, "From"
doesn't just "normally" represent the author -- that is the RFC
definition of this REQUIRED field.  Second, DMARC is basically an
informational protocol.  Only the "p" attribute defines "acceptance
requirements", and only if non-default.  Third, even "p=reject" does
not "affect use of From".  It affects *delivery* of syntacticly and
semantically conformant Internet messages.  (DMARC is a private
protocol by agreement between MXes, and absent a change to the
author's host's terms of service, I don't see why anybody other than
participating MXs should be expected to know or do anything about it.)

 > It might be nice to have DMARC embracing use of a special group
 > syntax,

A reasonable suggestion in context of other proposals.

 > since it seems recognizable special group would be fairly unlikely
 > to be useful spoofing users.

But I have to disagree with your premise.  To the extent that it *is*
recognizable, it is likely to be "recognized" as an alternative form
of "From" applied to some *authentic* authors for unknown reason, not
as evidence of inauthenticity.  Because spam filtering removes many
inauthentic messages and mailing lists generate large numbers of
authentic messages, the conservative assumption (from the point of
view of security against phishing) is that, as seen by recipients,
most authors so marked will indeed be authentic.

Furthermore, I believe spoofable users *will* get spoofed through such
devices.  For example, in Japan, where I live, a standard fraud starts
with a phone call, "Hello?  It's me, it's me, and I need $3000 in a
big hurry!"  "Oh, Johnny, you poor thing, do you have a cold?  Your
voice sounds funny!  Be more careful.  Where should I take the money?
I can get it in 30 minutes."  I am not kidding; in bad years victims
lose hundreds of millions of dollars to this scam.

I see no reason to suppose the demographic that falls prey to this
fraud is absent elsewhere, or that it contains no email users.

It would be very useful to know what the "conversion rate" for
phishing is for three cases of interest:

A.  a phishing message from a non-acquaintance, displayed by the MUA
    "in the usual way" (eg, with "avatar" from the contact list);
B.  a phishing message from an acquaintance, displayed by the MUA "in
    the usual way"; and
C.  a phishing message from an acquaintance, with a different display
    containing a familar mitigation.  (The user may not be aware that
    it is a mitigation, of course, expecially if they have experience
    with Outlook and similar MUAs that apply a similar mitigation to
    Sender.)

But without data, I am quite sure that C lies strictly between A and
B, and I think the conservative stance is to assume that it's too
close to B ("acquaintance phishing") to risk until data shows
otherwise.

Note that we *do* have data on the ability of the high tech spammers
and scammers to push millions of exploitative messages in minutes.
And *they* probably have data on conversion rates.  If that data shows
an advantage to Type C messages, it would be highly irresponsible to
fail to prepare for a sudden, directed onslaught of Type C messages
once such mitigations become common.

---

From: Douglas Otis <[email-masked]>
Date: Tue, 30 Sep 2014 16:08:43 -0700

Dear Stephen,

Thank you for your response, but it seems to overlook a few important issues.

The view of DMARC being a private agreement between MX servers assumes services to the general public are normally categorized as common carriers to ensure against discrimination and franchise abuse.  Unfortunately, this category is not always applied where appropriate.  For example, a major US provider handles email for other ISPs that offer general Internet access for millions of customers where the email-address provided maintains access.  As it happens, this provider also offers their own ad funded group discussion lists, but their assertion of "p=reject" applied against non-transactional messaging from other domains disrupts competitors' mailing-lists or legitimate invoicing services that are nevertheless exchanging RFC compliant messages.

Currently there is NO RFC requirement that From domains use SPF to authorize all possible IP addresses used by outbound MX servers OR be signed by matching DKIM domains.  To overcome misuse of DMARC policies, email reliant services are then FORCED to alter the content of their From header fields into non-compliant forms as their only means to ensure the integrity of their own service.  Such behavior clearly moves DMARC well beyond the realm of private agreements.  As this persists, it can no longer be seen as just a response to an exigent situation.

> lose hundreds of millions of dollars to this scam.

We are dealing with phishing problems on a daily basis.  Currently, SPF is being used by the Kelihos peer-to-peer botnet variant as a means to create DMARC valid messages using a simple tactic of looking up reverse domains of compromised computers and assuming SPF has authorized their gateway.   DMARC is also not effective against look-alike attacks either, which is another scheme not addressed.

At least adopting a standard DMARC Group exception should give potential victims a clue something is different about the message.  Valid mailing-list messages will not be ending up in their spam folder, where currently it is becoming increasingly common for people to respond to messages received there.  Will the D in DMARC soon stand for Dangerous?

I suspect the eventual solution spoofing will be to use DANE to authenticate email, Caller-ID, web sites, etc.

>    with Outlook and similar MUAs that apply a similar mitigation to

From: Dmarc-Noncompliant:[email-masked]; would be similar?

> once such mitigations become common.

In the era of botnets, spammers are able to create DMARC valid messages without much difficulty.   Many years ago, Paul Vixie warned against being too aggressive with a concern it would drive them in deeper.  By this, I think he was worried about weaknesses in DNS and BGP, abused even now often without our knowledge.  I meet with someone speaking on behalf a government who said IPv6 is much safer because it now uses BGPsec.  :-)

Regards,
Douglas Otis

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Wed, 01 Oct 2014 15:59:12 +0900

> Thank you for your response, but it seems to overlook a few

Indeed I overlooked them, but that's because in the context of this
working group I consider them unimportant.  As follows:

 > The view of DMARC being a private agreement between MX servers
 > assumes services to the general public are normally categorized as
 > common carriers to ensure against discrimination and franchise
 > abuse.

[...]

 > To overcome misuse of DMARC policies, email reliant services are
 > then FORCED to alter the content of their From header fields into
 > non-compliant forms as their only means to ensure the integrity of
 > their own service.  Such behavior clearly moves DMARC well beyond
 > the realm of private agreements.

[...]

 > SPF is being used by the Kelihos peer-to-peer botnet variant as a
 > means to create DMARC valid messages [...].   DMARC is also not
 > effective against look-alike attacks [...].

Abuse of common carrier status is not IETF business, is it?  As for
private decisions and agreements having external effects, what else is
new?  The important content of "private agreement" is that outsiders
can make all the "standards" they like, but the parties need not adopt
them and can continue their behavior.  Whether DMARC is effective or
not is also outside of the scope of this WG.

Bottom line: DMARC is a non-IETF standard prevalent on the Internet,
and the WG charter takes that as given as far as I can see.  I grant
that discussion of these issues provides context, but if I choose not
to discuss them, I don't see a problem for conducting WG business.

 > At least adopting a standard DMARC Group exception should give
 > potential victims a clue something is different about the message.

"Should", yes, I agree in some sense.  Unfortunately, at least one HCI
paper I read (I am not a specialist) says many potential victims are
clueless, and most lack some important clues for protecting their own
security.

What worries me is that users will pick up on a spurious correlation
(the same research provides evidence for that) and interpret the
visible artifacts (such as a DMARC-noncompliant tag) incorrectly.
Specifically, they'll interpret as just another example of computer
breakage, and ignore it in favor of their trust relationship with the
apparent author.

Remember, *we* know that a great majority of all messages getting that
tag are spam, but *the users* never notice them -- many are discarded
automatically and those in the spam bucket get discarded without
thought -- unless the message is actually from a trusted party, which
validates the "computer is broken" hypothesis from that point of view.

 > In the era of botnets, spammers are able to create DMARC valid
 > messages without much difficulty.

Who needs a botnet?  Anybody with access to any domain's DNS can do
that.  I suppose your point is that somebody with a botnet is able to
send From-aligned messages from a Yahoo! address.  I don't see how
anything authentication-based can deal with such malicious mail, by
definition -- the bot must be able to authenticate to send it.  To
combat such abuse, you need to examine content of outgoing messages as
well as incoming messages.  "Defense in depth" is the watchword here.

But that doesn't make DMARC "D for dangerous"or useless.  The point of
Yahoo!'s and AOL's use case (thanks to Alessandro Veseley here for
making me aware of this) is that the "acquaintance recommendation"
spam or phish is based on the ability to spoof a *particular* user to
another user, and this is based on an *external* (stolen) contact
list.  A bot can't do that arbitrarily: because of the way message
submission works, it needs to authenticate to pass DMARC p=reject.

I suppose if you have both that list and the list of bots, you could
match them up and use the "right" bot for this contact list (although
the number of matches would be a smallish fraction of all contacts, I
suppose).  (Not to mention that if you have such a 'bot, it seems more
sensible to just use its contact list if you're going to try
"acquaintance recommendation spam".)  So apparently that isn't what is
happening, as p=reject was effective to combat the April attacks.