Subject: [dmarc-ietf] Sending email on behalf of?

From: Jason Bodnar <[email-masked]>
Date: Mon, 9 Mar 2015 23:15:23 +0000

The company I work for makes software for non-profits. It's often used to host fundraising races and events. Part of the software allows people signed up for the event ("participants") to send email to their friends and family asking that they make a donation to support them in the event.

When Yahoo and AOL published their DMARC records we started having deliverability problems. Typical an email sent by a participant would look something like this:

HELO [email-masked]
MAIL FROM: [email-masked]
RCPT TO: [email-masked]
DATA
Sender: Non-profit-name <[email-masked]>
From: J Doe <[email-masked]>
To: Mamma Doe <[email-masked]>
Reply-To: J Doe <[email-masked]>
Subject: Please help me find a cure for cancer
...

This, of course, does not work well with DMARC because of:

From: J Doe <[email-masked]>

so we changed our emails to:

HELO [email-masked]
MAIL FROM: [email-masked]
RCPT TO: [email-masked]
DATA
Sender: Non-profit-name <[email-masked]>
From: J Doe <[email-masked]>
To: Mamma Doe <[email-masked]>
Reply-To: J Doe <[email-masked]>
Subject: Please help me find a cure for cancer
...

which now is delivered but, unfortunately, often appears in Mamma Doe's inbox as:

From: Non-profit-name <[email-masked]> on behalf of J Doe <[email-masked]>

According to the non-profits we work with, many people who receive these emails are wary of them due to what the From looks like in their email clients. Are there any options for us to send email on behalf of participants who have email from ESPs with DMARC reject records AND have a meaningful From in the recipient's mail client?

The DMARC draft says:

DMARC authenticates use of the RFC5322 [MAIL].From domain by requiring that it matches (is aligned with) an Authenticated Identifier. The RFC5322 [MAIL].From domain was selected as the central identity of the DMARC mechanism because it is a required message header field and therefore guaranteed to be present in compliant messages, and most MUAs represent the RFC5322 [MAIL].From field as the originator of the message and render some or all of this header field's content to end users.

But this seems contrary to information from OpenSPF:

http://www.openspf.org/Best_Practices/Webgenerated

The key component is to ensure that the SMTP "MAIL FROM" address is from your domain. After that, adding "Sender:" or "Reply-To:" headers is good etiquette and help direct replies to the proper address.

Thank you.

--
Jason Bodnar
Staff Software Engineer
Blackbaud, Inc.

---

From: Steve Atkins <[email-masked]>
Date: Mon, 9 Mar 2015 17:01:04 -0700

> According to the non-profits we work with, many people who receive these emails are wary of them due to what the From looks like in their email clients. Are there any options for us to send email o…

No, there aren't. Yahoo and AOL have published policy that they do not allow their users to use your service (at least not using their AOL or Yahoo email addresses).

If you want to avoid the "on behalf of" bits you might try removing the Sender: header. Whatever you do, though, it's going to look at least a little like mail that should be treated with suspicion - you'll need to compare your options and decide how to minimize that, possibly with some A/B testing of different options.

It's a problem a lot of ESPs who serve small customers with Yahoo and AOL addresses are seeing.

> The key component is to ensure that the SMTP "MAIL FROM" address is from your domain. After that, adding "Sender:" or "Reply-To:" headers is good etiquette and help direct replies to the proper add…

SPF is all about tying a responsible domain to a sending IP address. It doesn't really bother itself with the From: field much. Similarly, DKIM is about attaching a responsible domain to a message, one that needn't be related to any of the other email headers. It's not until you add DMARC to the mix that there's any requirement for those domains to have anything to do with the domain in the From: field.

Cheers,
  Steve
-- 
Having an Email Crisis? (800) 823-9674 

Steve Atkins - Word to the Wise - [email-masked]

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Tue, 10 Mar 2015 13:29:10 +0900

Jason Bodnar writes:

 > According to the non-profits we work with, many people who receive
 > these emails are wary of them due to what the From looks like in
 > their email clients. Are there any options for us to send email on
 > behalf of participants who have email from ESPs with DMARC reject
 > records AND have a meaningful From in the recipient's mail client?

Almost surely not.

As Steve Atkins points out, this is precisely what DMARC "p=reject" at
those providers is *intended* to prevent.[1]  If you can do it,
spammers and phishers can do it too.  There is a way to avoid DMARC[2]
that works in clients that implement the RFCs extremely flexibly, but
very few users have clients that handle such messages well, and some
clients handle them absymally badly.  I can't recommend it in your
application.

In my experience, the best you can do is

0. Tell your non-profit clients that it's a problem with the
   "recommender's" mailbox provider (and the recipient's client),
   which will occur no matter what third party sends the email on
   their behalf.  Perhaps they have suggestions on better phrasing for
   the display name (see part 2).

1. For each email, check for a restrictive DMARC policy for the
   apparent sender, and if not, send the message with the "optimal"
   message header.  (For bonus points you might be able to cache the
   policy for popular sending domains, but caching for long periods is
   risky as the TTL on the DMARC policy record is usually only a few
   minutes.)

2. If the policy is restrictive, "preformat" the display name part of
   the header as well as you can, and use your address as the address
   part.

   Example:
   From: "John Doe ([email-masked]) via NPO" <[email-masked]>

   You may even be able to omit the quotation marks in the display
   name part, but that's a little risky because of the "." (IIRC it's
   technically OK in a comment, i.e., in parentheses, but I've seen
   clients and MTAs complain anyway.)

I suspect that you can still get screwed up here, as some MUAs
automatically add the address with display name to a contact list, and
will proceed to use the display name from the contact list in
preference to the display name in From.  Then you could get this in
Mama Doe's mail client:

   From: "John Doe ([email-masked]) via NPO" <[email-masked]>

   Jane Doe ([email-masked]) recommended you as a potential
   contributor to Truly Worthy Cause.

because "[email-masked]" is the address used to look up the display name
in the contact list, and the client just stores the display name in
>From verbatim, as an alternative "real name" for that address in the
contact list.  That looks even more suspicious to me.  In that case,
the strategy you are already using is probably the best available.

Footnotes: 
[1]  Yahoo! and AOL use p=reject because spammers stole contact lists
from their users, and use those stolen lists to do exactly what you
want to do, but without having the address owner's permission.

[2]  Eg, send the actual message with the headers you want to present
as a MIME message/rfc822 attachment, and put your real address in the
outer message's From field.

---

From: "M. Hammer" <[email-masked]>
Date: Tue, 10 Mar 2015 08:21:38 -0400

Jason,

You can pattern your approach on what we at American Greetings do for 
our card notifications. The non-profit should send as themselves 
(without a sender field) and put something in the subject line that 
indicates the individual who is making the request through the 
non-profit. You can use the reply-to field to allow the recipient to 
reply directly to the originator of the request if they choose.  This 
allows the non-profit to publish their own SPF/DKIM/DMARC records 
without worrying about the records of other domains.

Both Steve and Stephan are correct in what they wrote in response to 
your question.

Mike

On 3/9/2015 7:15 PM, Jason Bodnar wrote:
> _______________________________________________
> dmarc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dmarc
>

---

From: Hector Santos <[email-masked]>
Date: Tue, 10 Mar 2015 08:51:18 -0400

> SPF is all about tying a responsible domain to a sending IP address. It doesn't really bother itself with the From: field much. Similarly, DKIM is about attaching a responsible domain to a message,…

Small reminder point -- Independent of DMARC, DKIM does have a single 
requirement to bind the 5322.from header at a minimum.  So that 
"separation" was never really there as DKIM was updated to describe as 
such. It was not possible to make that "separation."

DMARC, like all the rest, SSP, ADSP, etc, is just a policy layer; 
basically an enforcement of a 1st party singer requirement.   The 
problem we face (for nearly 10 years now) is that we lack 3rd party 
policies.

DMARC is ready for it.  Just got to do it.  When ATPS was done, there 
was no wide industry support for ADSP but we knew it was desirable. 
It was replaced with a report-generating "Super ADSP version called 
DMARC that did get industry interest.

Since DMARC does allow for authentication/authorization extensions, 
IMO, its time to restart and push an ATPS version for DMARC.  Feasible 
or not at a higher scale (which I am not convinced it is not), the 
IETF should not restrict this practical protocol framework, automated, 
non-subjective, plug and play, "lower cost" solution.  Allow 
implementators to explore the management of 3rd party policies.  Not 
everyone is going to need to have high-end scale versions of this in 
order to make it useful.

-- 
HLS

---

From: "John Levine" <[email-masked]>
Date: 10 Mar 2015 16:14:13 -0000

>   From: "John Doe ([email-masked]) via NPO" <[email-masked]>

I would try some tests particularly at AOL before I did that.  AOL
mail tends to reject anything that looks like a munged AOL address.

The least painful approach may be to tell people with AOL addresses,
sorry, please get a different address.

R's,
John

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Wed, 11 Mar 2015 01:42:16 +0900

> sorry, please get a different address.

Oh, boy, oh, boy, am I glad I *can* tell my users to get another
address or suffer.

But I don't see how the OP can.  First of all, they aren't his users,
they are his client's users.  Second of all, I can't imagine the
client telling *its* donors they have to get a new address to
recommend donation to friends and family.  And third (the killer) the
recipients aren't going to recognize the new address, and so it's
going to look as suspicious as the stupid Outlook-style headers.

Brother-I-feel-your-pain-ly y'rs,

Steve

---

From: Terry Zink <[email-masked]>
Date: Tue, 10 Mar 2015 16:55:31 +0000

> And third (the killer) the

What's "stupid" about Outlook style headers? How should it look?

-- Terry

-----Original Message-----
From: dmarc [mailto:[email-masked]] On Behalf Of Stephen J. Turnbull
Sent: Tuesday, March 10, 2015 9:42 AM
To: John Levine
Cc: [email-masked]
Subject: Re: [dmarc-ietf] Sending email on behalf of?

Steve

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Wed, 11 Mar 2015 03:34:38 +0900

Terry Zink writes:

 > > And third (the killer) the recipients aren't going to recognize
 > > the new address, and so it's going to look as suspicious as the
 > > stupid Outlook-style headers.
 > 

Technically, maybe nothing, in the OP's case.  Depends on how much
involvement the putative author (John Doe) has in composition or
approval of the message text.  If "none", then I suppose the "on
behalf of" in the From field as displayed by Outlook has the semantics
it normally has in English: the NPO wrote and sent the message as the
fully responsible agent of John Doe, with no intervention from
Mr. Doe.  In most cases I see, however, Sender is a robot Mediator,
typically a mailing list.  In those cases the Outlook header display
is the equivalent of "From the White House Mailroom on behalf of
Barack Obama".

Practically, the OP's client NPO wants the message to look like a
personal invitation from an existing donor to another potential donor
who is an acquaintance.  This is being done with the permission (at
least) of the existing donor.  So I don't see why Outlook (or any MUA)
would fail to display it that way.

As far as I can see, if the message is legitimate (ie, has the
permission of the existing donor and owner of the intended From
address), the "on behalf of" style is unuseful to anyone, and clearly
confusing to many non-technical users.  I don't know of any cases
where it's useful, given that illegitimate messages can (and do) avoid
suspicious display by the simple expedient of omitting the Sender
field.  "Stupid" is perhaps unnecessarily derogatory, but it's an idea
that has proven to be more trouble than it's worth in practice, and it
should be retired (or at least made an option).

How should it look?  How about

    From: John Doe <[email-masked]>
    Sender: NPO <[email-masked]>

or just

    From: John Doe <[email-masked]>

for that matter?  Sender is not very useful to the 99.44% of mail
users who aren't RFC nerds, since no MUA I know has a "report delivery
problem to sender" function (let alone a hotkey for it), and those
users aren't going to know that a problem with message delivery should
be reported to Sender (rather than From).

Regards,

---

From: "MH Michael Hammer (5304)" <[email-masked]>
Date: Tue, 10 Mar 2015 18:43:29 +0000

> Sent: Tuesday, March 10, 2015 2:35 PM
> To: Terry Zink
> Cc: [email-masked]
>  > > And third (the killer) the recipients aren't going to recognize  > > the new
> address, and so it's going to look as suspicious as the  > > stupid Outlook-style
> headers.
>  >
> involvement the putative author (John Doe) has in composition or approval
> of the message text.  If "none", then I suppose the "on behalf of" in the
> From field as displayed by Outlook has the semantics it normally has in
> English: the NPO wrote and sent the message as the fully responsible agent
> of John Doe, with no intervention from Mr. Doe.  In most cases I see,
> however, Sender is a robot Mediator, typically a mailing list.  In those cases
> the Outlook header display is the equivalent of "From the White House
> Mailroom on behalf of Barack Obama".
> 

If it had the format you suggest it would almost certainly be fraudulent. If it was legitimate it would be from @eop.gov.

> Practically, the OP's client NPO wants the message to look like a personal
> invitation from an existing donor to another potential donor who is an
> acquaintance.  This is being done with the permission (at
> least) of the existing donor.  So I don't see why Outlook (or any MUA) would
> fail to display it that way.
> 

How does the recipient know that the NPO has permission? In fact, how does the recipient know that it is really the NPO if it is spoofing the existing donor's email address? 

> As far as I can see, if the message is legitimate (ie, has the permission of the
> existing donor and owner of the intended From address), the "on behalf of"
> style is unuseful to anyone, and clearly confusing to many non-technical
> users.  I don't know of any cases where it's useful, given that illegitimate
> messages can (and do) avoid suspicious display by the simple expedient of
> omitting the Sender field.  "Stupid" is perhaps unnecessarily derogatory, but
> it's an idea that has proven to be more trouble than it's worth in practice, and
> it should be retired (or at least made an option).
> 

This is meaningless to the recipient in terms of authenticating the relationship or the From: if the From: email address is "spoofed".

>     From: John Doe <[email-masked]>

This is meaningless to the recipient in terms of authenticating the relationship or the From: if the From: email address is "spoofed". It falls into the category of : "If you could read my mind".

> for that matter?  Sender is not very useful to the 99.44% of mail users who
> aren't RFC nerds, since no MUA I know has a "report delivery problem to
> sender" function (let alone a hotkey for it), and those users aren't going to
> know that a problem with message delivery should be reported to Sender
> (rather than From).
> 

Sender in its present incarnation is not particularly useful, period.

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Wed, 11 Mar 2015 04:32:47 +0900

MH Michael Hammer (5304) writes:

 > How does the recipient know that the NPO has permission? In fact,
 > how does the recipient know that it is really the NPO if it is
 > spoofing the existing donor's email address?

He doesn't, and he doesn't.  That's my point: absent some form of
authentication, he doesn't know, and Outlook's use of "on behalf of"
doesn't help one bit.

 > This is meaningless to the recipient in terms of authenticating the
 > relationship or the From: if the From: email address is "spoofed".

Terry specifically asked what I don't like about Outlook's header
display.  There was no question about authentication AFAICS, and I
certainly didn't try to answer any such question.

 > Sender in its present incarnation is not particularly useful,

I don't disagree.  I just think Outlook's display makes it worse than
useless.

---

From: Terry Zink <[email-masked]>
Date: Tue, 10 Mar 2015 21:14:05 +0000

> I don't disagree.  I just think Outlook's display makes it worse than

The Outlook client is used in many places - it hooks up with the Exchange MTA but also with multiple other mail services like Yahoo Mail, Hotmail, Gmail... anything that supports mail. It has also been around for as long as I can remember, going back into the 1990's and has a very large install-base. It pre-dates any email authentication standard, and supports 2822 headers like Sender, Reply-To, etc.

Saying Outlook's display is "worse than useless" fails to consider the context in which was developed (in the absence of advice it did what it could), the massive if-then-else pipeline that must take authentication into account, whether or not it can trust email headers, checking the recipient email address (Hotmail, Gmail, Yahoo all have slightly different implementations of Authentication-Results stamping), how to deploy many changes to multiple versions in production, how to augment the experience with Exchange, backwards compatibility, adding public documentation to MSDN, etc.

Seeing as how there isn't any consensus on how MTAs should display things, Outlook's implementation of displaying multiple sender/from/reply-to  fields is on-par with almost anyone else's.

--Terry

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Wed, 11 Mar 2015 11:19:42 +0900

Terry Zink writes:

 > Seeing as how there isn't any consensus on how MTAs should display
 > things, Outlook's implementation of displaying multiple
 > sender/from/reply-to  fields is on-par with almost anyone else's.

That criterion ("no consensus") is like saying that since there's no
consensus on what constitutes great art, my daughter's finger painting
is on-par with Picasso's Guernica.

We have testimony here (and I can confirm that from observation of
Mailman lists, as well as report my personal "Ugh!" reaction) that
Outlook's "on behalf of" treatment of the Sender field causes
confusion and annoyance for third party services.

The ball is in your court, tell us about the benefits, don't tell us
that pros and cons don't matter.  I'm perfectly willing to believe
that it has benefits in environments I have not experienced, I just
can't think of any.  Especially not in the modern environment where
malicious agents will spoof any field that might be displayed to
support their frauds.  This is the attitude of most in the Mailman
community.  It won't ameliorate our pain, but at least it will help me
to reduce the open hostility my community shows to Outlook over this
issue, if you can come up with plausible arguments "pro" this display.

BTW, I consider this subthread mildly on-topic, as trying to
understand the protocols we propose requires understanding the threat
models, and it is fundamental to DMARC that header display is part of
any threat model (to the point where DMARC interdicts whole message
for the sole purpose of ensuring that "From" is not displayed to the
recipient).

Regards,
Steve

---

From: Terry Zink <[email-masked]>
Date: Wed, 11 Mar 2015 20:14:46 +0000

Hi, Stephen,

I understand your point that some UX displays are better than others. However, I'm not going to get into a debate about the merits and drawbacks, and under what scenarios, about how Outlook vs. how other mail clients choose to display Sender:, From:, Reply-To:, and other fields. It's not within my area of ownership nor have I looked any usability studies surrounding showing it as X vs. Y. Rather, we have Internet RFCs that define how to handle email but no RFCs about how to display email.

There is a lot of overlap in some mail clients, but there is also ambiguity about how to show certain things. To say "My users like this, and don't like that" is good anecdotal evidence but what proportion of mail flow is that? How representative is it for everyone else? How much does the existing user base understand or even care about how something like this is displayed? Is it good enough? These are not questions that are easily answered.

Thanks.

-- Terry

-----Original Message-----
From: Stephen J. Turnbull [mailto:[email-masked]] 
Sent: Tuesday, March 10, 2015 7:20 PM
To: Terry Zink
Cc: [email-masked]
Subject: Re: [dmarc-ietf] Sending email on behalf of?

Regards,
Steve

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Thu, 12 Mar 2015 14:12:29 +0900

Terry Zink writes:

 > There is a lot of overlap in some mail clients, but there is also
 > ambiguity about how to show certain things. To say "My users like
 > this, and don't like that"

"Like" and "dislike" are *not* what was reported here.  Mail that we
should presume is legitimate (based on the OP's testimony) was
suspected of being fraudulent merely because of the "on behalf of"
display.  So "on behalf of" appears to be generating Type II errors
(false positive for abuse), while DMARC is intended to prevent Type I
errors (false negative for abuse).

 > is good anecdotal evidence but what proportion of mail flow is
 > that? How representative is it for everyone else? How much does the
 > existing user base understand or even care about how something like
 > this is displayed? Is it good enough? These are not questions that
 > are easily answered.

Nor are they the questions I'm interested in here.  I want to know if
there are use cases where displaying "on behalf of" is *useful*, or if
when it matters, it primarily induces user *mistakes*.

Of course to Microsoft's Outlook developers, those are crucial
questions.  We aren't Outlook developers (not even you), so we don't
actually care.  I think we do care about user psychology in deciding
whether a particular message is abusive, though.

---

From: Hector Santos <[email-masked]>
Date: Thu, 12 Mar 2015 14:43:58 -0400

> On Mar 12, 2015, at 1:12 AM, Stephen J. Turnbull <[email-masked]> wrote:
> 
> Terry Zink writes:
> 

Actually, I do care because the desire is for consistent and persistent behavior across the MUA board.  When dealing with subjective concepts such as User Psychology, then its becomes much harder to address due to lack of persistency and consistency. 

We are looking for generic, reproducible, verifiable automatic computer software logic, that removes the burden from the user, including operators, making the complex decision about what is good or bad.  The exceptions are always taken into account.  Most of the time, you are going to depend on what the backend does consistently and persistently for all user types because the backend has to worry about a group of MUA types -- there is not just one type of course, even if there is a monopoly, there are far too many MUAs.  This includes the old school RFC-based MUAs. The current trend is to move users to online MUAs, but that has always been a direction with the older methods still viable. They are not going away. 

--
Hector Santos
http://www.santronics.com