Subject: [dmarc-ietf] DMARC alignment conflicts with RFC 5322 on the use of the From and Sender header fields

From: Jesse Thompson <[email-masked]>
Date: Tue, 2 Jun 2020 16:44:28 +0000

I'm relaying these DMARC questions/concerns on behalf of an email admin at another university.  I quickly searched this list's archives for the Sender header vs DMARC alignment issue and don't see much aside from a conversation in May 2015.  Is it worth further discussion and/or an issue in Trac?  I think I know the answer to the second concern, but I'll defer to people more adept at explaining the nuance.

See below...

Thanks,
Jesse Thompson
UW-Madison

"
I don't see on the list of issues the most fundamental problem of DMARC, namely that it conflicts with RFC 5322 on the use of the From and Sender header fields [1] and possibly with RFC 6326 as to the significance of DKIM fail [2].  The former is the more serious problem. Making DMARC alignment part of a standard for Internet messages requires a revision of RFC 5322. I'd love to see this resolved.

[1]
The "From:" field specifies the author(s) of the message,
   that is, the mailbox(es) of the person(s) or system(s) responsible
   for the writing of the message.  The "Sender:" field specifies the
   mailbox of the agent responsible for the actual transmission of the
   message.

[2]
signature verification failure does not force rejection of the message;
"

---

From: Dave Crocker <[email-masked]>
Date: Tue, 2 Jun 2020 10:00:55 -0700

> I don't see on the list of issues the most fundamental problem of DMARC, namely that it conflicts with RFC 5322 on the use of the From and Sender header fields [1] and possibly with RFC 6326 as to…

As one of the folk who created the Sender: construct in RFC 733, I'll 
suggest that the concern raised here has merit, though dealing with the 
fact isn't straightforward.  It's an issue I've been wrestling with for 
a couple of years.

In reality, all of the current email anti-abuse authentication methods 
concern the email operator, not the author.

The problem is that, in the message, the only identifier that is 
reliably present is the rfc5322.From field email address.

The underlying design choice in RFC733 -- 45 years ago -- was in making 
Sender: optional when it's content is the same as the From: field.  It 
never occurred to us that one of them might need changing along the 
handling path.  The lesson to future designers is to strongly resist 
conflating otherwise-independent semantics for "efficiency".

DMARC enforcement requires that the DKIM/SPF domain be the same as the 
author From:.  That is, the folk doing email operations have to be able 
to sign the DMARC aligned domain.

Hence the From: field is now really the Sender: field.  The From: field 
fixup hacks that are needed by intermediaries, to avoid DMARC-based 
rejections, makes this fact painfully clear, even as they serve to 
undermine recipient use of the From field for author-related message 
management.

Given the depth and momentum of DMARC's effects, I don't think it's 
realistic to try to fix this via changes to the From: field.

The only suggestion I've been able to formulate is:  create a new field, 
such as Author:.

(Give it a simple, clean, appropriate name, rather than something like 
Original-From.)

d/

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Brandon Long <[email-masked]>
Date: Tue, 2 Jun 2020 10:11:50 -0700

On Tue, Jun 2, 2020 at 10:01 AM Dave Crocker <[email-masked]> wrote:

> > I'm relaying these DMARC questions/concerns on behalf of an email admin
> at another university.  I quickly searched this list's archives for the
> Sender header vs DMARC alignment issue and don't see much aside from a
> conversation in May 2015.  Is it worth further discussion and/or an issue
> in Trac?  I think I know the answer to the second concern, but I'll defer
> to people more adept at explaining the nuance.
> >
> > See below...
> >
> > Thanks,
> > Jesse Thompson
> > UW-Madison
> >
> > "
> > I don't see on the list of issues the most fundamental problem of DMARC,
> namely that it conflicts with RFC 5322 on the use of the From and Sender
> header fields [1] and possibly with RFC 6326 as to the significance of DKIM
> fail [2].  The former is the more serious problem. Making DMARC alignment
> part of a standard for Internet messages requires a revision of RFC 5322.
> I'd love to see this resolved.
>
>

And if the mail client displays the Author, then we're kind of back to
square one
with displaying unvalidated data to the user.

Which is to say, DMARC is just policy, but what makes it different from the
previous policy attempts
is that it requires the visible author/operator to be authenticated.

There's no reason that DMARC couldn't have included the sender or tried to
have some kind of
PRA like spf v2... but that's not the goal.

At some point, you're going up against the existing mail client design and
of course
how users act, and of course all of that is messy.  The "clean" strictness
and mechanical nature
of DMARC is ultimately up against the fuzzy ux design and fuzzier humans.

On top of that, the author domains basically want to control who can send
on their behalf, which is
a reasonable goal as well.

Brandon

---

From: Dotzero <[email-masked]>
Date: Tue, 2 Jun 2020 13:35:01 -0400

On Tue, Jun 2, 2020 at 12:44 PM Jesse Thompson <jesse.thompson=
[email-masked]> wrote:

> I'm relaying these DMARC questions/concerns on behalf of an email admin at
> another university.  I quickly searched this list's archives for the Sender
> header vs DMARC alignment issue and don't see much aside from a

We went through this with SenderID and it's use of Sender in PRA. Back in
the day I upset the folks at Microsoft responsible for evangelizing
SenderID by sending emails to them using their name an email in the From
field and myself in the Sender field. My emails would consistently get a
neutral rating under SenderID because of how PRA worked.

As part of the original DMARC team, the goal was to make clear whether the
email was authorized by the domain being used, hence the reliance on SPF
and DKIM. These are clearly under the domain owner/administrator's control
to the extent they choose to exercise that control. There was much
discussion in the community at the time to use thei= field to enable more
granular signing. it never gained traction. Because the intent was to
protect end users from fake emails purporting to be from (primarily)
commercial domains such as financial institutions, greeting card companies,
etc., the Sender field was not a significant issue. Also, when the Sender
is in the same domain as the From, there is no DMARC issue.

Michael Hammer

If you really want to enable someone to send on your behalf then a) give
them direct access to send from your account; b) give them your private
DKIM signing key (which allows them to sign for any account in that domain
or c) delegate a subdomain to them so they can generate their own SPF or
DKIM records.

---

From: Dave Crocker <[email-masked]>
Date: Tue, 2 Jun 2020 11:01:03 -0700

> with displaying unvalidated data to the user.

No we aren't.

Your comment implies that what is displayed to the user is important in 
anti-abuse efforts, but there is no data to support that view, and some 
significant data to support the view that that's wrong.  (cf, the 
extensive literature review that was done during early BIMI discussions.)

What matters is what is done by your filtering engine -- and what is in 
the message content -- not what is displayed to the user in the From 
field.  I understand that DMARC has been useful, but I believe this is 
confusing correlation with causation.  The cause is down in the 
filtering engine.

> There's no reason that DMARC couldn't have included the sender or 
> tried to have some kind of

But the Sender: field is not reliably present and, of course, DMARC 
needs an identifier that is reliably present.

> At some point, you're going up against the existing mail client design 
> and of course
> how users act, and of course all of that is messy.  The "clean" 
> strictness and mechanical nature

DMARC is a triumph of infrastructure operator demands over end-user 
experience.  it's created a markedly Procrustean email identification 
environment.

The standards and the practice, for 45 years, have permitted certain 
freedoms in the From: field and DMARC eliminated them.

It's easy to be cavalier about this, since some operators run highly 
controlled environments and have no incentives for paying attention to 
those who have used those freedoms legitimately, for 45 years.

> On top of that, the author domains basically want to control who can 
> send on their behalf, which is

That's a rather small subset of domain owners.  While DMARC was original 
developed for that select community, its vastly broader use results in 
over-applying that restriction.

AGAIN, I'm not suggesting changing DMARC or the current reality for the 
From: field.

RATHER, I'm suggesting making it possible for recipients to regain 
usefulness of the author information that the From: field was intended 
to provide but no longer does.

FOR THOSE FEW DOMAINS that really want to get strict about who gets to 
claim to be an author associated with a domain, then do something like 
add a DMARC option that prohibits use of the Author: field.  (Note that 
just having DKIM and ARC pre-sign a non-existent Author field 
accomplishes this, too...)

d/

---

From: "Pete Resnick" <[email-masked]>
Date: Tue, 02 Jun 2020 13:12:29 -0500

> needs an identifier that is reliably present.

Dave, could you explain that? Coding-wise, there's surely no reason that 
an implementation can't say, "if 5322.sender is present then sender = 
5322.sender else sender = 5322.from". So you could say that the 
identifier of sender is reliably present, since it's taken from 
5322.from if 5322.sender isn't present. But maybe I'm missing something. 
Please explain.

pr
-- 
Pete Resnick https://www.episteme.net/
All connections to the world are tenuous at best

---

From: Dave Crocker <[email-masked]>
Date: Tue, 2 Jun 2020 11:29:00 -0700

On 6/2/2020 11:12 AM, Pete Resnick wrote:
> Dave, could you explain that? Coding-wise, there's surely no reason 
> that an implementation can't say, "if 5322.sender is present then 
> sender = 5322.sender else sender = 5322.from". So you could say that 
> the identifier of sender is reliably present, since it's taken from 
> 5322.from if 5322.sender isn't present. But maybe I'm missing 
> something. Please explain.
>

Not sure what you are asking.  What I mean is that it isn't always present.

If Sender: contains the same address as From:, then Sender does not have 
to be present, and often/usually isn't.

So when someone comes along and changes From: -- such as to hack around 
the DMARC problem for intermediaries -- the semantic of the Sender 
information is lost.

d/

---

From: "Pete Resnick" <[email-masked]>
Date: Tue, 02 Jun 2020 14:32:05 -0500

On 2 Jun 2020, at 13:29, Dave Crocker wrote:

> If Sender: contains the same address as From:, then Sender does not 
> have to be present, and often/usually isn't.

Well, that's the field, not its value.

> So when someone comes along and changes From: -- such as to hack 
> around the DMARC problem for intermediaries -- the semantic of the 
> Sender information is lost.

If you do change the From, you should always add a Sender. (Or is your 
point that implementer's don't, and that's the problem?)

What I'm missing is why the lack of an actual Sender: header field is 
problematic.

pr

---

From: Dave Crocker <[email-masked]>
Date: Tue, 2 Jun 2020 12:51:01 -0700

> point that implementer's don't, and that's the problem?)

Except that there's nothing that specifies that and I believe it isn't 
common practice.  Worse, creating that field in mid-stream does not fix 
the problem that now the author information is lost.

The actual requirement is to have the Sender field always be present at 
the time of original posting, and have DMARC work from the Sender 
field.  But since none of that is going to happen, I'm looking for a 
path to providing clean original-author information that is practical.

> What I'm missing is why the lack of an actual Sender: header field is 

Because DMARC should have focused on the Sender field, not the From 
field, since it is really about the email operator, not the author.

d/

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 2 Jun 2020 13:36:09 -0700

> extensive literature review that was done during early BIMI discussions.)

That's a curious assertion given all of the energy that's gone into
complaining about but never really resolving the "display name" problem
over the years.  I thought that was a key part of the vector of many
successful phishing attacks.

I suppose it's possible that operators came up with this problem and
decided it needs solving, with no user complaints like "I was fooled by
this fake From, can't you do something about that?" on which to base that.

Hasn't M3AAWG at least had something other than anecdata that this is a
true source of pain?

> those who have used those freedoms legitimately, for 45 years.

No reply here, just felt like citing this again.

-MSK

---

From: Dave Crocker <[email-masked]>
Date: Tue, 2 Jun 2020 13:46:17 -0700

On 6/2/2020 1:36 PM, Murray S. Kucherawy wrote:
> On Tue, Jun 2, 2020 at 11:01 AM Dave Crocker <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
> complaining about but never really resolving the "display name" 
> problem over the years.  I thought that was a key part of the vector 
> of many successful phishing attacks.

In the world of Internet protocol standards, there is a common problem 
in discussing anything involving users, failing to distinguish between 
the mathematics of abuse from the actual effects.  That is, if I lie 
about the author, that's abuse in an absolute sense.  But that can be 
quite different from whether that lie has any effect on the recipient.

So, yes, lots of people have been upset constantly over the years about 
all sorts of abusive behaviors.  However there appears to be no actual 
evidence that lying in the From field affects end user behaviors, and 
certainly none that lying in the From field about the domain name does.

And since my notes on this thread seem to be having a difficult time 
being understood, I'll stress that my reference is specifically to 
end-user behavior.  There other abuse factors, separate from that, which 
DMARC apparently correlates usefully with, which is why it apparently 
really is useful to the filtering engine.  But not the recipient user.

> decided it needs solving, with no user complaints like "I was fooled 
> by this fake From, can't you do something about that?" on which to 
> base that.

Exactly.

> Hasn't M3AAWG at least had something other than anecdata that this is 
> a true source of pain?

No.

As I mentioned in the previous note, there was a literature survey done 
at the start of the BIMI work, and it produced no evidence to support 
claims of improved end user behavior.

The canonical example of this issue was the EV web domain name exercise.

> No reply here, just felt like citing this again.

ditto.

d/

---

From: Seth Blank <[email-masked]>
Date: Tue, 2 Jun 2020 14:30:47 -0700

As an individual:

On Tue, Jun 2, 2020 at 1:46 PM Dave Crocker <[email-masked]> wrote:

> However there appears to be no actual evidence that lying in the From
> field affects end user behaviors, and certainly none that lying in the From
> field about the domain name does.
>

There are decades of data that prove just this. On the abuse side,
Microsoft, Google, Proofpoint, Mimecast, and others (including Valimail)
have all published reams of research reports over the years. On the
marketing side, there's another decade or two of data about how properly
crafting the From materially impacts open rates on messages, which means
user behavior is certainly impacted by what's in the From and display name.

There's more data here than can be meaningfully summarized. So to pick one
at random about usage of these methods in abuse, read page 11 of this
report:
https://www.proofpoint.com/sites/default/files/pfpt-us-tr-q117-threat-report.pdf

And on the marketing side, after a 2 second google search, here's some A/B
testing:
https://blog.influenceandco.com/how-to-optimize-your-email-open-rate-with-friendly-froms

> this fake From, can't you do something about that?" on which to base that.

The history of DMARC is the exact opposite. There was a mountain of phish
impersonating well known companies that was defrauding consumers to the
tune of hundreds of millions of dollars, and the companies involved got
together and asked the major mailbox providers to work with them to
determine the appropriate signals needed to prevent the phishing using
their domains. DMARC is the result of a multi-year comprehensive data
investigation here.

> As I mentioned in the previous note, there was a literature survey done at
> the start of the BIMI work, and it produced no evidence to support claims
> of improved end user behavior.
>

Trust indicators that require users take appropriate action are doomed to
fail, and as you mentioned the data concurs. See your EV example and the
reason that padlock icons are going away.

But the flipside is not true. What users see can certainly trick them into
doing the wrong thing, especially if they believe they're doing the right
thing, and especially if a wide net is cast. This is why CEO-CFO and gift
card scams are so prevalent and effective. Again, grabbing a random example
from another 2 second google search, a few years ago the FBI said this type
of scam resulted in $2.3 billion worth of damages:
https://www.fbi.gov/contact-us/field-offices/phoenix/news/press-releases/fbi-warns-of-dramatic-increase-in-business-e-mail-scams

Or:
https://ottawa.ctvnews.ca/city-treasurer-sends-128-000-to-fraudsters-in-email-phishing-scam-1.4370829
(although it's unclear if DMARC would have solved this attack, the point is
that the treasurer thought it was from the mayor).

M3AAWG has shared mountains of data that DMARC solves a materially
significant problem, and this has been presented on again and again and
again. Governments are increasingly mandating it, and more industry
organizations are requiring it for all members. This is a source of real
pain which goes far beyond anecdotes.

Seth (hatless, and trying to understand your comments)

---

From: Seth Blank <[email-masked]>
Date: Tue, 2 Jun 2020 14:42:29 -0700

Also, from literally today:
https://www.justice.gov/usao-sdtx/pr/man-admits-spoof-email-fraud-scheme-and-more

---

From: Dave Crocker <[email-masked]>
Date: Tue, 2 Jun 2020 15:01:00 -0700

Wow. I'll ask folk to reread my text, here, carefully, since it 
specified something quite narrow and concrete, but is somehow being 
taken to have meant something broad and general:

> On Tue, Jun 2, 2020 at 1:46 PM Dave Crocker <[email-masked] 
>> field affects end user behaviors, and certainly none that lying in 
>> the From field about the domain name does.

And again, there are all sorts of threats and all sorts of bad 
behaviors, but the question is whether a particular kind of bad actor 
behavior affects recipient end-user behavior.

And the specific kind is lying about the From: field domain name.

Please point to specific research -- not an extended report with lots of 
varying content.

On 6/2/2020 2:30 PM, Seth Blank wrote:
> There are decades of data that prove just this.

As I said, we did an extensive literature search at the beginning of the 
BIMI and there was no supporting research.

So now let's look at the purported counter-example you provided:

> On the abuse side, Microsoft, Google, Proofpoint, Mimecast, and others 
> (including Valimail) have all published reams of research reports over 
> the years. On the marketing side, there's another decade or two of 
> data about how properly crafting the From materially impacts open 
> rates on messages, which means user behavior is certainly impacted by 
> what's in the From and display name.
>
> There's more data here than can be meaningfully summarized. So to pick 
> one at random about usage of these methods in abuse, read page 11 of 
> this report: 

Doesn't contain the word 'behavior'.

Doesn't contain 'from:'

Only 'author' is reference to malware creators, not recipients.

'Recipeint' gets a brief sidebar reference to mail pretending to be from 
a top executive. Another sidebar with the word explains 'spoofing' as 
impersonation (which, of course, is what it means in the real world, but 
not in the email abuse world, which has a much broader definition.

I'll stop now and note that the reference you gave appears to have 
nothing to do with the specific behavioral issue I cited.

d/

---

From: "Kurt Andersen (b)" <[email-masked]>
Date: Tue, 2 Jun 2020 15:26:12 -0700

I'm sorry to pile on but could not restrain myself:
https://www.bmj.com/content/327/7429/1459?ijkey=c3677213eca83ff6599127794fc58c4e0f6de55a&keytype2=tf_ipsecsha

I get Dave's point, but at the same time, it is well known that copy tweaks
can have significant effects on conversion rates. Whether the specific
tweak of the 5322.From header field's SMTP domain from <rando-domain> to
<unprotected-by-dmarc-enforcement-domain> has an effect separate from any
other subterfuges that might be being employed by the miscreant is probably
hard to disambiguate, especially since in the real world, it is rare to
have bare SMTP addresses in that header field.

--Kurt

On Tue, Jun 2, 2020 at 3:01 PM Dave Crocker <[email-masked]> wrote:

> Wow. I'll ask folk to reread my text, here, carefully, since it specified
> something quite narrow and concrete, but is somehow being taken to have
> meant something broad and general:
>
> And again, there are all sorts of threats and all sorts of bad behaviors,
> but the question is whether a particular kind of bad actor behavior affects
> recipient end-user behavior.
>

---

From: Dave Crocker <[email-masked]>
Date: Tue, 2 Jun 2020 15:28:50 -0700

> https://www.justice.gov/usao-sdtx/pr/man-admits-spoof-email-fraud-scheme-and-more

Oh my. Is it really that difficult to understand the difference between 
choosing to take an action, versus being affected by your taking that 
action.

Let's keep this simple:

1. Most users do not see the domain name in the From: field.  So how 
could using a cousin domain or any other misbehavior with the domain 
name, cause the recipient to be tricked?

2. Nothing in that article demonstrates that a recipient was tricked 
into misbehavior by the presence of a problematic From: field domain name.

d/

---

From: Dotzero <[email-masked]>
Date: Tue, 2 Jun 2020 18:41:55 -0400

On Tue, Jun 2, 2020 at 5:31 PM Seth Blank <seth=
[email-masked]> wrote:

> https://blog.influenceandco..com/how-to-optimize-your-email-open-rate-with-friendly-froms
> <https://blog.influenceandco.com/how-to-optimize-your-email-open-rate-with-friendly-froms>
>

Actually Seth, you are flat out wrong. I was there and part of it. It was
not about signaling. It was implemented at the MTA  level and was about
preventing the "badness" from reaching the end user rather than signaling
to the end user. Google experimented with displaying "keys" and Microsoft
experimented with displaying "shields". Neither of those efforts were
integral to the DMARC effort. My own experience is that a significant
percentage of end users will click on just about anything. This was
validated in the 2007 timeframe during some phishing runs where the bad
guys actually left some tracking code on a fake WWW landing page the email
links led to. It was also validated during the Storm Worm when the links
used IP Addresses. This issue has been validated at other points and times.
Individual sending organizations and receiving domains have been generally
reluctant to release data because it might expose company confidential
information. Aggregated isn't so useful because there are significant
variations in company efforts - not just with DMARC - that impact outcomes.
So far, signaling to the end user doesn't have a particularly good track
record.

DMARC started as a private effort among a handful of private parties. when
it was successful in stopping direct domain abuse for a handful of sending
domains at a handful of receivers we started discussing whether the
approach could be codified as a standard to enable others to benefit from
the approach. The origins and history are important in understanding why
DMARC is what it is.

Michael Hammer

---

From: Seth Blank <[email-masked]>
Date: Tue, 2 Jun 2020 15:53:14 -0700

On Tue, Jun 2, 2020 at 3:42 PM Dotzero <[email-masked]> wrote:

> to the end user.

Michael, there's a crossed wire here-- I didn't mean to communicate that
DMARC is in any way about signaling. I'm in complete agreement here. The
point I was trying to make is that consumers are susceptible to fraud, and
the system needs to stop these messages before they ever get in front of a
user. The signal I was talking about is from the data: when something tries
to authenticate to an MTA but then tell the user it's someone else. That's
what alignment fixes and what's so powerful about DMARC.

> Google experimented with displaying "keys" and Microsoft experimented with
> displaying "shields". Neither of those efforts were integral to the DMARC
> effort. My own experience is that a significant percentage of end users
> will click on just about anything. This was validated in the 2007 timeframe
> during some phishing runs where the bad guys actually left some tracking
> code on a fake WWW landing page the email links led to. It was also
> validated during the Storm Worm when the links used IP Addresses. This
> issue has been validated at other points and times. Individual sending
> organizations and receiving domains have been generally reluctant to
> release data because it might expose company confidential information.
> Aggregated isn't so useful because there are significant variations in
> company efforts - not just with DMARC - that impact outcomes. So far,
> signaling to the end user doesn't have a particularly good track record.
>

I'm in complete agreement with this, and didn't mean to convey otherwise.

---

From: Dave Crocker <[email-masked]>
Date: Tue, 2 Jun 2020 16:02:08 -0700

On 6/2/2020 3:53 PM, Seth Blank wrote:
> The point I was trying to make is that consumers are susceptible to 
> fraud,

Of course they are.  Unfortunately, that point is irrelevant, because it 
isn't the question at hand.

> and the system needs to stop these messages before they ever get in 
> front of a user.

Exactly.  And that's why claiming that making the From: field domain 
name better (or worse) in changing recipient user behavior is wrong.

> The signal I was talking about is from the data: when something tries 
> to authenticate to an MTA but then tell the user it's someone else. 
> That's what alignment fixes and what's so powerful about DMARC.

Seth, your statement is so confused, I'm not sure I can fix it up. 
"Authenticate to the MTA?"  DKIM and DMARC don't do that.

d/

---

From: Seth Blank <[email-masked]>
Date: Tue, 2 Jun 2020 17:13:19 -0700

> isn't the question at hand.

Dave, this is exactly the point where I think we're on different pages. The
From: domain matters because its contents affect user behavior. Unless I'm
deeply misunderstanding your earlier posts (and I'm glad to be wrong here),
you don't appear to believe this to be true.

Alignment matters, because it ensures that the domain which is
authenticated matches what the user sees in the inbox (because, rightly or
wrongly, inboxes show the contents of the From: header field). When this
match fails, a message can be rejected before it's ever in front of a user
and capable of causing confusion or fraud.

The point is NOT to change user behavior due to what is presented in the
From:, it is to prevent manipulation of user behavior by only allowing
From: domains to be displayed if they have been authenticated.

Your argument seems to be that you don't believe that spoofing the From:
domain leads to user impact, or am I completely misunderstanding you?

Seth

---

From: Dave Crocker <[email-masked]>
Date: Tue, 2 Jun 2020 17:26:11 -0700

On 6/2/2020 5:13 PM, Seth Blank wrote:
> On Tue, Jun 2, 2020 at 4:02 PM Dave Crocker <[email-masked] 
> Dave, this is exactly the point where I think we're on different 
> pages. The From: domain matters because its contents affect user 
> behavior.

Apparently I wasn't simple enough, so let's reduce this to the absurd 
reality that typically applies:

      If a user doesn't see it, how can it affect their behavior?

> authenticated matches what the user sees in the inbox (because, 
> rightly or wrongly, inboxes show the contents of the From: header field).

Except that most users don't see the From: domain name.

> When this match fails, a message can be rejected before it's ever in 
> front of a user and capable of causing confusion or fraud.

Exactly.  What matters is that unalignment is presumed to demonstrate 
bad faith by the originator.  THAT is what significant.  And it's 
significant to the filtering engine, not the recipient user.

>
> The point is NOT to change user behavior due to what is presented in 
> the From:, it is to prevent manipulation of user behavior by only 
> allowing From: domains to be displayed if they have been authenticated.

Yeah, but that's quite different from saying that a user who sees a bad 
from: field is manipulated.

>
> Your argument seems to be that you don't believe that spoofing the 
> From: domain leads to user impact, or am I completely misunderstanding 
> you?

Where is the clear and credible research data that says that a bad From: 
field domain name specifically tricks end users?

d/

Dave Crocker

---

From: Seth Blank <[email-masked]>
Date: Tue, 2 Jun 2020 17:45:47 -0700

Thanks for bearing with me, Dave.

On Tue, Jun 2, 2020 at 5:26 PM Dave Crocker <[email-masked]> wrote:

> When this match fails, a message can be rejected before it's ever in front
> of a user and capable of causing confusion or fraud.
>
> Exactly.  What matters is that unalignment is presumed to demonstrate bad
> faith by the originator.  THAT is what significant.  And it's significant
> to the filtering engine, not the recipient user.
>

Yes, we're agreed 100% here.

> field domain name specifically tricks end users?

There's a lot of clear and generally consistent data that shows From:
header field spoofing leads to outsized impact on end users. However, if by
"credible" you mean peer reviewed and not presented by someone with
something to sell in preventing the problem, that may be missing (although,
it only tends to be systems with a part to play in preventing abuse that
are even capable of seeing and distinguishing the issues) and could be an
interesting independent study to run.

Seth

---

From: Dave Crocker <[email-masked]>
Date: Tue, 2 Jun 2020 17:57:26 -0700

On 6/2/2020 5:45 PM, Seth Blank wrote:
> header field spoofing leads to outsized impact on end users.

Odd that I've never seen it.  Odd that it didn't surface during the 
literature search that was done when BIMI was started.

Again:  Please point to work that is specific to this issue and, just in 
case it is part of a larger tome, please point to the specific place in 
the document that is relevant to this issue.

> However, if by "credible" you mean peer reviewed and not presented by 
> someone with something to sell in preventing the problem, that may be 
> missing (although, it only tends to be systems with a part to play in 
> preventing abuse that are even capable of seeing and distinguishing 
> the issues) and could be an interesting independent study to run.

People with something to sell often do serious research.  And they often 
document it.  But this is quite different from marketing literature or 
hallway discussion.  I'm asking to see the research writeups.  (I made 
that plural since you are so firm in saying there is lots of supporting 
research.)

d/

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 3 Jun 2020 18:38:01 +0200

On Tue 02/Jun/2020 19:00:55 +0200 Dave Crocker wrote:
>> another university.  [...]
>>
>> "
>> part of a standard for Internet messages requires a revision of RFC 5322. I'd
>> love to see this resolved.
> 
> [...]
> 
> DMARC enforcement requires that the DKIM/SPF domain be the same as the author
> From:.  That is, the folk doing email operations have to be able to sign the
> DMARC aligned domain.
> 
> Hence the From: field is now really the Sender: field.  The From: field fixup
> hacks that are needed by intermediaries, to avoid DMARC-based rejections, makes
> this fact painfully clear, even as they serve to undermine recipient use of the
> From field for author-related message management.
> 
> [...]
> 
> The only suggestion I've been able to formulate is:  create a new field, such
> as Author:.

+1, that's the proper way to fix the issue Jesse relayed.

Re-senders who rewrite From: should copy its value to Author: unless such field
is already present.

MUAs should be discouraged from displaying or using Author:, unless
(verifiably) signed by a trusted domain or otherwise configured by the user.

> (Give it a simple, clean, appropriate name, rather than something like

Yes, and let's note that the issue is so fundamental that solving it also
solves the long-standing problem of how to standardiz mailing lists behavior
with DMARC.

Best
Ale
--

---

From: Dave Crocker <[email-masked]>
Date: Wed, 3 Jun 2020 09:43:16 -0700

> (verifiably) signed by a trusted domain or otherwise configured by the user.

Why?

d/

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 3 Jun 2020 19:20:36 +0200

>> (verifiably) signed by a trusted domain or otherwise configured by the user.

That avoids the dreaded back-to-square-one path that Brandon conjectured.  It
prevents attacks based on this field, while maintaining the DMARC paradigm.

I, for example, would configure to display Author: in the listing of
[dmarc-ietf] and similar folders.  Reply-to-Author would also be a useful
button, if not abused.  It's fine to fulfill advanced users' wishes as long as
average user behavior is not forced to change.

Best
Ale
--

---

From: Dave Crocker <[email-masked]>
Date: Wed, 3 Jun 2020 10:27:52 -0700

> prevents attacks based on this field, while maintaining the DMARC paradigm.

The barrier you specified sounds reasonable.  But it isn't.

Any signature put in place when the Author: field is created is likely 
broken by the time it gets to the recipient.  That's the entire problem 
that necessitates rewriting the From: field.

We do not require 'signatures' on Subject: or the Body or Date:. This is 
just one more field.

The concern about square one is misguided, apparently mostly because it 
continues the erroneous belief that the validation work is done for the 
end user, rather than the filtering engine. Besides that, it ignores the 
fact that authentication mechanisms are presumably still in place.

Users are tricked by the content of the message, not by the From: (or 
Author:) field.

On the other hand, a clean From: (or Author:) field enables consistent 
MUA organizing of messages.  Messing with the From: (or Author:) field 
by intermediaries defeats that.

d/

-- 

Dave Crocker

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 3 Jun 2020 20:30:38 +0200

On Wed 03/Jun/2020 19:27:52 +0200 Dave Crocker wrote:
> Any signature put in place when the Author: field is created is likely broken
> by the time it gets to the recipient.  That's the entire problem that
> necessitates rewriting the From: field.

That depends on who creates the Author: field.  I'd imagine it can be created
on rewriting From:.  If it exists already at that time, one can still check (by
ARC?) if it was signed, and, in case, sign it in turn.

> We do not require 'signatures' on Subject: or the Body or Date:. This is just
> one more field.

Right.  To sign Author: wouldn't be a DKIM or DMARC rule.  It's just a hint for
MUAs.  Rather than thoughtless treating Author: as From: by default, do some
serious checking and/or trust user settings.

> continues the erroneous belief that the validation work is done for the end
> user, rather than the filtering engine. Besides that, it ignores the fact that
> authentication mechanisms are presumably still in place.

Some authentication results are displayed to the end user.  They are important
in edge cases.

> Users are tricked by the content of the message, not by the From: (or Author:)
> field.

The content is only seen if the message is opened.  In the message listing I
only see the display name.  So, it is important that the display name comes
from a trusted field.  That is to say, from From:, at least in the default
configuration.

When the message is open, on edge cases the user should first look at the
authentication results, which shows the domain name on a decent MUA.

> On the other hand, a clean From: (or Author:) field enables consistent MUA
> organizing of messages.  Messing with the From: (or Author:) field by
> intermediaries defeats that.

Intermediaries already mess up when they rewrite the From:.  Adding an Author:
allows that mess to be partially undone.

Experience seems to indicate that mailing list software reacts more quickly
than MUAs.  MUAs will not add an Author: field any time soon, especially if it
has to be a copy of From:, with zero added information.

And then an Author: field is only needed by mailing lists and similar
minorities, when the From: is rewritten.  Recall DMARC was adopted because the
amount of such traffic is negligible.  In addition, I'd dare hypothesize that
users who subscribe to mailing lists are more skilled than average (?)  They
should be able to configure a MUA to deviate from the "safe" behavior in
certain cases.

Best
Ale
--

---

From: "Stan Kalisch" <[email-masked]>
Date: Thu, 04 Jun 2020 10:56:18 -0400

> ARC?) if it was signed, and, in case, sign it in turn.

I, too, was wondering whether ARC was really the only practical way to attempt this, assuming you don't think it deviates enough from ARC's purpose.

Thanks,
Stan

---

From: "John Levine" <[email-masked]>
Date: 4 Jun 2020 16:32:26 -0000

In article <[email-masked]>,
Stan Kalisch  <[email-masked]> wrote:
>I, too, was wondering whether ARC was really the only practical way to attempt this, assuming you
>don't think it deviates enough from ARC's purpose.

It took me a while to understand why ARC works the way it does but now
that I do, I don't think anything simpler would do.

In particular, anything that lets you say "I'm a mailing list" is out
since spammers can do that too. Also, real mailing lists tend to have
lousy spam filtering and leak a lot of spam, so you can't just
whitelist everything from even real lists.

-- 
Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

---

From: Jim Fenton <[email-masked]>
Date: Thu, 4 Jun 2020 21:40:01 -0700

On 6/2/20 10:35 AM, Dotzero wrote:
>
> As part of the original DMARC team, the goal was to make clear whether
> the email was authorized by the domain being used, hence the reliance
> on SPF and DKIM. These are clearly under the domain
> owner/administrator's control to the extent they choose to exercise
> that control. There was much discussion in the community at the time
> to use thei= field to enable more granular signing. it never gained
> traction. Because the intent was to protect end users from fake emails
> purporting to be from (primarily) commercial domains such as financial
> institutions, greeting card companies, etc., the Sender field was not
> a significant issue. Also, when the Sender is in the same domain as
> the From, there is no DMARC issue.

I'm all confused about why alignment matters. Back around the time DKIM
was standardized, we were concerned about phishing attacks from
look-alike domains, i.e., substitutions of 1 for l, that might be
registered by attackers and sign their messages with valid DKIM signatures.

But now a lot of people don't see anything but the Friendly Name; the
email address isn't displayed at all. For that (apparently increasing)
proportion of users, the From or Sender addresses aren't visible; the
attacker might as well use any Friendly Name of their choosing with any
domain they can sign for there. So they get DMARC alignment, but what
has it accomplished?

-Jim

---

From: Dotzero <[email-masked]>
Date: Fri, 5 Jun 2020 01:39:37 -0400

On Fri, Jun 5, 2020 at 12:40 AM Jim Fenton <[email-masked]> wrote:

> was standardized, we were concerned about phishing attacks from look-alike
> domains, i.e., substitutions of 1 for l, that might be registered by
> attackers and sign their messages with valid DKIM signatures.
>
> domain they can sign for there. So they get DMARC alignment, but what has
> it accomplished?
>
> -Jim
>

The goal of DMARC was (and is) to mitigate direct domain abuse. Nothing
more and nothing less. It helps receiving systems identify a (correctly)
participating domain's mail. That is why a DMARC policy is often described
as a sending domain's request and local policy is so important (and can
override that request).

For attackers that deploy DMARC it simply means that they are self
identifying their malicious messages as theirs.

Much has incorrectly been attributed to SPF/DKIM/DMARC. For example: "It
stops spam" - It does not. "It stops phishing" - It does not. The modest
goal is to stop direct domain abuse. It can do this remarkably well. On the
other hand it creates an incentive for attackers to compromise
participating domains. This has led to the long standing discussion (more
lately lapsed) between Dave Crocker and my self about reputation. My
position is that long term, reputation systems are of limited value because
of domain compromise or even sending policy change. To put it another way,
"What have you done to me today?". Dave has in the past had greater faith
in reputation.

For Sending domains, SPF/DKIM/DMARC is only one set of tools in protecting
their brand from abuse. It protects end users from abuse. In fact, in many
cases the individuals most susceptible to falling prey to such abuse may
not even be customers of that sending domain. No, that greeting card you
received isn't legit (Nobody loves you). No, that retailer isn't giving you
a $200 gift card. This is why other tools like takedowns are so important
and why the removal of registrant information from domain registrations has
enabled abusers.

Just a few thoughts.

Michael Hammeer

---

From: Hector Santos <[email-masked]>
Date: Fri, 05 Jun 2020 09:31:50 -0400

On 6/5/2020 1:39 AM, Dotzero wrote:
>
> The goal of DMARC was (and is) to mitigate direct domain abuse.

+1, it was the goal of:

[1] The original proof of concept with DomainKeys' built-in o= policy 
tag for 1st party support, and

[2] The original DKIM draft augmented with the original SSP draft with 
extended o= tags that included 3rd party signer considerations, and

[3] My own DSAP with its full coverage of 1st party and 3rd party 
signer policies, and

[4] SSP/ASP when policy was separated from DKIM to create DKIM-BASE. 
SSP/ASP  was replaced with,

[5] ADSP which was limited to 1st party and ambiguous 3rd party signer 
support which was replaced with,

[6] DMARC with its limited 1st party only support, punting on 
addressing the 3rd party signer issue.

Short of reporting and SPF integration, DMARC did not bring any more 
to the DKIM Policy design picture.  All of the DKIM Policy proposals 
offered the same power for highly detectable, enforcible direct mail, 
exclusive and restrictive 1st policies. From day one, a few of the key 
cogs here stated for the most part, "No will ever use an exclusive 
DKIM policy, and if so, it will be an extremely narrow case with small 
guys, and we don't about them"  was the prevailing attitude about DKIM 
Policy protocols.

Thanks to a non-IETF Trade Group who filled in the abandoned IETF DKIM 
Policy ADSP model with "Super ADSP" aka DMARC, that all changed with 
domains, of all sizes, switching to restrictive, rejectable DMARC 
policy and the verifiers where beginning to honor it.  It brought the 
DKIM Policy theory into practice along with the unhandled indirect 3rd 
party resigner list problem.

> Nothing more and nothing less.

+1 for the most part.

-- 
HLS

---

From: Jim Fenton <[email-masked]>
Date: Fri, 5 Jun 2020 14:26:19 -0700

On 6/4/20 10:39 PM, Dotzero wrote:
> Nothing more and nothing less. It helps receiving systems identify a
> (correctly) participating domain's mail. That is why a DMARC policy is
> often described as a sending domain's request and local policy is so
> important (and can override that request).
I'm not clear on what kind of direct domain abuse you're referring to.
If we accept that domain names are either not visible or are ignored by
the recipient, the domain name doesn't matter much as long as the
attacker can get their message delivered, and DMARC doesn't apply
because they're using their domain.
> identifying their malicious messages as theirs.
No, DKIM and SPF do that. DMARC doesn't have anything to do with
identifying messages.
>
> For Sending domains, SPF/DKIM/DMARC is only one set of tools in
> protecting their brand from abuse. It protects end users from abuse.
> In fact, in many cases the individuals most susceptible to falling
> prey to such abuse may not even be customers of that sending domain.
> No, that greeting card you received isn't legit (Nobody loves you).
> No, that retailer isn't giving you a $200 gift card. This is why other
> tools like takedowns are so important and why the removal of
> registrant information from domain registrations has enabled abusers.

So maybe the core question here is, does the identity in the domain name
matter or not? It does to me personally because I look at it (whenever I
can -- my iPhone doesn't make it easy to display) and I pay attention to
it. But I know I'm not a typical user, and I also see increasing
evidence of mail client software that doesn't show anything but the
Friendly Name. So is there a "brand" associated with the email domain
name any more?

If the domain name doesn't matter, the binding to the From/Signer
address doesn't either.

-Jim

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 05 Jun 2020 18:37:52 -0400

> If the domain name doesn't matter, the binding to the From/Signer

If the domain name didn't matter, no one would bother to use 'real' domains in 
abusive mail.  They demonstrably do, so while one might have differences of 
opinion about how important they are (every MUA I use displays them, so let's 
also not draw too hasty conclusions about them not being displayed) I don't 
think it's a supportable that they don't matter.

Scott K

---

From: Jim Fenton <[email-masked]>
Date: Sat, 6 Jun 2020 12:25:56 -0700

> think it's a supportable that they don't matter.

And I receive a good deal of email with friendly names like "DHL
Express" or the names of friends (who apparently have suffered address
book compromise) but completely unrelated domain names.

I phrased my comment as a question because I really don't know the
answer to this, and have been reading comments from people asserting
opinions on both sides. It would simplify the discussion if the WG could
reach rough consensus on this. And if the domain name doesn't matter,
the WG really needs to rethink the utility of DMARC.

-Jim

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 06 Jun 2020 20:32:57 +0000

>the WG really needs to rethink the utility of DMARC.

I think the market has spoken on the utility of DMARC.

Scott K

---

From: "John Levine" <[email-masked]>
Date: 6 Jun 2020 20:45:11 -0000

In article <[email-masked]>,
>I think the market has spoken on the utility of DMARC.

There's no question that it was highly successful at Yahoo and AOL
after they let crooks steal their address books at reducing the amount
of spam their users received that forged addresses in those stolen
address books.  Of course, if you are not Verizon Media, who cares?

I gather it is also quite effective against phishes that for some
reason put the actual target's domain in the From: address, but
at this point I don't know how common that is relative to phishes
that put it in the From: comment, viz. Jim's question.

R's,
John

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 06 Jun 2020 17:23:20 -0400

> that put it in the From: comment, viz. Jim's question.

I'm not sure how important a question it is.

It used to be quite common.  If it's not anymore (I don't have access to a 
current data set big enough to really have an opinion), then I'd suggest that 
it's because abusers are, at least to some degree, deterred from doing so.

If things like DMARC, SPF, and DKIM do nothing more than get abusers to use 
different domains than they would otherwise, I think that's a win.  
Unfortunately it's quite difficult to measure the deterrent effect associated 
with these mechanisms.

I would expect that using different domains would make the filtering problem 
easier to solve, so even if the domain presented to end user doesn't matter (I 
think it does, but meh), pushing abusive mail to use other domains helps solve 
the filtering problem.

Scott K

---

From: Dave Crocker <[email-masked]>
Date: Sat, 6 Jun 2020 14:26:08 -0700

> different domains than they would otherwise, I think that's a win.

The issue here is DMARC, not SPF or DKIM, since DMARC is the only one of 
the 3 that restricts the choice of domain name.

With that in mind, I'll ask you why you think the kind of change you 
cite is a win.

d/

---

From: Dotzero <[email-masked]>
Date: Sat, 6 Jun 2020 17:28:21 -0400

On Fri, Jun 5, 2020 at 5:26 PM Jim Fenton <[email-masked]> wrote:

> I'm not clear on what kind of direct domain abuse you're referring to. If
> we accept that domain names are either not visible or are ignored by the
> recipient, the domain name doesn't matter much as long as the attacker can
> get their message delivered, and DMARC doesn't apply because they're using
> their domain.
>
>
> The type of direct domain abuse where someone attempts to send a message
using <[email-masked]> in the From email address field. As I wrote
earlier, the combination of SPF/DKIM/DMARC is a tool that accomplishes a
narrow goal. It is not a silver bullet that solves all forms of abuse. It
can be used to mitigate a specific type of abuse.

> As with SPF and DKIM, some abusers were quick to implement DMARC in
addition to SPF and/or DKIM on the theory that it makes their email appear
more legitimate to receivers. Just one more nail in the coffin.

> it. But I know I'm not a typical user, and I also see increasing evidence
> of mail client software that doesn't show anything but the Friendly Name.
> So is there a "brand" associated with the email domain name any more?
>
There is. Don't get hun up on what is displayed to the end user. Think
about the reporting aspect. In my previous incarnation we were able to
initiate takedowns and/or blocking by 3rd parties much more quickly based
on DMARC reports than simply waiting for end user complaints to customer
service or abuse@.

> If the domain name doesn't matter, the binding to the From/Signer address
> doesn't either.
>
> -Jim
>
It does matter for the specific abuse scenario. Those particular abusive
mail streams never get to the end user recipient. I'm basing this on my
experience on a corpus of billions of emails sent for what had been
previously highly abused domains/brands. For other types of abuse we
implemented other types of mitigation approaches. Collectively those
approaches reduced abuse by over 95%. The goal was to reduce ROI for the
bad guys to the point that they would look for greener pastures. You are
implying/assuming that DMARC solves the problem of a wider scope of abusive
email types than it does. The Display Name (Mail From) is a particularly
thorny problem to solve in that it is not tied to anything in that it is a
free form field into which anything can be entered.

Michael Hammer

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 06 Jun 2020 17:42:37 -0400

> With that in mind, I'll ask you why you think the kind of change you

1.  I think the domain displayed to the end user matters.  In my sample size 
of 1, it matters to me.  I know I'm not the average user, but independent of 
the question of how many users it matters to, there are some.

2.  When abusers use different domains to send mail, it adds more information 
for filters to work on, so even if this is all about filtering, that works 
better too.

Scott K

---

From: Jim Fenton <[email-masked]>
Date: Sat, 6 Jun 2020 15:03:28 -0700

> the question of how many users it matters to, there are some.
Same with me, but again I'm not the average user.
> for filters to work on, so even if this is all about filtering, that works 

But when abusers use different domains, the DMARC policy that applies is
controlled by them and is therefore meaningless. And the reports, if any
(probably none), are sent back to the attacker or their designate.

Filtering might be done based on the DKIM signing domain or the
envelope-from domain if SPF is used, but neither of those require DMARC.

-Jim

---

From: Alessandro Vesely <[email-masked]>
Date: Sun, 7 Jun 2020 12:18:44 +0200

On Sun 07/Jun/2020 00:03:28 +0200 Jim Fenton wrote:
>>>> If things like DMARC, SPF, and DKIM do nothing more than get abusers
>>>> to use different domains than they would otherwise, I think that's a
>>>> win.>>> The issue here is DMARC, not SPF or DKIM, since DMARC is the only one of

+1, but then we're mailing list subscribers (leaving aside this list's topic.)

> Filtering might be done based on the DKIM signing domain or thesimilar

The From: domain was chosen because that's the field that users can see.  Now
we conjecture that users don't actually see it.  Oh boy.  Certainly, if the
From: domain is not visible we could filter on X-Filter-On-Me: and gracefully
avoid the mailing list problem.

On closer view, we seem to be discovering that the From: domain is obscured by
the display name.  We always neglected the display name.  Furthermore, by
letting the mailing list problem be dodged by creative From: rewriting, such as

    From: [email-masked] <[email-masked]>

we are granting full citizenship to devious display names.  Some clients (e.g.
Thunderbird) can show only display name for people in the address book.[*]  A
close, perhaps formally easier, subject is the IDN homograph attack.[†]

Would it make sense to ban, say, the use of the at sign (@) in display names?

Best
Ale
-- 

[*]
https://support.mozilla.org/en-US/kb/names-bug-no-email-addresses-are-displayed

[†] https://en.wikipedia.org/wiki/IDN_homograph_attack

---

From: "Douglas E. Foster" <[email-masked]>
Date: Sun, 07 Jun 2020 13:16:21 GMT

1) The original assertion, that DMARC creates a conflict with prior specifications, appears to be undefended and incorrect.   For From Addressing, It merely establishes some boundaries that the sender and the recipient choose to consider appropriate.    For DKIM, it creates a use-case for a technology that was initially defined without any use cases.    Consequently, I think this topic is ready for closure.

2) Some of the discussion appeared to resolve around the assertion that DMARC can have no value.   Since that view is not universal, I think the project can continue with those who do believe that it adds value.

3) Some of the discussion has been about how to prevent soclal engineering of the recipient user.  This is an important topic, but not directly related to the project.  IETF would do well to establish some recommendations about how MUAs should behave, so that trust data can be displayed to the user.   A typical user will have access to at least three different email clients: one on his cell phone, a product-specific web page, and a desktop product like Outlook or Thunderbird.    If I wanted an organization policy that controlled when Friendly Name was displayed or DMARC status was displayed, I would have to find and distribute plug-ins to all of these products.   As best as I have been able to tell, no such plug-ins even exist for Outlook and the other products do not accept extensions.   There is an opportunity here for valuable standardization.

----------------------------------------
From: Alessandro Vesely <[email-masked]>
Sent: 6/7/20 6:19 AM
To: [email-masked]
Subject: Re: [dmarc-ietf] DMARC alignment conflicts with RFC 5322 on the use of the From and Sender header fields

Best
Ale
--

[*]

_______________________________________________
dmarc mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/dmarc

---

From: "John Levine" <[email-masked]>
Date: 7 Jun 2020 13:03:41 -0400

In article <[email-masked]> you write:
>-=-=-=-=-=-
>1) The original assertion, that DMARC creates a conflict with prior specifications, appears to be undefended and incorrect.

It should not be controversial that DMARC can only describe a subset
of valid Internet mail. The problem arises when people then assert
that somehow it is our fault that we are sending mail that DMARC can't
describe, typically in ways we've been using for decades, long before
anyone thought of DMARC.

>2) Some of the discussion appeared to resolve around the assertion that DMARC can have no value.

It clearly has value to Verizon, and it apparently has value to banks
and Paypal. I can't see that it has much value for me or my users,
since it has screwed up all the mailing lists we use, and for whatever
reason we're not big phish targets.

R's,
John

PS: My bank chronically sends out real mail that looks like a total
phish, e.g., it says there's a dubious charge on your card, click this
button if it's real or that button if it's not, with the URL for
neither button having any connection to any domain the bank owns.
I know it's real because I know enough about the bank business to
recognize the subcontractor they use, but jeez.

---

From: "Stan Kalisch" <[email-masked]>
Date: Sun, 07 Jun 2020 13:53:54 -0400

On Sun, Jun 7, 2020, at 9:16 AM, Douglas E. Foster wrote:
> 3) Some of the discussion has been about how to prevent soclal engineering of the recipient user. This is an important topic, but not directly related to the project. IETF would do well to establish some recommendations about how MUAs should behave, so that trust data can be displayed to the user.

Assuming this can be practically done, I would rephrase this, "...[E]stablish how MUAs should display trust data to users."

Which is a question many have struggled to answer, given the history of users understandably wanting to pull their hair out over trust data and warnings.

Stan

---

From: "John Levine" <[email-masked]>
Date: 7 Jun 2020 15:52:28 -0400

In article <[email-masked]> you write:
>> 3) Some of the discussion has been about how to prevent soclal engineering of the recipient user. This is an important
>topic, but not directly related to the project. IETF would do well to establish some recommendations about how MUAs should
>behave, so that trust data can be displayed to the user.
>

We have decades of experience that tells us that the IETF is hopeless
at UI design, and our intuition is usually wrong.

In particular, displaying warnings that "this may be bad" or even
"this is extremely bad" is known not to work. No matter what you say,
people will click through any warning to get to their kitten GIFs or
porn or whatever.

---

From: "Stan Kalisch" <[email-masked]>
Date: Sun, 07 Jun 2020 16:23:24 -0400

> people will click through any warning to get to their kitten GIFs or

I didn't know the history of the IETF's approach to UI, in particular, but I'm aware of the research on the nastiness of solving the UI problem.  I mostly wanted to clarify that the problem is, indeed, *how* to show that data to users, and that no one has actually ever been able to solve that problem.

Thanks,
Stan

---

From: Jim Fenton <[email-masked]>
Date: Sun, 7 Jun 2020 14:20:19 -0700

On 6/7/20 6:16 AM, Douglas E. Foster wrote:
> 2) Some of the discussion appeared to resolve around the assertion
> that DMARC can have no value.   Since that view is not universal, I
> think the project can continue with those who do believe that it adds
> value.
>
That's not how it works; this working group is discussing an IETF
standards-track specification. This is based on the rough consensus of
the entire working group; you can't just go forward with a subset of
people who agree.

-Jim (with apologies to the WG chairs if I'm venturing into their territory)

---

From: "John Levine" <[email-masked]>
Date: 7 Jun 2020 17:40:45 -0400

In article <[email-masked]> you write:
>I didn't know the history of the IETF's approach to UI, in particular, but I'm aware of the research on the nastiness of
>solving the UI problem.  I mostly wanted to clarify that the problem is, indeed, *how* to show that data to users, and that no
>one has actually ever been able to solve that problem.

I believe the real question is *whether* to show trust data to users
and the consensus seems to be don't bother, it only confuses them.

R's,
John

---

From: Dave Crocker <[email-masked]>
Date: Sun, 7 Jun 2020 14:53:08 -0700

On 6/7/2020 10:53 AM, Stan Kalisch wrote:
> Assuming this can be practically done, I would rephrase this, 
> "...[E]stablish how MUAs should display trust data to users."
>
Since there has been a demonstrated lack of efficacy in this sort of 
display, there needs to be an objective basis for knowing that any new 
such requirement will be useful.  Good luck with that.

On 6/6/2020 2:42 PM, Scott Kitterman wrote:
> t

I might be wrong, but I believe the IETF does not intend to make global 
standards on the basis of possible utility for only one engineer working 
on the specification.  Or two.  Or three.

cf, above.

d/

---

From: "Stan Kalisch" <[email-masked]>
Date: Sun, 07 Jun 2020 18:07:20 -0400

> and the consensus seems to be don't bother, it only confuses them.

I mean, I can't dispute that that's a fair question.  Confusion obviously works against the goals behind showing the user the data in the first place.

Stan

---

From: "Douglas E. Foster" <[email-masked]>
Date: Sun, 07 Jun 2020 23:04:24 GMT

I am trying to play by the rules and not chase topics outside the one assigned, but since several have jumped on my comment, I will follow up briefly.

Dave Crocker wrote
Since there has been a demonstrated lack of efficacy in this sort of display, there needs to be an objective basis for knowing that any new such requirement will be useful.

Every email filtering product that I have examined has provided a user-signaling system, using one or more of the following:
tagging the subject, adding text as a body header or body footerconverting the suspect message into an attachment of  a replacement message,soft-quarantining, where the user has unrestricted control to release the message from quarantine.
Given that market reality, I conclude that most vendors and their customers believe that user-signalling is useful.   The signalling system does not have to prevent every mistake for the signal to be useful.

The problem with all current notification methods is that they are relatively primitive, often communicating nothing substantive about the suspicious message characteristics.  They also work against other security mechanisms.   
Any form of tagging breaks DKIM signatures, reducing the credibility of the message if it is auto-forwarded for any reason.   

The tagging also becomes tattooed to the message and its replies, rather than being restricted to the trust domain that assigned the tag.   One example should suffice:  a message was tagged with [SPAM?] because the sender had an error in his SPF record and I was trying to enforce SPF.   Then when my staff replied, the originator treated the reply as spam because the word SPAM was still in the subject line when he received it!   
We need a user notification mechanism that is local to the trust domain and does not break DKIM signatures.  You have already done the heavy lifting for this interoperability problem with Authentication-Results and ARC   I am not expecting a "Standard" that requires every implementation to notify every user in the same way.   I am looking for a guidance document that helps vendors to deliver products which permit an organization to implement a notification policy which they find to be effective and appropriate.   IETF is the right organization to take this on because the email filter, the mail system, and the multiple MUAs are almost always a multi-vendor configuration.

df

---

From: Scott Kitterman <[email-masked]>
Date: Sun, 07 Jun 2020 19:08:30 -0400

On Sunday, June 7, 2020 5:53:08 PM EDT Dave Crocker wrote:
> > 1.  I think the domain displayed to the end user matters.  In my sample
> > size of 1, it matters to me.  I know I'm not the average user, but
> > independent of t
> 

Conveniently, I'm not proposing any new requirements.

Scott K

---

From: Dave Crocker <[email-masked]>
Date: Sun, 7 Jun 2020 16:14:25 -0700

On 6/7/2020 4:04 PM, Douglas E. Foster wrote:
> Given that market reality, I conclude that most vendors and their 
> customers believe that user-signalling is useful.   The signalling 
> system does not have to prevent every mistake for the signal to be useful.

What you are describing has nothing at all to do with system-provided 
trust assertions.

d/

---

From: Dave Crocker <[email-masked]>
Date: Sun, 7 Jun 2020 16:17:15 -0700

On 6/7/2020 2:40 PM, John Levine wrote:
> I believe the real question is*whether*  to show trust data to users

It's not that it 'seems to be'. It isn't nearly that soft.

It is that there have been multiple efforts over the years and none has 
demonstrated efficacy.

Anyone claiming the contrary needs to provide objective data to 
substantiate a claim of efficacy.

Adding a capability or relying one carries an affirmative obligation to 
provide a basis for knowing that the cost is justified. So far, there 
isn't one, for end-user trust notifications.

d/

---

From: "Stan Kalisch" <[email-masked]>
Date: Sun, 07 Jun 2020 19:45:06 -0400

On Sun, Jun 7, 2020, at 7:04 PM, Douglas E. Foster wrote:
> The problem with all current notification methods is that they are relatively primitive, often communicating nothing substantive about the suspicious message characteristics.

And you propose the average user can understand, much less take the time to understand, the substance?

Stan

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 7 Jun 2020 20:04:14 -0700

On Sun, Jun 7, 2020 at 6:58 AM Douglas E. Foster <
[email-masked]> wrote:

> 1) The original assertion, that DMARC creates a conflict with prior
> specifications, appears to be undefended and incorrect.   For From
> Addressing, It merely establishes some boundaries that the sender and the
> recipient choose to consider appropriate.    For DKIM, it creates a
> use-case for a technology that was initially defined without any use cases.
>    Consequently, I think this topic is ready for closure.
>

 I claim your assertions here are contradictory.  Specifically, by
establishing "some boundaries", it is creating a conflict with prior
specifications which established none, possibly intentionally.

3) Some of the discussion has been about how to prevent soclal engineering
> of the recipient user.  This is an important topic, but not directly
> related to the project.  IETF would do well to establish some
> recommendations about how MUAs should behave, so that trust data can be
> displayed to the user.   A typical user will have access to at least three
> different email clients: one on his cell phone, a product-specific web
> page, and a desktop product like Outlook or Thunderbird.    If I wanted an
> organization policy that controlled when Friendly Name was displayed or
> DMARC status was displayed, I would have to find and distribute plug-ins to
> all of these products.   As best as I have been able to tell, no such
> plug-ins even exist for Outlook and the other products do not accept
> extensions.   There is an opportunity here for valuable standardization.
>

The IETF has routinely punted, at least in the email space, on the idea of
prescribing or proscribing user interface behaviors because we are protocol
engineers, not human factors experts.  Are you claiming that's changed?

-MSK

---

From: Dave Crocker <[email-masked]>
Date: Sun, 7 Jun 2020 20:13:01 -0700

On 6/7/2020 8:04 PM, Murray S. Kucherawy wrote:
>
>      If I wanted an organization policy that controlled when Friendly
>     Name was displayed or DMARC status was displayed, I would have to
>     find and distribute plug-ins to all of these products.   As best
>     as I have been able to tell, no such plug-ins even exist for
>     Outlook and the other products do not accept extensions. There is
>     an opportunity here for valuable standardization.
>
>
> The IETF has routinely punted, at least in the email space, on the 
> idea of prescribing or proscribing user interface behaviors because we 
> are protocol engineers, not human factors experts.  Are you claiming 
> that's changed?

A 'friendly' amendment, or rather addition:

      The origination side of an email exchange has no authority over 
the reception side.  Different administrative authorities. Compliance 
with DMARC is voluntary -- and receiving administrations often implement 
behaviors that differ from what is requested via DMARC.

      There is no basis for believing that requests about MUA display 
will achieve meaningful support on the receive side, nevermind whether 
they would be at all useful.

d/

---

From: "Douglas E. Foster" <[email-masked]>
Date: Mon, 08 Jun 2020 06:03:21 GMT

Stan Kalisch asks:  And you propose the average user can understand, much less take the time to understand, the substance?

Yes.   I believe users are worried about spam, and want to make intelligent decisions about whether or not email can be trusted.  Unfortunately, our present software denies them access to the available information needed to make intelligent decisions.

Dave Crocker observes:       There is no basis for believing that requests about MUA display will achieve meaningful support on the receive side, nevermind whether they would be at all useful. 

I was not talking about the sender.   I was talking entirely about the receiving organization:    its spam filter communicating to its MUA to communicate information to the end user based on its local policy.

Dave Crocker also observes about end-user signaling failures:       It's not that it 'seems to be'. It isn't nearly that soft.  It is that there have been multiple efforts over the years and none has demonstrated efficacy.

    Then lets restate that assertion in all its ugly elitism, and put it into an RFC:
   
Incontrovertible research shows that humans will always act on malicious email, and cannot be taught to do otherwise.   Organizations should deploy email if and only if they have automated tools which provide perfect protection from unwanted email.     End user training is useless.

I have a higher opinion about my users than that.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 8 Jun 2020 01:11:00 -0700

On Sun, Jun 7, 2020 at 11:04 PM Douglas E. Foster <
[email-masked]> wrote:

> Stan Kalisch asks:  And you propose the average user can understand, much
> less take the time to understand, the substance?
>
> Yes.   I believe users are worried about spam, and want to make
> intelligent decisions about whether or not email can be trusted.
> Unfortunately, our present software denies them access to the available
> information needed to make intelligent decisions.
>

A study presented some years ago now, I think back when DKIM was young, (if
I can find a citation, I will send it along) found that a statistically
significant -- it was more like 18% -- portion of their test subjects would
willingly click on links found in their spam folders if the email found
there looked legitimate.

That's right, they weren't just clicking links in their inboxes, they were
clicking links in a partition of their inbox expressly created, and named,
to store stuff the receiving system thought was probably dangerous.  The
theory, as I recall, was that they were worried they were missing something
important.

Who were these users?  As I recall, the study was run by a collaboration of
banks attempting to ascertain the gullibility of their typical customers.

That seems to be data antithetical to the notion that users are universally
worried about spam and want to make intelligent decisions.  Moreover, these
particular users were presented with information clearly marking these
messages as possibly dangerous, insofar as they had to click through to
their spam folders first.

They did it anyway.

Dave Crocker also observes about end-user signaling failures:       It's
> not that it 'seems to be'. It isn't nearly that soft.  It is that there
> have been multiple efforts over the years and none has demonstrated
> efficacy.
>
> Incontrovertible research shows that humans will always act on malicious
> email, and cannot be taught to do otherwise.   Organizations should deploy
> email if and only if they have automated tools which provide perfect
> protection from unwanted email.     End user training is useless.
>

I wonder on what basis.  Given the contortions through which we went to
produce even the vague text in Appendix D of RFC 6376, we didn't know then
what would work.  I don't think today is any different, or we'd be doing it
already.

-MSK

---

From: Дилян Палаузов <[email-masked]>
Date: Mon, 08 Jun 2020 11:11:46 +0000

Hello,

when a message is wrongly evalutated as spam, and is left therefore
unnoticed, it is nobody’s fault.  You can signal the users as you want,
including the users, which just redirect mails on your host, and do not
utilize the “Spam” store there.

A message is either likely spam (subject to signaling), or it is not
likely spam.

When the message is likely spam, you can signal the sender by sending a
non delivery report.  That report can contain information, entered by
the intended recipient, like snail mail address or phone number. 

If you signal the sender (by rejecting at SMTP level) you do not need a
signaling method on the recipients’ side, that works across all MUAs. 
Besides, there is no “nobody is fault for the overseen signaled
message”.

If you run a server with 10 local users and 10 000 redirecting
addresses (aliases), you have to use a signaling method, that does not
break the DKIM-signature, and works for redirected emails.  For users
that utilize just redirecting there is no “neither likely spam, nor
unlikely spam” category.  And the other users also do not need such
category.

Greetings
  Дилян

On Sun, 2020-06-07 at 23:04 +0000, Douglas E. Foster wrote:
> I am trying to play by the rules and not chase topics outside the one
> assigned, but since several have jumped on my comment, I will follow
> up briefly.
> 
> Dave Crocker wrote
> display, there needs to be an objective basis for knowing that any
> new such requirement will be useful.
> 
> Every email filtering product that I have examined has provided a
> user-signaling system, using one or more of the following:
>  * tagging the subject, 
>  * adding text as a body header or body footer
>  * converting the suspect message into an attachment of  a
> replacement message,
>  * soft-quarantining, where the user has unrestricted control to
> release the message from quarantine.
> The problem with all current notification methods is that they are
> relatively primitive, often communicating nothing substantive about
> the suspicious message characteristics.  They also work against other
> security mechanisms.   
>  * Any form of tagging breaks DKIM signatures, reducing the
> credibility of the message if it is auto-forwarded for any reason.   
>    
>  * The tagging also becomes tattooed to the message and its replies,
> rather than being restricted to the trust domain that assigned the
> tag.   One example should suffice:  a message was tagged with [SPAM?]
> because the sender had an error in his SPF record and I was trying to
> enforce SPF.   Then when my staff replied, the originator treated the
> reply as spam because the word SPAM was still in the subject line
> when he received it!   
> We need a user notification mechanism that is local to the trust
> domain and does not break DKIM signatures.  You have already done the
> heavy lifting for this interoperability problem with Authentication-
> Results and ARC   I am not expecting a "Standard" that requires every
> implementation to notify every user in the same way.   I am looking
> for a guidance document that helps vendors to deliver products which
> permit an organization to implement a notification policy which they
> find to be effective and appropriate.   IETF is the right
> organization to take this on because the email filter, the mail
> system, and the multiple MUAs are almost always a multi-vendor
> configuration.
> 
> df
> 
> 
> 
>

---

From: "John Levine" <[email-masked]>
Date: 8 Jun 2020 11:02:51 -0400

In article <[email-masked]> you write:
>Yes.   I believe users are worried about spam, and want to make intelligent decisions about whether or not email
>can be trusted.  Unfortunately, our present software denies them access to the available information needed to make
>intelligent decisions.

When I said our intuitions about UI in the IETF are wrong, this is exactly what I had in mind.

As others have said in more detail, there is no evidence that users
make "intelligent" decisions about what is spam, and considerable
evidence that they don't.

We're not saying that users are stupid, beause they aren't. But we are
saying that their intuitions about how to treat their incoming mail
are a poor match for the reality of what's in the mail.

---

From: "Stan Kalisch" <[email-masked]>
Date: Mon, 08 Jun 2020 17:17:08 -0400

> Yes. I believe users are worried about spam, and want to make intelligent decisions about whether or not email can be trusted. Unfortunately, our present software denies them access to the availabl…

See, I believe they want to, too, but, anecdotally, I can think of a number of intelligent people I can't explain DMARC to in a substantive manner in a short period of time. And the research bears these kinds of anecdotes out.

What I've tried to establish here is that you really have to take the initiative if you want to come up with a system that can present the kind of data you want presented to the users. You're missing the point that a number of people with a great deal of experience have tried, and think it's either impossible or very unlikely. So simply asking the community to come up with a solution won't be enough, because the community has labored to find a solution for a very long time.

A good place for you to begin would probably be this paper:
http://www.usablesecurity.org/emperor/

Stan