Subject: [dmarc-ietf] Composition Kills: A Case Study of Email Sender Authentication

From: Scott Kitterman <[email-masked]>
Date: Tue, 21 Apr 2020 12:11:00 -0400

There is probably protocol improvement work that should be done based on:

https://www.usenix.org/system/files/sec20fall_chen-jianjun_prepub_0.pdf

Scott K

---

From: "Kurt Andersen (b)" <[email-masked]>
Date: Tue, 21 Apr 2020 11:11:47 -0700

Extracting from the abstract of a paper in process by Casey Deccio (now
researching SPF at BYU, formerly at Cisco):

Our techniques elicit SPF and DMARC validation activity of the servers,
> while minimizing spam and perceived abuse. We find that only 25% of mail
> servers and less than half of domains deploy SPF validation. Of domains
> that perform SPF validation, 7.6% exhibit inconsistent validation behavior
> across mail servers. We also find that 75% of organizations with mail
> servers for popular domains share DNS infrastructure with up to tens of
> thousands of others.

Most of his focus has been on SPF but it would be useful to loop him into
the DMARCbis discussions.

--Kurt

On Tue, Apr 21, 2020 at 9:11 AM Scott Kitterman <[email-masked]> wrote:

> _______________________________________________
> dmarc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dmarc
>

---

From: "John Levine" <[email-masked]>
Date: 21 Apr 2020 14:22:41 -0400

>https://www.usenix.org/system/files/sec20fall_chen-jianjun_prepub_0.pdf

I didn't see any protocol issues other than the well known DKIM
multiple From: headers (the Doug Otis feature) and l=.  They certainly
did find a lot of implementation bugs, some of which I found pretty
surprising, like Gmail allowing and misinterpreting NUL characters in
DKIM signature headers.

This sounds like we need more test suites and perhaps more reminders
that when you're writing security software, being forgiving of other
people's bugs will backfire on you.

R's,
John

---

From: Dave Crocker <[email-masked]>
Date: Wed, 22 Apr 2020 13:28:44 -0700

The paper concerns valid authentication and effective use of 
authenticated information.  That's an excellent goal.  The paper offers 
extensive technical analyses and empirical demonstrations of operational 
failures. Exercises like these are valuable and too infrequent.  Besides 
prompting specific repairs, work like this should motivate careful 
industry review of systems-level issues and standards-based documentation.

The paper does not adequately distinguish between claims of protocol 
design/specification errors, versus implementation errors, yet that 
distinction is essential.

The paper has a simplistic model of email anti-abuse processing and this 
distorts some of its analysis.  At the least, the paper needs to 
distinguish between theory and practice.

The paper has a flawed model of the role of recipient users in 
anti-abuse work and therefore the paper makes a basic error in its 
assessment of the importance of From: field validation.

The paper identifies a number of significant implementation deficiencies 
in various system.

Detailed comments:

> to authenticate the sender’s purported identity, as the basis for 
> displaying in email clients assurances of validity to userswhen they 
> read messages

That's a rather basic misunderstanding of how these mechanisms are 
actually used.

> In practice, attackers usually exploit SMTPby running their own email 
> servers or clients.

This is largely a non-sequitur.  It has built-in assumptions that aren't 
explained and are wrong with respect to any of the domain name-based 
protection mechanisms.  (And it isn't SMTP that is being exploited, per 
se. That's like saying that if I send a postal letter purporting to be 
the president of the United States, I've exploited the postal system. )

> SMTP’s design includes multiple “identities” when handling messages. 
Since this has been documented, they should cite it, for background.

> Both theMAIL FROMandFromheadersidentify the email sender, but they 
> have different meaningsin an SMTP conversation. The first represents 
> the user whotransmittedthe message, and is usually not displayed to 
> therecipient. 
It would be clearer and more helpful to say 'operator' or 'service 
provider' rather than 'user'.  A separate issue is whether their 
characterization of the differences between the two identifiers is valid 
in practice... The paper uses sender in a way that encourages confusion 
in the reader.  Especially since this is a technical paper, it should 
use language that carefully distinguishes roles, especially since 
terminology for making these distinctions is readily available.

> In addition, SMTP introduces multiple other sender identi-ties, such 
> as theHELOcommand,SenderandResent-Fromheaders.

I suspect that the failure to cite DKIM's d= field , until later, is a 
significant oversight.

> http://www.ietf.org/internet-drafts/draft-blank-ietf-bimi-00.txt

heh.  the URL isn't valid.  Should be:

https://www.ietf.org/archive/id/draft-blank-ietf-bimi-00.txt<https://datatracker.ietf.org/doc/draft-blank-ietf-bimi/>

> DKIM.DomainKeys Identified Mail (DKIM) uses cryp-tography to 
> authenticate senders...

I'm being too picky, right?  Formal DKIM semantics don't produce exactly 
that result.

> The general idea behind DKIM is to let senders sign parts ofmessages 
> so that receivers can validate them.
This is poorly written, wrong, or both.  'them' is ambiguous.  And the 
signing is to affix the d= value, not to validate any of the data that 
is part of the signature.

> it only need to have the same registered domain

Will typical readers understand what this means, in the context of this 
paper, since all of the domain name's components are 'registered'? It's 
not obvious what language would make the meaning clearer.

> If the email passes theDMARC verification, it enters the user’s inbox.

This is simply wrong.  It needs to place this phase of processing inside 
a complex filtering engine, which is the primary venue for using any of 
the mechanisms discussed in the paper.

> usually the MUA only displays theFromheader as the message sender.

These days, this isn't true either.  Most users only see the 
un-validated Display-Name.

> Thus, theFromheaderprovides the key identity relevant for gaining the 
> user’s trust

On its face, this seems a reasonable view.  In practice, it isn't.  Mail 
with obviously bogus email addresses in the rfc5322.From field are still 
effective for phishing, because what real users actually pay attention 
to is the body of the message and the identification information in it.  
That's why assertions about the display of validated source/author 
information to the end user are demonstrably wrong.

> Malicious usersof legitimate email providers exploit thefailure of 
> some email providers to perform sufficient valida-tion of emails 
> received from local MUAs. These attackers cansend emails with 
> spoofedFromheaders. The exploited emailproviders may automatically 
> attach DKIM signatures to theiroutgoing emails, enabling the attackers 
> to impersonate otherusers of the email provider

The writing of the paragraph's conclusion seems to be fundamentally 
misleading or wrong.  It seems to be saying that the DKIM signature will 
be for the domain in the From: field, which it won't.

> Security requirement.To achieve this goal,

I'm not sure exactly what goal they are referring to.

> (1) TheFromheader of the email thatSsends matches theauthenticated 
> username (other users ofScannot spoof Alice’saddress);

This requirement applies only in the presence of DMARC.

> (3) SPF/DKIM and DMARCcomponents inRconsistently authenticate the same 
> identifier

Except that SPF and DKIM are permitted to validate other identifiers, 
not just the one in the rfc5322.From field.

> This require-ment, although intuitive, implies a set of semantic 
> bindingrelations that every component in the email processing 
> chainmust respect.

This is imposing a requirement for deep, internal systems knowledge 
about features that are not, in fact, deeply embedded in email history 
or processing.  Arguably, the demand for having every component enforce 
this model is a basic mistake.

Rather the requirement is to prevent assumptions inside the system that 
serve to violate the policies needed at evaluation points.

 > robust principle

robustness

https://en.wikipedia.org/wiki/Robustness_principle

> be permissive in how they process malformed inputs

No, Postel did not direct accepting 'malformed' inputs.

And RFC 1122, Section 1.2.2 elaborated on this issue very helpfully:

> Software should be written to deal with every conceivable
> error, no matter how unlikely;

> 4.1 HELO/MAIL FROM confusion
This seems to imply that tightening is needed in DMARC, so that it uses 
an SPF domain that SPF has actually been validated? I think the issue is 
that SPF validation needs to inform DMARC what domain it has validated, 
rather than have DMARC decide which domain to fetch.

> SPF implementations treat “([email-masked]” as anempty MAIL FROM 
> address, and thus forward the resultsof checking HELO to the DMARC 
> component, because thestring in the parentheses can be parsed as a 
> comment ac-cording to RFC 5322 [10]. Some DMARC 
> implementations,however, may take it as a normal non-empty address,

1. This issues goes away if SPF supplies DMARC the domain name, rather 
than DMARC having to fetch it

2. I doubt this otherwise needs changes to language in the DMARC spec, 
but it's worth making sure.

> 4.3 Authentication results injection

Another focus on what results are communicated and how.

The paper asserts that AR is used as DMARC input.  I suspect that is 
rarely, if ever, true.  Yes? No?

> Generally, we can divideFromheader-relatedprocessing into two phases: 
> 1) parsing a MIME message to extract the Fromheader; 

The rfc5322.From: field is independent of MIME. MIME pertains only to 
the body.

> Microsoft:disregarded our report (which included our pa-per and a 
> video5demoing theA10attack) because the threatsrely on social 
> engineering, which they view as outside thescope of security 
> vulnerabilities.

That's oddly impressive.

> Improving MUA display
...

> We note however that experiences with suchapproaches for promoting 
> HTTPS (via browsers displayingtrusted icons for websites with valid 
> TLS certificates) havedemonstrated the challenges of ensuring that 
> users correctlyinterpret the icons and do not get fooled by imposters

"Challenges" is incorrect.  "Failure to be useful" is more appropriate 
language.

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 22 Apr 2020 18:08:37 -0400

> rather than have DMARC decide which domain to fetch.

The standard authentication results header field will do this in the form of:

Authentication-Results: mx.example.org; spf=pass smtp.mailfrom=example.com

> but it's worth making sure.

I agree it's worth making sure.  The input for DMARC from SPF/DKIM should a 
both a result and a domain.

There is some inconsistency in how SPF verifiers report SPF results when using 
HELO as a fallback for an null Mail From.  RFC 7208 section 2.4 is, IMO, 
reasonably clear that for a null Mail From, the SPF 'Mail From' identity is 
postmaster@[HELO], so it should be reported as an smtp.mailfrom= result, which 
the paper claims is the threat vector.  I think they are wrong.  As long as 
the input domain is checked and is in alignment for DMARC, there's no issue 
there.

I tried their exact scenario with two of my domains and here's what the SPF 
verifier reported:

Authentication-Results: mx.example.org; spf=pass (sender SPF authorized) 
smtp.mailfrom=kitterman.com (client-ip=72.81.252.18; helo=kitterman.org; 
envelope-from=([email-masked]; receiver=[email-masked])

While this is not ideal (for envelope-from=([email-masked].  The string 
should be quoted [per the 5322 CFWS definition], it shouldn't lead to any 
confusion about the identity.

I figured out this is wrong and the example below is right per various RFCs, 
but it took some investigation to dig it all back up.  A more thorough review 
by someone that knows more than me would surely be a good idea.

> rarely, if ever, true.  Yes? No?

For integration of open source components such as opendkim and opendmarc, this 
is the standard method of communicating results data between components (also 
true for communicating SPF results to opendmarc is you don't use its internal 
SPF implementation).

For what it's worth, I invested some time yesterday and today testing the 
various open source components I maintain (pyspf, pyspf-milter, policyd-spf-
python, dkimpy, dkimpy-milter, and authres) and found some behaviors in the 
cases they describe that is sub-optimal, I could not replicate any of the 
security relevant issues they describe.  I did get some new test cases. ...

As an example, if the example they used is presented to the authres module as 
the d= domain in for a DKIM verification, d=legitimate.com(.attacker.com, it 
returns d="legitimate.com(.attacker.com" [note the quoting, which is correct 
per the 5322 CFWS definition].  If it's parsing that, it fails to return a 
domain at all, which is a bug, but doesn't cause the reported problem.

Scott K

---

From: "John Levine" <[email-masked]>
Date: 22 Apr 2020 20:20:20 -0400

>distinguish between theory and practice.

Yup.

>than DMARC having to fetch it

Their assumptions about DMARC validators may be true in some cases but
not in general.  In OpenDMARC which is pretty widely used, the caller
tells the DMARC library what domain SPF checked, whether it was the
mail_from or helo, and what the result was.

>rarely, if ever, true.  Yes? No?

I'd say never.  To do DMARC rejects you have to do all of the
validation in the SMTP daemon, which is before anything has a chance
to create an A-R header.

>The rfc5322.From: field is independent of MIME. MIME pertains only to 

The display name can be a MIME encoded word, but that has nothing to do with
parsing the MIME parts in the body of the message.

I'm still impressed at some of the really simple bugs they exploited,
like NUL characters in header fields.

R's,
John

---

From: Juri Haberland <[email-masked]>
Date: Thu, 23 Apr 2020 09:38:18 +0200

> validation in the SMTP daemon, which is before anything has a chance

Of course this is possible with the Milter protocol introduced by 
Sendmail and used also by Postfix. The mail traverses during the SMTP 
phase through different milters, e.g. a SPF milter, followed by a DKIM 
milter, and every milter injects an AR header with its results. The last 
milter is a DMARC milter that processes the AR headers and signals the 
SMTP daemon do either accept or reject the message. This is how OpenDKIM 
& OpenDMARC work together.

Other projects do it all in one milter (rspamd?)...

   Juri

---

From: [email-masked]
Date: Thu, 23 Apr 2020 07:34:06 -0700 (PDT)

> & OpenDMARC work together.

This is done all the time, and not just by Sendmail and Postfix.

> Other projects do it all in one milter (rspamd?)...

Yes, but when you do it all in one go it's more difficult to customize.

				Ned