Subject: [dmarc-ietf] DMARC aggregate reports XML Schema inconsistencies

From: "Freddie Leeman" <[email-masked]>
Date: Wed, 31 Jul 2019 11:47:29 +0200

I've been processing millions of DMARC aggregate reports from a lot of
different organizations, and have been trying to make sense of them for
quite some time now. I've noticed that most of them, even those from large
parties like Google and Yahoo!, fail to follow the DMARC RFC guidelines
(Appendix C.  DMARC XML Schema). I've written a blog about this that can be
found here: https://www.uriports.com/blog/dmarc-reports-ietf-rfc-compliance/

The bottom line is that the RFC 7489 Appendix C is a mess and contradicts
itself numerous times in both schema and comments. I think it's important to
be clearer and stricter about the xml elements and their values. Too much of
this section is open to interpretation. 

Some examples: 

The report has an element with the name "policy_published". This name would
indicate that the elements within, contain the domain's published policy.
The comments however, mention "applied" and "apply". Most organizations that
send aggregate reports do not send failure reports and thus do not "apply"
the "fo" (Failure reporting options) element. This is why parties like
Google leave this element out of their reports. This particular element's
comment ("failure reporting options in effect") also implies that it is
optional. On the other hand, this element has a default "minOccurs" value of
1, so it should not be omitted.

It should also be clearer about what to do with policy elements that are
unspecified in the domain's DNS record. I think it is best to fill these
elements in the report with their respected default values. So when 'pct' is
not specified in the domain's policy, the report should state '100'. When
'sp' is not specified it should have the value of the 'p' element.

I've also noticed that most parties do not specify the PolicyOverrideType,
even when both SPF and DKIM alignment fails. So this element should be made
mandatory whenever alignment fails and the disposition doesn't follow the
domain's DMARC policy.

The RFC guidelines for aggregate reports should also state that empty
elements with a minOccurs of 0 should be omitted and not be left blank.

It should also be specified that if a message is not signed with DKIM the
'DKIMAuthResultType' should be omitted. And thus the 'DKIMResultType' 'none'
would never be used. Because when a message has no signatures, then it also
doesn't have a specified 'domain' (d=) (minOccurs 1) and 'selector' (s=)
(minOccurs 0). What happens now is that some organizations report non-signed
messages with the 'dkim' element and fill the 'domain' and 'selector' with a
bogus 'none' value.

There are also multiple mentions of MinOccurs="1", even though the document
specifies that unless otherwise specified in the schema, the minOccurs and
maxOccurs values for each element are set to 1. This adds to the confusion.

DMARC reporting capabilities are a valuable aspect of the DMARC mechanism.
It can help domain owners in setting up and hardening their DKIM/SPF/DMARC
policy. But unless these reports follow strict guidelines they just pile up
to a lot of inconsistent data open to interpretation and guesswork. Domain
owners should be able to understand the data without the need for a
spiritual voodoo DMARC guru (trademark pending) to make sense of it all.

Kind regards,

Freddie Leeman

---

From: "Kurt Andersen (b)" <[email-masked]>
Date: Wed, 31 Jul 2019 08:00:17 -0700

On Wed, Jul 31, 2019 at 2:47 AM Freddie Leeman <freddie=
[email-masked]> wrote:

> I’ve been processing millions of DMARC aggregate reports from a lot of
> quite some time now. I’ve noticed that most of them, even those from large
> (Appendix C.  DMARC XML Schema). I’ve written a blog about this that can be
> found here:
> https://www.uriports.com/blog/dmarc-reports-ietf-rfc-compliance/
>
>
>
> itself numerous times in both schema and comments. I think it’s important
> to be clearer and stricter about the xml elements and their values. Too
> much of this section is open to interpretation.
>

Freddie,

Thanks for your observations - would you mind proposing concrete language
to replace Appendix C? Speaking from experience, it can be helpful to be
able to do in-place comments/markup so I've posted
http://bit.ly/dmarc-rpt-schema - please make any adjustments in "Suggest"
mode or comments you feel appropriate. The invitation extends to all
members of this group.

--Kurt Andersen

---

From: "Freddie Leeman" <[email-masked]>
Date: Wed, 31 Jul 2019 17:42:12 +0200

Thanks Kurt, I’ve made the adjustments to the document.

--Freddie Leeman

Van: Kurt Andersen (b) [mailto:[email-masked]] 
Verzonden: woensdag 31 juli 2019 17:00
Aan: Freddie Leeman <[email-masked]>
CC: [email-masked]
Onderwerp: Re: [dmarc-ietf] DMARC aggregate reports XML Schema inconsistencies

On Wed, Jul 31, 2019 at 2:47 AM Freddie Leeman <freddie=[email-masked] <mailto:[email-masked]> > wrote:

I’ve been processing millions of DMARC aggregate reports from a lot of different organizations, and have been trying to make sense of them for quite some time now. I’ve noticed that most of them, even those from large parties like Google and Yahoo!, fail to follow the DMARC RFC guidelines (Appendix C.  DMARC XML Schema). I’ve written a blog about this that can be found here: https://www.uriports.com/blog/dmarc-reports-ietf-rfc-compliance/

The bottom line is that the RFC 7489 Appendix C is a mess and contradicts itself numerous times in both schema and comments. I think it’s important to be clearer and stricter about the xml elements and their values. Too much of this section is open to interpretation.

Freddie,

Thanks for your observations - would you mind proposing concrete language to replace Appendix C? Speaking from experience, it can be helpful to be able to do in-place comments/markup so I've posted http://bit.ly/dmarc-rpt-schema - please make any adjustments in "Suggest" mode or comments you feel appropriate. The invitation extends to all members of this group. 

--Kurt Andersen