Subject: [dmarc-ietf] Fwd: Eliot's review of the DMARC spec

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 22 May 2013 22:43:20 -0700

This got caught in list moderation and I accidentally discarded it.
However it was also sent direct to me, so I do still have it; re-sending.

---------- Forwarded message ----------
From: Eliot Lear <[email-masked]>
Date: Tue, May 21, 2013 at 3:09 AM
Subject: Eliot's review of the DMARC spec
To: [email-masked], [email-masked], Barry
Leiba <[email-masked]>

 I've been asked to take a look at the DMARC spec.  Let me start by saying
that I know that you guys have running code.  I beg your indulgence and ask
you to read through.  While this review may come across as a bit negative,
and while I have concerns about several components (potential downgrade
attack etc).

In general I'd say this document needs some cleanup, and in my opinion, it
is *not* ready to move forward as a proposed standard.  To start with, the
requirements section is horribly confused.  Some are not requirements but
specifications, others are vague.  They read like there was a negotiation
among the contributors on what to keep out and what to keep in, or perhaps
these are markers for the working group.  Whatever.

The whole section needs to be replaced with something simpler, like:

[1]  Here is what the specification is out to accomplish (and not
accomplish).
[1a] A formal description of the threat model
[2]  Here are the assumptions we make about sender capabilities
[3]  Here are the assumptions we're making about receiver capabilities.

Let me stress conciseness!!

The document is laden with terms that are either explained later or not
explained at all (see below for examples).

A clear overview of the entire system should be provided.  It needn't be
long; perhaps a page or two.  Yes, include a pretty ASCII picture (maybe a
swimming pool diagram or two).

Please also intersperse the examples.  Otherwise the reader has to jump all
over the document.

The Security Considerations section needs a serious edit cut.

More detailed comments below.  This list is incomplete and should receive
further review, once the document is updated.

Sections 2.1 – 2.3 are advertisements for the mechanism, and largely
vacuous.  I would be happy to read how DMARC actually helps with phishing
or scaling, but these sections don't go into enough detail to be
meaningful.  Particularly off putting is the statement:

   [...] the DMARC mechanism is viewed more importantly as a
   substantial step forward in terms of creating reliable and defensible
   message streams.

By whom and why is it more important?  Let's avoid the advertisements and
the appeal to some vague authority.

Section 3 needs a fair amount of work.  I would prefer to see more detailed
statements about what the goals are.

So for instance, taking each bullet point in turn:

   o  Minimize false positives.

Positive for what?  Phish?  Positive authentication?

   o  Provide robust authentication reporting.

By whom to whom?  And what is authentication reporting in this context?

   o  Reduce the amount of successfully delivered phish.

Is the intent here solely to avoid phish or also spam?  Are we talking
about "fraudulant and/or dangerous email?"  I'm nitpicking on phish because
if this is a major high level goal, I want to understand what precisely you
intend to minimize.

Section 3.2 really belongs in Security Considerations.

Also, a nit: these are goals not requirements, unless we can say who is
making them requirements.  And if it's not the IETF, then let's back off
that language.

Section 3.3 point 3 is full of forward references and use of jargon like
"joe job" attacks, leading one straight to Google.  I'm not documenting the
rest at the moment, but the document is laden with this sort of thing, and
really needs to be unladen.

Section 3.4:

1.  is not a requirement, but an assertion.
2.  is all mixed up.  What precisely is the requirement?  The last sentence
belongs in another sentence.
3.  Is not really a requirement but a statement that DNS will be used.
5.  "No action" is used without explaining what that means.  I know that
sounds obvious "take no action", but does that mean HALT?  Let the message
through?  What?
6.  This is obvious.  The reverse is hard and necessary.  How can a policy
be applied that functions across an entire domain?
7.  I don't understand why this is in the doc at all, and why it is a
requirement.
8.  This requires that state be maintained on a receiver, and could be an
attack vector.
9.  Is not a requirement.
10,11,12.  Requirements for aggregate report without actually saying what
is being aggregated or reported.
14.  If we're using DNS, this is obvious.

Section 4:

The glossary needs to be moved forward!!!

Organizational Domain: Question: what would happen if two sites develop two
separate lists of public suffixes that don't match?  In particular, could a
parent domain assert authority for messages sent from a child domain?  What
are the implications if a TLD is not listed?  Does anyone know the
breakdown of the Egyptian domains in Arabic?

4.2: Summary  => Overview.  This also needs to be moved forward, although
if you rewrite and shrink Section 3 as I recommend, it will have that
effect ;-)

Section 5:

A Domain Owner advertises DMARC participation by adding a DNS TXT
   record (described in Section 6) {R3, R15, R16} to one or more sending
   domains under its direct or indirect control (e.g. operated by a
   delegate by agreement with the Domain Owner).

I think you mean "A Domain Owner advertises DMARC participation of one or
more sending domains by advertising them through a DNS TXT record."

   A Mail Receiver MUST consider an arriving message to pass the DMARC
   test if and only if one or more of the underlying message
   authentication mechanisms pass with proper identifier alignment.

Please define what the test actually IS!!

Later on:

   A Domain Owner that does not advertise an SPF policy or sign with
   DKIM is making an implicit statement that the use cases those
   protocols satisfy are not to be considered when determining whether
   or not the message under evaluation is valid.  For example, not
   publishing an SPF policy is an implicit message from Domain Owners to
   Mail Receivers that successful path authorization is not to be taken
   as sufficient evidence that the Domain Owner authorized the message.

Either I'm confused or this example is backwards.  How can you do
successful path authorization in the first place without SPF?

Section 8.2:

For example, a TXT resource record at
   "*._report._dmarc.example.com" containing at least "v=DMARC1"
   confirms that example.com is willing to receive DMARC reports for any
   domain.

Any domain or any CHILD domain?

Section 8.4:

That last comment in (2) seems to indicate that there should be a way to
collapse the ABNF, but I leave it to Dave who is Mr. ABNF as to whether
that is the right thing to do (I guess the tradeoff would be a mandatory
order).

Section 9:

Steps 2 and 4 look both repetitive and they conflict.  One says that the
record must START with a version and the other simply requires its
inclusion.  ???

Section 11.3 as an example:

   Attention must be paid to the possible presence of the "pct" tag in
   the DMARC policy record.  If the tag is present, the Mail Receiver
   MUST NOT enact the requested policy ("p" tag or "sp" tag") on more
   than the stated percent of the totality of affected messages.

The first sentence is chatty, and reflects the overall informal tone of the
document.  This is a technical specification and not a casual conversation.

Section 12.2.1: EMail

Please justify your normative language in the following cases:

   In the case of a "mailto" URI, the Mail Receiver SHOULD communicate
   reports using the method described in [STARTTLS].

As a matter of security, RFC-2119 gives license to you to use MUST here.
Why not do it?

   The aggregate data MUST be present using the media type "application/
   gzip", and the filenames SHOULD be constructed using the following
   ABNF:

On what basis according to RFC-2119  MUST aggregate data be compressed?
Separately, why is the filename at all important, requiring a SHOULD?

Section 13:

   o  Is able to send and/or receive reports at least daily;

   o  Is able to send and/or receive reports using "mailto" URIs;

Receive?  Eh?  How can one receive a report at least daily?  Using a URI?

Section 15 Security Considerations:

15.1, 15.2, and elsewhere don't follow the best current practices specified
in RFC-3552, and those that have evolved since.  State the threat and state
the remediation (if any).  I would split 15.4 to two points: there is the
potential for a downgrade attack as SPF is only as strong as its weakest
link in the path, and that a poor deployment of SPF can lead to garbage
getting through.

I don't really see 15.9 as a security consideration, but more of a
deployment consideration.

15.10.4:

Open transport mechanisms should be
   avoided.

"Unencrypted"?

15.12:

Again, poorly stated threat.  Remediation seems to suggest DNSSEC but then
doesn't require its use.  At the same time, the text gives license to
implementations to defer message processing when DNSSEC isn't deployed.
This leads to interoperability issues.  Be clear: either require DNSSEC or
don't but don't NOT require it and then recommend deferrals.

Eliot

---

From: SM <[email-masked]>
Date: Thu, 23 May 2013 02:55:13 -0700

>From: Eliot Lear <[email-masked]>

[snip]

>I've been asked to take a look at the DMARC spec.  Let me start by 
>saying that I know that you guys have running code.  I beg your 
>indulgence and ask you to read through.  While this review may come 
>across as a bit negative, and while I have concerns about several 
>components (potential downgrade attack etc).

I strongly agree with what is mentioned in Eliot's review.

>In general I'd say this document needs some cleanup, and in my 
>opinion, it is not ready to move forward as a proposed standard.  To 
>start with, the requirements section is horribly confused.  Some are 
>not requirements but specifications, others are vague.  They read 
>like there was a negotiation among the contributors on what to keep 
>out and what to keep in, or perhaps these are markers for the 
>working group.  Whatever.

If the aim to have a proposed standard I suggest focusing on the 
protocol aspect instead of having a document which tries to say everything.

>[1]  Here is what the specification is out to accomplish (and not accomplish).

Agreed.

 From the Abstract:

   "This memo presents a proposal for a scalable mechanism by which an
    organization can express, using the Domain Name System, domain-level
    policies and preferences for message validation, disposition, and
    reporting with predictable and accurate results.

    The enclosed proposal is not intended to introduce mechanisms that
    provide elevated delivery privilege of authenticated email.  The
    proposal presents a mechanism for policy distribution that enables a
    continuum of increasingly strict handling of messages that fail
    multiple authentication checks, from no action, through silent
    reporting, up to message rejection."

The body of the draft diverges significantly from what is stated in 
the Abstract.  Instead of expressing policies and preferences parts 
of the draft imposes policies (on the receiver).

The Introduction Section discusses about DMARC from a high-level 
perspective and does not convey any information about the 
technologies used in the design.  I had to read up to page 15 to get 
an idea of what DMARC is about.  It is not clear how DMARC URIs fit 
into the picture.

Some text in the document sounds like marketing to me.  For example, 
in Section 2.1:

   "DMARC is designed to support the extreme scalability requirements
    that characterize the systemic problem of identifying the origination
    and legitimacy of email.  DMARC seeks to preserve the positive
    aspects of the current email infrastructure, such as the ability for
    anyone to communicate with anyone else without introduction."

I suggest focusing on how the protocol works instead of trying to 
convince the reader about scalability requirements.

In Section 2.2:

   "This document is significantly informed by ongoing efforts to enact
    large-scale, Internet-wide, anti-phishing measures.  Whereas DMARC
    can only be used to combat specific forms of exact-domain phishing
    directly, the DMARC mechanism is viewed more importantly as a
    substantial step forward in terms of creating reliable and defensible
    message streams."

This can be discussed under security considerations if the question 
is considered as in scope for the protocol.

It would be easier to move Section 3 out.  The text makes the 
document look like a compliance document instead of a protocol 
specification.  In my opinion security considerations are about what 
threats were identified, what was addressed, what was not addressed, 
together with explanations.  Saying that X is out of scope does not 
sound like consideration has been given to a security issue.

The terminology section mentions terms such as "cousin domains", 
"domain owner", etc.  Although such terms may be popular I don't 
think that it is a good idea to use them in a protocol document as 
they can be ambiguous.

In Section 5:

   "A Mail Receiver MUST consider an arriving message to pass the DMARC
    test if and only if one or more of the underlying message
    authentication mechanisms pass with proper identifier alignment."

The above conveys the idea.  It does not provide adequate information 
for the implementer.

   "A Mail Receiver implementing the DMARC mechanism MUST make a best-
    effort to adhere to the Domain Owner's published DMARC policy when a
    message fails the DMARC test."

The above uses a "MUST" to tell people that they ought to make a 
best-effort.  In my opinion that is incorrect usage of RFC 2119 key words.

   "Mail Receivers MAY deviate from a Domain Owner's published policy
    during message processing and SHOULD make available the fact and
    reason of the deviation to the Domain Owner via feedback reporting."

I read the above as "if you do not do what I say you have to provide 
me with a justification".  Let me rewrite the text:

   "Mail Receivers deviating from a Domain Owner's published policy
    during message processing and MUST make available the fact and
    reason of the deviation to the Domain Owner via feedback reporting."

If that text is not acceptable to mail receivers the RFC 2119 SHOULD 
will have the same effect.

In Section 7:

   "When this is done, the DNS domain name thus recorded MUST be encoded as an
    A-label, as described in Section 2.3 of [IDNA]."

The above is under "policy enforcement considerations".  It should be 
in a section discussing the DNS aspects of the specification.

Section 8.2 is about "Verifying External Destinations".  If I am not 
mistaken, IETF specifications are not about imposing a specific 
method.  Section 8.2 sounds like a tutorial for the implementer 
instead of a definition of what the "DMARC policy string" means.

   "A report receiver MUST publish such a record in its DNS if it wishes
    to receive reports for other domains."

The RFC 2119 usage is inappropriate here.  I would put it after the 
high-level view (see Eliot's message) of how DMARC works.

In Section 8.3:

   "The report SHOULD include the following data:

    o  Enough information for the report consumer to re-calculate DMARC
       disposition based on the published policy, message dispositon, and
       SPF, DKIM, and identifier alignment results. {R12}"

"enough information" is an ambiguous description of what the report 
should include.

In Section 11.2:

   "Heuristics applied in the absence of use by a Domain Owner of either
    SPF or DKIM (e.g., [Best-Guess-SPF]) SHOULD NOT be used, as it may be
    the case that the Domain Owner wishes a Message Receiver not to
    consider the results of that underlying authentication protocol at
    all."

The reference would have to be normative.

In Section 11.3:

   'If email is subject to the DMARC policy of "quarantine", the Mail
    Receiver SHOULD quarantine the message.  If the email is not subject
    to the "quarantine" policy (e.g., due to the "pct" tag), the Mail
    Receiver SHOULD apply local spam classification as normal.'

The usage of RFC 2119 key words in the above is incorrect.

In Section 12.2.1:

  'In the case of a "mailto" URI, the Mail Receiver SHOULD communicate
    reports using the method described in [STARTTLS].'

Has the usual STARTTLS issues been taken into consideration for the 
RFC 2119 recommendation?

   "The aggregate data MUST be an XML file subjected to GZIP compression.
    The aggregate data MUST be present using the media type "application/
    gzip", and the filenames SHOULD be constructed using the following
    ABNF:"

In Section 12.2.2 it mentioned that:

   'Where an "http" or "https" method is requested in a Domain Owner's
    URI list, the Mail Receiver MAY encode the data using the
    "application/gzip" media type ([GZIP]) or MAY send the Appendix C
    data uncompressed or unencoded.'

The data exchange format depends on the URI scheme being used.  In my 
opinion that makes the implementation more complex.

   'For the GZIP file itself, the extension MUST be "gz"; for the XML
    report, the extension MUST be "xml".'

I would ask why such a RFC 2119 requirement is in the draft.

   "Email streams carrying DMARC feedback data MUST conform to the DMARC
    mechanism, thereby resulting in an aligned "pass" (see Section 4.3)."

As far as I am aware the IETF does not do conformance.

   'The RFC5322.Subject field for individual report submissions SHOULD
    conform to the following ABNF:'

The Subject field is specified as structured text in the above.

In Section 12.2.2:

   "The header portion of the POST or PUT request SHOULD contain a
    Subject field as described in Section 12.2.1."

This sounds like sending RFC 5322 formatted messages over HTTP.

In Section 15.8:

   "Many systems are able to scan the SMTP reply text to determine the
    nature of the rejection, thus providing a machine-detectable reason
    for rejection allows automated sorting of rejection causes so they
    can be properly addressed."

I don't think that it is a good idea to discuss about that as part of 
a protocol specification.

I suggest giving careful consideration to privacy.  The draft 
discusses about that in Section 15.10.1.  The draft would require a 
detailed review to understand what information is being exchanged and 
how that can affect users.

In Section 15.12:

   "The DMARC mechanism and its underlying technologies (SPF, DKIM)
    depend on the security of the DNS.  To reduce the risk of subversion
    of the DMARC mechanism due to DNS-based exploits, serious consideration
    should be given to the deployment of DNSSEC in parallel to the deployment
    of DMARC."

I would ask whether anyone who has deployed this mechanism is 
actually using DNSSEC.  If not the above is not saying anything to 
ensure secure use of the protocol.

   "S/MIME, or Secure Multipurpose Internet Mail Extensions, is a
    standard for encryption and signing of MIME data in a message.  This
    was suggested and considered as a third security protocol for
    authenticating the source of a message."

I could not understand the relationship between S/MIME and what this 
specification attempts to do.

   "It has been suggested in several message authentication efforts that
    the Sender header field be checked for an identifier of interest, as
    the standards indicate this as the proper way to indicate a re-
    mailing of content such as through a mailing list.  Most recently, it
    was a protocol-level option for DomainKeys, but on evolution to DKIM,
    this property was removed."

That discussion would be better in a DKIM-related document instead of 
adding it to this draft.

   'DMARC has been characterized as a "super-ADSP" of sorts.'

It is not a good idea to include criticism of other protocols in a 
draft that specifies a protocol unless it is relevant to the protocol 
being specified.

I suggest dropping the idea of using the "public suffix list" as 
previous attempts to use it have not been successful in the IETF.

Regards,
-sm

---

From: "John Levine" <[email-masked]>
Date: 23 May 2013 18:15:05 -0000

I agree with pretty much everything Eliot said.  Fortunately, having
written a DMARC implementation that handles everything other than
sending reports (the stuff is in a database, I could do it if I wanted
to) I believe the problems can be fixed with little or no change to the
bits on the wire.

DMARC does two separate things.  One is the policy stuff, what a
domain would like you to do if you see mail with their domain on the
From: line and no corresponding DKIM or SPF authentication.  The other
is the report stuff, requesting per-message and daily aggregate
reports from the world about the authentication results of mail with
your domain on the From: line.

The two parts have practically nothing to do with each other other
than sharing the _dmarc DNS record and it's quite possible to use
one without the other.  I've collected lots of interesting stats
for domains that will never publish a DMARC policy.

As far as what it does, or what it's good for, the policy stuff does
one very specific thing, denouncing unauthenticated mail with your
domain on the From: line.  For some senders, that category of mail
matches up well with a narrow but significant category of phishing.
For others it may match up with phish-like behaviors in which company
employees send mail from outside to avoid routing mail through the
corporate servers (a big issue for stockbrokers and the like where the
servers log everything.)  For a lot of senders, e.g., ISPs, the policy
isn't useful at all since the authentication model doesn't match what
their users do.  I would strenuously resist any attempts to claim that
it does more, particularly attempts to make it the FUSSP of the month,
often from people who should know better.

The reporting stuff is useful to see what sort of mail purporting to
be from you is arriving at the sites that send reports.  For some
domains, it may be useful to see how much you're being phished.  If
you're a stockbroker, it could be useful to find people evading the
server logging by sending from gmail.  Again, I would resist attempts
to claim that it does more than what it does, send reports which may
tell you interesting things about your mail (and in the security
section, note that it can also tell you interesting things about other
people's mail, e.g., the mailing lists that your users subscribe to.)

All but one of the technical comments seem on target.  I think the
protocol is OK, but the description needs to be edited down be a
description of what it does, leaving out all the reasons it's
wonderful.

>8.  This requires that state be maintained on a receiver, and could be an

I implement it as perform policy if (time mod 100) < pct, which I
think is what everyone else does, no DMARC state needed.  It's worth
mentioning this as an adequate implementation.

R's,
John

---

From: Tim Draegen <[email-masked]>
Date: Thu, 23 May 2013 15:11:38 -0400

On May 23, 2013, at 1:43 AM, Murray S. Kucherawy <[email-masked]> wrote:
> [Requirements] read like there was a negotiation among the contributors on what to keep out and what to keep in, or perhaps these are markers for the working group.

Ha!  Eliot, you've nailed it.  Looking back, the document is largely the result of a long negotiation between organizations that all share a desire to combat fraud, and yet brought very different perspectives/emphasis to the effort.

Against that backdrop, a hearty +1 to your's and SM's comments.  Converting the DMARC memo into a proper protocol document means chopping away a great deal of the "result of long negotiation" bits, but hopefully those bits can be stitched together into supplementary documentation, as they're valuable as reference and maybe to clarify future discussion.

=- Tim

---

From: Matt Simerson <[email-masked]>
Date: Thu, 23 May 2013 14:24:35 -0700

On May 23, 2013, at 11:15 AM, "John Levine" <[email-masked]> wrote:

> I agree with pretty much everything Eliot said.

+1

> Fortunately, having

I have written two implementations of DMARC (one merely validates, the other (Mail::DMARC) is mostly complete and does validation as well as send/receive aggregate reports. As an implementer, my "indulgence" is heartily extended. I would much prefer a more straight forward and concise specification. 

> one without the other.

I wonder whether DMARC shouldn't be two specifications? It seems that the validation portions of DMARC are well defined, straight forward to implement, and could easily be implemented by most modern MTAs (whether by milter, Amavis, or SpamAssassin). 

The reporting aspects OTOH, are complex and encumbered with technical issues, security issues and sticky legal questions about information disclosure. While the reporting is valuable, its value will certainly be diminished by the number of organizations that implement it.

> be from you is arriving at the sites that send reports.

+1

> For some domains, it may be useful to see how much you're being phished.

It is also useful to see how many DMARC deploying sites fail to validate the DKIM signatures. For the DMARC receiver, reporting is a burden.  But as a domain author, DMARC reporting is very useful. 

> mentioning this as an adequate implementation.

I implemented with rand:

$result->reason( type => 'sampled_out' ) if rand(100) >= $policy->pct;

Matt

---

From: "John R Levine" <[email-masked]>
Date: 23 May 2013 19:11:45 -0400

> I wonder whether DMARC shouldn't be two specifications? It seems that 
> the validation portions of DMARC are well defined, straight forward to 
> implement, and could easily be implemented by most modern MTAs (whether 
> by milter, Amavis, or SpamAssassin).
>
> The reporting aspects OTOH, are complex and encumbered with technical 
> issues, security issues and sticky legal questions about information 
> disclosure. While the reporting is valuable, its value will certainly be 
> diminished by the number of organizations that implement it.

I don't see any benefit in that.  I expect the spec will be somewhat 
shorter when we're done, and in most cases I expect that people who 
implement one will at least partially implement the other.

> $result->reason( type => 'sampled_out' ) if rand(100) >= $policy->pct;

rand() is usually seeded from the stuff including clock so it amounts to 
the same thing.  The point of course is that you can do the percent stuff 
by rolling dice, not by a rolling average.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
"I dropped the toothpaste", said Tom, crestfallenly.