Subject: [spfbis] Result of record evaluation with non-implemented mechanism

From: Alessandro Vesely <[email-masked]>
Date: Tue, 12 Jan 2016 17:09:35 +0100

Hi,
RFC 7208 doesn't seem to be very clear on the requirements for "exists"
mechanisms.  I don't know if that deserves an errata.

The spec defines mechanisms, but Section 4.6.4 "DNS Lookup Limits" does not
mention "exists", while Section 5.7 "exists" does not mention return values.

OTOH, dmarcian reports that "exists" mechanisms are not fully supported by all
high-volume receivers.  In fact, google.com returns permfail for the record in
the first bullet of Appendix D.1, unless a match is found before "exists".

According to Section 2.6.7 "Permerror", Google signal an error condition that
definitely requires DNS operator intervention to be resolved. Tools which parse
DMARC aggregate feedback correctly report an issue, to no avail :-O

You see,  SPF receivers are divided into two categories: those with a loaded
check_host() function and those who never reject on fail.  Google never reject
on fail.  Why do they return permerror, then?  Wouldn't "none" be fine?

4.6.2.  *Mechanisms*

The second paragraph there may be a good point to pin an errata:

OLD
   When a mechanism is evaluated, one of three things can happen: it can
   match, not match, or return an exception.

NEW
   When a mechanism is evaluated, one of three things can happen: it can
   match, not match, or return an exception.  Non-implemented mechanisms
   MUST NOT return an exception.

Better ideas?

Ale

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 12 Jan 2016 22:56:28 -0500

On Tuesday, January 12, 2016 05:09:35 PM Alessandro Vesely wrote:
> OTOH, dmarcian reports that "exists" mechanisms are not fully supported by
> all high-volume receivers.  In fact, google.com returns permfail for the
> record in the first bullet of Appendix D.1, unless a match is found before
> "exists".
> 
> According to Section 2.6.7 "Permerror", Google signal an error condition
> that definitely requires DNS operator intervention to be resolved. Tools
> which parse DMARC aggregate feedback correctly report an issue, to no avail
> :-O
> 
> check_host() function and those who never reject on fail.  Google never
> reject on fail.  Why do they return permerror, then?  Wouldn't "none" be
> fine?
> 
> 4.6.2.  *Mechanisms*
> 

The way the RFC is designed, mechanisms aren't optional.  What you're asking 
for is not something that should be done via errata.  4.6.2 is correct and 
exists is no different than any other mechanism.

Scott K

---

From: S Moonesamy <[email-masked]>
Date: Thu, 14 Jan 2016 07:36:41 -0800

Hi Alessandro,
>mechanisms.  I don't know if that deserves an errata.

Section 5 lists "exists" as a designated sender mechanism.

>    MUST NOT return an exception.

The suggested text is about non-implemented mechanisms.  There is a 
"do not use" mechanism.  Would that be a non-implemented 
mechanism?  There isn't an explanation for the "MUST NOT" in the 
suggested text.

Regards,
S. Moonesamy

---

From: Tim Draegen <[email-masked]>
Date: Thu, 14 Jan 2016 11:15:52 -0500

> On Jan 12, 2016, at 11:09 AM, Alessandro Vesely <[email-masked]> wrote:
> 
> high-volume receivers.

Speaking for dmarcian.com's SPF tool, Hotmail used to skip exists mechanisms due to the impact of expanding macro resolutions and trying to cache it all.  Search RFC 7208 for "cache" for some related text.

Two things:

1. It's not the exists: mechanism itself, its the use of macros that can be painful to DNS caching.  Apparently Hotmail saw macro-based DNS explosions in the context of the exists: mechanism, and act accordingly.  However, other mechanisms can use macros, too.

2. Hotmail's infrastructure has been moved (or is just about finished moving) to all-Outlook.  Part of this transition means no more skipping of exists.

I'm OK with removing the exists: mechanism warning shown to users of dmarcian's SPF tool.  One thing I'm not sure about is if warning about the use of macros should replace it.

Is concern about DNS caching infrastructure still relevant, or are these concerns rooted in an older time?  I'll be asking more "DNS people" about this, as I'd like to give correct guidance.

-= Tim

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 14 Jan 2016 21:15:32 +0100

On Thu 14/Jan/2016 17:15:52 +0100 Tim Draegen wrote: 

> I'm OK with removing the exists: mechanism warning shown to users of
> dmarcian's SPF tool.  One thing I'm not sure about is if warning about the
> use of macros should replace it.

Yes, please.  It would clean up the interface a bit.  Would it be possible to
turn it on only after actual SPF permerror's show up in aggregate reports?

Ale

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 14 Jan 2016 21:50:34 +0100

Hi SM,

On Thu 14/Jan/2016 16:36:41 +0100 S Moonesamy wrote: 
> The suggested text is about non-implemented mechanisms.  There is a "do not
> use" mechanism.  Would that be a non-implemented mechanism?  There isn't an
> explanation for the "MUST NOT" in the suggested text.

Yeah, that would have been a bit freaky.  Yet, a cute evaluator could optimize
an evaluation in some ways.  Consider a record ending like so:

   ...   ?exists:%{ir}.blah ?all

There is no reason to evaluate the exists.

Now what if you had:

   ...   ?exists:%{ir}.blah -all

Suppose you know your internal policy only differentiates between "pass" and
"not-pass".  Getting "neutral" or "fail" wouldn't make a real difference,
except for the formal result name that you write on reports.  Does it really
hurt to put "none" as in "nothing relevant here"?  The note at the end of
Section 4.8 seems to allow some leeway in the reported result, but only for
malformed names.

There are some other head-scratchers before the example in the first bullet of
Appendix D.1 can be used in practice.  One is the query quota that a whitelist
may impose.

Let me use this post as an occasion to apologize to Google for having wrongly
imagined that their implementation skips proper evaluation of mechanisms.  In
fact, the resulting DNS queries fail precisely because it doesn't.

Ale