Subject: Re: [spfbis] #22: RFC 4408: Section 2.5.7 PermError on invalid domains after macro expansion,

From: Arthur Thisell <[email-masked]>
Date: Tue, 4 Sep 2012 07:58:11 -0700 (PDT)

> My claim is that if invalid domains MUST result in
> PermError, then there is _no good way_ to use %h.  That is, such that
> authorized senders pass and attackers match -all.

It is important to remember that even if the WG charter allowed it, we are not creating a new protocol and arguments based on "this makes more sense to do" get outweighed by "this is what is actually being done".  Indeed, it is my opinion that even before MARID really got started, the deployment of SPF was so large that major changes couldn't be done.

Some history on this permerror stuff.

First, SPF was developed outside of the IETF and largely done based on documents that wouldn't even pass as I-Ds.  In order to understand the implementations of SPF in the real world, we need to look past the quality of these documents and accept them as what was used to develop off of.

Before MARID, permerror was called "unknown" (and temperror was "error").  This wasn't a bad name for what Meng Weng Wong had envisioned for SPF.  He viewed an SPF pass as being the most important thing (see, for example, http://openspf.org/whitepaper.pdf )  In both that document, and his first I-D ( http://tools.ietf.org/html/draft-mengwong-spf-00 ), he envisioned extending SPF via unknown mechanisms.  So, something like this is was used an example of a valid SPF record:

    v=spf1 a mx ptr domainkeys:_dk.%{d} -all

When an implementation hit the mechanism "domainkeys" and it didn't know what to do with it, the implementation returned "unknown" and plopped the mechanism into the Received-SPF header for later processing.

It didn't matter that the SPF evaluation didn't hit the -all at the end.  You could get an SPF pass via the initial mechanisms, and domainkeys would help you get further if you knew what to do with it.

In early drafts (pre IETF involvement), there were conflicting sentences about what to do with invalid domain names.  Some places said they should trigger "unknown" (e.g. permerror), some places didn't.   Some implementations did it one way, others did it the other way.

OK back to today, where we are trying to decide what should be done with invalid domain names and whether they should cause a permerror or not. RFC 4408 documented this mess by having one sentence in the permerror definition that said:

   Be aware that if the domain owner uses macros
   (Section 8), it is possible that this result is due to the checked
   identities having an unexpected format.

No where does it actually say that an implementation must generate a permerror, just that in the real world, this can happen.

So, we can't use logic to argue what RFC 4408bis "should" do, we need to go out and find out what the situation is in the real world.   How many implementations trigger permerror when the get an invalid domain and how many just don't match?   If almost all do one thing, maybe we can shift the standard a bit to bless that behavior.   For that matter, what do implementations actually define as an invalid domain name?  Is "localhost" an invalid domain name?  What about 1.2.3.4?  Do implementations trigger permerror on only a subset of invalid domain names?

Note that for SPF records, section 4.3 already takes care of this kind of thing.  In 4408bis section 2.1 (the HELO identity), there is a new sentence that duplicates the requirements in section 4.3.

So, again, I am no longer able to easily do much work (e.g., it would take me a month or two), so I can't answer the question "what do actual implementations do?"   If someone else can do the work, that would be great, if not, then I think the best that this WG can do is add some clarifying text about what we know used to be the situation.

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 05 Sep 2012 12:53:27 +0200

On Tue 04/Sep/2012 16:58:11 +0200 Arthur Thisell wrote:
> 
> RFC 4408 documented this mess by having one sentence in the 
> permerror definition that said:
> 
> No where does it actually say that an implementation must generate
> a permerror, just that in the real world, this can happen.
> 
> So, we can't use logic to argue what RFC 4408bis "should" do, we
> need to go out and find out what the situation is in the real
> world.   How many implementations trigger permerror when the get an
> invalid domain and how many just don't match?

It is very impractical to test implementations running on a remote
host, because they won't necessarily let the client know the
evaluation result.  It is possible to examine source code, or test
locally.  But then let me recall that there is a test suite provided
by OpenSPF.org that has various cases already prepared.

The relevant test cases are invalid-domain-empty-label,
invalid-domain-long, and invalid-domain-long-via-macro, for which the
testsuite indicates permerror or fail as valid results.
Unfortunately, running the testsuite only reports whether the test was
successful, possibly along with the amount of time it took to complete it.

I tested Courier-MTA, and it returns a non-compliant "unknown" result
for those three cases.  I'll ask on spf-discuss, it will hopefully
take less than a month or two...

> If almost all do one thing, maybe we can shift the standard a bit
> to bless that behavior.

One thing that changed since RFC 4408 is the possibility to get a
report whenever SPF evaluation results in *error.  The usefulness of
that is that it will report real errors, such as DNS servers hiccups
or an occasional syntax error slipping in.  Blessing permerror would
taint such streams with spurious attack reports.

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 07 Sep 2012 09:59:10 +0200

On Wed 05/Sep/2012 12:53:27 +0200 I wrote:
>> How many implementations trigger permerror when the get an 
> I tested Courier-MTA, and it returns a non-compliant "unknown"
> result for those three cases.  I'll ask on spf-discuss, it will
> hopefully take less than a month or two...

I got two responses, one for each possibility.  Unless Scott has
already written something to tackle this issue, I propose to leave it
for -08.

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 07 Sep 2012 09:19:06 -0400

> already written something to tackle this issue, I propose to leave it

I don't think we've reached a consensus on this.  BTW, unknown is the pre-
MARID equivalent of permerror, so you should put courier-mta in that camp.

Scott K

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 07 Sep 2012 20:36:45 +0200

On Fri 07/Sep/2012 15:19:06 +0200 Scott Kitterman wrote:
> On Friday, September 07, 2012 09:59:10 AM I wrote:
> 
>> Unless Scott has already written something to tackle this issue,
>> I propose to leave it for -08.
> 
> I don't think we've reached a consensus on this.

So do I.  However, I reckon you have accumulated quite some local
changes, so I think it's better to get -07 out and checked.  Perhaps,
it will be easier to make our mind up about catch #22 at that point...

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 07 Sep 2012 14:38:18 -0400

> it will be easier to make our mind up about catch #22 at that point...

Interesting timing for your suggestion.

Scott K