Subject: [spfbis] Last Call: <draft-ietf-spfbis-4408bis-19.txt> (Sender Policy Framework (SPF) for Authorizing Use of Domains in Email, Version 1) to Proposed Standard

From: The IESG <[email-masked]>
Date: Mon, 19 Aug 2013 06:19:16 -0700

The IESG has received a request from the SPF Update WG (spfbis) to
consider the following document:
- 'Sender Policy Framework (SPF) for Authorizing Use of Domains in Email,
   Version 1'
  <draft-ietf-spfbis-4408bis-19.txt> as Proposed Standard

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
[email-masked] mailing lists by 2013-09-02. Exceptionally, comments may be
sent to [email-masked] instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract

   Email on the Internet can be forged in a number of ways.  In
   particular, existing protocols place no restriction on what a sending
   host can use as the "MAIL FROM" of a message or the domain given on
   the SMTP HELO/EHLO commands.  This document describes version 1 of
   the Sender Policy Framework (SPF) protocol, whereby ADministrative
   Management Domains (ADMDs) can explicitly authorize the hosts that
   are allowed to use its domain names, and a receiving host can check
   such authorization.

   This document obsoletes RFC4408.

The file can be obtained via
http://datatracker.ietf.org/doc/draft-ietf-spfbis-4408bis/

IESG discussion can be tracked via
http://datatracker.ietf.org/doc/draft-ietf-spfbis-4408bis/ballot/

The following IPR Declarations may be related to this I-D:
https://datatracker.ietf.org/ipr/1698/

---

From: Måns Nilsson <[email-masked]>
Date: Mon, 19 Aug 2013 17:05:21 +0200

Subject: [spfbis] Last Call: <draft-ietf-spfbis-4408bis-19.txt> (Sender Policy?Framework (SPF) for Authorizing Use of Domains in Email, Version 1) to Proposed Standard Date: Mon, Aug 19, 2013 at 06:19:16AM -0700 Quoting The IESG ([email-masked])
> beginning of the Subject line to allow automated sorting.

I strongly OPPOSE draft-ietf-spfbis-4408bis-19.txt being published as
RFC unless substantial parts are reworked.

* The charter disallows major protocol changes -- removing the SPF RR type
is a direct charter violation; since SPF is being used on the Internet.

* The overloading of the TXT record is a hack at best, aimed at
circumventing DNS management systems vendors that fail to ship
support. Breaking the DNS model with specific resource records is not
the way to get better application support. (besides, the major argument
at the time was "it's so hard and takes ages to get a RR type", which
isn't true anymore and also, the RRtype is allocated, what's the fuss? )

* The empirical data that was gathered and the conclusions from which
that where published as RFC 6686 are IMNSHO flawed and rushed in that they
set far too optimistic deadlines for adaptation before declaring failure.

The IESG should send draft-ietf-spfbis-4408bis-19 back to spfbis wg and tell
the wg that instead of deprecating SPF it should be algorithmically
preferred while maintaining support for TXT.

Thanks, 
-- 
Måns Nilsson     primary/secondary/besserwisser/machina
MN-1334-RIPE                             +46 705 989668
It was a JOKE!!  Get it??  I was receiving messages from DAVID LETTERMAN!!
YOW!!

---

From: Dotzero <[email-masked]>
Date: Mon, 19 Aug 2013 15:03:06 -0400

The issue Måns Nilsson raises was discussed extensively on the SPFbis
list prior to as well as during last call on the list and I believe
the appropriate decision was reached by the working group. If there is
any doubt in the minds of the IESG regarding whether the working group
reached the correct decision, I would urge those IESG members to
review the threads in the archives related to this issue.

Several related issues, including a race condition, were identified
and the solution to go with TXT only records is IMHO the correct one
under the circumstances. The relatively small uptake of Type 99
records in the wild (both on the publishing side AND on the validation
side) in comparison to the implementation for TXT records made a
compelling case for the decision of the working group.

With regard to the limitations of the working group charter, some
significant change was required to eliminate the race condition
regardless of what that change would be. The decision of the working
group (IMHO - I do not want to put words into anyones mouth) was to go
with the approach which had the least impact on what is arguably a
very large installed existing base on both the sender AND the
validator sides of implementation.

Based on this I would ask that tehe IESG move
draft-ietf-spfbis-4408bis-19.txt to Proposed Standard.

Michael Hammer

On Mon, Aug 19, 2013 at 11:05 AM, Måns Nilsson
<[email-masked]> wrote:
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
>
> iEYEARECAAYFAlISNDEACgkQ02/pMZDM1cXK+gCfYQ1Mv1CHjy9DDn7sA7DC7dF3
> b48An1b49Zqf/du3dvN6pmj6in+CEujB
> =soFG
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> spfbis mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/spfbis
>

---

From: Andrew Sullivan <[email-masked]>
Date: Mon, 19 Aug 2013 16:08:02 -0400

Note that I am not the shepherd for this draft, but I am the WG
co-chair.

> is a direct charter violation; since SPF is being used on the Internet.

That argument doesn't work, because the WG had to make a major
protocol change in this area no matter what, since 4408 has an
interoperability problem.  If you wish to argue that the WG decided on
the wrong protocol change, that line is open to you; but nobody can
argue that the change is wrong because of our charter.  

Best regards,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Pete Resnick <[email-masked]>
Date: Mon, 19 Aug 2013 15:48:36 -0500

Speaking in my capacity as responsible AD for this WG and document, and 
the one who is going to have to judge the consensus of this Last Call 
and report to the IESG.

> argue that the change is wrong because of our charter.

To reinforce this point: Nowhere does the charter "disallow major 
protocol changes"; words to that effect do not appear in the charter. It 
does say that features in current use cannot be removed, but it also 
says that errors can be corrected. In this case the WG concluded that 
fixing the interoperability problem fell under the auspices of 
"correction of errors". The chairs agreed, and I saw no reason to disagree.

>> isn't true anymore and also, the RRtype is allocated, what's the fuss? )

This issue *was* considered and extensively discussed on the spfbis 
list. AFAICT, the conclusion the WG came to was *not* based on the 
argument that "it's so hard and takes ages to get an RR type". (Yes, the 
original decision to use TXT included that argument, but that was not 
the basis for the decision in the current WG.) And I don't know that 
anyone in the WG would disagree that "the use of the TXT record is a 
hack". The conclusion was that it was a hack that we are stuck with due 
to the current deployment profile. I think you will have to find 
something that the WG missed in that discussion to be convincing that a 
substantial change is needed.

There is no doubt the consensus in the WG was rough on the above two 
issues, but it was, by my reading, solid.

>> set far too optimistic deadlines for adaptation before declaring failure.

I think you're going to need substantially more explanation (and perhaps 
some data) to make a convincing case that RFC 6686 needs to be 
reconsidered, thereby affecting this last call. The above states a 
conclusion, but provides no data or explanation. I don't know how to 
evaluation this.

pr

-- 
Pete Resnick<http://www.qualcomm.com/~presnick/>
Qualcomm Technologies, Inc. - +1 (858)651-4478

---

From: Pete Resnick <[email-masked]>
Date: Mon, 19 Aug 2013 15:55:11 -0500

My apologies: A typo rendering a sentence incoherent that I missed 
before hitting "Send":

On 8/19/13 3:48 PM, Pete Resnick wrote:
> I think you're going to need substantially more explanation (and 
> perhaps some data) to make a convincing case that RFC 6686 needs to be 

Of course, I meant, "I don't know how to *evaluate* this."

pr

---

From: Dave Crocker <[email-masked]>
Date: Mon, 19 Aug 2013 13:59:39 -0700

> Of course, I meant, "I don't know how to *evaluate* this."

 From earlier exchanges about this concern, the assertion that I recall 
is that 7 years is not long enough, to determine whether a feature will 
be adopted.  That is, failure to gain deployment traction after 7 years 
from the time of publication should not be taken as a sufficient waiting 
period.

I do not recall anyone (else) showing support for that view, but 
certainly not any substantial constituency.

d/

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 19 Aug 2013 14:04:10 -0700

On Mon, Aug 19, 2013 at 1:59 PM, Dave Crocker <[email-masked]> wrote:

> From earlier exchanges about this concern, the assertion that I recall is
> that 7 years is not long enough, to determine whether a feature will be
> adopted.  That is, failure to gain deployment traction after 7 years from
> the time of publication should not be taken as a sufficient waiting period.
>
> I do not recall anyone (else) showing support for that view, but certainly
> not any substantial constituency.
>
>
Moreover:

What is the premise for seven years being "not long enough"?  And what does
constitute "long enough"?  And upon what is that last answer based?

It would be wonderful if the boundaries for this test were written down
somewhere, so that we would've had that information when we did the
research for RFC6686.

-MSK

---

From: Dave Crocker <[email-masked]>
Date: Mon, 19 Aug 2013 14:06:07 -0700

On 8/19/2013 2:04 PM, Murray S. Kucherawy wrote:
> Moreover:
>
> What is the premise for seven years being "not long enough"?  And what
> does constitute "long enough"?  And upon what is that last answer based?
>

Well, they were written down 15 years ago, but they haven't gained 
traction yet, though I remain hopeful.

d/

---

From: Hector Santos <[email-masked]>
Date: Mon, 19 Aug 2013 17:10:27 -0400

Hi,

I'm having a hard time with both sides of the argument, especially the 
supposed existence of an "interop problem" which seems to only highlight 
how to "procedurally" stump the SPF type advocates with a "error 
correction" standpoint.   What is that error by the way?

I don't believe there was an adequate answer from the advocates of 
removing the SPF RR type and the repeated assertion that its been 
discussed at length has not been convincing it was appropriately 
addressed. It all seem to be a "Shut up" approach to the problem (always 
suggest that its been discussed already). This seems to be one of the 
reasons why the issue will not go away.

My take is that the initial MARID design expectations were being met. 
This fact is a very important design consideration in all this; the 
original expectation for a TXT/SPF migration, although very slowly, 
there were some deployments administratively (publishers) and 
technically (processors). I would like to point out the lack of a 
majority does not/should not represent a lack of interest.

In my estimation, the WG did receive positive support expressing sincere 
technical interest in adding support for the SPF record type primarily 
because there was some level of adoption discovered during the "interop" 
report work.  Please include us (SSI) as having interest in 
enabling/adding SPF record support.

I recommend the following to be considered to basically allow 
implementators to decide:

1) Continue with the TXT/SPF migration plan, fix up this plan,

2) Address any technical protocol issue with using an SPF type,

3) Add implementation designs notes on the pros and cons.

This will allow coders to add the optimized logic for usage.  It will 
also allow for new problem solving seeds to be laid down. It will 
hopefully get the DNS software vendors to finally add direct support for 
unnamed TYPE support (query and passthru) not only for SPF but for 
future DNS application protocols.

Finally, which is what I have been trying to get - why hasn't the IETF 
taking this issue (unnamed type support) very serious?

This is an integration DNS issue - not just an SPF issue. If the DNS 
vendors don't address this, this, to me, would be the only logical and 
feasible reason to remove the SPF type or even bother with any new RR 
type concept.  The interop problem cited to exist within RFC 4408 is not 
convincing to me.

Overall, as an early adopter, the only reason my package doesn't have 
SPF support was purely for short term optimizations and lack of servers 
with unnamed type processing support - no mas.   If we remove SPF 
support, then we lost it forever.  I don't think we have viewed a 
possible long term solution from an integrated protocol standpoint.

--
HLS

---

From: Pete Resnick <[email-masked]>
Date: Mon, 19 Aug 2013 16:47:55 -0500

I will let the document shepherd/editor address particular points in 
this and other messages, but on one procedural point:

On 8/19/13 4:10 PM, Hector Santos wrote:
> removing the SPF RR type...

That's an appropriate issue to raise during Last Call, and I expect the 
shepherd to elaborate on why the WG came to its conclusion, and you to 
follow up with more explanation if you still think it is inadequate. 
However:

> ...and the repeated assertion that its been discussed at length has 
> not been convincing it was appropriately addressed. It all seem to be 
> a "Shut up" approach to the problem (always suggest that its been 
> discussed already). This seems to be one of the reasons why the issue 
> will not go away. 

The above is *not* appropriate to raise. The first part is attributing 
motives to folks, which is out of line. Even if the motives of folks 
asserting these things *were* malicious, I am expected to call the 
consensus on the basis of the technical arguments and to ignore the 
motives, and that is what I plan to do. And the last sentence above is 
trying to divine the psychological state of the IETF as to why or why 
not it continues to discuss an issue, an equally inappropriate and 
unproductive thing to do during Last Call.

Let's stick to issues and not delve into these areas please.

pr

---

From: S Moonesamy <[email-masked]>
Date: Mon, 19 Aug 2013 16:42:42 -0700

Hi Hector,
At 14:10 19-08-2013, Hector Santos wrote:
>I'm having a hard time with both sides of the argument, especially 
>the supposed existence of an "interop problem" which seems to only 
>highlight how to "procedurally" stump the SPF type advocates with a 
>"error correction" standpoint.   What is that error by the way?

In a message dated February 27, 2012, the SPFBIS Chairs commented 
that the discussion about Issue #9 (SPF RRTYPE) has revealed an 
interoperability concern in the existing RFC (4408).

 From RFC 6686:

   "RFC 4408 itself included a faulty transition plan, likely because of
    the late addition of a requirement to develop one -- it said:

       An SPF-compliant domain name SHOULD have SPF records of both RR
       types.  A compliant domain name MUST have a record of at least
       one type.

    which means both can claim to be fully compliant while failing
    utterly to interoperate."

The consensus of the SPFBIS WG was that this is an interoperability 
issue and it would have to be corrected.  That is what was considered 
as an error correction.

>addressed. It all seem to be a "Shut up" approach to the problem 
>(always suggest that its been discussed already). This seems to be 
>one of the reasons why the issue will not go away.

I personally do not think that it is appropriate to ask any working 
group participant to "shut up".  I welcome hearing arguments and I 
expect a working group to carefully consider them.

Regards,
S. Moonesamy (as document shepherd)

---

From: Ted Lemon <[email-masked]>
Date: Tue, 20 Aug 2013 00:16:01 +0000

On Aug 19, 2013, at 5:10 PM, Hector Santos <[email-masked]> wrote:
> This will allow coders to add the optimized logic for usage.  It will also allow for new problem solving seeds to be laid down. It will hopefully get the DNS software vendors to finally add direct support for unnamed TYPE support (query and passthru) not only for SPF but for future DNS application protocols.

It would help if you would go review the working group discussion on this point; if you had done so, I do not believe you could make this point with a straight face, because it was soundly refuted by the working group.   I tried to make the same point to the working group, and they successfully argued me around.   Although I'm sure you're aware that I am quite stubborn, it's possible that they would not succeed in arguing you around as they did me.  But you ought at least to do us the service of allowing them to try by reviewing the discussion yourself.

---

From: Måns Nilsson <[email-masked]>
Date: Tue, 20 Aug 2013 05:01:52 +0200

> expect a working group to carefully consider them.

The repeated assertions of "This has been discussed already" are in effect
"shut up", but slightly more polite. I complied until last call. As was
recommended by wg chairs.

-- 
Måns Nilsson     primary/secondary/besserwisser/machina
MN-1334-RIPE                             +46 705 989668
Yes, but will I see the EASTER BUNNY in skintight leather at an IRON
MAIDEN concert?

---

From: S Moonesamy <[email-masked]>
Date: Mon, 19 Aug 2013 22:12:18 -0700

>preferred while maintaining support for TXT.

I read the SPFBIS charter ( 
https://datatracker.ietf.org/wg/spfbis/charter/ ) 
again.  The charter violation the above may be referring might be about:

  "Specifically out-of-scope for this working group:

   * Removal of existing features that are in current use."

There is a message from the Responsible Area 
Director at 
http://www.ietf.org/mail-archive/web/spfbis/current/msg02167.html 
which might shine some light about that part of the charter.

Both RR Type 16 and RR Type 99 are in use on the 
Internet.  Tony Hansen mentioned during the IETF 83 session that:

   "when you write a protocol, there is definite harm if there is a
    choice in what you publish and what you look for.  He is of
    the view that there is a definite bug in RFC 4408."

Fixing that bug falls within the scope of the SPFBIS charter, specifically:

   "Changes to the SPF specification will be limited to the correction
    of errors, removal of unused features, addition of any enhancements
    that have already gained widespread support, and addition of
    clarifying language. "

The fourth paragraph (see quoted text) states 
that the conclusions from that where RFC 6866 
were flawed and rushed.  The explanation given is 
because the working group declared failure too 
soon.  The SPFBIS WG discovered a bug during the 
review of the SPF specification.  One of the main 
considerations for the decision was to fix the 
bug.  The working group had to choose one RRTYPE 
as the conclusion was that it was the appropriate 
way to fix the bug and ensure interoperability.

The comments posted up to now for the Last Call 
points to a disagreement about the choice of RRTYPE.

Regards,

---

From: Dave Crocker <[email-masked]>
Date: Mon, 19 Aug 2013 22:16:40 -0700

> recommended by wg chairs.

There is a fundamental difference between telling someone to shut up and 
asking the person to familiarize themselves with the group's history 
before speaking up.

It's a question of who is being required to carry the primary burden for 
making their case.  In the current circumstance, the burden is on the 
person lodging the concern, namely you, rather than on the group.

The alternative is, effectively, a denial of service attack on the 
group's progress, by requiring that it be (potentially constantly) 
required to re-visit old and resolved topics.

Since there is, in fact, an extensive -- possibly even complete -- 
public archive of discussions, placing the burden on the person lodging 
the concern is singularly reasonable.

As others have attempted to point out, this reduces to requiring that 
someone raising a concern during the last call start with the group's 
context and, therefore, either point out what the group missed or 
explain how it assessed reality incorrectly or otherwise made a 
problematic engineering choice.

d/

---

From: Hector Santos <[email-masked]>
Date: Tue, 20 Aug 2013 09:30:13 -0400

On 8/19/2013 7:42 PM, S Moonesamy wrote:>
> issue and it would have to be corrected.  That is what was considered as
> an error correction.

I have a few questions and points:

May I ask why was the above was not an area for clarification as oppose 
as the presumed stated technical reason for removal?

There was adequate information for the expected and original optimal 
migration plan but it could of been further codified and clarified. It 
would of been on par with BIS level of work.  Issue #9 should not have 
been a reason for removal.  I reported issue #25 [1] regarding the 
complexity of the recommended parallel processing approach.  I believe 
most folks agreed the ideal and optimal migration approach was:

     Query SPF type first,
     Fallback to TXT secord.

It was common sense, at least to me.

Second, I was under the impression interop reports (RFC 6686) were not 
making any conclusions or recommendations?  Is that a correct basic 
understanding of interop reports?  They were observations, collection of 
available data and while it might be eventually used to rethink a 
protocol design, I don't think the above RFC 4408 statement is a serious 
"error" in the functional description to justify removal. There are 
other parts of 4408 which helped clarify the migration path.

But overall, a correction (not removal) would of suffice. It would of 
been on par for BIS-like corrections and protocol updates.

Third, I believe removal required a more deeper IETF discussion about 
the initial presumed engineering expectation that DNS servers (all top 
DNS servers, including and especially Microsoft DNS servers) would 
eventually directly support a new registered SPF type or at the very 
least support RFC 3597 (Handling of Unknown DNS Resource Record (RR) 
Type) [2].

If this is no longer the expectation, then it would make sense to remove 
the SPF type but also be aware that in general, this will also  nix (not 
help) any future idea for successfully adopting new RR types.
It would be added statement that TXT based applications are acceptable. 
  I think the DNS community continues to have a problem with this.

>> (always suggest that its been discussed already). This seems to be one
>> of the reasons why the issue will not go away.
>

SM, Pete, thank you for the opportunity to clarify my point. For the 
record, there was no intent to imply it occurred but quite frankly when 
it is repeatedly stated, this deeply divided issue has not be resolved 
at any point,  it does have an intimidation factor.  As Mr. Crocker 
stated, the burden is on the those who oppose the removal. But my 
question was always why was the decision made to remove in the first 
place done when in fact it was quite obvious it would not have industry 
wide endorsement. The burden should of been to justify removal. Now it 
has become difficult to effectively add it back. This is why I posed the 
question in two forums to get community input over the last few years. 
It was quite obvious to me that the DNS community would be against this 
removal.  So in this vain, it was prematurely removed in the WG without 
early IETF/IESG/DNS concerns adequately dealt with.  Unfortunately, we 
were advise to raise the issue again in LC, but by that point, all the 
IETF procedural moves were made making it it a very high burden to add 
something that should not been removed in the first place.

The only reason our own early adoption package did not support the SPF 
type was because we could not; the Microsoft DNS server (NT 4.0 at the 
time) had not directly support the SPF type but more importantly it did 
not support RFC 3597.  Unfortunately, not even MS DNS 5.0 which did add 
direct support for other new RR types (SRV, DNSSEC, AAAA, etc).

In my engineering opinion, the implications of the SPF type removal are 
wide spread as it implies a no confidence in the future of DNS servers 
to support safe, transparent passthru, recursive queries of unnamed types.

If such is the case with the IETF DNS community,  then it makes 
engineering sense to remove the SPF type but also keep in mind new RR 
types are also unrealistic as well. All future DNS DOMAIN policy based 
applications will be TXT only based which may be the acceptable reality 
today.  It wasn't yester-years (during MARID).  The endorsement of DKIM 
as an IS specification is testament of this TXT based solution 
acceptability in the market place.

To me, that would be the reason for removal. Otherwise, if we still have 
confidence in the future of RFC 3597 support in DNS servers across the 
board out of the box, then we should correct and clarify the SPF/TXT 
query method migration path in RFC 4408-bis.

As stated a number of times in the SPF WG, I would support SPF type in 
our implementation once Microsoft DNS server supported at the very least 
unnamed RR type processing.  It can't be provisional software, i.e. 
custom, that would not cut it. It needs to be out of the box for wide 
network query support.

Thanks

--
HLS

[1] http://www.ietf.org/mail-archive/web/spfbis/current/msg00586.html
[2] http://tools.ietf.org/html/rfc3597

---

From: Hector Santos <[email-masked]>
Date: Tue, 20 Aug 2013 10:16:27 -0400

On 8/20/2013 1:12 AM, S Moonesamy wrote:

> There is a message from the Responsible Area Director at
> http://www.ietf.org/mail-archive/web/spfbis/current/msg02167.html which
> might shine some light about that part of the charter.
>
> Both RR Type 16 and RR Type 99 are in use on the Internet.  Tony Hansen
> mentioned during the IETF 83 session that:
>

This doesn't seem to be correct. My apology if I don't follow or see the 
logic.   The only real issue as it was since day zero in MARID was the 
infrastructure support for recursive passthru queries which is what RFC 
3597 (Handling of Unknown DNS Resource Record (RR) Type).  Without this, 
which allows for servers to delay/plan direct new RR type support yet 
still not error out on an unnamed type, there COULD NOT be any 
reasonable expectation to even only suggest a dual query migration 
approach, either in serial or parallel. It is very important to consider 
the mindset when it was first considered and written for RFC 4408 
because to me, that is the mindset that has changed.

If we don't think the infrastructure, the DNS vendors primarily, will 
support RFC 3497, then it seems proper to me to finally forget the idea.

But if we still think its desirable and possible, then I would prefer to 
keep the protocol feature/option, corrected of course, and allow 
implementers the design choice to support it.

The way I see it, if more implementations did begin to add support of a 
clarified BIS specification, that doesn't mean they may enabled it 
(default ON) out of the box.  I would want to initially offer the lowest 
overheard operational setup out of the box.  It will still be a long 
term migration path, shorter if we can encourage the major DNS vendors 
to add at least RFC 3597 support.  Without, no new RR type will work 
across the board.

> The fourth paragraph (see quoted text) states that the conclusions from
> that where RFC 6866 were flawed and rushed.  The explanation given is
> because the working group declared failure too soon.  The SPFBIS WG
> discovered a bug during the review of the SPF specification.  One of the
> main considerations for the decision was to fix the bug.  The working
> group had to choose one RRTYPE as the conclusion was that it was the
> appropriate way to fix the bug and ensure interoperability.
>
> The comments posted up to now for the Last Call points to a disagreement
> about the choice of RRTYPE.

I hope I am reading this incorrectly.  It may be too procedural oriented 
to me at this point.

I would like to see a simple just distinctive consensus on what the 
entire IETF/DNS community believes is the future of DNS servers offering 
unnamed RR type processing because that is the main barrier for any kind 
success. We knew this as far back as MARID.  I don't quite understand 
why this key point seems to be left out, instead MARID was deemed off 
topic. That was an error because if there is no future in unnamed RR 
type support, then it really doesn't matter and the problem is solved. 
TXT only will be the only fast entry point for new DNS DOMAIN policy 
applications.

If the community still believes it possible, then clarifying and 
codifying the SPF/TYPE lookup procedure seems to me to be the only real 
correction to make here.

Thanks

--
HLS

[1] http://tools.ietf.org/html/rfc3597

---

From: Andrew Sullivan <[email-masked]>
Date: Tue, 20 Aug 2013 10:26:37 -0400

This seems like business for the spfbis list, not the ietf list, so
that's where I'm replying.  I am writing my impressions as one
co-chair who had to make a consensus determination, but I have not
consulted with SM on these remarks so they should be considered mine
alone.

On Tue, Aug 20, 2013 at 09:30:13AM -0400, Hector Santos wrote:
> the complexity of the recommended parallel processing approach.  I
> believe most folks agreed the ideal and optimal migration approach
> was:
> 
>     Query SPF type first,
>     Fallback to TXT secord.
> 

The argument at the time -- and this is prominent in the archives, I
note -- was that the additional query for RRTYPE 99 was a waste, given
that nobody seemed to be using it.  Moreover, the above strategy
incorporates additional complexity for all SPF software.  Given that
some change to the protocol was required anyway, and given that the
TXT record couldn't seriously be deprecated when the overwhelming
majority of actual deployment used it, the WG concluded that the
additional complexity and additional load on the network for no
apparent benefit might as well be removed.  I am aware that you didn't
agree with that conclusion (and, frankly, if I had my druthers, we'd
have come to a different conclusion; but I was persuaded by the
evidence).  I nevertheless came to the conclusion that this was rough
consensus, as did some others who seemed to have a clue about the DNS.

> Third, I believe removal required a more deeper IETF discussion
> about the initial presumed engineering expectation that DNS servers
> (all top DNS servers, including and especially Microsoft DNS
> servers) would eventually directly support a new registered SPF type
> or at the very least support RFC 3597 (Handling of Unknown DNS
> Resource Record (RR) Type) [2].

There was, as near as I can tell, never any argument that most or even
an important minority of DNS servers were incapable of supporting
RRTYPE 99.  The argument was, instead, that many provisioning tools
don't have such support, and are unlikely to get it any time soon.
That position seemed to me to be pretty strong, so I'd be interested
if you have counterarguments.

> It would be added statement that TXT based applications are

I do not think that this draft sets any kind of precedent, because as
a general strategy the IETF has clearly settled on scoped TXT records
using underscore labels.  There are in fact problems with them, but
they're nothing compared to the way SPF treads on the namespace.
So I'm not sure why this seems to be an issue.

Best,

A

-- 
Andrew Sullivan
[email-masked]

---

From: S Moonesamy <[email-masked]>
Date: Tue, 20 Aug 2013 10:02:57 -0700

Hi Hector,
At 06:30 20-08-2013, Hector Santos wrote:
>May I ask why was the above was not an area for clarification as 
>oppose as the presumed stated technical reason for removal?

The SPFBIS WG had a session at IETF 83.  The minutes for that session 
is at 
http://www.ietf.org/proceedings/83/minutes/minutes-83-spfbis.txt 
Item C on the agenda is about "SPF RR Type and TXT RR (issue 
#9)".  Andrew Sullivan, as SPFBIS Chair, explained that:

   'The were people on the mailing list in favor of using the TXT RR
    appeared to be a modest majority and there were people who argued
    for SPF RR Type on the grounds of DNS hygiene.  The Chair explained
    that it appears as an empirical matter, overwhelmingly, the TXT RR
    is used but RRTYPE 99 does not qualify under the unused provision
    of the SPFBIS charter.'

Pete Resnick, as Area Director, commented that:

   'the deal with the IESG, as a general statement, the working can removed
    what is unused and put in what is widely deployed. He pointed out that
    saying that TXT RR is part of an experiment is contrary to the agreement
    made with the IESG.  His opinion is that either way, the proposal is stuck
    with TXT RR and that it is not an issue.  The only issue is whether the
    proposal can remove the SPF RRTYPE as unused.'

On February 26, 2012, Barry Leiba, as a participant, posted a message 
( http://www.ietf.org/mail-archive/web/spfbis/current/msg00654.html ) 
which I will quote:

   "These two statements make a pretty compelling case that there is, indeed,
    an error in RFC4408.  There are, of course, multiple ways to resolve it,
    and I have no immediate opinion on which is best."

My opinion, as one of the SPFBIS WG Chairs, was that there was an 
error.  As Barry Leiba mentioned, there were several possible ways to 
fix that.  The SPFBIS Chair stated at the meeting that:

   "The conclusion reached by the Chair was that the document will say
    SHOULD NOT publish RRTYPE 99 and MUST NOT query RRTYPE 99."

That statement was posted to the SPFBIS mailing list ( 
http://www.ietf.org/mail-archive/web/spfbis/current/msg01369.html 
).  Nobody disagreed.

>migration plan but it could of been further codified and clarified. 
>It would of been on par with BIS level of work.  Issue #9 should not 
>have been a reason for removal.  I reported issue #25 [1] regarding 
>believe most folks agreed the ideal and optimal migration approach was:
>
>     Query SPF type first,
>     Fallback to TXT secord.
>

Both Issue #9 and Issue #25 (see SPFBIS tracker)  were 
well-discussed.  Was every technical point carefully considered?  I 
don't think so as it is possible that a technical point may have been 
missed.  There is an opportunity for anyone to comment during the 
Last Call if the person believes that a technical point has been 
missed or was not addressed appropriately by the working group.  In 
my opinion the SPFBIS WG carefully considered all the comments which 
were made in reaching its decision.

>Second, I was under the impression interop reports (RFC 6686) were 
>not making any conclusions or recommendations?  Is that a correct 
>basic understanding of interop reports?  They were observations, 
>collection of available data and while it might be eventually used 
>to rethink a protocol design, I don't think the above RFC 4408 
>statement is a serious "error" in the functional description to 
>justify removal. There are other parts of 4408 which helped clarify 
>the migration path.

 From the Introduction Section of RFC 6686:

   "In line with the IESG's request to evaluate after a period of time,
    this document concludes the experiments by presenting evidence
    regarding both deployment and comparative effect of the two
    protocols.  At the end, it presents conclusions based on the data
    collected."

In my opinion RFC 6686 does make conclusions (see Section 6 of RFC 
6686).  There is some background information about the RRTYPE issue 
in Appendix A.

>But overall, a correction (not removal) would of suffice. It would 
>of been on par for BIS-like corrections and protocol updates.

Andrew Sullivan, as SPFBIS WG Chair, mentioned that "We have to do 
_something_, though any action would introduce a backward 
incompatibility ( 
http://www.ietf.org/mail-archive/web/spfbis/current/msg03595.html ).

>Resource Record (RR) Type) [2].

There were comments about RFC 3567 during the working group 
discussions (see 
http://www.ietf.org/mail-archive/web/spfbis/current/msg00545.html 
http://www.ietf.org/mail-archive/web/spfbis/current/msg00820.html ).

>If this is no longer the expectation, then it would make sense to 
>remove the SPF type but also be aware that in general, this will 
>also  nix (not help) any future idea for successfully adopting new RR types.
>It would be added statement that TXT based applications are 
>acceptable.  I think the DNS community continues to have a problem with this.

Noted.

>record, there was no intent to imply it occurred but quite frankly 
>when it is repeatedly stated, this deeply divided issue has not be 
>resolved at any point,  it does have an intimidation factor.  As Mr. 
>Crocker stated, the burden is on the those who oppose the removal. 
>But my question was always why was the decision made to remove in 
>the first place done when in fact it was quite obvious it would not 
>have industry wide endorsement. The burden should of been to justify 
>removal. Now it has become difficult to effectively add it back. 
>This is why I posed the question in two forums to get community 
>input over the last few years. It was quite obvious to me that the 
>DNS community would be against this removal.  So in this vain, it 
>was prematurely removed in the WG without early IETF/IESG/DNS 
>concerns adequately dealt with.  Unfortunately, we were advise to 
>raise the issue again in LC, but by that point, all the IETF 
>procedural moves were made making it it a very high burden to add 

There are two SPFBIS WG Chairs.  The way we work is: I read the 
mailing list messages, the other Chair reads the mailing list 
messages, each chair reaches his own conclusion and we discuss about 
it.  Once the Chairs agree a message is posted to the WG mailing 
list.  Let's assume that the Chairs did not carefully consider the 
comments or their determination was incorrect.  There is a Last Call 
where the matter can be raised again.  That is the advice that was 
given.  There is indeed a high burden as the person raising the issue 
again will have to read the mailing list messages to determine 
whether they have been carefully considered.

Regards,

---

From: S Moonesamy <[email-masked]>
Date: Tue, 20 Aug 2013 10:30:41 -0700

Hi Hector,
At 07:16 20-08-2013, Hector Santos wrote:
>This doesn't seem to be correct. My apology if I don't follow or see 
>the logic.   The only real issue as it was since day zero in MARID 
>was the infrastructure support for recursive passthru queries which 
>is what RFC 3597 (Handling of Unknown DNS Resource Record (RR) 
>Type).  Without this, which allows for servers to delay/plan direct 
>new RR type support yet still not error out on an unnamed type, 
>there COULD NOT be any reasonable expectation to even only suggest a 
>dual query migration approach, either in serial or parallel. It is 
>very important to consider the mindset when it was first considered 
>and written for RFC 4408 because to me, that is the mindset that has changed.

This would be a process issue then as the SPFBIS Chairs determination 
was made on the basic of the text from the SPFBIS Charter which was 
cited.  A person can raise an objection (I am okay with that as 
that's part of the work) with substantive comments to support the objection.

>I hope I am reading this incorrectly.  It may be too procedural 
>oriented to me at this point.
>
>entire IETF/DNS community believes is the future of DNS servers 
>offering unnamed RR type processing because that is the main barrier 
>for any kind success. We knew this as far back as MARID.  I don't 
>quite understand why this key point seems to be left out, instead 
>MARID was deemed off topic. That was an error because if there is no 
>future in unnamed RR type support, then it really doesn't matter and 
>the problem is solved. TXT only will be the only fast entry point 
>for new DNS DOMAIN policy applications.
>
>codifying the SPF/TYPE lookup procedure seems to me to be the only 
>real correction to make here.

My reading of  the SPFBIS Charter is that the working group was not 
tasked to work on the future of DNS servers.  That does not mean that 
arguments about the future of DNS servers are not relevant.

There are several questions:

  (a) Was there an error in RFC 4408?

  (b) What was the error in RFC 4408?

  (c) Why was there an error in RFC 4408?

  (d) What should be done about the error?

There isn't anything that can be done about question (c) except not 
to repeat the same mistake.

Is there disagreement on the answers to questions (a) and (b)?

Regards,

---

From: Måns Nilsson <[email-masked]>
Date: Wed, 21 Aug 2013 12:00:56 +0200

Subject: Re: [spfbis] Last Call: <draft-ietf-spfbis-4408bis-19.txt> (Sender Policy?Framework (SPF) for Authorizing Use of Domains in Email, Version 1) to Proposed Standard Date: Tue, Aug 20, 2013 at 10:30:41AM -0700 Quoting S Moonesamy ([email-masked]
 
> tasked to work on the future of DNS servers.  That does not mean
> that arguments about the future of DNS servers are not relevant.

The codification of a pessimistic view of the ability of the DNS installed
base to adapt to new RRtypes is - in itself - a statement that influences
the future of DNS. That this was not completely understood in terms of
influence weight is one of the reasons for this debate.
 
>  (a) Was there an error in RFC 4408?

Yes. 
 
>  (b) What was the error in RFC 4408?

The TXT rrtype was not properly decommissioned; it lacked definitiveness,
and neither publication instructions nor selection/query algorithm
satisfy this (originally intended, I suppose and believe) sunset view
of the TXT record rôle vis à vis SPF.

>  (d) What should be done about the error?

4408bis needs a better defined depreciation statement for TXT,
accompanied by publication and  query methods that will ensure the
continous phasing-out of SPF data in TXT records.

A clarification here will help in making DNS UI vendors doing the right
thing. I'm quite confident that presently, the UI vendors sleep soundly
after reading 4408 and continuing to ignore SPF/99. That is not 
desirable. 
 
> Is there disagreement on the answers to questions (a) and (b)?

Apparently. 

-- 
Måns Nilsson     primary/secondary/besserwisser/machina
MN-1334-RIPE                             +46 705 989668
I'm using my X-RAY VISION to obtain a rare glimpse of the INNER
WORKINGS of this POTATO!!

---

From: S Moonesamy <[email-masked]>
Date: Wed, 21 Aug 2013 19:20:11 -0700

At 09:22 21-08-2013, Ted Lemon wrote:
>I am curious if the folks who did the analysis of query rates know 
>the answers to the following questions:
>
>1. Per unit of mail delivered (as opposed to per domain), for what 
>percentage of delivered mail for which SPF TXT records exist do SPF 
>RRtype records _also_ exist?   I wasn't clear on whether an attempt 
>was made to come up with an answer to this question.
>
>2. Per unit of mail received, for what percentage of received mail 
>does the receiver currently issue SPF RRtype queries.
>
>The reason I ask these questions is that the rationale for the 
>decision made by the working group was that the data supported it, 
>and I think that was a good rationale, but only if the data 
>_actually_ supports it.   But I don't think that the data was 
>analyzed on the basis of units of mail delivered, but rather on the 
>basis of number of queries seen.

I don't recall whether the SPFBIS WG considering the angle mentioned 
above.  It would help if someone could provide some answers to the 
above questions.

Regards,

---

From: Dave Crocker <[email-masked]>
Date: Wed, 21 Aug 2013 19:25:48 -0700

On 8/21/2013 7:20 PM, S Moonesamy wrote:
> above.  It would help if someone could provide some answers to the above
> questions.

I suggest, instead, that Ted review the research that the wg already did 
and if he believes it inadequate he should explain how so, what is 
needed beyond it, and why it is essential to expend the added effort and 
time.

d/

---

From: "John Levine" <[email-masked]>
Date: 22 Aug 2013 03:02:40 -0000

>>does the receiver currently issue SPF RRtype queries.

I'm not aware of any statistics, and it would be a challenge to
collect any meaningful ones since there's no straightforward way to
get information about any mailstream other than your own, but it's not
hard to make some estimates.

The SMTP mail world has a few giants and a very long tail.  The giants
are Google, Yahoo, and Microsoft, each of which does mail under their
own familiar names, and also provides outsourced mail for huge numbers
of other people.  Then there's some pretty big mail systems like
Comcast, AOL, at&t, and GMX (mail.com and a lot of other private and
vanity domains.)  Then there's a very long tail of about 100,000 other
mail servers.  The number of domains vastly exceeds the number of
servers since most mail servers handle mail for many domains.

None of the big three publish or check type 99 records, and I'm not
aware of anyone in the next tier who does either.  If you weighted the
numbers by mail volume, most if not all of the systems that publish
and check type 99 are out in the tail, so the percentage would be
considerably less than you'd get from just counting domains.

This is just an estimate, but it's an educated one based on a lot of
experience so if anyone disagrees, please provide data.

R's,
John

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 21 Aug 2013 23:24:55 -0400

> above.  It would help if someone could provide some answers to the

I don't think it's either possible or useful to answer those questions.

The key point is we needed to pick a common mandatory to publish/mandatory to 
receive RRTYPE.  I cannot imagine any way to look at the data that would make 
it reasonable to pick RRTYPE SPF.

Scott K

---

From: Ted Lemon <[email-masked]>
Date: Thu, 22 Aug 2013 03:28:57 +0000

On Aug 21, 2013, at 7:25 PM, Dave Crocker <[email-masked]> wrote:
> I suggest, instead, that Ted review the research that the wg already did and if he believes it inadequate he should explain how so, what is needed beyond it, and why it is essential to expend the added effort and time.

I did review the research back when we had the discussion during the last call (or possibly before), and I am fairly sure that the research I am asking about did not in fact happen.   But my point, as I expressed to John earlier, was not to say that spfbis made a mistake, but rather that if people want to _assert_ that spfbis made a mistake, there's some obvious homework they should do before making that assertion.

---

From: Dave Crocker <[email-masked]>
Date: Wed, 21 Aug 2013 20:35:29 -0700

On 8/21/2013 8:28 PM, Ted Lemon wrote:
>> I suggest, instead, that Ted review the research that the wg
>> already did and if he believes it inadequate he should explain how
>> so, what is needed beyond it, and why it is essential to expend
>> the added effort and time.
>
> I did review the research back when we had the discussion during the
> last call (or possibly before), and I am fairly sure that the
> research I am asking about did not in fact happen.   But my point,
> as I expressed to John earlier, was not to say that spfbis made a
> mistake, but rather that if people want to _assert_ that spfbis made
> a mistake, there's some obvious homework they should do before
> making that assertion.

On that I entirely agree.

To wit, my point is not whether the specifics you are suggesting were
done. There is an infinite number of possible things to research for an
activity.  We could go on for a long time inventing things to study.

So the first requirement when finding an existing set of work deficient
is to carefully explain what the deficiency is and how the deficiency
affects might affect decisions that were based on it.

If there is then a recommendation for additional research, it needs to
be explained both in its nature and how it will remedy the deficiencies,
in terms of having a better basis for decision-making.

d/

---

From: Ted Lemon <[email-masked]>
Date: Thu, 22 Aug 2013 03:57:41 +0000

> in terms of having a better basis for decision-making.

If you can figure out who is triggering most of the queries, and get a sufficiency of them to install SPF RRtype RRs, then you can make the argument that spfbis ought not to deprecate the SPF RRtype.   If you can't, then your argument is pretty empty.   E.g., we haven't just been sitting on our hands waiting for someone to adopt IPv6—we've been actively working to make it happen.

But we didn't see that kind of effort with the SPF RRtype, either during the years it's been deployed prior to proposing that it be deprecated, nor in response to the proposal that it be deprecated.   Which suggests to me that the people who don't want it to be deprecated aren't serious about it—they are trying to win an argument, not change the world.

This is a perfectly understandable position, because in fact it just isn't worth the kind of effort that would be needed.   And my point is that the proponents of keeping the SPF RRtype, de facto, agree with this.   I count myself in that number—all things being equal, I would prefer that the SPF RRtype _not_ be deprecated.   But it's not worth my time to do what it would take to get a different outcome, because it's just not that important, and we have better things to do.

Anyone who disagrees with what I just said should evidence that by working furiously to get SPF RRtype adoption going with at least the big three, not by castigating spfbis for being pragmatic.

---

From: Dave Crocker <[email-masked]>
Date: Wed, 21 Aug 2013 21:16:38 -0700

On 8/21/2013 8:57 PM, Ted Lemon wrote:
>> If there is then a recommendation for additional research, it needs
>> to be explained both in its nature and how it will remedy the
>> deficiencies, in terms of having a better basis for
>> decision-making.
>
> If you can figure out who is triggering most of the queries, and get
> a sufficiency of them to install SPF RRtype RRs, then you can make
> the argument that spfbis ought not to deprecate the SPF RRtype.   If
> you can't, then your argument is pretty empty.   E.g., we haven't
> just been sitting on our hands waiting for someone to adopt
> IPv6—we've been actively working to make it happen.

By 'you' I hope you don't mean me.  The IETF does not promote adoption 
of its technologies.  Folks in industry do that.

However whether they do it or not does not affect the /criteria/ for 
success.  Adoption is the measure, not whether the technology was well 
marketed.  If the community did a poor job of marketing and got poor 
results, that is its own measure of the technology's worth.

And the need for such marketing efforts varies. For example, there was 
essentially no marketing effort for MIME and yet it garnered very large 
market adoption very quickly.

By reasonable measures, especially given the high levels of industry 
marketing effort so far, the very low adoption of of DNSSec and IPv6 
actually warrant calling them failures (so far.)

In any event, anti-abuse industry trade groups, like M3AAWG and APWG, 
have ongoing discussions about de jure and de facto standards, like DKIM 
and SPF.  So there's been plenty of community review of the technologies 
by potential adopters.  If they chose not to implement a port of a spec, 
it was an informed choice.

> Anyone who disagrees with what I just said should evidence that by
> working furiously to get SPF RRtype adoption going with at least the
> big three, not by castigating spfbis for being pragmatic.

They should have done that over the last seven years.  Not now.

d/

---

From: Mark Andrews <[email-masked]>
Date: Thu, 22 Aug 2013 17:03:58 +1000

In message <[email-masked]>, T
ed Lemon writes:
> If you can figure out who is triggering most of the queries, and get a
> sufficiency of them to install SPF RRtype RRs, then you can make the
> argument that spfbis ought not to deprecate the SPF RRtype.   If you
> can't, then your argument is pretty empty.   E.g., we haven't just been
> sitting on our hands waiting for someone to adopt IPv6-we've been
> actively working to make it happen.
>
> But we didn't see that kind of effort with the SPF RRtype, either during
> the years it's been deployed prior to proposing that it be deprecated,
> nor in response to the proposal that it be deprecated.   Which suggests
> to me that the people who don't want it to be deprecated aren't serious
> about it-they are trying to win an argument, not change the world.

Well there has been.  Named complains about there not being both
SPF and TXT records present.  I've got code in the BIND review queue
to answer SPF queries by looking at the TXT records for unsigned
zones if there is no SPF record (see below).  One can argue about
whether to make the synthesised SPF records identifyable or not.
The process is very similar to DNS64 synthesis.

That aside why did there have to be until Microsoft fully supports
SPF / unknown records?  They are slowly moving in that direction.
Once that happens would be the time to push the people to publish
SPF records in ernest, until that happens you just need to deploy
code that supports both.

Additionally please point out the text in RFC 4408 that says there
was a deadline to the migration.  I saw RFC 4408 as testing whether
the *contents* of the record worked or not.  It was never as far as
I was concerned a test of whether one could migrate to SPF or not
quickly.

If I had know that there would be a deadline imposed on migration
then I would have added lots of other code to BIND to push the
process along.

> This is a perfectly understandable position, because in fact it just
> isn't worth the kind of effort that would be needed.   And my point is
> that the proponents of keeping the SPF RRtype, de facto, agree with this.
> I count myself in that number-all things being equal, I would prefer
> that the SPF RRtype _not_ be deprecated.   But it's not worth my time to
> do what it would take to get a different outcome, because it's just not
> that important, and we have better things to do.
>
> working furiously to get SPF RRtype adoption going with at least the big
> three, not by castigating spfbis for being pragmatic.
>

diff --git a/bin/named/include/named/query.h b/bin/named/include/named/query.h
index a14d516..88fbdd6 100644
--- a/bin/named/include/named/query.h
+++ b/bin/named/include/named/query.h
@@ -67,6 +67,8 @@ struct ns_query {
 	unsigned int			dns64_aaaaoklen;
 	unsigned int			dns64_options;
 	unsigned int			dns64_ttl;
+	dns_rdataset_t *		spf;
+	dns_rdataset_t *		spfsig;
 };
 
 #define NS_QUERYATTR_RECURSIONOK	0x0001
diff --git a/bin/named/include/named/server.h b/bin/named/include/named/server.h
index b586f5c..50571a6 100644
--- a/bin/named/include/named/server.h
+++ b/bin/named/include/named/server.h
@@ -174,7 +174,9 @@ enum {
 
 	dns_nsstatscounter_rpz_rewrites = 40,
 
-	dns_nsstatscounter_max = 41
+	dns_nsstatscounter_spf = 41,
+
+	dns_nsstatscounter_max = 42
 };
 
 void
diff --git a/bin/named/query.c b/bin/named/query.c
index 6ba5093..194b294 100644
--- a/bin/named/query.c
+++ b/bin/named/query.c
@@ -359,6 +359,11 @@ query_reset(ns_client_t *client, isc_boolean_t everything) {
 		client->query.dns64_aaaaoklen =  0;
 	}
 
+	if (client->query.spf != NULL)
+		query_putrdataset(client, &client->query.spf);
+	if (client->query.spfsig != NULL)
+		query_putrdataset(client, &client->query.spfsig);
+
 	query_freefreeversions(client, everything);
 
 	for (dbuf = ISC_LIST_HEAD(client->query.namebufs);
@@ -623,6 +628,8 @@ ns_query_init(ns_client_t *client) {
 	client->query.dns64_sigaaaa = NULL;
 	client->query.dns64_aaaaok = NULL;
 	client->query.dns64_aaaaoklen = 0;
+	client->query.spf = NULL;
+	client->query.spfsig = NULL;
 	query_reset(client, ISC_FALSE);
 	result = query_newdbversion(client, 3);
 	if (result != ISC_R_SUCCESS) {
@@ -2182,6 +2189,170 @@ query_addrdataset(ns_client_t *client, dns_name_t *fname,
 	CTRACE("query_addrdataset: done");
 }
 
+static isc_boolean_t
+isspf(const dns_rdata_t *rdata) {
+        char buf[1024];
+        const unsigned char *data = rdata->data;
+        unsigned int rdl = rdata->length, i = 0, tl, len;
+
+        while (rdl > 0U) {
+                len = tl = *data;
+                ++data;
+                --rdl;
+                INSIST(tl <= rdl);
+                if (len > sizeof(buf) - i - 1)
+                        len = sizeof(buf) - i - 1;
+                memcpy(buf + i, data, len);
+                i += len;
+                data += tl;
+                rdl -= tl;
+        }
+
+        if (i < 6U)
+                return(ISC_FALSE);
+
+        buf[i] = 0;
+        if (strncasecmp(buf, "v=spf1", 6) == 0 &&
+	    (buf[6] == 0 || buf[6] == ' '))
+                return (ISC_TRUE);
+        return (ISC_FALSE);
+}
+
+static isc_result_t
+query_spf(ns_client_t *client, dns_name_t **namep, dns_rdataset_t *rdataset,
+	  isc_buffer_t *dbuf)
+{
+	dns_name_t *name, *mname;
+	dns_rdata_t *spf_rdata;
+	dns_rdatalist_t *spf_rdatalist;
+	dns_rdataset_t *spf_rdataset;
+	dns_rdataset_t *mrdataset;
+	isc_buffer_t *buffer;
+	isc_region_t r;
+	isc_result_t result;
+	const char syn[] = " comment=synthesised";
+
+	CTRACE("query_spf");
+	name = *namep;
+	mname = NULL;
+	mrdataset = NULL;
+	buffer = NULL;
+	spf_rdata = NULL;
+	spf_rdataset = NULL;
+	spf_rdatalist = NULL;
+	result = dns_message_findname(client->message, DNS_SECTION_ANSWER,
+				      name, dns_rdatatype_spf,
+				      rdataset->covers,
+				      &mname, &mrdataset);
+	if (result == ISC_R_SUCCESS) {
+		/*
+		 * We've already got an RRset of the given name and type.
+		 * There's nothing else to do;
+		 */
+		CTRACE("query_spf: dns_message_findname succeeded: done");
+		if (dbuf != NULL)
+			query_releasename(client, namep);
+		return (ISC_R_SUCCESS);
+	} else if (result == DNS_R_NXDOMAIN) {
+		/*
+		 * The name doesn't exist.
+		 */
+		if (dbuf != NULL)
+			query_keepname(client, name, dbuf);
+		dns_message_addname(client->message, name, DNS_SECTION_ANSWER);
+		*namep = NULL;
+		mname = name;
+	} else {
+		RUNTIME_CHECK(result == DNS_R_NXRRSET);
+		if (dbuf != NULL)
+			query_releasename(client, namep);
+	}
+
+	if (rdataset->trust != dns_trust_secure)
+		client->query.attributes &= ~NS_QUERYATTR_SECURE;
+
+	result = dns_message_gettemprdataset(client->message, &spf_rdataset);
+	if (result != ISC_R_SUCCESS)
+		goto cleanup;
+	result = dns_message_gettemprdatalist(client->message,
+					      &spf_rdatalist);
+	if (result != ISC_R_SUCCESS)
+		goto cleanup;
+
+	dns_rdataset_init(spf_rdataset);
+	dns_rdatalist_init(spf_rdatalist);
+	spf_rdatalist->rdclass = dns_rdataclass_in;
+	spf_rdatalist->type = dns_rdatatype_spf;
+	spf_rdatalist->ttl = rdataset->ttl;
+
+	for (result = dns_rdataset_first(rdataset);
+	     result == ISC_R_SUCCESS;
+	     result = dns_rdataset_next(rdataset)) {
+		dns_rdata_t rdata = DNS_RDATA_INIT;
+		dns_rdataset_current(rdataset, &rdata);
+		if (!isspf(&rdata))
+			continue;
+		result = isc_buffer_allocate(client->mctx, &buffer,
+					     rdata.length + sizeof(syn));
+		if (result != ISC_R_SUCCESS)
+			goto cleanup;
+		result = dns_message_gettemprdata(client->message, &spf_rdata);
+		if (result != ISC_R_SUCCESS)
+			goto cleanup;
+		dns_rdata_init(spf_rdata);
+		isc__buffer_putmem(buffer, rdata.data, rdata.length);
+		isc_buffer_putuint8(buffer, sizeof(syn) - 1);
+		isc_buffer_putstr(buffer, syn);
+		isc__buffer_usedregion(buffer, &r);
+		dns_rdata_fromregion(spf_rdata, dns_rdataclass_in,
+				     dns_rdatatype_spf, &r);
+		ISC_LIST_APPEND(spf_rdatalist->rdata, spf_rdata, link);
+		spf_rdata = NULL;
+		dns_message_takebuffer(client->message, &buffer);
+	}
+	if (result != ISC_R_NOMORE)
+		goto cleanup;
+
+	if (ISC_LIST_EMPTY(spf_rdatalist->rdata))
+		goto cleanup;
+
+	result = dns_rdatalist_tordataset(spf_rdatalist, spf_rdataset);
+	if (result != ISC_R_SUCCESS)
+		goto cleanup;
+	client->query.attributes |= NS_QUERYATTR_NOADDITIONAL;
+	spf_rdataset->trust = rdataset->trust;
+	query_addrdataset(client, mname, spf_rdataset);
+	spf_rdataset = NULL;
+	spf_rdatalist = NULL;
+	inc_stats(client, dns_nsstatscounter_spf);
+	result = ISC_R_SUCCESS;
+
+ cleanup:
+	if (buffer != NULL)
+		isc_buffer_free(&buffer);
+
+	if (spf_rdata != NULL)
+		dns_message_puttemprdata(client->message, &spf_rdata);
+
+	if (spf_rdataset != NULL)
+		dns_message_puttemprdataset(client->message, &spf_rdataset);
+
+	if (spf_rdatalist != NULL) {
+		for (spf_rdata = ISC_LIST_HEAD(spf_rdatalist->rdata);
+		     spf_rdata != NULL;
+		     spf_rdata = ISC_LIST_HEAD(spf_rdatalist->rdata))
+		{
+			ISC_LIST_UNLINK(spf_rdatalist->rdata,
+					spf_rdata, link);
+			dns_message_puttemprdata(client->message, &spf_rdata);
+		}
+		dns_message_puttemprdatalist(client->message, &spf_rdatalist);
+	}
+
+	CTRACE("query_spf: done");
+	return (result);
+}
+
 static isc_result_t
 query_dns64(ns_client_t *client, dns_name_t **namep, dns_rdataset_t *rdataset,
 	    dns_rdataset_t *sigrdataset, isc_buffer_t *dbuf,
@@ -5993,7 +6164,7 @@ query_find(ns_client_t *client, dns_fetchevent_t *event, dns_rdatatype_t qtype)
 	dns_rpz_st_t *rpz_st;
 	isc_boolean_t resuming;
 	int line = -1;
-	isc_boolean_t dns64_exclude, dns64;
+	isc_boolean_t dns64_exclude, dns64, spf;
 	dns_clientinfomethods_t cm;
 	dns_clientinfo_t ci;
 
@@ -6026,6 +6197,7 @@ query_find(ns_client_t *client, dns_fetchevent_t *event, dns_rdatatype_t qtype)
 	resuming = ISC_FALSE;
 	is_zone = ISC_FALSE;
 	is_staticstub_zone = ISC_FALSE;
+	spf = ISC_FALSE;
 
 	dns_clientinfomethods_init(&cm, ns_client_sourceip);
 	dns_clientinfo_init(&ci, client);
@@ -6879,10 +7051,37 @@ query_find(ns_client_t *client, dns_fetchevent_t *event, dns_rdatatype_t qtype)
 	iszone_nxrrset:
 		INSIST(is_zone);
 
+		if (spf) {
+			/*
+			 * Restore the answers from the previous SPF lookup.
+			 */
+			if (rdataset != NULL)
+				query_putrdataset(client, &rdataset);
+			if (sigrdataset != NULL)
+				query_putrdataset(client, &sigrdataset);
+			rdataset = client->query.spf;
+			sigrdataset = client->query.spfsig;
+			client->query.spf = NULL;
+			client->query.spfsig = NULL;
+			if (fname == NULL) {
+				dbuf = query_getnamebuf(client);
+				if (dbuf == NULL) {
+					QUERY_ERROR(DNS_R_SERVFAIL);
+					goto cleanup;
+				}
+				fname = query_newname(client, dbuf, &b);
+				if (fname == NULL) {
+					QUERY_ERROR(DNS_R_SERVFAIL);
+					goto cleanup;
+				}
+			}
+			dns_name_copy(client->query.qname, fname, NULL);
+			spf = ISC_FALSE;
+		} else
 #ifdef dns64_bis_return_excluded_addresses
-		if (dns64)
+		        if (dns64)
 #else
-		if (dns64 && !dns64_exclude)
+			if (dns64 && !dns64_exclude)
 #endif
 		{
 			/*
@@ -6947,6 +7146,34 @@ query_find(ns_client_t *client, dns_fetchevent_t *event, dns_rdatatype_t qtype)
 			}
 			dns64 = ISC_TRUE;
 			goto db_find;
+		} else if (result == DNS_R_NXRRSET &&
+			   client->message->rdclass == dns_rdataclass_in &&
+			   qtype == dns_rdatatype_spf &&
+			   !dns_db_issecure(db)) {
+			/*
+			 * Look to see if there are TXT records for this
+			 * name.
+			 */
+			INSIST(client->query.spf == NULL);
+			INSIST(client->query.spfsig == NULL);
+			client->query.spf = rdataset;
+			client->query.spfsig = sigrdataset;
+			query_releasename(client, &fname);
+			dns_db_detachnode(db, &node);
+			rdataset = NULL;
+			sigrdataset = NULL;
+			type = qtype = dns_rdatatype_txt;
+			rpz_st = client->query.rpz_st;
+			if (rpz_st != NULL) {
+				/*
+				 * Arrange for RPZ rewriting of any A records.
+				 */
+				if ((rpz_st->state & DNS_RPZ_REWRITTEN) != 0)
+					is_zone = rpz_st->q.is_zone;
+				rpz_st_clear(client);
+			}
+			spf = ISC_TRUE;
+			goto db_find;
 		}
 
 		/*
@@ -7762,7 +7989,18 @@ query_find(ns_client_t *client, dns_fetchevent_t *event, dns_rdatatype_t qtype)
 		    dns_name_equal(client->query.qname, dns_rootname))
 			client->query.attributes &= ~NS_QUERYATTR_NOADDITIONAL;
 
-		if (dns64) {
+		if (spf) {
+			qtype = type = dns_rdatatype_spf;
+			result = query_spf(client, &fname, rdataset, dbuf);
+			dns_rdataset_disassociate(rdataset);
+			dns_message_puttemprdataset(client->message, &rdataset);
+			if (result == ISC_R_NOMORE) {
+				goto iszone_nxrrset;
+			} else if (result != ISC_R_SUCCESS) {
+				eresult = result;
+				goto cleanup;
+			}
+		} else if (dns64) {
 			qtype = type = dns_rdatatype_aaaa;
 			result = query_dns64(client, &fname, rdataset,
 					     sigrdataset, dbuf,
diff --git a/bin/named/statschannel.c b/bin/named/statschannel.c
index d3c74f9..8573f74 100644
--- a/bin/named/statschannel.c
+++ b/bin/named/statschannel.c
@@ -217,6 +217,7 @@ init_desc(void) {
 		       "RateSlipped");
 	SET_NSSTATDESC(rpz_rewrites, "response policy zone rewrites",
 		       "RPZRewrites");
+	SET_NSSTATDESC(spf, "synthesised SPF responses", "SynSPF");
 	INSIST(i == dns_nsstatscounter_max);
 
 	/* Initialize resolver statistics */
diff --git a/bin/tests/system/spf/clean.sh b/bin/tests/system/spf/clean.sh
index 2e3a0ee..5810930 100644
--- a/bin/tests/system/spf/clean.sh
+++ b/bin/tests/system/spf/clean.sh
@@ -14,3 +14,4 @@
 
 rm -f ns1/named.run
 rm -f ns1/named.memstats
+rm -f dig.out.test*
diff --git a/bin/tests/system/spf/ns1/spf.db b/bin/tests/system/spf/ns1/spf.db
index ffa850a..7956e5f 100644
--- a/bin/tests/system/spf/ns1/spf.db
+++ b/bin/tests/system/spf/ns1/spf.db
@@ -14,8 +14,9 @@
 
 @ 0 IN SOA . . 0 0 0 0 0
 @ 0 IN NS .
-@ 0 IN TXT "v=spf1 -all"
-@ 0 IN SPF "v=spf1 -all"
-x 0 IN TXT "v=spf1"
-y 0 IN SPF "v=spf1"
+@ 0 IN TXT "v=spf1 -all comment=txt"
+@ 0 IN SPF "v=spf1 -all comment=spf"
+x 0 IN TXT "v=spf1 comment=txt-only"
+y 0 IN SPF "v=spf1 comment=spf-only"
 y 0 IN TXT "a non spf record"
+z 0 IN TXT "a non spf record"
diff --git a/bin/tests/system/spf/tests.sh b/bin/tests/system/spf/tests.sh
index 6acd283..494a657 100644
--- a/bin/tests/system/spf/tests.sh
+++ b/bin/tests/system/spf/tests.sh
@@ -41,5 +41,44 @@ n=`expr $n + 1`
 if [ $ret != 0 ]; then echo "I:failed"; fi
 status=`expr $status + $ret`
 
+echo "I:checking that the SPF record is returned if there is both SPF and v=spf1 TXT records present ($n)"
+ret=0
+$DIG -p 5300 @10.53.0.1 -t spf -q spf > dig.out.test$n
+grep "ANSWER: 1" dig.out.test$n > /dev/null || ret=1
+grep "comment=spf" dig.out.test$n > /dev/null || ret=1
+grep "status: NOERROR" dig.out.test$n > /dev/null || ret=1
+n=`expr $n + 1`
+if [ $ret != 0 ]; then echo "I:failed"; fi
+status=`expr $status + $ret`
+
+echo "I:checking that we synthesis a SPF record if there is a v=spf1 TXT record ($n)"
+ret=0
+$DIG -p 5300 @10.53.0.1 -t spf -q x.spf > dig.out.test$n
+grep "ANSWER: 1" dig.out.test$n > /dev/null || ret=1
+grep "comment=txt-only" dig.out.test$n > /dev/null || ret=1
+grep "status: NOERROR" dig.out.test$n > /dev/null || ret=1
+n=`expr $n + 1`
+if [ $ret != 0 ]; then echo "I:failed"; fi
+status=`expr $status + $ret`
+
+echo "I:checking that a lone SPF record at a node is returned ($n)"
+ret=0
+$DIG -p 5300 @10.53.0.1 -t spf -q y.spf > dig.out.test$n
+grep "ANSWER: 1" dig.out.test$n > /dev/null || ret=1
+grep "comment=spf-only" dig.out.test$n > /dev/null || ret=1
+grep "status: NOERROR" dig.out.test$n > /dev/null || ret=1
+n=`expr $n + 1`
+if [ $ret != 0 ]; then echo "I:failed"; fi
+status=`expr $status + $ret`
+
+echo "I:checking that a SPF record is not synthesised if there are no v=spf1 TXT records ($n)"
+ret=0
+$DIG -p 5300 @10.53.0.1 -t spf -q z.spf > dig.out.test$n
+grep "ANSWER: 0" dig.out.test$n > /dev/null || ret=1
+grep "status: NOERROR" dig.out.test$n > /dev/null || ret=1
+n=`expr $n + 1`
+if [ $ret != 0 ]; then echo "I:failed"; fi
+status=`expr $status + $ret`
+
 echo "I:exit status: $status"
 exit $status
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]

---

From: Ted Lemon <[email-masked]>
Date: Thu, 22 Aug 2013 15:03:01 +0000

On Aug 22, 2013, at 12:03 AM, Mark Andrews <[email-masked]> wrote:
> SPF and TXT records present.

I mean advocacy, not a warning message from a daemon that nobody with any power to do anything sees.   While Dave points out that the IETF doesn't do advocacy, IETF participants certainly do as part of their work.   Nobody has done any advocacy on this point.   If you care so deeply about it, why haven't you?    Or is it the case that you have, and it hasn't worked?

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 22 Aug 2013 14:42:19 -0400

Ted Lemon <[email-masked]> wrote:
>I mean advocacy, not a warning message from a daemon that nobody with
>any power to do anything sees.   While Dave points out that the IETF
>doesn't do advocacy, IETF participants certainly do as part of their
>work.   Nobody has done any advocacy on this point.   If you care so
>deeply about it, why haven't you?    Or is it the case that you have,
>and it hasn't worked?

It's not correct that nobody has done any advocacy.  

The major FOSS SPF libraries all support type SPF and have done so since shortly after it was allocated (or in the case of Mail::SPF it's initial release).  This was all done as free time work.  No one was paid for it.

I have run what I have reason to believe is the most popular web based SPF  validator.   Since very shortly after the type was allocated it has checked for it and included in its results. You've got no idea how many e-mails I get from people that are confused about it.  I also did this in my 'spare' time.

You may have noted FAQ entries on openspf.org presenting type SPF as the right thing to do.  I wrote that.

We tried.  It didn't work out for a variety of reasons that I won't rehash, but it's not like we didn't try. What I didn't do was engage in intellectually dishonest advocacy to push people to favor type SPF over TXT knowing it would work less well for them.

Scott K

---

From: Hector Santos <[email-masked]>
Date: Thu, 22 Aug 2013 15:20:02 -0400

Dishonest?  Ouch!

I don't think I was dishonest in my engineering agreements with others 
that new RR types should be used and there was really no reason to 
remove it first and make it a burden on the rest of the community to 
add it back.

I won't call this dishonest, but the burden should of been on you to 
remove first by asking the entire interfacing community first, not 
just with WG.  This was highly predictable, yet the WG went on to 
remove it without consulting the related IETF groups.  I understand 
that is the current procedure, but this decision does not provide the 
design confidence that new RR types are still valid to consider. If 
you believe they still are, then in my opinion, the SPFBIS should not 
had removed it.

And lets just say that after all these discussions, someone writes a 
serious and accepted RFC addressed at the DNS industry, Levine is 
write a I-D for unnamed types, etc, and we begin to get big traction 
and finally the changes to the infrastructure begin, SPFBIS will be 
without its SPF type99 and that would be unfortunate.

I don't think that is dishonest.

--
HLS

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 22 Aug 2013 16:22:05 -0400

Hector,

Not everything is about you. 

Scott K

Hector Santos <[email-masked]> wrote:
>> The major FOSS SPF libraries all support type SPF and have done so
>since shortly after it was allocated (or in the case of Mail::SPF it's
>initial release).  This was all done as free time work.  No one was
>paid for it.
>> 
>> I have run what I have reason to believe is the most popular web
>based SPF  validator.   Since very shortly after the type was allocated
>it has checked for it and included in its results. You've got no idea
>how many e-mails I get from people that are confused about it.  I also
>did this in my 'spare' time.
>> 
>> We tried.  It didn't work out for a variety of reasons that I won't
>rehash, but it's not like we didn't try. What I didn't do was engage in
>intellectually dishonest advocacy to push people to favor type SPF over
>TXT knowing it would work less well for them.
>> 
>> Scott K

---

From: Mark Andrews <[email-masked]>
Date: Fri, 23 Aug 2013 06:23:14 +1000

In message <[email-masked]>, T
ed Lemon writes:
> I mean advocacy, not a warning message from a daemon that nobody with any
> power to do anything sees.   While Dave points out that the IETF doesn't
> do advocacy, IETF participants certainly do as part of their work.
> Nobody has done any advocacy on this point.   If you care so deeply about
> it, why haven't you?    Or is it the case that you have, and it hasn't
> worked?

We can make it refuse to load the zone without being fixed.  We can make
it a MUST for if there isn't a SPF record. i.e. you can only disable the
warning if you are missing the TXT version.

And by the way we issue this only on master servers which is where the
data is entered.

We prevent lots of other DNS issues this way for our users.

You can't load a zone without the address record for in zone servers being
present.

We block the loading of zones without required glue being present.

We block the loading of zones for other reason.

All these blocks are for master zones.  Slave servers don't perform the
checks.

SERVFAIL from the master server is a powerful tool.  You see it and you
have to fix it.

Mark

---

From: Hector Santos <[email-masked]>
Date: Thu, 22 Aug 2013 16:33:43 -0400

So who was dishonest? The group of SPF type advocates or a specific 
individual?

Nonetheless, maybe I am wrong, but its not cool to be labeling anyone 
dishonest.

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 22 Aug 2013 17:06:08 -0400

I didn't have anyone in particular in mind, but I think that writing in 
4408bis that one MUST use RRTYPE SPF and MAY/SHOULD use RRTYPE TXT is an 
intentionally suboptimiized design (I know others disagree).  I think putting 
that in an RFC would not be intellectually honest.  Someone reading such a 
spec might go and implement just the SPF part of the spec and then wonder why 
it seems like SPF usage is so much lower than they'd heard.

Note that I'm not accusing anyone of not telling the truth.  Relax.  It's fine.

Scott K 

On Thursday, August 22, 2013 16:33:43 Hector Santos wrote:
> >>>> I mean advocacy, not a warning message from a daemon that nobody
> >> 
> >> with
> >> 
> >>>> deeply about it, why haven't you?    Or is it the case that you
> >> 
> >> have,
> >> 
> >>>> and it hasn't worked?
> >>>

---

From: Ted Lemon <[email-masked]>
Date: Thu, 22 Aug 2013 21:43:31 +0000

> warning if you are missing the TXT version.

Who is this "we," Mark?  I'd be happy if ISC decided to do that—it would help Nominum to sell more servers!

---

From: Mark Andrews <[email-masked]>
Date: Fri, 23 Aug 2013 08:29:34 +1000

In message <[email-masked]>, T
ed Lemon writes:
> Who is this "we," Mark?  I'd be happy if ISC decided to do that=97it would =
> help Nominum to sell more servers!

We in this case would be a request from the IETF to the DNS nameserver
vendors.

Raising the default to fail rather than warn I think I can argue
that ISC to does alone.  That would just be making people think
about the "MUST publish both" that was in 4408.  At the moment they
really don't make that decision because most people are not aware
of the SPF RR type, they think TXT record.

If the SPF WG had wanted a fast migration they could have ask for
nameserver vendors to issue warning when SPF was not present or to
refuse to load unless configured otherwise.

Fail with no config file fix needs IETF backing.  The failure log
message would need to reference a RFC MUST.

As for nameservers not answering the IETF should ask ICANN to get
COM and all the other ICANN affiliated zones to audit delegated
nameservers for RFC 4035 Section 4.1.1 compliance with respect to
SPF records and are causing operational problems.  A nameserver
that fails to answer is out of spec.  The technical contacts for
the zones should be contacted to request a fix.

If after a month the servers are not fixed, the delegations should
be withdrawn.  Withdrawing delegation is the prescribed fix for
nameservers which are cause operation problem and are not being
fixed.

This is consistent with RFC 1033 COMPLAINTS.

We can play "Whack a Mole" or we can ask those that are supposed to
be managing the DNS to actually manage the DNS.

Mark

---

From: Ted Lemon <[email-masked]>
Date: Thu, 22 Aug 2013 22:35:42 +0000

> We in this case would be a request from the IETF to the DNS nameserver

How would "the IETF" make such a request?

---

From: Dave Crocker <[email-masked]>
Date: Thu, 22 Aug 2013 15:57:09 -0700

> We in this case would be a request from the IETF to the DNS nameserver

"The IETF" doesn't do such things.

d/

---

From: Mark Andrews <[email-masked]>
Date: Fri, 23 Aug 2013 09:00:11 +1000

In message <[email-masked]>, T
> How would "the IETF" make such a request?

The same way it makes all its other requests of vendors.  They get written
down in RFC format and it get ratified by a IETF last call.

Mark

---

From: Ted Lemon <[email-masked]>
Date: Thu, 22 Aug 2013 23:42:09 +0000

> down in RFC format and it get ratified by a IETF last call.

Okay, then we've already made that request, and it's been ignored.

---

From: Måns Nilsson <[email-masked]>
Date: Fri, 23 Aug 2013 01:44:00 +0200

Subject: Re: [spfbis] Last Call: <draft-ietf-spfbis-4408bis-19.txt> (Sender Policy Framework (SPF) for Authorizing Use of Domains in Email, Version 1) to Proposed Standard Date: Thu, Aug 22, 2013 at 03:57:09PM -0700 Quoting Dave Crocker ([email-masked]
> "The IETF" doesn't do such things.

As Mark explained in a later message, the IETF codifies behaviour by
using 2119 terms like MUST and SHOULD. 

Vendors and operators are from what I can tell eager to get what
amounts to a perceived IETF seal of approval by actually implementing
RFC specifications. It mostly works like that. (look, for instance, at
the archives for the payload wg, where vendors arrive in a steady stream
urging standardisation work in order to increase interoperability. )

Returning to the issue at hand: The fuzzy language in 4408, which
supposedly is to be clarified in 4408bis is a major contributor to the
mess we see today. As I've written before, this language (original 4408)
allows the vendors of DNS management systems to ignore SPF/99. That, as
far as I can tell (but I did not participate in getting 4408 published
and I have not read the archives; my interpretation is from reading 4408),
does not seem to have been the intention of the 4408 authors.

The charter of spfbis, feature removal notwithstanding, seems spot on to 
correct this language, and thus 4408bis should be directing implementors
to not ignore SPF/99 but instead implement it in name servers,
full-service resolvers, provisioning systems, and SPF validator suites.

Further, validators should actively prefer SPF rrtype in validation
queries. I believe that TXT will be with us for a long time, so validation
suites should be configurable to ask for TXT records. These configuration
options, though, must not allow for the disabling of SPF record lookup. 

Text on suitable TTL values for SPF records (regardless of rrtype used)
could perhaps be useful. A long TTL (24h+) would -- providing the resolver
chain is not completely brain-dead -- allow the validation data to be
cached and blocking behaviour associated with non-compliant middleboxes
avoided for repeated lookups.

-- 
Måns Nilsson     primary/secondary/besserwisser/machina
MN-1334-RIPE                             +46 705 989668
My vaseline is RUNNING...

---

From: Mark Andrews <[email-masked]>
Date: Fri, 23 Aug 2013 10:24:48 +1000

In message <[email-masked]>, T
ed Lemon writes:
> > The same way it makes all its other requests of vendors.  They get writte=
> n

Where were the words "Nameservers SHOULD enforce the presence of
SPF records"?

RFC 4408 Section 3.1 was directed to "Domain owners".

That said we do warn about it.  The warnings do get noticed. 

	https://lists.isc.org/pipermail/bind-users/2013-July/091176.html

And people notice the feature change.

	https://lists.isc.org/pipermail/bind-users/2013-May/090700.html

We have code that has passed technical review that will synthesize
SPF records when there TXT v=spf1 records.  There is still debate
about whether to commit it or not.  This would address part of the
web forms don't support SPF records.

One could add a recommendation that servers do such synthesis with
a sunset date so that there is some presure on getting the web
interfaces fixed.

Mark

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 22 Aug 2013 21:45:39 -0700

On Thu, Aug 22, 2013 at 3:29 PM, Mark Andrews <[email-masked]> wrote:

> [...]

Doesn't this approach presuppose rather a lot of cooperation from the DNS
vendors (i.e., just about all of them), and a pretty "keep totally current"
posture among pretty much all operators?  That seems like a high bar.

> the zones should be contacted to request a fix.

Why would ICANN want to get involved in this?

> be managing the DNS to actually manage the DNS.
Unfortunately, I don't think either the IETF or ICANN wield such hammers.

-MSK

---

From: Mark Andrews <[email-masked]>
Date: Fri, 23 Aug 2013 15:03:23 +1000

In message <[email-masked]>
> posture among pretty much all operators?  That seems like a high bar.

ISC added SPF support in the first .0 release after RFC 4408 was
published.  We had coded it before RFC 4408 was published.  You had
support from ISC and other DNS vendors.

> Why would ICANN want to get involved in this?

I think ICANN would prefer to not have to deal with this because
it is a thorny issue.  However they are responsible for establishing
the contracts gTLD registries and registrars work under.  It is
within their purview to ensure that checks like these get made.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 22 Aug 2013 22:49:22 -0700

> support from ISC and other DNS vendors.

You appear to have ignored the second part, which is a pretty strong
dependence that operators will upgrade quickly once this is out, and be
comfortable with this sort of thing forced upon them.

> within their purview to ensure that checks like these get made.

They have no such contracts with the ccTLDs, and as you said, they probably
don't want to get involved in this particular issue either way.

-MSK

---

From: Mark Andrews <[email-masked]>
Date: Fri, 23 Aug 2013 16:14:36 +1000

In message <[email-masked]>
> comfortable with this sort of thing forced upon them.

Lots of operators keep current.  The package maintainers help here
in that there are new packages available within days of a release
for *BSD and Linux.  We ship Windows binaries so they are available
when we do a release.  The appliance vendors that incorporate named
aren't far behind either.

Yes there is a long tail, but most operators are reasonably current.

---

From: Ángel <[email-masked]>
Date: Sat, 24 Aug 2013 00:00:32 +0200

> be managing the DNS to actually manage the DNS.

It's much easier than this, and can be accomplished by the community,
without a controversial top-down imposition by ICANN.

a) Create a "RRTYPE 99 wall of shame site", to be used as central place
for reports and checks of broken nameservers.

b) Attempt to reach out those dns administrators. This will need a few
people, but can be mostly automated, and given a proper FAQ at (a),
shouldn't be a problem.

After several months:
c) Start rejecting mails from domains whose nameservers don't support
SPF 99. Just to be explicit: not to return a spf record to the query
would be perfectly fine, but leaving the mail server to timeout would
not.
This is completely justified as a "rejection for policy reasons". And a
very reasonable one, given that the nameservers blocking the delivery
while the smtpd is waiting for a reply are a problem for mail
administrators.

In order to avoid false positives due to the occasionally dropped
packet, the following algorithm could be used:

i) Following the spf process, if the type 99 query times out, record the
domain and the current time at a local db.
ii) If there were at least 3 failed checks for this domain with the
earliest being no less than 24 hours ago, return a permanent error with
a message pointing to a page explaining the condition at (a) site.
iii) Else return a temporary failure.

Purge the records after one week, or when a spf reply is received.

You may insist on a broken nameserver which impacts MTAs, but in that
case we will refuse serving your emails.

I expect most sysadmins to happily update their nameservers to rfc3597
compliant ones, but c) ensures that domains with broken nameservers (due
to ignorance, malice, lazyness... or sudden death of the only one in the
company knowing what dns is) won't harm the email ecosystem (as well as
providing a sidechannel for alerting the dns admins).

---

From: Mark Andrews <[email-masked]>
Date: Sat, 24 Aug 2013 10:22:53 +1000

In message <[email-masked]>, =?ISO-8859-1?Q?=C1
> providing a sidechannel for alerting the dns admins).

That helps with legitimate email.  It doesn't help with spoofed
email using domains with addresses that don't send email and have
a non rfc compliant nameserver or a misconfigured nameserver.

---

From: Ángel <[email-masked]>
Date: Mon, 26 Aug 2013 22:58:46 +0200

> a non rfc compliant nameserver or a misconfigured nameserver.

Quite the opposite! For spoofed mail, bouncing them *is* the appropiate
step. You don't want to accept such mail as if it had no spf.
Plus, the spammers wouldn't be interested in spoofing such domains
(after they notice that leads to them being dropped).

It depends highly on volume, but it can be expected that such domain
pool would only shrink, so a "nightly blacklist" should end up covering
all of them.

---

From: Pete Resnick <[email-masked]>
Date: Tue, 27 Aug 2013 15:33:54 -0500

I probably should have sent out this message over the weekend, but I was 
hoping I would complete a bigger message soon. Since I'm still working 
on that, a quick note to level set:

I have been reading all of the Last Call responses as they have come in. 
I am in the process of reviewing the comments and making a summary of 
the issues and responses. I have asked SM as document shepherd to do the 
same. (Being far more diligent than I, SM has completed his review 
already and is waiting on me.) We will be comparing notes and posting a 
summary of the issues and how they were addressed, along with whether we 
think they are resolved. That will give folks who disagree with my 
assessment one last chance to say, "Pete, you completely misunderstood 
what my objection / response was saying and you should re-evaluate your 
consensus determination on this issue." After that all shakes out, I'll 
make the final call on consensus.

An important thing to note is that I am seeing scant little I would call 
a "new argument" since late last week, so I would suggest that you are 
probably not adding to my understanding of the consensus by continuing 
to post. I'd like to think that everyone would be able to notice that 
things have gotten repetitive, both the folks who are repeating things 
as well as the folks who think they have to keep answering, but there 
does seem to be a bit of "getting the last word in" going on. Doing so 
isn't likely to add to my understanding or sway my opinion of the 
consensus (since I'm making a list of independent issues and their 
answers, not counting posts), but it does make it take longer for me to 
get through my review. So I'd recommend posting judiciously, at least 
until you see my summary.

Thanks,

pr