Subject: Re: [spfbis] Last call of draft-ietf-spfbis-4408bis-19 - migration strategy

From: S Moonesamy <[email-masked]>
Date: Tue, 20 Aug 2013 22:43:16 -0700

At 11:58 20-08-2013, Patrik Fältström wrote:
>As the bottle is opened, I hereby state my 
>objection to Section 3.1 of 
>draft-ietf-spfbis-4408bis-19 for the reasons 
>explained below. I do agree (as stated below) 
>that the section of RFC 4408 that specify how to 
>use both SPF and TXT resource record types 
>include an error as it does not lead to interoperability.
>
>As the intention with use of both TXT and SPF 
>originally was to migrate from TXT to SPF I 
>instead of what is outlined in the draft suggest 
>that a proper migration strategy is laid out 
>(look up mandatory to implement SPF and fall 
>back to TXT). This instead of deprecation of the SPF record.
>
>In general I do believe, for example when 
>looking at IPv6 and DNSSEC and similar 
>technologies, that the lifetime of RFC 4408 is 
>too short to deprecate any of the proposed 
>records that are in use, specifically as RFC 
>4408 explicitly do allow use of both.

I posted the a message about "Revisiting past 
technical arguments where consensus was reached" 
(see SPFBIS Charter) in response to a comment 
about the RFC 4408 transition strategy ( 
http://www.ietf.org/mail-archive/web/spfbis/current/msg01558.html 
).  I have been reviewing some of the comments 
which have been made about the topic (see thread 
at 
http://www.ietf.org/mail-archive/web/spfbis/current/msg00651.html 
).  There is a comment from the Responsible Area 
Director ( http://www.ietf.org/mail-archive/web/dnsext/current/msg13064.html ):

   "This discussion should have happened at SPFBIS *chartering* time, as
    it is crystal clear from the charter that existing features currently
    in use in SPF are not going away. Indeed, the TXT record was specifically
    mentioned in the charter."

The Responsible Area Director also mentioned ( 
http://www.ietf.org/proceedings/83/minutes/minutes-83-spfbis.txt ) that:

   "The only issue is whether the proposal can remove the SPF RRTYPE
    as unused."

The decisions taken were to:

   1. Fix the error

   2. Not to put in a migration strategy

I suggest that the working group carefully 
consider whether there should be a migration 
strategy or not and whether there are substantive 
arguments to support the decision.

Regards,
S. Moonesamy (as document shepherd)

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 21 Aug 2013 02:12:59 -0400

On Tuesday, August 20, 2013 22:43:16 S Moonesamy wrote:
>     in use in SPF are not going away. Indeed, the TXT record was
> specifically mentioned in the charter."
> 

I'm a bit confused by this message.  Is this meant to be purely a recitation 
of past events or, in the last paragraph, are you asking us to take this up 
now?  I think we've done that already.

Scott K

---

From: S Moonesamy <[email-masked]>
Date: Wed, 21 Aug 2013 00:00:38 -0700

Hi Scott,
>now?  I think we've done that already.

I am asking the working group to review the discussion and comment 
about whether there are strong arguments to support the decision not 
to have a migration strategy.

Regards,

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 21 Aug 2013 03:32:12 -0400

On Wednesday, August 21, 2013 00:00:38 S Moonesamy wrote:
> >I'm a bit confused by this message.  Is this meant to be purely a
> >recitation of past events or, in the last paragraph, are you asking us to
> >take this up now?  I think we've done that already.
> 

There is no migration strategy.

The comparison to IPv6 in the last call discussion is almost apt.  This 
migration would be like IPv6 migration if IPv4 addresses weren't running out.  
In fact, not that good.

There are real operational issues experienced with trying to use type SPF.  
That's why, at least in the open source world, people are backing away from 
it.  Users complain about issues and there is an obvious fix.

It's impossible to write an RFC that captures the flavor of the suggested 
migration strategy without engaging in magical thinking or not caring about 
the installed base (which we're required by charter to do).  If we were to 
follow the approach of having 4408bis say something like:

 - MUST publish type SPF, should publish type TXT
 - MUST check type SPF, should check type TXT

Then a few things would happen:

 - We would have, on paper solved the interoperability bug of not having a 
common type mandated for publishing and verifications.

 - Many people would know were weren't really serious and continue using TXT

 - New implementers, without experience in the protocol would likely read the 
text and conclude that supporting type SPF was more important and they should 
code/publish that first.

 - These same new implementers would then be left wondering where are all they 
to other people doing SPF.  They might have read that SPF covered more than 
half of the email on the internet and wonder why their local statistics were 
substantially smaller (hint: it would be because 4408bis told them to deploy 
SPF in a way that's completely the opposite of interoperating with the 
installed base).

In case you're left in doubt, I think this would be a ridiculous situation.

One counter to the "more queries won't affect DNS, so who cares about doubling 
hte query load" is that for email processing it's not DNS load that drives 
performance, it's DNS latency.  Latency can hang the entire mail process.  In 
high volume operations, latency is what kills.  So even if the dual SPF/TXT 
transitional model has no real effect on DNS (not that I buy that) it's still a 
performance issue in the mail stack and it substantially complicated the 
code/design requirement.

I do wonder where these people who claim DNS query loads don't matter where 
when Doug Otis was busy claiming 111 queries potentially from an SPF check 
would melt the internet?

I would also suggest turning it around a bit.  Why would people migrate?  "It 
makes the DNS more pure" isn't something many administrators would find 
motivating.  As far as  I know there really aren't any operational incentives 
to migration.

That's my limit for 3:30AM (local).

Scott K

---

From: "John Levine" <[email-masked]>
Date: 21 Aug 2013 14:25:21 -0000

>There is no migration strategy.

I entirely agree with Scott's arguments.

Regardless of what one might think about the technical issues, there
is no way the very large base of SPF users with TXT records that work
just fine, and always will work just fine, is going to migrate
anywhere.

R's,
John

---

From: Dotzero <[email-masked]>
Date: Wed, 21 Aug 2013 10:31:33 -0400

+1 to Scott's arguments. We have had these discussions ad nauseum.

Mike

On Wed, Aug 21, 2013 at 10:25 AM, John Levine <[email-masked]> wrote:
> _______________________________________________
> spfbis mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/spfbis

---

From: Mark Andrews <[email-masked]>
Date: Thu, 22 Aug 2013 00:54:55 +1000

> just fine, and always will work just fine, is going to migrate

You get the nameserver vendors to add code look for TXT SPF records
and add SPF SPF records when they find them if there isn't a record
there when loading / updating a master zone.

I'll write the code for BIND even if I have to do it on my own time.
The only reason it isn't written completely at the moment is that
their isn't a document recommending that it be done.  It is trivial
to keep the records in sync as spf supports extensions that are
supposed to be ignored and we just tag the copy by adding a extra
string to the end e.g. " original=txt".  If we were copying from a
SPF record then it is " original=spf".

Named already checks for "SHOULD publish both" in 4408 and warns
when they are not both present.  When the message comes up on
bind-users about what does this message mean we point to the text
that says publish both and that is what usually happens.

Now can we please stop saying things are impossible when they are
not.  You just need to be a little proactive.

Mark
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]

---

From: "John R Levine" <[email-masked]>
Date: 21 Aug 2013 11:07:04 -0400

> there when loading / updating a master zone.

Believe it or not, you are not the first person to think of that hack.

I did it for a while, and found that it caused more problems than it 
solved.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
"I dropped the toothpaste", said Tom, crestfallenly.

---

From: Tim Draegen <[email-masked]>
Date: Wed, 21 Aug 2013 11:19:36 -0400

> +1 to Scott's arguments. We have had these discussions ad nauseum.

Also +1 to Scott's arguments.

I'm a bit of an odd soul, though, as I think all of the SPF vs TXT record discussion is simply wrong.  There should not be a "type SPF", as SPF records are just text strings.

If I had a magic wand, I'd wave it and change where SPF is discovered to be of the form "_spf.example.org", with TXT records serving out text spf records.  Problem solved!

=- Tim

---

From: Dave Crocker <[email-masked]>
Date: Wed, 21 Aug 2013 08:48:22 -0700

>   +1 to Scott's arguments. We have had these discussions ad nauseum.

+1

Working on migration is useful when the market has expressed interest in 
migration.  Absent that, it's an exercise in waste and delay.

And we are profoundly lacking in any market demonstration of needing to 
change from the current use of the TXT record for SPF, no matter how 
architecturally impure or operationally offensive it might seem.

Frankly, being required to pursue such a thing would mostly demonstrate 
the IETF as a denial of service attack on utility.

d/

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Mark Andrews <[email-masked]>
Date: Thu, 22 Aug 2013 09:49:26 +1000

> the IETF as a denial of service attack on utility.
 
Bumpkin.

We already have code that looks for both records.   We already have
support in nameservers and resolvers.  We can make authoritative
servers for the zone publish SPF when they see a TXT/v=spf1 record.

Where is the denial of service attack?

Mark

---

From: Mark Andrews <[email-masked]>
Date: Thu, 22 Aug 2013 10:02:07 +1000

In message <[email-masked]>, "John R Levine" wri
> I did it for a while, and found that it caused more problems than it 
 
With code to identify the original record or without?

I was thinking "with code to identify the original record".  The
SPF format supports extensions.  Adding " original=<type>", where
<type> is spf or text, to the end of the replicated record allows
for fully automated replication to be done.

Mark

---

From: "John Levine" <[email-masked]>
Date: 22 Aug 2013 00:40:35 -0000

>With code to identify the original record or without?

Without, but the problems had nothing to do with the contents of the
type 99 records.

R's,
John

---

From: Mark Andrews <[email-masked]>
Date: Thu, 22 Aug 2013 10:55:52 +1000

> Without, but the problems had nothing to do with the contents of the

Yet you don't enumerate the problems.

Did you modify the nameserver to make the changes?

When I say put it in the nameserver.  That covers loading master
zones from disk, updating the zone using UPDATE, fixing the external
tools that manipulate/check the zone the zone like dnssec-signzone,
named-checkzone and named-compilezone.  It also mean have the inline
signer code add the records.  This also means fix NSEC/NSEC3 records
add RRSIGs etc.

A simpler step would be to just refuse to serve the zone if SPF
records are not present when v=sfp1 TXT records are.  Not accept
updates that put the zone into this state.  Make it a concious
decision to break the SHOULD in RFC 4408.  I believe most people
break that SHOULD unintentionally.

Mark

---

From: "John R Levine" <[email-masked]>
Date: 21 Aug 2013 21:03:16 -0400

> Yet you don't enumerate the problems.

I was a while ago and I don't remember exactly what went wrong, but it 
was the same sort of stuff that Scott described.

> Did you modify the nameserver to make the changes?

No, of course not.  The input from my DNS server is generated by scripts 
from a database, and it was a three line tweak to one of the scripts.  The 
servers published type 99 just fine, the problems were elsewhere.

Regards,

---

From: Commerco WebMaster <[email-masked]>
Date: Wed, 21 Aug 2013 19:15:09 -0600

> Without, but the problems had nothing to do with the contents of the

Actually, I think some of the problems did have something to do with the 
contents of the SPF (type 99) RRs insofar as, if memory serves, there 
were concerns posted about the synchronization issue when both were 
supported by a DNS publisher (e.g., which do you choose to believe if 
they are different, etc).

The other major problem discussed, as I recall, was that abandoning TXT 
might cause some trouble with older hardware which did not have the 
ability to support or handle an SPF (type 99) RR.  That problem is at 
least partially addressed by the DNS changes that Mark Andrews 
mentioned.  Addressed, in that a modern version DNS server (resolver) 
will go check the SPF RR and, finding no answer, will go check the TXT 
RR.  Thus continuing to even breath life into devices which probably 
should get updated or be replaced and doing it without breaking things.

Best,

Alan M.

---

From: "John R Levine" <[email-masked]>
Date: 21 Aug 2013 21:20:33 -0400

> contents of the SPF (type 99) RRs insofar as, if memory serves, there were 
> concerns posted about the synchronization issue when both were supported by a 
> DNS publisher (e.g., which do you choose to believe if they are different, 
> etc).

In my case, it was various things in other places doing unfortunate things 
with type99 queries and/or replies.  It was a while ago, I don't remember 
the details.

Regards,

---

From: Tim Draegen <[email-masked]>
Date: Thu, 22 Aug 2013 11:16:10 -0400

On Aug 21, 2013, at 9:20 PM, "John R Levine" <[email-masked]> wrote:
>> Actually, I think some of the problems did have something to do with the contents of the SPF (type 99) RRs insofar as, if memory serves, there were concerns posted about the synchronization issue when both were supported by a DNS publisher (e.g., which do you choose to believe if they are different, etc).
> 
> In my case, it was various things in other places doing unfortunate things with type99 queries and/or replies.  It was a while ago, I don't remember the details.

I've already brought up my own experience long ago, but since we're rehashing this..

While implementing SPF checking for a high volume email server, it turned out everything worked nicely in a lab, but in field testing, there were enough DNS servers out there that just did nothing with queries of type SPF.  That is, timeouts had to be relied upon to supply a response.

Even with a huge pool of connections available to service SPF lookup requests, under volume eventually the entire pool would find itself in some state of waiting for a reply to a type 99 lookup.  This is after caching responses, caching if a DNS server would be known to stall, and blocking incoming connections at an IP-reputation level.

Just to roll SPF checking out, the fix was to make type 99 lookups optional and off by default.  Otherwise I would had to implement a bunch of hacks...   all of which added up to "why bother, the world is using TXT records anyway".

=- Tim

PS. I've written this before, but I think a type-specific DNS record for text-based SPF records is wrong.  If anything, migration to prefixing DNS labels using "_spf." is superior.  But this better approach is out of scope and we're all OK with this.

---

From: S Moonesamy <[email-masked]>
Date: Thu, 22 Aug 2013 07:37:22 -0700

Hi Tim,
At 08:19 21-08-2013, Tim Draegen wrote:
>If I had a magic wand, I'd wave it and change where SPF is 
>discovered to be of the form "_spf.example.org", with TXT records 
>serving out text spf records.  Problem solved!

The SPFBIS WG did not discuss the above alternative.  I will have to 
discuss with Andrew Sullivan to see whether it falls within the scope 
of the charter or not.

I suggest discussing the above alternative.

Regards,

---

From: Tim Draegen <[email-masked]>
Date: Thu, 22 Aug 2013 12:06:03 -0400

On Aug 22, 2013, at 10:37 AM, S Moonesamy <[email-masked]> wrote:
> The SPFBIS WG did not discuss the above alternative.  I will have to discuss with Andrew Sullivan to see whether it falls within the scope of the charter or not.
> 

I don't believe the above falls in scope.  From a clean-room development perspective (starting from scratch), I'd prefer using "_spf." prefix when it comes to "How to discover SPF data".

Given the scope of the charter and what already exists in terms of How to discover SPF data:

1. SPF TXT
2. SPF type-99

It's clear to me that #1 is the way to go, for all reasons already argued.

I guess my point is, even if the charter allowed us explore, from scratch, How to discover SPF data..  going the "SPF type-99" route wouldn't be on my list of viable schemes.

HTH,
=- Tim

---

From: "John Levine" <[email-masked]>
Date: 22 Aug 2013 17:09:51 -0000

>discuss with Andrew Sullivan to see whether it falls within the scope 

I'm not Andrew, but it clearly does not.  It's a change that is
incompatible both with the existing spec and with current practice.

Once again, if we were doing this from scratch, it wouldn't be an
unreasonable idea, but we aren't, so it's not.

R's,
John

---

From: Hector Santos <[email-masked]>
Date: Thu, 22 Aug 2013 13:12:04 -0400

S Moonesamy wrote:
>> If I had a magic wand, I'd wave it and change where SPF is discovered 
>> to be of the form "_spf.example.org", with TXT records serving out 
>> text spf records.  Problem solved!
> 
> discuss with Andrew Sullivan to see whether it falls within the scope of 
> the charter or not.
> 

This was discussed as I recall and/or it was obvious in the overhead 
concerns, or the similar overhead was raised in the floods of list 
discussions.

Idea:

     Query TXT at _SPF.example.com
     Fallback Query TXT at example.com

It would be promoting a migration path with an ideal expected short 
term high redundancy overhead in failures.  When you consider that 
some of the domains organize their SPF network with includes and 
redirects, we are talking about even more TXT calls.

I prefer a minimal amount, like one call.

Maybe what is possibly needed is a yet another TXT based discovery 
call to tell the world where all these TXT-based DNS application 
records are located.

--
HLS

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 22 Aug 2013 14:48:04 -0400

>I suggest discussing the above alternative.

We did discuss it both when RFC 4408 was written and in SPFbis.  It wasn't done for 4408 because, at the time, it was felt it would be a barrier to deployment since most provisioning systems didn't allow domain names with an underscore.  Now you can't mandate it without breaking backward compatibility. 

Scott K

---

From: Ángel <[email-masked]>
Date: Wed, 21 Aug 2013 21:49:12 +0200

I came late to the discussion, and thinking it closed wasn't going to
reopen it, but since it has sprung again,  I am adding my views on the
subject.

I have read the «Obsoleting SPF RRTYPE» thread (starting Apr 23) ands
its follow-ups (when obvious from its subject) as well as the «Issue #9»
threads of February 2012 (where contrary to what I expected from later
mails, I don't see consensus, btw. The thread finished with Feb and then
continued on June 30/Jul 01 just to word type 99 removal, with no
objections).

IMHO the solution should be pushed into the dns software.

* A spf implementation MUST perform one query for SPF RR to a resolver
which complies with the below algorithm.

* A dns resolver when receiving a query for SPF (99) recordset MUST:
a) Perform the usual resolution process with that record type. If
this yields a positive answer, return that. Skip the other steps.
b) If the answer was a timeout/No data/Not implemented [also 
include SERVFAIL?], retry the query changing the type from SPF (99) 
to TXT (16).
c) Filter out all the answers not beginning with v=spf1
d) If there's any answer left, return that
else return the spf null record.

* A dns authoritative server responsible for TXT and SPF records
that when loading its zone, finds a TXT record beginning with 
v=spf1 and there's no SPF record for that domain MUST store it 
with type SPF as it is a legacy record really intended for SPF 
usage (it is implementation-defined if the record is also published
with TXT type or not). It is recommended to emit a diagnostic when 
fixing the zone.

* A dns authoritative server that is asked for a SPF record but 
doesn't have one such record for that domain MUST return the spf 
null record, using as ttl the default negative-ttl for the domain.

The spf null record is defined as a record with TYPE SPF and content
"v=spf0". Its semantic

- The bigger TXT deployment base and provisioning tools not supporting
SPF arguments are fixed on next dns server upgrade.
- The resolver update means that the update of an intermediate caching 
server already improves the situation.
- An old authoritative server forcing the use of legacy TXT records will
look nicely to the internet as soon as you put a modern dns cache in
front of it.
- SPF records inside TXT on old dns servers continue working.

The only problem remaining is the passthru server/firewall
rejecting/dropping the unknown RRTYPE (aka. rfc 3597 uncompliance).
- If you are checking SPF records, you must have a resolver supporting
SPF with TXT fallback (which may be embedded with the libspf, or needed
to install separatedly if your normal resolver is one of the buggy ones)
and a clean network which isn't filtering out spf queries (else, talk
with your network admin).
- If you are publishing type 99 records but your outbound dns server /
firewall strips then, you need to have that fixed, it should be simple
to discover that you have problems when you use a spf validator on your
domain (because you are testing it, right?).

Yes, this is still a kludge, but a smaller one than doubling the dns
requests and storage, and moving in the right direction. (However, the
resolver changes might be skipped)
Unlike TXT, which is a shared space, the spf record allows for more
tweaks, and adding special code for it is not that different than if eg.
spfbis mandated that spf records generate additional section
processing. 

Regards