Subject: Re: [spfbis] #24: RFC 4408: Reasonable DNS error limits

From: Alessandro Vesely <[email-masked]>
Date: Fri, 18 Jan 2013 12:14:46 +0100

On Wed 07/Mar/2012 22:55:28 +0100 Scott Kitterman wrote:
> On Wednesday, March 07, 2012 09:36:11 PM Murray S. Kucherawy wrote:
>> 
>> If for example I post a policy of "v=spf1 mx -all", and then I advertise "n"
>> MXes, and each of those have "m" IP addresses, that gives me m x n IP
>> addresses to compare to the client address.
> 
> What RFC 4408 doesn't say (it's left to the reader to infer from paragraph 
> 10.1) is if you have more than 10 mx records, don't use an mx mechanism in 
> your SPF record.  We could be explicit about that.  There are ways around this 
> problem that don't require protocol changes.
> [...]
> Instead of using mx in an SPF record, publish records for another unique 
> hostname, e.g. mx.example.com, and a records from it pointing at all the 
> various mx* IP addresses:
> 
> $ dig a mx.example.com +short
> 
> mx.example.com     120     IN    A   192.0.2.2
> mx.example.com     120     IN    A   192.0.2.3
> mx.example.com     120     IN    A   192.0.2.4
> mx.example.com     120     IN    A   192.0.2.5
> mx.example.com     120     IN    A   192.0.2.6
> mx.example.com     120     IN    A   192.0.2.7
> mx.example.com     120     IN    A   192.0.2.8
> ...
> mx.example.com     120     IN    A   192.0.2.22

With those 21 addresses, the DNS record seems to get near to the UDP
packet size limit.  On a TCP connection, one can return a gabillizion
IPs from an A record lookup.  Assuming each IPv4 address increases the
record size by 16 bytes, +20% TCP overhead, at 100Mbps, I reckon a
whole /8 could get through before the 20-sec time limit elapses.

Is that worth an additional paragraph in Section 3.4 and 5.3?  I
attempt that in the attached patch.  Please comment on it.
--- draft-ietf-spfbis-4408bis-09.orig.xml	2013-01-18 12:13:12.000000000 +0100
+++ draft-ietf-spfbis-4408bis-09.xml	2013-01-18 12:13:12.000000000 +0100
@@ -509,6 +509,13 @@
             then DNS answers ought to fit in UDP packets.  Note that when
             computing the sizes for queries of the TXT format, one has to take
             into account any other TXT records published at the domain name.
+          </t>
+          <t>
+            DNS records of different types, when used for SPF purposes, undergo
+            a similar restriction.  For example, types A and AAAA as used in
+            the "a" mechanism (see <xref target="mech-a"/>).
+          </t>
+          <t>
             Records that are too long to fit in a single UDP packet could be
             silently ignored by SPF verifiers due to firewall and other issues
             that cause DNS over TCP to be less reliable than DNS over UDP.
@@ -1059,6 +1066,12 @@
           &lt;ip&gt; is compared to the returned address(es). If any address
           matches, the mechanism matches. 
         </t>
+        <t>
+          An UDP DNS reply might hold 20 type A records, or 10 type AAAA ones,
+          but is likey to get truncated at quantities near or above those
+          values, and thus require fallback to TCP connections.  Publishers are
+          encouraged to carefully test their SPF records in such cases.
+        </t>
       </section>
       <section title="&quot;mx&quot;" anchor="mech-mx">
         <t>

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 18 Jan 2013 14:51:29 -0500

On Friday, January 18, 2013 12:14:46 PM Alessandro Vesely wrote:
> >> If for example I post a policy of "v=spf1 mx -all", and then I advertise
> >> "n" MXes, and each of those have "m" IP addresses, that gives me m x n
> >> IP addresses to compare to the client address.
> > 
> > your SPF record.  We could be explicit about that.  There are ways around
> > this problem that don't require protocol changes.
> > [...]

What if we just changed the last sentence before the first hunk of your patch 
from:

> Note that when computing the sizes for queries of the TXT format, one has to
> take into account any other TXT records published at the domain name.

to

> Note that when computing the sizes for queries of the TXT format, one has to
> take into account the size of all records returned with the query, including
> any other TXT records published at the domain name.

I think that accomplishes the clarification you are after without the tutorial 
on how DNS works.

Scott K

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 19 Jan 2013 12:47:35 +0100

On Fri 18/Jan/2013 20:51:29 +0100 Scott Kitterman wrote:
>> On a TCP connection, one can return a gabillizion IPs from an A
>> record lookup.
>> 
>> Is that worth an additional paragraph in Section 3.4 and 5.3?
> 

It doesn't, because they are actually two queries.  For example, on
that base I'd conclude that the record short-spf-record.tana.it is
perfectly good.  Indeed, http://www.kitterman.com/spf/validate.html
tells me the "SPF record passed validation test with pySPF", with no
complaints.

http://dmarcian.com/spf-survey/short-spf-record.tana.it instead says:
"WARNING: DNS: Truncated UDP Reply, SPF records should fit in a UDP
packet, retrying TCP"   -- BTW, I'm amazed at how sharply the
flattening comes out in that case.  Bravo!

I think Tim's diagnosis is correct, but cannot find a statement in the
spec that supports it, since all records returned by the TXT query (1
record, that is) fit UDP size...

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 19 Jan 2013 10:05:34 -0500

On Saturday, January 19, 2013 12:47:35 PM Alessandro Vesely wrote:
> > What if we just changed the last sentence before the first hunk of your
> > patch> 
> > from:
> >  Note that when computing the sizes for queries of the TXT format, one has
> >  to take into account any other TXT records published at the domain name.> 
> > to
> > 
> >  Note that when computing the sizes for queries of the TXT format, one has
> >  to take into account the size of all records returned with the query,
> >  including any other TXT records published at the domain name.
> > 
> > I think that accomplishes the clarification you are after without the
> > tutorial on how DNS works.
> 

That's because (surprise to me) the server that hosts that now supports EDNS 
0, it was able to do it all in UDP.  I'll fix that.

OK.  How about:

>  Note that when computing the sizes for queries of the TXT format, one has
>  to take into account the size of all records returned with the query,
>  including any other TXT records published at the domain name.  It is more
> reliable if all answers for all queries needed to process the SPF record can
> be returned in a single UDP packet.

Scott K

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 19 Jan 2013 21:17:30 +0100

>   be returned in a single UDP packet.

Yeah, that's the idea, more or less... I think that the reliability
gain obtained by UDP optimization is the same for TXT records as for
any other type.  In that case, I could try and reword the whole
paragraph so as to avoid that dangling "more reliable", and s/a single
UDP packet/UDP packets/.

You didn't say anything about the paragraph proposed for Section 5.3,
did you?  That was:

 An UDP DNS reply might hold 20 type A records, or 10 type AAAA ones,
 but is likey to get truncated at quantities near or above those
 values, and thus require fallback to TCP connections.  Publishers are
 encouraged to carefully test their SPF records in such cases.

Besides repeating the same concept, the latter also gives rough
estimates.  The other types (mx and ptr) already have their own limits.

---

From: Franck Martin <[email-masked]>
Date: Sat, 19 Jan 2013 20:48:18 +0000

>> be returned in a single UDP packet.

Replace reliable by efficient in the above, otherwise you may infer that
DNS fallback to TCP is not a reliable standard.

---

From: Scott Kitterman <[email-masked]>
Date: Sun, 20 Jan 2013 18:04:24 -0500

Franck Martin <[email-masked]> wrote:

>>> >  to take into account any other TXT records published at the
>domain
>>>name.> 
>>> > to
>>> > 
>>That's because (surprise to me) the server that hosts that now
>supports
>>EDNS 
>>>  Note that when computing the sizes for queries of the TXT format,
>one
>>>has
>>>  including any other TXT records published at the domain name.  It
>is
>>>more

That's the point.  Operationally it's not reliable because it's unfortunately common for port 53 TCP to be blocked. 

Scott K

---

From: Franck Martin <[email-masked]>
Date: Sun, 20 Jan 2013 23:38:26 +0000

On 1/20/13 3:04 PM, "Scott Kitterman" <[email-masked]> wrote:

>That's the point.  Operationally it's not reliable because it's
>unfortunately common for port 53 TCP to be blocked.
>

Yes and my point is that there is nothing wrong with the protocol, it is
an operational issue. People are free to shoot themselves in the foot.

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 21 Jan 2013 10:38:54 -0500

> an operational issue. People are free to shoot themselves in the foot.

Which is why there's no 2119 mustard around the point.  People are, but we 
shouldn't require people to be experts in DNS to write an SPF record.  The 
guidance to fit into UDP is in RFC 4408 and I think that extending the advice 
to include mentioning other records is a reasonable thing to do.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 21 Jan 2013 10:40:52 -0500

On Saturday, January 19, 2013 09:17:30 PM Alessandro Vesely wrote:
> >   reliable if all answers for all queries needed to process the SPF record
> >   can be returned in a single UDP packet.
> 

I hesitate to give specific guidance because there are quite a number of 
variables involved.

With the change in the first paragraph, it seemed to me the change for the 
second was no longer needed.

Scott K

---

From: Andrew Sullivan <[email-masked]>
Date: Tue, 22 Jan 2013 18:13:57 -0500

No hat.

> estimates.  The other types (mx and ptr) already have their own limits.

I see no reason this document should have recommendations about how to
run DNS in general.  There already are documents about message size
and the DNS.

Moreover, the entire thing seems to hang on the dubious principle that
you can usefully operate on the Internet today without EDNS0.  

Best,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 22 Jan 2013 18:26:24 -0500

> you can usefully operate on the Internet today without EDNS0.

It is not rare to see problems due to TCP fallback being blocked.  If EDNS0 
were ubiquitous, I don't think it would come up.  It's most reliable to fit an 
answer into a UDP packet still seems to me to be good advice (and is 
consistent with RFC 4408).

Scott K

---

From: Franck Martin <[email-masked]>
Date: Tue, 22 Jan 2013 23:34:19 +0000

>consistent with RFC 4408).

Considering that now google uses 2048bit DKIM keys (that don't even fit in
an UDP packet), I have already seen a duh! moment. I suspect the problem
will go away relatively quickly now except for the machines, pass the door
with a sign saying "Beware of the Leopard".

---

From: Andrew Sullivan <[email-masked]>
Date: Tue, 22 Jan 2013 20:01:20 -0500

No hat.

On Tue, Jan 22, 2013 at 06:26:24PM -0500, Scott Kitterman wrote:
> It is not rare to see problems due to TCP fallback being blocked.  

This is true, but if we're going to say anything here it ought to be,
"Blocking TCP fallback violates the DNS protocol and may negatively
affect SPF."  It's a flat-out violation of the protocol, and always
has been.  If people aren't going to use the protocol we depend on,
we're screwed anyway.

> If EDNS0 
> were ubiquitous, I don't think it would come up.  

Unfortunately not true, as we have learned at length with DNSSEC
deployment.  The problem is mostly that people have extremely
incorrect ideas about how big their buffers are, and misstate them in
their EDNS0 buffer parameters.  This causes EDNS0 to fail sometimes
even where it's available.  Again, however, I think, this is stuff
that simply doesn't belong in the SPF document, because DNSOP is
already busy explaining (or has already explained) all of this in
various places.  If we just need references, I'll dig them up (I guess
we probably do, so I suppose I just volunteered to do work).

> It's most reliable to fit an 

I am perfectly happy if the document says, "This will be most reliable
if all responses fit inside the traditional DNS UDP limit.  Note that
DNSSEC may strain that limit anyway."  More than that seems to me to
be borrowing trouble.

Best,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 22 Jan 2013 22:22:35 -0500

> DNSSEC may strain that limit anyway."  More than that seems to me to

I think the change I proposed earlier is in line with this (slightly modified 
here):

> take into account the size of records returned with the any associated
> query, including any other TXT records published at the domain name.

The total discussion on record length would be:

   The published SPF record for a given domain name SHOULD remain
   small enough that the results of a query for it will fit within
   512 octets.  This UDP limit is defined in <xref target="RFC1035"/>
   section 2.3.4.  This will keep even older DNS implementations from
   falling over to TCP.  Since the answer size is dependent on many
   things outside the scope of this document, it is only possible to
   give this guideline: If the combined length of the DNS name and
   the text of all the records of a given type is under 450 characters,
   then DNS answers ought to fit in UDP packets.  Note that when
   computing the sizes for queries of the TXT format, one has to take
   into account any other TXT records published at the domain name.
   Records that are too long to fit in a single UDP packet could be
   silently ignored by SPF verifiers due to firewall and other issues
   that cause DNS over TCP to be less reliable than DNS over UDP.

   Note that when computing the sizes for queries of the TXT format, one has
   to take into account the size of records returned with any associated
   query, including any other TXT records published at the domain name.

This is very similar to what's in RFC 4408.  We could also add something about 
DNSSEC here and the references you are going to dig up if you want.

Scott K

---

From: Andrew Sullivan <[email-masked]>
Date: Tue, 22 Jan 2013 22:58:05 -0500

No hat

>    the text of all the records of a given type is under 450 characters,

Apart from saying "450 octets (practically speaking, ASCII
characters)" here, I'm completely content with this.  I don't think
the additional references are needed.  People operating DNS services
need to pay attention to those other RFCs anyway, IMO, and the
references will just be clutter.  Emphatically, though, this is no
hat: if people feel otherwise, I'd like them to provide text with the
required references.  (If people want me to dig them up, I will.  But
I'm not going to extra work when I don't think it needed.)

A

-- 
Andrew Sullivan
[email-masked]

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 23 Jan 2013 00:31:35 -0500

On Tuesday, January 22, 2013 10:58:05 PM Andrew Sullivan wrote:
> > I think the change I proposed earlier is in line with this (slightly
> > modified> 
> > here):

Thanks.   I think that's enough.

Scott K

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 23 Jan 2013 09:52:07 +0100

>    The published SPF record for a given domain name SHOULD remain

What does /SPF record/ mean there?  Is that a TXT record beginning
with "v=spf1" or any DNS record queried during an SPF evaluation?

How about:

     For efficiency and security reasons, the DNS records that a
     domain publishes for SPF  --that is, the policy record itself as
     well as any record it references even indirectly-- are subject
     to size restrictions.
?

>    512 octets.  This UDP limit is defined in <xref target="RFC1035"/>

It may make sense to relax that to 4096 or whatever number we deem to
be safe nowadays, accepting Andrew's offer to dig up a suitable ref.
(Doing so would enable publishers to replace IP4s with longer IP6s
without increasing the total number of queries, for example.)

>    into account any other TXT records published at the domain name.

I'd rephrase the latter sentence as:

     For TXT records in particular, especially the first one, one has
     to take into account the size of any other TXT records published
     at the same domain name.

>    that cause DNS over TCP to be less reliable than DNS over UDP.

Although that is  a /could be/, not a MAY, would we consider an
implementation to be conforming if it aborts SPF evaluation when it
finds unreasonably long records?  That's why I mentioned security in
the proposed rewording above.

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 23 Jan 2013 23:57:29 -0500

>      well as any record it references even indirectly-- are subject

I don't think this helps.  This isn't a formal restriction, but an operational 
limitation that's inferred from the way DNS works (or not) today.  There's no 
MUST about UDP packet size and I don't think there should (SHOULD) be.

> without increasing the total number of queries, for example.)

No.  I think we decided it wasn't safe to do this yet.  512 octets is still 
demonstrably more reliable so we want it if we can get it.  For a variety of 
reasons, this will probably change, but I don't think we can write the spec 
banking on the pace of that change.

>      to take into account the size of any other TXT records published

First doesn't matter.  TXT records are returned in an arbitrary sequence so 
first is a meaningless concept.

> the proposed rewording above.

Record size is not, so far as I'm aware, a security issue.  To the extent a 
potential DoS threat exists, it's based on numbers of queries generated.  If 
you think we need to add a new security consideration for maximum record size, 
I'd like to see the threat analysis behind that recommendation.

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 25 Jan 2013 01:10:27 -0800

> with "v=spf1" or any DNS record queried during an SPF evaluation?

Why does it matter?  Regardless of the RRTYPE used (though it's always TXT
now in this document), the size caveat applies.  But if you prefer to avoid
confusion with the deprecated SPF RRTYPE, we could change "SPF record" to
"SPF policy".

>      well as any record it references even indirectly-- are subject

I don't think this adds any clarity, and as Andrew mentioned it gets into
DNS esoterica that probably don't belong here.  I believe we already refer
normatively to RFC1035; we don't need to restate much of what it and its
later friends said.

> without increasing the total number of queries, for example.)

If we say 4096, we also have to talk about EDNS0, because RFC1035 says
512.  I don't want to drag in a ton of DNSOP WG references if we can avoid
it.

Just to clarify here: Is the security reason alluded to the fact that SPF
can't be evaluated if TCP fallback occurs, or is there some other threat?

In any case, maybe it's safer to drop the specific 512 reference, and just
talk more generally about avoiding truncation that would force fallback to
TCP, which may not be possible.  If people are worried about that, they can
read up on it in RFC1035.  (We could include a reference to that section.)

> >    into account any other TXT records published at the domain name.
To be precise, even if the combined length is over 450 characters, you're
still inside a UDP packet easily.  It's the RFC1035 limit you hit, which is
way smaller than the UDP limit.

>      to take into account the size of any other TXT records published

Given that they can be returned in arbitrary order, I think this only
complicates the matter for the reader.

> the proposed rewording above.

We'd need to define "unreasonably long".

I'm not sure "silently ignored" is right.  Why would an SPF verifier
silently ignore a reply that comes back with the truncation bit on, for
example?  As I understand it, you always get the count of available
answers, and then as many of the matching RRs that fit in the reply.  So if
there are two or more answers, the one starting "v=spf1" might not be
included in the set, but if it is, you're guaranteed that it's all there.

But all of that is detail about "avoid truncation" that we don't really
need to provide, IMHO.

-MSK

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 25 Jan 2013 11:57:49 +0100

On Fri 25/Jan/2013 10:10:27 +0100 Murray S. Kucherawy wrote:
> On Wed, Jan 23, 2013 at 12:52 AM, Alessandro Vesely <[email-masked]
> <mailto:[email-masked]>> wrote:
> 
> Why does it matter?  Regardless of the RRTYPE used (though it's always
> TXT now in this document), the size caveat applies.  But if you prefer
> to avoid confusion with the deprecated SPF RRTYPE, we could change
> "SPF record" to "SPF policy".

No, I meant including other record types, such as type-A, that can be
triggered by an "a:" mechanism.  There is lots of hype, on lists such
as spf-help, on the size of type-TXT (or type-SPF), while the size of
the other record types is never mentioned.  I just wandered if that's
an unintentional omission.

> Just to clarify here: Is the security reason alluded to the fact that
> SPF can't be evaluated if TCP fallback occurs, or is there some other
> threat?

The threat is only theoretical.  I don't know of any good reason why
one might want to set up a terabyte type-A record, assuming that is
even possible.  However, my ignorance is not a valid rationale to
exclude that such beasts exist.  Nor spfbis is the place to make such
statement.  Hence, an apparently innocent SPF record could
theoretically reference a number (10) of third parties oversize
records, and thus lend itself to DoS attacks.

The question is, why does the spec put a SHOULD on the size of the
"SPF record" only?

> In any case, maybe it's safer to drop the specific 512 reference, and
> just talk more generally about avoiding truncation that would force
> fallback to TCP, which may not be possible.  If people are worried
> about that, they can read up on it in RFC1035.

I agree.  The protocol is right to recommend efficiency.  Handy advice
is welcome, but advice that is likely to become obsolete before the
spec will be revised should be avoided.

Testing the record seems to be necessary anyway.  Since it is also
sufficient, there is no reason to say more.  We need to specify what
tests a good test site should carry out, though.

---

From: Andrew Sullivan <[email-masked]>
Date: Fri, 25 Jan 2013 09:36:04 -0500

No hat.

> read up on it in RFC1035.  (We could include a reference to that section.)

Well, 4096 is as a practical matter impossible on the Internet anyway,
but you might be ok somewhere in the 1500 range.  But EDNS0 isn't a
DNSOP reference, it's part of the DNS protocol (indeed, the IESG just
approved it as Internet Standard).  We probably need a reference
anyway to that, but I don't think we should have a lot of guidance
about how to operate the zone in here.  

> complicates the matter for the reader.

Inded.  In the DNS, there is no "first one".

> example?

Indeed, if it conforms even remotely to how DNS is supposed to work,
it ought to reissue the query using TCP.

I really strongly do not think this document is a good place to try to
educate people about how to operate the DNS, particularly since SPF
is one of the poster children for DNS abuse in the DNS community.  If
we want to have a big messy fight during IETF LC, then we can lard
this draft with a lot of DNS advice.  Otherwise, let's not.

Best,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 25 Jan 2013 09:48:59 -0500

Andrew Sullivan <[email-masked]> wrote:

>SPF
>just

I think 4408 got the amount of advice pretty much right.  We might improve the wording a bit,  but we should not jump in and be more expansive. 

>this draft with a lot of DNS advice.  Otherwise, let's not.
Agreed. 

Scott K

---

From: "John Levine" <[email-masked]>
Date: 25 Jan 2013 15:03:36 -0000

>this draft with a lot of DNS advice.  Otherwise, let's not.

Absolutely.  SPF documents chronically suffer from gratuitous advice
about stuff unrelated to the SPF protocol, from how to run a mail
server to how to run DNS and more, all which might be described
as somewhat above our pay grade.

We need to tell people how to publish SPF records, and how to retrieve
them and interpret them the context of a domain name and IP address.
Then we need to stop.

R's,
John

---

From: Dotzero <[email-masked]>
Date: Fri, 25 Jan 2013 10:30:44 -0500

+1

On Fri, Jan 25, 2013 at 10:03 AM, John Levine <[email-masked]> wrote:
> _______________________________________________
> spfbis mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/spfbis

---

From: S Moonesamy <[email-masked]>
Date: Thu, 31 Jan 2013 05:47:00 -0800

Hi Scott,

There had been some discussion about Issue #24 over the last 
week.  The issue is listed as open in the tracker [1].  Is the issue resolved?

Regards,
S. Moonesamy

1. http://trac.tools.ietf.org/wg/spfbis/trac/ticket/24

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 31 Jan 2013 09:22:49 -0500

> week.  The issue is listed as open in the tracker [1].  Is the issue

I think it will be resolved with the next revision.  I think we've discussed 
it sufficiently.  Next week looks to be good for me to take a stab at making 
progress on the remaining issues.

Scott K

---

From: S Moonesamy <[email-masked]>
Date: Thu, 31 Jan 2013 06:53:55 -0800

Hi Scott,
>progress on the remaining issues.

Thanks for the feedback.

Regards,
S. Moonesamy