Subject: [dnsext] EDNS Page Option draft

From: "George Barwood" <[email-masked]>
Date: Sat, 15 Aug 2009 13:22:19 +0100

I have submitted an internet draft

http://www.ietf.org/id/draft-barwood-dnsext-edns-page-option-00.txt

Abstract

   Describes an EDNS option to allow large DNS responses to be sent
   using small UDP packets.

It differs from my original informal proposal:

(1) An authentication mechanism is included.

(2) A server cookie is used to pass process state information, giving
maximum flexibility on the server side, and simplifying the client side.

I hope this may be a basis for solving many of the DNS transport
issues that have recently been discussed here.

You comments would be very much apprciated.

George

---

From: Alex Bligh <[email-masked]>
Date: Sat, 15 Aug 2009 14:27:48 +0100

>    Describes an EDNS option to allow large DNS responses to be sent
...
> You comments would be very much apprciated.

The data that the server would put in a reply may change between the
initial request and the first followup request, and between each subsequent
followup request. I presume the idea is that when the server issues a
cookie, it also might memorise a snapshot of the response to be issued (or
use some other cookie indexed mechanism to reconstruct it). That is going
to use some memory on the server end, so there should be some indication
how long it must keep possible responses to cookies, i.e. how long the
cookie should be valid for. For instance, it is unreasonable to expect that
a cookie should continue to work after 24 hours. Is there a possible DoS
vector where application state memory is used up by issuing many initial
requests (possibly with forged source addresses)?

Latency of a query would be increased from RTT (for fragmented responses)
to n x RTT, where n is the number of pages.

I don't think that you avoid amplification DoS attacks in any case (except
to the extent responses would be larger than 4096 bytes), as any server
must support EDNS0@4096 as well as this, and a miscreant would simply make
an EDNS0@4096 query without the page option.

If it is the case that a resolver is going to ask for all the pages anyway
(and I think this must be the most likely scenario), is there not a far
simpler solution? The querying client provides a secret key (per your
draft) which also indicates that it can accept paged replies, and the
resolved sends all the pages back as a set of UDP packets, each with a page
number, the total number of pages, and the secret key. The secret key
retains the protection against blind spoofing, each looking like a DNS
response packet. Yes, in an environment with p% packet loss, you end up
with 1-(1-p)^n packet loss for n pages, (i.e. approximately n x p packet
loss for small p); however n should hopefully not be large (to reach 4096
bytes anyway), and retries are possible. No state is kept on the server,
and no cookies are necessary.

-- 
Alex Bligh

---

From: Nicholas Weaver <[email-masked]>
Date: Sat, 15 Aug 2009 07:02:16 -0700

On Aug 15, 2009, at 6:27 AM, Alex Bligh wrote:

>
>
> --On 15 August 2009 13:22:19 +0100 George Barwood <[email-masked] 
> > wrote:
>

This does not have to be.  There are two reasons the response could  
change:

a)  The data itself changed (in which case, the cookie is a timestamp  
on data freshness)

b)  The server deliberately gives out partial/different information  
each time because of a randomized algorithm.  In this case, the cookie  
represents the seed state for the pRNG used to give out this  
particular instance of a response, which allows the server to  
reconstruct things comptelety.

> to n x RTT, where n is the number of pages.

This I think is a mistake.  Rather, there should be an EDNS0/EDNS1  
where you have TWO parameters: the size of the response receivable  
(the current EDNS0 size) and the size of the PACKET receivable, and  
just fragment the response at user-level by splitting it into multiple  
UDP packets.

And if you lose any single packet, oh well, you don't try for  
reliability and you'd conclude you need reliability so you fall back  
to TCP anyway.

I agree that the paging strategy doesn't buy you anything over a UDP  
user-level fragmentation with one-shot delivery.

---

From: "George Barwood" <[email-masked]>
Date: Sat, 15 Aug 2009 17:00:40 +0100

----- Original Message ----- 
From: "Alex Bligh" <[email-masked]>
To: "George Barwood" <[email-masked]>; 
<[email-masked]>
Cc: "Alex Bligh" <[email-masked]>
Sent: Saturday, August 15, 2009 2:27 PM
Subject: Re: [dnsext] EDNS Page Option draft

> cookie, it also might memorise a snapshot of the response to be issued

That's one possibility, but there are others, e.g. reconstructing the 
response
each time, and all sorts of hybrid strategies.

> (or  use some other cookie indexed mechanism to reconstruct it). That is 
> going
> cookie should be valid for.

It's up to the server, however the cookie need not be valid for more than a 
few
seconds, and if some event occurs, the server can restart the transfer.

> For instance, it is unreasonable to expect that

Yes, the server needs to guard against that, as mentioned in the Security 
section.

"Servers should seek to minimise or eliminate per-client state, to mitigate 
DoS attacks."

> to n x RTT, where n is the number of pages.

I expect "n" not to exceed 2 in normal operation.

That is, provided servers are sensible about sending unrequested NS RRsets -
my suggestion is not to perform additional section processing for these.

> an EDNS0@4096 query without the page option.

A server under heavy attack may choose to not respond to such queries, that 
is to
queries that do not have an EDNS Page option. That encourages clients to
implement the EDNS Page option :)

---

From: Alex Bligh <[email-masked]>
Date: Sat, 15 Aug 2009 22:52:48 +0100

--On 15 August 2009 07:02:16 -0700 Nicholas Weaver 
<[email-masked]> wrote:

> a)  The data itself changed (in which case, the cookie is a timestamp on
> data freshness)
>
> b)  The server deliberately gives out partial/different information each
> time because of a randomized algorithm.  In this case, the cookie
> represents the seed state for the pRNG used to give out this particular
> instance of a response, which allows the server to reconstruct things
> comptelety.

I suppose the worst possible case is either the secondary of a large
zone with a constant trickle of IXFR, or a TLD zone (or similar)
built real time from a database, in each case where the average
interval between changes is smaller than a reasonable maximum for
RTT x 2 x a reasonable maximum of pages (i.e. where a change
is likely across the few seconds it might take to sample). Clearly the
cookie could be used as a timestamp here, but in order for
a deterministic answer to be present, there is going to have to
be a substantial change to the way the nameserver works.

-- 
Alex Bligh

---

From: "George Barwood" <[email-masked]>
Date: Sun, 16 Aug 2009 00:47:27 +0100

----- Original Message ----- 
From: "Alex Bligh" <[email-masked]>
To: "Nicholas Weaver" <[email-masked]>
Cc: "Nicholas Weaver" <[email-masked]>; "George Barwood" 
<[email-masked]>; <[email-masked]>; "Alex Bligh" 
<[email-masked]>
Sent: Saturday, August 15, 2009 10:52 PM
Subject: Re: [dnsext] EDNS Page Option draft

> is likely across the few seconds it might take to sample).

For this scenario, servers could keep version information at
the RRset level. Alternatively copies of changed RRsets, can be
kept for a short period after an update, if there are outstanding
EDNS Page transactions active on them. There are many possibilities.

My guess is that in most cases a very simple approach will be
adequate - e.g. simply restart outstanding transactions after an update.

Page transactions should not very common : DNSKEY RRsets
( which are the obvious cause of large responses ) tend to be quite
stable. A TLD only has a single DNSKEY RRset, so TLDs are not
a bad case at all.

Longer term, I don't doubt the ingenuity of programmers to come
up with good solutions.

Note: in the draft, the MAC is shown after the other data.
This is an error : the MAC type (at least) should be adjacent
to the secret key (maybe two copies : one before, one after),
to reduce the possibility of fragmentation attacks.
I will correct this in the next version.

> Clearly the

---

From: Robert Elz <[email-masked]>
Date: Sun, 16 Aug 2009 08:43:29 +0700

Date:        Sat, 15 Aug 2009 13:22:19 +0100
    From:        "George Barwood" <[email-masked]>
    Message-ID:  <5722A99A87EA478BAC81D71ED194B500@localhost>

  | I have submitted an internet draft
  | http://www.ietf.org/id/draft-barwood-dnsext-edns-page-option-00.txt

I haven't been paying much attention to DNS drafts/rfcs recently,
but I saw this one come past, and couldn't avboid taking a look.

Sorry, but it is hard to imagine a worse proposal, this is just
hopeless in almost every way.

Let's start from the beginning ...

1.  Introduction

   Large UDP packets are undesirable for 3 reasons:
draft-barwood-dnsext-edns-page-option-00.txt

   (1) Although the IP protocol specifies a means by which large IP 
   packets are split into fragments and then re-assembled, in practice
   this mechanism cannot be relied on, and large UDP packets may fail
   to be delivered

Of course UDP packets may fail to be delivered, any IP packets may
fail to be delivered - but fragmentation is (and always has been) a part
of IP, if for some reason it cannot be relied upon, it needs to be
fixed (even though it should be avoided where possible).

   (3) The large ratio between the request size and the response size
   allows malicious programs to mount "amplification" attacks.

That is because of the nature of the DNS, your proposal does not
affect that really (I guess it allows a victim to avoid asking for
more responses after the first, when it is unknown, but the alternative
is that now the served is hanging onto all that data waiting for
some unspecified further time for the next request from the client,
which is never going to arrive - if the server gives up too soon,
then a legitimate request can never succeed).   a DOS attack on a
major DNS server is a far worse outsome than an amplification
attack on some random node.

   (4) Re-assembling fragments requires buffer resources, which opens
   up denial of service attacks.

That is not avoided by this, IP reassembly is still a required part
of IP, you're just removing one legitimate client - unless you can
remove every possible use of fragmentation, hosts still need to
reassemble incoming datagrams.

The resources used to reassemble the incoming DNS response will be
approximately the same either weay (just spent in a different place),
the big DNS reply still needs to be reassembled somewhere - this proposal
will just take much longer to do it, so comsume the resources for
longer.

   Instead, it is possible to use TCP, but this is undesirable, as TCP
   imposes significant overhead and state that is vulnerable to denial
   of service attack.

Now you're trying to replace TCP as well - that makes no sense at
all.   There is little extra TCP state then your proposal requires,
and TCP does its job far better and more effeciently.

Please resist the temptation to attempt to invent alternatives to
TCP, withoug a LOT of effort, they're almost all doomed to failure.

   The option includes an authentication mechanism that ensures that
   blind spoofing of the response is not possible.

Just the DNS ID & question is reasonably good for that (though
not perfect).   I wouldn't really call what you have included as
"authentication" either.

2. Protocol

2.1 Initial request

   The client signals support in it's initial request by including
   an EDNS Page option containing the following data:

   - Secret key length ( up to 16 bytes )
   - Secret key

It is sent in clear text, it is not a secret - this is what
other protocols call a nonce, not a secret key.   I cannot
imagine why it needs the additional complication of being
variable length, and if it is to be variable length, why the
maximum length needs to be 16.

2.2 Server response

   The server responds with an EDNS Page option containing the following
   option data:

   - Secret key length
   - Secret key ( up to 16 bytes )
   - Total response size ( 16 bits )
   - Data offset ( 16 bits )
   - Response data length ( 16 bits )
   - Response data
   - Cookie length ( 1 byte )
   - Cookie
   - MAC Type ( 1 byte )
   - MAC Length ( 1 byte )
   - MAC ( Message authentication code )

Obviously at some time the exact format of that would need to be
specified, if this were to go forward (but hopefully that will
never actually be needed).

But ...
 
   The original Question is also sent.

from this I assume that the question is in every answer fragment,
which them makes the interpretation and reassembly of the reply
harder - and adds to the overheads.

   The cookie has data that allows the server to send further data. Many
   different implementations are possible, and this document does not
   attempt to enumerate all these possibilities, except to note that
   allowance must be made for server updates and restarts.

This limits the protocol to strict lock step, send one request, get
one answer piece, send next request, get next piece, ... for long
requests, with a remote server, that's going to mean perhaps 5 seconds
or so (that's just 10 trips - assuming a 5K response, not very big,
with a 500ms RTT which is just one satellite leg).   That's absurd,
TCP would be done and finished ages ago, and any rational client will
have abandoned the request already.   If you're looking for a way to
kill DNSSEC, this would be it.

3.  Security Considerations

   The secret key may expose internal state to an attacker who controls
   a name server. It is essential that a cryptographically strong source
   of random numbers be used to generate the secret key. This must be 
   seeded from data that cannot be guessed by an attacker, such as
   thermal noise or other random physical fluctuations.

And for clients that have no such source of randomness?  Thay'r enot
permitted to use this protocol?

   [RFC2181]  P. Vixie, "Extension Mechanisms for DNS (EDNS0)", 
              RFC 2181, August 1999.

EDNS0 is not rfc2181.   And de-facto is not "de factor"

As a proposal, this isn't worth fixing, TCP is a much better solution
to the problem.   To make this workable, you'd need to allow the whole
answer in one burst of packets (just like a fragmented UDP datagram),
then if you want to be able to refetch just a part of the reply that was
missed, you need just about all of TCP's mechanism to deal with it.

Forget this proposal.

kre

---

From: Paul Vixie <[email-masked]>
Date: Sun, 16 Aug 2009 04:54:32 +0000

> From: Robert Elz <[email-masked]>
> Date: Sun, 16 Aug 2009 08:43:29 +0700
> 

+1.

---

From: "George Barwood" <[email-masked]>
Date: Sun, 16 Aug 2009 08:13:01 +0100

----- Original Message ----- 
From: "Robert Elz" <[email-masked]>
To: "George Barwood" <[email-masked]>
Cc: <[email-masked]>
Sent: Sunday, August 16, 2009 2:43 AM
Subject: Re: [dnsext] EDNS Page Option draft

> hopeless in almost every way.

I'm afraid your response is wrong in almost every point.
Thanks for your interest, nevertheless, the fault may be mine, in that I
probably did not sufficiently stress that the server can have no
per-client state (which is the whole point of the design).
I will fix that in the next draft.

> fixed (even though it should be avoided where possible).

The point is that the non-delivery is systematic - retrying will not 
succeed.

Even if delivery does work, fragmentation is inherently susceptible to Dos 
attack,
since the IP level cannot check that a fragment is bogus ( AFAIK, if anyone 
knows
how IPsec might practically be incorporated into DNS, that might be 
interesting ).

> affect that really

Yes it does. It limits the amplification factor, that is the amount of 
traffic an
attacker can generate in response to a small request. As described in the 
draft,
a server can dynamically reduce the amplification in response to an attack.

> (I guess it allows a victim to avoid asking for
> then a legitimate request can never succeed).

I assume by "served" you mean "server". No, the server is not "hanging on"
to the data, that is the whole point, there does not need to be any 
per-client
state, as described in the security section.

A server might have some per-client state, as an optimization, but this can
(and should) be time and space limited.

>  a DOS attack on a  major DNS server is a far worse outsome than
>  an amplification attack on some random node.

An amplification attack can exhaust the transmit capability of the
authoritative server, so it can be serious. It can have other serious
consequences as well, and such attacks ( although normally using
open recursive servers ) have been seen in the wild.

This imposes extra costs on providing authoritative servers, as the transmit
capability must be massively over-provisioned, or risk DoS attack.

> That is not avoided by this,

Yes it is. The difference is that bogus packets are immediately discarded.

>  IP reassembly is still a required part

That's true - in fact this proposal is not intended to rule out large 
packets
/ fragmentation entirely, just to provide a fall back that works smoothly
and reliably (against DoS) when the path MTU is not large ( typically
around 1500 bytes, the IPv6 minimum MTU is 1280 bytes ).

One point I didn't mention is the possibility of a server setting DF=1
and then responding to an ICMP error message by reducing the packet
size ( as occurs with TCP ). I don't know if this works theoretically
or practically, or indeed whether any existing DNS servers have this
capability - that is sending a truncated response in response to a
transmit error.

> and TCP does its job far better and more effeciently.

TCP needs per-client state, this proposal does not.

> TCP, withoug a LOT of effort, they're almost all doomed to failure.

This is nothing like TCP. Think of it more as intelligent truncation,
which allows the client to efficiently and reliably continue with UDP
when the MTU is not large.

> not perfect).   I wouldn't really call what you have included as

It's not good enough, especially when fragmentation occurs behind
a NAT device, since the ID field only protects the first fragment, and port
randomization is normally reversed by NAT.

[ copy of draft snipped ]

Regards,
George

---

From: Alex Bligh <[email-masked]>
Date: Sun, 16 Aug 2009 08:53:13 +0100

Goerge,

> kill DNSSEC, this would be it.

If we ignore the point about amplification, as I pointed out, you
could just sent all the UDP answers in one go.

You could solve the amplification issue simply by saying the first
request is answered with one packet, and the second request (if
one is needed) is answered with n-1 packets. As this would require
the "secret key" (or nonce), we know (but see below) its source
is not forged. The RTT time here is maximum 2 x RTT, which is no
worse than TCP.

> permitted to use this protocol?

I am not quite sure "secret key" (nonce or whatever) is trying to achieve.
The concern appears to be that a rogue nameserver could get hold of
the internal state of the client, presumably to subsequently forge
answers to queries to a different name server. As this is a proposal
intended (mainly) to address DNSSEC queries, that problem is mostly
solved already. However, if the key used was something like
H(real-secret,query,incrementing-counter), where "real-secret" is
a true secret (i.e. known only to the client, not the 'secret" in
George's proposal), it's hard to see how internal state would be
revealed if H() was an effective hash function.

-- 
Alex Bligh

---

From: Robert Elz <[email-masked]>
Date: Sun, 16 Aug 2009 18:46:07 +0700

Date:        Sun, 16 Aug 2009 08:13:01 +0100
    From:        "George Barwood" <[email-masked]>
    Message-ID:  <02DA7F9C1CE84E4AB0CE185185B5B55B@localhost>

  | I'm afraid your response is wrong in almost every point.

You didn't address the most important point, which is the lock
step nature of the proposal, which makes it totally unworkable.

  | Thanks for your interest, nevertheless, the fault may be mine, in that I
  | probably did not sufficiently stress that the server can have no
  | per-client state (which is the whole point of the design).

Yes, I could see that the server could encode some kind of a hash of the
answer in  the cookie it sent back, so it can tell whather or not the
data has changed since its last (part) reply.   (I'd kind of hate to
imagine the likely client implementation bugs when that happens though).

But that's not nearly enough, recall that RRs are not ordered, and servers
typically don't send the RRs in the same order in two consecutive answers
to the same question.   That means, that not only a hash of the data needs
to be remembered per client, but the exact layout of the generated answer,
or that answer itself (the easy way) so a subsequent reply can continue from
where the previous one left off.   You seem to be assuming that just the offset
will allow a stateless server to continue its reply - it wouldn't.  I doubt
it is possible to encode all of the information that would be needed in a
16 byte cookie - making the cookie big enough would mean ending up including
almost no useful data in a limited size reply, not making it big enough
means state is required at the server.

  | The point is that the non-delivery is systematic - retrying will not 
  | succeed.

Only if someone has deliberately broken fragmentation/reassembly, not for the
more typical case of a frag just being lost due to congestion or whatever.
In the former case, whatever has been broken needs to get fixed.

  | Even if delivery does work, fragmentation is inherently susceptible to Dos 
  | attack,

Sure - it is also trivially easy to protect from, well enough for the
net to continue to operate, and for the attack to be mostly useless (ie:
not perfect protectionm but good enough).  If that wasn't true, the internet
would have collapsed by now.   Fragmentation has lots of problems, and is
best avoided wherever possible, but it isn't nearly as bad as you're making out.

  | I assume by "served" you mean "server".

Yes, one of my common typos when I am in a hurry - sorry...

  | No, the server is not "hanging on" to the data,

As above, I doubt what you're thinking is implementable.

  | That's true - in fact this proposal is not intended to rule out large 
  | packets / fragmentation entirely, just to provide a fall back that works
  | smoothly and reliably (against DoS) when the path MTU is not large
  | ( typically around 1500 bytes, the IPv6 minimum MTU is 1280 bytes ).

Yes, I know what MTU's typically are - the point was that avoiding
using fragmentation doesn't make fragmentation safer (assuming the problems
are serious enough to really worry about).   Whatever fragmentation
attack someone wants to carry out doesn't need to involve the DNS at all.

You aren't going to succeed in having fragmentation elimiinated from
IP, just to avoid a minor DoS attack possibility.

  | One point I didn't mention is the possibility of a server setting DF=1
  | and then responding to an ICMP error message by reducing the packet
  | size ( as occurs with TCP ).

PMTUD - for IPv6 it is more or less required (ie: unavoidable) unless
the large packet sender deliberately fragments at a smallish size.

  | I don't know if this works theoretically  or practically,

It works theoretically (though it is complicated for UDP).
Practically, idiots who filter ICMP messages cause problems.

  | TCP needs per-client state, this proposal does not.

I suspect they both do.

  | This is nothing like TCP.

I know, it isn't - yet.   The problem is that as you find that you
need to solve all of the other problems (like avoiding the lock
step, one packet in flight at at time design - which is horrid)
you'll end up making it more and more like TCP - when you have
multiple packets in transit at once, you have to deal with reordering,
and arranging a method to have retransmitted just what needs
retransmitting, and flow control, and ...   and what you will end
up with is something that has to do just about all that TCP has to
do, just (probably) done in a different way.

This isn't uncommon, people keep making these proposals for UDP
based protocols to supposedly "solve" the problems of using TCP
for some problem or other.   I don't think I've ever seen one
succeed.

For really large DNS answers, TCP is the right solution.

  | Think of it more as intelligent truncation,
  | which allows the client to efficiently and reliably continue with UDP
  | when the MTU is not large.

But that is not a rational aim.

  | It's not good enough, especially when fragmentation occurs behind
  | a NAT device, since the ID field only protects the first fragment, and port
  | randomization is normally reversed by NAT.

This depends what you're really worried about.   If it is just
the reassembly cost, then sure - but that's not (or shouldn't be
with a reasonable implementation) a problem serious enough for
all of this, if it is DNS spoofing, then protecting fragments is
not relevant any more, only the entire answer (and as Alex Bligh
said in a later reply, since the whole point of this is dealing with
the large replies that DNSSEC generates, DNS spoofing is irrelevant
anyway, that's what DNSSEC is all about).

kre

---

From: "George Barwood" <[email-masked]>
Date: Sun, 16 Aug 2009 19:25:35 +0100

----- Original Message ----- 
From: "Robert Elz" <[email-masked]>
To: "George Barwood" <[email-masked]>
Cc: <[email-masked]>
Sent: Sunday, August 16, 2009 12:46 PM
Subject: Re: [dnsext] EDNS Page Option draft

> step nature of the proposal, which makes it totally unworkable.

I don't think that's the case. It's not expected that more than 2 response
packets would ever be needed. However, by adding a page count to the 
response,
and a page number to the follow-up request, the follow-ups could all be 
fetched
in parallel. That leads to 2 x RTT as the overall time, assuming no packets 
are
lost. However congestion control may then be a concern.

> to the same question.

I would expect authoritative servers to be deterministic.
Maybe you are referring to load balancing servers?
I think for that case DNSSEC may be problematic, but in any case, I don't
see a reason to shuffle the DNSKEY RRset, which is the case that generates
large responses.

> That means, that not only a hash of the data needs

If random shuffling of RRsets was required (and that's an odd thing to do I 
think),
that could still be accomodated by careful programming.

> In the former case, whatever has been broken needs to get fixed.

Fragmentation is bad when there is congestion, because a single lost packet 
can
generate multiple new transmissions, worsening the congestion.

> best avoided wherever possible, but it isn't nearly as bad as you're 

I'm not sure how bad I painted it. But at least we agree it isn't ideal.

> As above, I doubt what you're thinking is implementable.

Well I guess that can only be settled by more detailed work, but I
think it is really not that hard. Programming problems often seem hard until
they have been solved, then with hindsight they are easy.

> attack someone wants to carry out doesn't need to involve the DNS at all.

Rome wasn't built in a day, you have to start somewhere.
Having a robust DNS seems worthwhile to me.
This is in the recently amended charter for the working group.

> do, just (probably) done in a different way.

The difference will be that many transactions can be done in 1 RTT.
If 2 round trips is the worst case, I think you are exaggerating the 
problem.

> for some problem or other.   I don't think I've ever seen one

Ok.

> For really large DNS answers, TCP is the right solution.

Yes - but these are not expected ( excluding IXFR etc of course ).

> But that is not a rational aim.

It seems rational to me. It should improve performance and reliability.
I accept that the increased complexity may be too high a price to pay.
That's an unclear judgement call.

> anyway, that's what DNSSEC is all about).

Not entirely. Spoofing attacks against DNSSEC can still result in
denial of  service. It's especially a problem with a DNSSEC-aware,
non-validating cache. A validating client of the cache has no way of
telling the cache that it has bad ( spoofed ) data.

I suspect it's also non-trivial for validating caches to proceed
smoothly when a signature fails due to spoofing.

Regards,
George

---

From: Mark Andrews <[email-masked]>
Date: Mon, 17 Aug 2009 11:42:08 +1000

In message <7A092320F96440F186F89464389FEC7D@localhost>, "George Barwood" write
s:
> packets would ever be needed.

You wanted to get through 512 byte limited devices.  This is up to
9 DNS pages compared to 2 to 3 (IPv4) or 4 (IPv6) fragments with
IP fragmentation assuming a 4096 byte DNS response.

Whatever is done still needs to work through 512 byte limited devices.

> However, by adding a page count to the response,
> fetched in parallel. That leads to 2 x RTT as the overall time, assuming
> no packets are lost. However congestion control may then be a concern.
> 

There are lots of answers that want to exceed 512 bytes that need to be
shuffled.
 
> think), that could still be accomodated by careful programming.

Random shuffling is needed as it is the only way to get balanced
load in the presence of a dead server.

Lots of things can be accomodated by careful programming and extra
memory.  The question is "is it worth it".  We have a simple system
that is working though it is is not optimal.  It works through 512
byte limiting devices.  It doesn't require the server to hold any
state.

> can generate multiple new transmissions, worsening the congestion.

> > the large packet sender deliberately fragments at a smallish size.

IPv6 DNS/UDP responses are fragmented by the sender at network MTU.

IPv4 stacks which set DF on UDP without the application requesting
is are broken and need to be fixed.

DF is therefore a non-issue.

> If 2 round trips is the worst case, I think you are exaggerating the 

No.  Robert is not exaggerating the problem.  If you only have
static data your simple scheme will work as the server really doesn't
have to hold extra data or state.

In the real world you have caches and dynamic zones which result
in constantly changing data sources with no short term stability.
For your scheme to work you need to be able to hold the data used
to create the response until you are sure there will not be a request
for it.  You need to do this for every paged response, i.e. every
response > 512 bytes.  You also need to hold/pass state to reference
that data and have it replayed in the original order so you can
send the relevent page.

> Yes - but these are not expected ( excluding IXFR etc of course ).

No.  They my not be common by they are expected.
 
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]

---

From: "W.C.A. Wijngaards" <[email-masked]>
Date: Mon, 17 Aug 2009 10:38:18 +0200

Hi George,

My opinion about the piece.  I hope that IP fragmentation is not
broken, and effort to fix IP itself is the first concern.

If it turns out that IP is badly broken.  And that the only
recourse is to have another big bad layer violation in the
DNS.  Then, something like this draft can be the solution.

It would need to fix the stuff that is broken.  That is the
fragmentation.  Thus I would like to echo Alex Bligh's idea
that this draft would be better with no cookies and state,
and just replying with several smaller packets.

I note that is no backwards compatibility considerations in
the draft.

Overall, I do not get the impression that fragmentation is
badly broken.  The only reports we have seen are adsl box
tests that indicate those proxies are broken (must use
their routing capabilities).  There may be misconfigured or
misguided operators, but fragmentation problems seem sporadic.

Therefore, I do not think this is something for the workgroup
to take up at this time.

Best regards,
   Wouter

---

From: Florian Weimer <[email-masked]>
Date: Mon, 17 Aug 2009 08:39:22 +0000

* Mark Andrews:

> You wanted to get through 512 byte limited devices.

Wouldn't they block this unknown protocol extensions anyway?

-- 
Florian Weimer                <[email-masked]>
BFK edv-consulting GmbH       http://www.bfk.de/
Kriegsstraße 100              tel: +49-721-96201-1
D-76133 Karlsruhe             fax: +49-721-96201-99

---

From: Mark Andrews <[email-masked]>
Date: Mon, 17 Aug 2009 19:51:42 +1000

In message <[email-masked]>, Florian Weimer writes:
> * Mark Andrews:
> 
> >> I don't think that's the case. It's not expected that more than 2 respon=
> se

You have some which do and some which don't.  You can't work through
those that do so we can effectively ignore them.
 
> --=20
> Kriegsstra=DFe 100              tel: +49-721-96201-1

---

From: Florian Weimer <[email-masked]>
Date: Mon, 17 Aug 2009 10:16:05 +0000

> those that do so we can effectively ignore them.

Why can't we ignore those who *still* restrict DNS to 512 bytes, too?

---

From: "George Barwood" <[email-masked]>
Date: Mon, 17 Aug 2009 11:18:57 +0100

----- Original Message ----- 
From: "W.C.A. Wijngaards" <[email-masked]>
To: "George Barwood" <[email-masked]>
Cc: <[email-masked]>
Sent: Monday, August 17, 2009 9:38 AM
Subject: Re: [dnsext] EDNS Page Option draft

> broken, and effort to fix IP itself is the first concern.

Nicholas Weaver has done some investigation, which suggests it is broken

http://ops.ietf.org/lists/namedroppers/namedroppers.2009/msg01513.html

and has stated

"At least for DNS, fragmented UDP does not get through
a shockingly high percentage of the time, when non-fragmented
UDP gets through just fine. To my mind, this suggests that,
as an authority, whenever possible, make sure you fit in the
Ethernet MTU, unless you are happy with TCP fallback on those queries."

http://ops.ietf.org/lists/namedroppers/namedroppers.2009/msg01693.html

It would actually be quite surprising to me if it was not broken.
This capability is apparently not required for current internet use,
hence it has evolved to a state where it doesn't work.

It may be that TCP fallback is adequate, but I think it's worth considering
an alternative solution.

I worry that resolvers may start establishing a TCP connection before
the UDP attempt has failed - since the failure is a "slow" timeout.

I think that could lead to problems.

George

---

From: "W.C.A. Wijngaards" <[email-masked]>
Date: Mon, 17 Aug 2009 12:45:09 +0200

Hi George,

> http://ops.ietf.org/lists/namedroppers/namedroppers.2009/msg01513.html

I see the results but those are mostly end-users and are testing
the proxied-dns resolver on their adsl box.  And it is my understanding
also the tests from Osterweil and Bellis show the errors in the
end-user adsl box.  So, there is ample evidence that for end-users
their proxied dns service has troubles (all over the place).
- From Osterweil, it became clear the 1500-point changes things.
- From Bellis, we know that that non-proxied DNS packets are OK.

.org is talked about but seems to be solved by not appending NS
RR sets to DNSKEY queries (making the reply is 1334 bytes).
I see no evidence of IP fragmentation problems.  Am I
missing something?

Best regards,
   Wouter

---

From: "George Barwood" <[email-masked]>
Date: Mon, 17 Aug 2009 13:55:05 +0100

----- Original Message ----- 
From: "W.C.A. Wijngaards" <[email-masked]>
To: "George Barwood" <[email-masked]>
Cc: <[email-masked]>
Sent: Monday, August 17, 2009 11:45 AM
Subject: Re: [dnsext] EDNS Page Option draft

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hi George,
> 

Bellis and Weaver seem to not agree on this.

Bellis: "All 24 units could route DNSSEC queries transparently
to upstream resolverswithout flag or length limitations."

http://download.nominet.org.uk/dnssec-cpe/DNSSEC-CPE-Report.pdf

Weaver: "The situation is similar for the end user's connection. 
On a query directly to our server from the applet, bypassing 
the recursive resolver, the EDNS large request is successful 
86% of the time (3635 out of 4211), the medium is successful 
95% of the time (4019 out of 4211), and the small is successful 
99% of the time (4179 out of 4211). This suggests that for the end user
there is a significant problem with middleboxes. Many middleboxes 
either can't NAT UDP fragments or do not reassemble fragmented 
DNS traffic before analysis, but a significant number simply assume 
that DNS packets are smaller than 1400B."

suggesting that the 24 units tested were not fully representative, AFAICS.
Perhaps Ray Bellis could confirm that responses > 1500 bytes were tested.
 
> I see no evidence of IP fragmentation problems.  Am I

Responses larger than 1500 bytes seem inevitable during algorithm upgrade / key rollover.
My worry is that the protocol goes up a creek and then finds itself without a paddle.

Besides that, support for proxies that impose a 512 byte limit on responses (even over TCP),
and/or do not respect Z bits etc, seems desirable. I hope to include that in the next
version of the draft.

Best regards,
George

> Best regards,
>   Wouter
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (GNU/Linux)
> Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/
> 
> iEYEARECAAYFAkqJNLUACgkQkDLqNwOhpPhKcwCfWyW3a3ktU3LJzclYFSp1twSk
> e7YAoIpn/PoF1e141R/OS8pUjZ5hku1F
> =cCce
> -----END PGP SIGNATURE-----
>

---

From: Nicholas Weaver <[email-masked]>
Date: Mon, 17 Aug 2009 06:30:37 -0700

> - From Bellis, we know that that non-proxied DNS packets are OK.

Actually, those tests are for the recursive resolvers in the ISP/ 
network, not just the end box.

The client generates a query which, from our server, will generate a  
huge amount of crud in terms of completely unrelated CNAMEs.  But  
almost all this crud is meaningless, so the recursive resolver should  
strip it out.

  edns_large.netalyzr.icsi.berkeley.edu
  edns_medim.netalyzr.icsi.berkeley.edu

Compare the results with a dig directly to our authority
dig +dnssec edns_large.netalyzr.icsi.berkeley.edu @roland.icir.org

from one without.

Thus only the actual recursive resolver receives a large response, the  
part returned to the client should be much smaller because there are a  
lot of just totally pointless glue records involved.  Making it a test  
of the recursive resolver, not the end client.

End boxes are tested separately, and those are more likely to have a  
"can't do DNS >512" failure mode.

One big problem is the simple firewall rules, such as:
DENY all
Allow DNS UDP
will not work, because that will cause all the fragments beyond the  
first one to drop, unless the firewall is stateful with respect to  
fragments.

But since TCP goes through many hoops to avoid fragmentation these  
days, and every other UDP protocol I can think of is happy to limit  
itself to <1500B packets, this doesn't prove a problem for anything  
else.

> I see no evidence of IP fragmentation problems.  Am I

This "Not appending NS Rrsets" is specifically the effect of the  
recommendation:  make sure your responses DO NOT fragment, because  
otherwise you will have problems.  But if you make sure they don't  
fragment, you're generally OK.

Yet if DNSSEC gets deployed, you will end up with a lot pushing that  
1500B limit, and serious breakage whenever it goes over.  The Internet  
IS broken for UDP fragments, its just everybody else doing UDP has  
already implicitly understood this or does not do behavior which  
tickles this.

---

From: Alex Bligh <[email-masked]>
Date: Mon, 17 Aug 2009 14:55:52 +0100

> http://ops.ietf.org/lists/namedroppers/namedroppers.2009/msg01693.html

What would be interesting is what percentage of the time
a) DNSSEC is supported
b) EDNS0 @4096 does not get through
c) EDNS0 @1200(ish) does get through

I don't believe we have seen data on that. IE broken DNS and middleboxen
might have a high correlation with resolver inability to support DNSSEC.
If that's the case, fragmentation wouldn't be a worry.

This statement (from the first link, and thus by Nick Weaver):
> The more sophisticated version, which checks both large and medium, has
> far fewer data points. Of these, 87% (1946 out of 2232) were able to
> receive the large EDNS response, while 97.2% (2170 out of 2232) were able
> to receive the medium sized EDNS response.

Suggests that nearly 10% of EDNS capable hosts do not support large
EDNS packets (i.e. ones requiring fragmentation). If these were
all behind legacy firewalls (for instance) that didn't pass
DNSSEC through anyway, then no amount of EDNS Page Options will
help (even if the legacy firewall passed them through).

-- 
Alex Bligh

---

From: Alex Bligh <[email-masked]>
Date: Mon, 17 Aug 2009 14:57:22 +0100

> - From Bellis, we know that that non-proxied DNS packets are OK.

And moreover, from Bellis we know that where the "proxied-dns resolver
on adsl box" is used, it isn't likely to support DNSSEC anyway. So
EDNS Page Option won't help.

-- 
Alex Bligh

---

From: Nicholas Weaver <[email-masked]>
Date: Mon, 17 Aug 2009 07:38:31 -0700

On Aug 17, 2009, at 6:55 AM, Alex Bligh wrote:

>
>
> --On 17 August 2009 11:18:57 +0100 George Barwood <[email-masked] 
> > wrote:
>

We didn't test B because the response has EDNS if the resolver's  
request has EDNS, and if EDNS was being stripped/blocked, it would  
failback and be counted as "Non EDNS".

A and C are tested for, although C was only tested for later on (so  
fewer data points).

If you have particular data analysis you would like to see on this, we  
can, we're currently starting to dig through the Mass Pile of Data.

However, Bind's glue policy is moderately distinct, I wonder if we can  
find the set of glue policy etc which doesn't have EDNS?!?  Hmm, need  
to think about it and think if its meaningful.

> If that's the case, fragmentation wouldn't be a worry.

ALMOST all resolvers (don't have the numbers) which advertise EDNS  
advertise also request DNSSEC (simply because thats Bind's default  
behavior).

And the fragmentation problem was ONLY measured for those who do have  
EDNS availability >2000B.  So yes, fragmentation IS a worry.  :(

> help (even if the legacy firewall passed them through).

This is probably more likely devices which are fragmentation related,  
because they DID allow EDNS @1200ish, so they either don't care about  
DNS at all or are aware of EDNS, but some stupid rule doesn't allow  
fragments, or the part which cares about DNS doesn't handle fragments.

---

From: Alex Bligh <[email-masked]>
Date: Mon, 17 Aug 2009 16:58:51 +0100

Nick,

> We didn't test B because the response has EDNS if the resolver's request
> has EDNS, and if EDNS was being stripped/blocked, it would failback and
> be counted as "Non EDNS".
>
> A and C are tested for, although C was only tested for later on (so fewer
> data points).
>
> find the set of glue policy etc which doesn't have EDNS?!?  Hmm, need to
> think about it and think if its meaningful.

I suppose the question is "of all the resolvers claiming DNSSEC support
and EDNS0 support that actually respond at 1200 bytes-ish, what percentage
do not respond at 4096 bytes-ish?" (i.e. what percentage are broken
by fragmentation.

> because they DID allow EDNS @1200ish, so they either don't care about DNS
> at all or are aware of EDNS, but some stupid rule doesn't allow

I'm trying to square this with Ray Bellis et al.'s data. I suppose
one possibility is that the end user devices concerned are so
broken that when a DNS query is sent to them, they are copying the
whole query across (together with the OPT RR for EDNS0) and are sending
that to the resolver, but fail to reassemble fragments. That's the
only way I can see that they can (as Ray maintains) pass UDP
fragments on port 53 OK provided they aren't addressed to the unit,
still appear to be EDNS0 capable, and still result in your tests
showing lots of problems with fragmented EDNS0 but not
unfragmented EDNS0.

-- 
Alex Bligh

---

From: Paul Vixie <[email-masked]>
Date: Mon, 17 Aug 2009 15:59:31 +0000

> Date: Mon, 17 Aug 2009 10:38:18 +0200

unless we think DNS is the last wide-area UDP application who
needs or will need payloads larger than path MTU, we're better
off fixing IP fragmentation or adding UDP fragmentation (which
is the approach i'm working on) than adding DNS fragmentation.

---

From: Nicholas Weaver <[email-masked]>
Date: Mon, 17 Aug 2009 09:32:48 -0700

On Aug 17, 2009, at 8:58 AM, Alex Bligh wrote:

>> request
>> fewer

We did that.  Its fragmentation, not 1300B EDNS, that causes most of  
the problems for recursive resolvers.  (Unfortunately, that test was  
added later so only the more recent tests have that)

For end hosts, rather than recursive resolvers, its about 1/3 of the  
problem cases is EDNS0 at 1300B, 2/3rds is fragmentation.

> showing lots of problems with fragmented EDNS0 but not

On our test of end-user devices, its probably that we are testing a  
lot more different devices, and a lot more legacy devices.

The test for end-user devices is a direct UDP request to our server,  
bypassing all other infrastructure.

---

From: Nicholas Weaver <[email-masked]>
Date: Mon, 17 Aug 2009 09:34:41 -0700

> is the approach i'm working on) than adding DNS fragmentation.

It might indeed be the case that DNS IS this case.

After all, anyone else doing a UDP protocol NOW is going to run into  
the same problem, and rather than trying to work around broken  
middleboxes where some 10% of the time things fail, is going to either  
do path MTU discovery or the hack that works 99%+ of the time: assume  
Ethernet MTU and subtract a few bytes for safety.

---

From: Paul Vixie <[email-masked]>
Date: Mon, 17 Aug 2009 16:50:02 +0000

> From: Nicholas Weaver <[email-masked]>
> Date: Mon, 17 Aug 2009 09:34:41 -0700
> 

and we're ok with that?  a datagram environment so hostile that new apps are
encouraged to just find a way to do their work with small payloads and/or TCP?

> After all, anyone else doing a UDP protocol NOW is going to run into the
> same problem, and rather than trying to work around broken middleboxes
> where some 10% of the time things fail, is going to either do path MTU
> discovery or the hack that works 99%+ of the time: assume Ethernet MTU
> and subtract a few bytes for safety.

the solution we use for DNS has to be available to other apps, and if it is,
then those middleboxes will be on the run, instead of us.

---

From: Alex Bligh <[email-masked]>
Date: Mon, 17 Aug 2009 17:56:32 +0100

> problem cases is EDNS0 at 1300B, 2/3rds is fragmentation.

So, of those that could receive DNSSEC, and have EDNS0 enabled,
but exhibit failures with large packets (between 46 and 49% of
all end users, 58% of resolvers), around 13% have problems
with large packets, of which in 2/3 this is down to fragmentation
(8% of end-users/resolvers that might otherwise work).

> bypassing all other infrastructure.

Implying at least circa 10% of end users can't receive fragmented UDP
packets at all. Depressing.

It's very tempting to suggest we ensure *every* DNS packet requires
UDP fragmentation in order to make people fix their middleboxes,
but ok, count me in the "something must be done" camp. However, think
that something isn't EDNS Page Option in current form.

-- 
Alex Bligh

---

From: Nicholas Weaver <[email-masked]>
Date: Mon, 17 Aug 2009 11:07:53 -0700

> (8% of end-users/resolvers that might otherwise work).

Close, but not quite.  Of these recursive resolvers, almost ALL these  
failures are due to fragmentation rather than a middlebox which  
assumes DNS <=512B.

For END HOST systems directly doing a request to an arbitrary internet  
host for DNS, about 13% have problems, of which ~2/3rds are due to  
fragmentation, and ~1/3rds are due to a middlebox either not  
understanding EDNS or not able to handle DNS replies >512B.

> packets at all. Depressing.

More depressing is that its 10% of the recursive resolvers!?!  Which  
is just really sad.

> that something isn't EDNS Page Option in current form.

My thought is simply do a user-level fragmentation:  An additional  
EDNS RR that is EDNS MTU rather than EDNS message size, with the DNS  
authority's response simply breaking the packet up at the application  
level (application-level fragmentation), with no notion of  
indidividual packet reliability, just a mini-header on the 2nd and  
subsequent packets that is the transaction ID and the sequence # of  
the reply.

You don't lose any reliability over working IP fragmentation (the  
additional overhead is UDP header + mini-header per additional packet  
after the first, and both have an all-or-nothing failure mode), but  
work even when you don't have IP fragmentation working.

---

From: [email-masked]
Date: Mon, 17 Aug 2009 20:54:22 +0200

> suggesting that the 24 units tested were not fully representative, 
AFAICS.

The 24 units were representative of the top-selling units in the UK (ADSL 
integrated) and US (dual-ethernet) markets.

> Perhaps Ray Bellis could confirm that responses > 1500 bytes were 
tested.

They were indeed tested.

I've separately proposed that we should do some more indepth investigation 
to find out what's causing the 15% loss of fragmented packets shown by 
Netalyzr.   I plan to try and start that up once I get back from vacation.

Ray

---

From: [email-masked]
Date: Mon, 17 Aug 2009 21:08:41 +0200

> showing lots of problems with fragmented EDNS0 but not

For the _proxies_ our tests showed that most units did indeed just blindly 
copy the OPT RR from stub to recursor, even if the proxy itself was 
subsequently unable to reassemble the returned fragments.

Ray

---

From: Matthew Dempsky <[email-masked]>
Date: Mon, 17 Aug 2009 13:48:07 -0700

> subsequently unable to reassemble the returned fragments.

Are you implying that's desirable or undesirable behavior?  I read
your tone to imply it's undesirable, but it seems to be the behavior
suggested by draft-ietf-dnsext-dnsproxy-06.

---

From: Michael Graff <[email-masked]>
Date: Mon, 17 Aug 2009 15:48:06 -0500

> EDNS Page transactions active on them. There are many possibilities.

This touches on my main reason for not liking this page thing.  It puts
a lot of complication in a resolver, where the most complexity already
lives.

The most straight forward method here is to remember the item sent back
to the client for a short time (2-3 seconds) and then send the next page
from that cache.  However, this means state must be maintained on the
server in such a way that we don't know when the client is done with it.
 After all, just because all parts of the response are transmitted does
not mean one can free the state -- there could be packet loss.

I would rather just use TCP than this, which I think will cause more
complication and problems than it solves.

- --Michael

---

From: Alex Bligh <[email-masked]>
Date: Mon, 17 Aug 2009 22:20:16 +0100

--On 17 August 2009 13:48:07 -0700 Matthew Dempsky <[email-masked]> 
wrote:

>> For the _proxies_ our tests showed that most units did indeed just
>> blindly copy the OPT RR from stub to recursor, even if the proxy itself
>> was subsequently unable to reassemble the returned fragments.
>

I am not the author, but it seems to me that it is the combination
of forwarding the OPT RR saying EDNS0@4096 is supported AND dropping
the fragments which is bad. Para 4.4.3 of the above draft says:

   Therefore (irrespective of which of the methods above is in use)
   proxies SHOULD be capable of forwarding UDP packets up to a payload
   size of at least 4096 octets.

"Proxying" the OPT RR and allowing the replies through, whilst yucky
and possibly (*) contrary to RFC2671 para 4.1 "OPT RRs shall never
be cached, forwarded, or stored in or loaded from master files.",
is far less dangerous.

(*)=this would depend what is meant by 'forwarded' - I presume
it means in the sense of an old DNS "forwarder", as opposed to
IP forwarding or some L4 proxy behaviour.

-- 
Alex Bligh

---

From: Matthew Dempsky <[email-masked]>
Date: Mon, 17 Aug 2009 14:34:59 -0700

> be cached, forwarded, or stored in or loaded from master files.",

This is perhaps deviating from the thread subject, but the reason I
brought it up is I'm concerned that recommending DNS proxies to
forward OPT records without any inspection limits what new EDNS option
mechanisms can be introduced.  E.g., in this thread there was a
suggestion that the server could fragment the DNS response at the UDP
level instead of the IP level.  This would certainly require proxy
support to work correctly, and could not be introduced if proxies are
going to break the functionality.

I'd kind of like to see some mechanism for proxies to know which EDNS
options are safe to include in forwarded queries without needing to be
aware of all options in use.  E.g., suppose (going forward) even
numbers are reserved for options that are safe to forward while
proxies should strip out odd numbers that they don't know about.  (The
PNG file format uses something similar: chunk type tags are 4
characters long, and the case of each character is a flag to indicate
to tools whether the chunk can be handled opaquely or not.)

---

From: Nicholas Weaver <[email-masked]>
Date: Mon, 17 Aug 2009 15:42:01 -0700

> encouraged to just find a way to do their work with small payloads  

Its NOT hostile to new apps.

ITs pretty simple:  Ethernet MTU - small safety factor pretty much  
always works, anything more and you WILL always have fragmentation  
happening.

Can anyone name any major UDP application other than DNSSEC that  
relies on sending >1500B packets that are fragmented in the IP layer,  
rather than doing any fragmentation in the application layer?

Especially since this isn't a problem for most DNS, only DNSSEC.

So what will happen?  People turn on DNSSEC, get either failures and  
timeouts and fallbacks to TCP on the resolver end, greatly increased  
load on the authority end, and either the authorities work around  
things by making their responses small, or the resolvers turn off  
DNSSEC.

If we actually want DNSSEC over UDP, we should resign ourselves to  
doing an application-level fragmentation when responses exceed the  
Ethernet MTU.

Because there is now enshrined in the de-facto Internet reality two  
limits:  The Ethernet MTU, and devices that can't handle fragments.

---

From: Mark Andrews <[email-masked]>
Date: Tue, 18 Aug 2009 09:26:01 +1000

In message <06A65CB0596049F7A085A8C45494823F@localhost>, "George Barwood" write
> http://ops.ietf.org/lists/namedroppers/namedroppers.2009/msg01693.html

Lots of which is *not* the result of IP fragmentation but is the
result of broken APPLICATION layer DNS proxies.  Application layer
DNS proxies are in a position to adjust the EDNS UDP size advertised
upstream and the size returned to the client.  They don't have to
just pass through the packet and hope that the packet they emit is
compatible with the buffer sizes they are using.

Yes there are firewalls which limit DNS to 512 bytes.  The way to
fix that is to talk to the firewall vendors to change the defaults.
Unless the message is signed the firewalls can even adjust the EDNS
size to something they are more comfortable with like a DNS proxy
can.

Yes there are firewalls which don't handled IP fragments well.  The
way to fix that is to talk to the firewall vendors.   They can
re-assemble the packet before applying filter rules or let through
all fragments.  Either of these strategies will deal with the issue.

> hence it has evolved to a state where it doesn't work.

Fragmented IP works through NATs.  It works with all the end boxes.
Some firewalls block fragments but that is purely configuration.

IP fragmentation isn't the ogre here.  Bad middleboxes are.  We
just need to talk to the middleboxes vendors so that new products
are fixed.  We can handle the existing broken ones.  We just don't
want the problem to get significantly bigger than it is which is a
matter of education and not protocol change.
 
> the UDP attempt has failed - since the failure is a "slow" timeout.

Current iterative clients go to TCP on truncation.  TCP connect
timeouts are much worse than UDP timeouts.  Sane clients at least
make sure the server is up via UDP before going to TCP.

---

From: Mark Andrews <[email-masked]>
Date: Tue, 18 Aug 2009 09:58:59 +1000

> showing lots of problems with fragmented EDNS0 but not

They copy the query sometime adjusting the id.  The answer then
comes into a buffer smaller than the EDNS UDP size and the resulting
malformed packet is sent back to the original client.  The original
client then rejects it.  That at least is what my Netgear WGR614v6
does.

Mark

e.g.
% dig +dnssec se dnskey @192.168.1.1
;; Warning: Message parser reports malformed message packet.

; <<>> DiG 9.3.6-P1 <<>> +dnssec se dnskey @192.168.1.1
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 13407
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 8, AUTHORITY: 0, ADDITIONAL: 1

;; QUESTION SECTION:
;se.                            IN      DNSKEY

;; ANSWER SECTION:
se.                     3600    IN      DNSKEY  257 3 5 AwEAAeeGE5unuosN3c8tBcj1/q4TQEwzfNY0GK6kxMVZ1wcTkypSExLC BPMS0wWkrA1n7t5hcM86VD94L8oEd9jnHdjxreguOZYEBWkckajU0tBW wEPMoEwepknpB14la1wy3xR95PMt9zWceiqaYOLEujFAqe6F3tQ14lP6 FdFL9wyCflV06K1ww+gQxYRDo6h+Wejguvpeg33KRzFtlwvbF3AapH2G XCi4Ok2+PO2ckzfKoikIe9ZOXfrCbG9ml2iQrRNSM4q3zGhuly4NrF/t 9s9jakbWzd4PM1Q551XIEphRGyqcbA2JTU3/mcUVKfgrH7nxaPz5DoUB 7TKYyQgsTlc=
se.                     3600    IN      DNSKEY  256 3 5 AwEAAZzN62+7wyMKUs06PMCvJf2FV/mo7phe+7447lPcvJvOF2uZ0nou 87Yp1waA2FPR84VWmK5qduytrNGpmBl1HM2FsDbdllA+TdgJvGETXHkX Hpxu5FkUtlTzdLmiKZ4Q/0JmHlftHGGQ5EBhbDCZbbtORXesvJvxnRDR 6WCngcV5
se.                     3600    IN      DNSKEY  256 3 5 AwEAAc9B2MqxtXW/bQUYBV5zHf1z8e8MRSVWleDlQ3XZaOMlki4g4qdX hJ2rHN69ZzmvMj2DjTvLd8lMTN54NiHntofBIg0ZJ9pi0pcDemeeVQ7B h7T1OYzaM2rnId7xfSaoXBdX+6tMhkB/3CXM/cgqgar33emkhzdPUDdS 8FYNBGuL
se.                     3600    IN      DNSKEY  256 3 5 AwEAAeplhwBO0eadnSP6xMiFQyQ8PpVjHeutOzHSn5a47h48aPwnTUFp JzAWbHElXtweFU9g5gMtatNhHdofXz7ndsEoFArNS2M4DapSPaklrZGB SIdSMAUrhKsDPsmwIBqLHQgYVeAx4De4aEobnpTcVX/221w1+zpckwgh F79trWZD
se.                     3600    IN      DNSKEY  257 3 5 AwEAAdKc1sGsbv5jjeJ141IxNSTdR+nbtFn+JKQpvFZETaY5iMutoyWH a+jCp0TBBAzB2trGHzdi7E55FFzbeG0r+G6SJbJ4DXYSpiiELPiu0i+j Pp3C3kNwiqpPpQHWaYDS9MTQMu/QZHR/sFPbUnsK30fuQbKKkKgnADms 0aXalYUuCgDyVMjdxRLz5yzLoaSO9m5ii5cI0dQNCjexvj9M4ec6woi6 +N8v1pOmQAQ9at5Fd8A6tAxZI8tdlEUnXYgNwb8eVZEWsgXtBhoyAru7 Tzw+F6ToYq6hmKhfsT+fIhFXsYso7L4nYUqTnM4VOZgNhcTv+qVQkHfO OeJKUkNB8Qc=
se.                     3600    IN      RRSIG   DNSKEY 5 1 3600 20090822001951 20090816001810 33786 se. uvhU3jWa+Jq04rctYryXFhXpUyasEcEDbkcMkwZ7qQ7GYwhCNsWkzy0U tt9US2BFD7M6z2NILqrgfhbEZCvmTMFxnbpW9Oz8qfLyskh43ERKLJ3N Dl7yWk/cMVhLseIHu9ih9r89upYLXipLidnrAoVsjFnhS3kZKQXWMdiO 1VE=
se.                     3600    IN      RRSIG   DNSKEY 5 1 3600 20090914000000 20090804080536 8779 se. gGtYyS7qZ89UgHHxzyfs754dtdpbipQV+SfwLxWDsBWNz8qZ2RHO82vY aqEBVcA3xo1onmRdDrijFvy28//HmcC3Jb7ZkMh5UAxo+GFHMZnSo0aJ UhHGNsy2Ul2YjqRUKJoOmlMkX/GhcrOUCZFkgmZPEalrlk74nnYSdB5M 4KbdCGJM2Wm82X5gO7ahe+ozqfGOu+pxM2wMOgdthFSOYkA1QVjKLkgr fc8/xo04WWCHudz/ldmng4myVzE4Zv4TJYuqOk9HXjKmdtWIsIQlrCWP 2SU2lFvElGJ/DvOMPj2z1QtPpbjNb8+q8LpahhJlljVSjG14mnqxKob0 LLC0QQ==

;; Query time: 606 msec
;; SERVER: 192.168.1.1#53(192.168.1.1)
;; WHEN: Tue Aug 18 09:57:18 2009
;; MSG SIZE  rcvd: 1500

%

---

From: Mark Andrews <[email-masked]>
Date: Tue, 18 Aug 2009 10:45:20 +1000

In message <[email-masked]>, Nicholas W
> already implicitly understood this or does not do behavior which  

No.  Border firwalls are broken.  Fragmented IP cross the Internet
quite fine.  The packets get to the destination host/net.

Mark

---

From: Mark Andrews <[email-masked]>
Date: Tue, 18 Aug 2009 11:13:00 +1000

> be cached, forwarded, or stored in or loaded from master files.",

draft-ietf-dnsext-dnsproxy need a MUST NOT advertise a EDNS UDP
size it is incapable of returning to the client.

---

From: [email-masked]
Date: Tue, 18 Aug 2009 07:41:51 +0200

> size it is incapable of returning to the client.

Well, not really.  It already points out that it behoves the client to set 
their own EDNS0 buffer size in accordance with their MTU, with the 
implication that the client should also take the proxy's "buffer MTU" into 
account.

Ray

---

From: [email-masked]
Date: Tue, 18 Aug 2009 07:37:55 +0200

> be cached, forwarded, or stored in or loaded from master files.",

See also Section 3 ("Transparency Principle") [added emphasis]

"The role of the proxy should therefore be no more and no less than to
 receive DNS requests from clients on the LAN side, forward those
 verbatim to one of the known upstream recursive resolvers on the WAN
 side, and ensure that the *whole response* is returned verbatim to the
 original client.

 It is RECOMMENDED that proxies should be as transparent as possible,
 such that any "hop-by-hop" mechanisms or newly introduced protocol
 extensions operate as if the proxy were not there."

In other words, a proxy can blindly copy request packets if it wants, but 
if it does it had better damned well ensure that it blindly copies the 
whole response too.

Ray

---

From: Mark Andrews <[email-masked]>
Date: Tue, 18 Aug 2009 17:04:49 +1000

In message <[email-masked]
rg.uk>, [email-masked] writes:
> This is a multipart message in MIME format.
> --=_alternative 001F4BFDC1257616_=
> Content-Type: text/plain; charset="US-ASCII"
> 

The client has no way to know the DNS proxy's buffer sizes and it
is unreasonable for us to expect the client to discover this
especially when the DNS proxy is advertising itself in the DHCP
offer as a recursive nameserver.

The DNS proxy MUST adjust the EDNS buffer size to reflect it's own
buffers.  It MUST NOT just copy the request/response without doing
so.  If the DNS proxy want to do this in a minimal manner it will
use the minimum of the recieved EDNS UDP size and its own buffer
sizes.  This will allow the reply to be passed through without
having to completely rewrite it in the event that the client's EDNS
UDP size is smaller than the DNS proxy's buffer size.  It will also
allow clients that are attempting to discover what size request
they can send over UDP to work as the effective MTU of the outgoing
path will be recorded in the response the client sees.

A DNS proxy that tells the recursive server that it can accept EDNS
responses that are ethernet sized is infinitely better that a DNS
proxy that lies to the recursive server by copying the 4096 EDNS
size the client sent.

Mark

---

From: Alex Bligh <[email-masked]>
Date: Tue, 18 Aug 2009 09:04:34 +0100

--On 18 August 2009 17:04:49 +1000 Mark Andrews <[email-masked]> wrote:

> buffers.

I'm not sure that is right. Better alternatives would include
acting as (say) a caching resolver rather than proxying at all.
Isn't it the other way around, i.e. that it must not pass through
OPT RRs indicating support for an EDNS0 buffer size which it
then will not support?

Note that the proxy may not be able to adjust its EDNS buffer
size without relying on fragmentation working upstream
as it may not know what the upstream MTU is (e.g. it
may pass over a segment with smaller MTU than ethernet MTU).

> It MUST NOT just copy the request/response without doing
> so.

There was a long discussion about MUST/SHOULD status of
things in this draft; precatory words were favoured over
language of obligation in many places where doing the
opposite resulted in "bad things happening" (I can't
remember the other examples off the top of my head).
As I understand the argument, it was that encouraging
vendors to conform was better than whipping them for
not doing so, resulting in them ignoring the whole
draft, particularly as some of the behaviours were
already proscribed by other RFCs.

-- 
Alex Bligh

---

From: Florian Weimer <[email-masked]>
Date: Tue, 18 Aug 2009 08:25:46 +0000

> buffers.  It MUST NOT just copy the request/response without doing

Ahem, it's much easier to just use a 64K buffer.

---

From: Mark Andrews <[email-masked]>
Date: Tue, 18 Aug 2009 18:37:39 +1000

> may pass over a segment with smaller MTU than ethernet MTU).

That should not be a problem as these boxes are capable of correctly
NATing UDP bigger that physical MTU.  They either reassemble today
to do this or hold out of order fragments until the initial fragment
arrives then track the fragment.  In either case they can still
adjust the EDNS UDP size as the packet transits them.

---

From: Doug Barton <[email-masked]>
Date: Tue, 18 Aug 2009 11:42:28 -0700

[email-masked] wrote:
> Well, not really.  It already points out that it behoves the client to
> set their own EDNS0 buffer size in accordance with their MTU, with the
> implication that the client should also take the proxy's "buffer MTU"
> into account.

This leads naturally into a thought that has been percolating in my
brain throughout these fragmentation threads. It would be really nice
if there were a command-line tool that admins could use to test this
(where "this" is "what is my largest possible unfragmented UDP
packet?"). Then at least admins who had sufficient clue to use the
tool could set up there name servers appropriate and/or work to find
the cause of things not being what they think they should be.

I would love to jump into the fray on this immediately, but atm I'm
busy trying to find ways to feed my family. :)

Doug

---

From: Nicholas Weaver <[email-masked]>
Date: Tue, 18 Aug 2009 12:02:47 -0700

> the cause of things not being what they think they should be.

dig edns_large.{anything}.netalyzr.icsi.berkeley.edu
     edns_medium.{anything}.netalyzr.icsi.berkeley.edu
     edns_small.{anything}.netalyzr.icsi.berkeley.edu

Basically a crude test, but gets you in the ballpark of small, ~1400B,  
~1800B, and fragmentation almost always hits at the Ethernet MTU of  
1500B

---

From: Duane Wessels <[email-masked]>
Date: Tue, 18 Aug 2009 19:44:34 +0000 (UTC)

> packet?"). Then at least admins who had sufficient clue to use the

See also  https://www.dns-oarc.net/oarc/services/replysizetest

    $ dig +short rs.dns-oarc.net txt

---

From: David Conrad <[email-masked]>
Date: Tue, 18 Aug 2009 10:37:03 -1000

Hi,

> limits:  The Ethernet MTU, and devices that can't handle fragments.

An informal, wildly unscientific survey...

So, in the relatively near future, the root zone will be signed.   
Current plans are to NOT make any special effort to reduce root  
response size (see ns.iana.org for a reasonable facsimile for what the  
signed root is going to look like).  Last I checked, about 70% of the  
queries hitting the root had DO=1, with somewhere around 75% of those  
queries advertising a buffer size of 4096.

On the fateful date the root gets signed, what do you think is going  
to happen?

a) there will be much rejoicing
b) no one but a few geeks will notice
c) a percentage of folks will have resolution problems (what  
percentage?)
d) some root server instances will fall over from increased load but  
no one other than geeks will notice
e) some root server instances will fall over and non-geeks will notice  
due to performance degradation
f) something else (what?)

(combinations of answers are obviously ok)

Respond publicly or privately to me, I'll summarize the responses.

Thanks,
-drc

---

From: Mark Andrews <[email-masked]>
Date: Wed, 19 Aug 2009 09:38:05 +1000

In message <[email-masked]>, David Conrad
> (combinations of answers are obviously ok)

a by b + c (sites that never talk to ORG, SE, BR or any other signed TLDs).
f more firewalls will be re-configured to handle large UDP.  There will
  be questions about why my lookups are slow.

---

From: Eric Osterweil <[email-masked]>
Date: Wed, 19 Aug 2009 11:33:31 +0200

> the cause of things not being what they think they should be.

Try dnsfunnel:
	http://www.vantage-points.org/download.html

It's bundled w/ our DNSSEC suite.  We announced this at IETF 75 and it  
will find the exact size of messages that fit over the path from  
nameservers to you:

$ dnsfunnel questsys.com

208.67.177.139  4096B   0.291743
208.67.177.139  2304B   0.268944
208.67.177.139  1408B   0.326089 (truncated)
208.67.177.139  1856B   0.245980 (truncated)
208.67.177.139  2080B   0.234383
208.67.177.139  1968B   0.215011 (truncated)
208.67.177.139  2024B   0.269964
208.67.177.139  1996B   0.217236
208.67.177.139  1982B   0.214775
208.67.177.139  1975B   0.215307 (truncated)
208.67.177.139  1978B   0.227221 (truncated)
208.67.177.139  1980B   0.216392 (truncated)
208.67.177.139  1981B   0.216609
208.67.177.139  1980B   0.217926 (truncated)
208.67.177.139  4096B   0.216612
---------------------------------

208.67.177.8    4096B   * * *
208.67.177.8    2304B   * * *
208.67.177.8    1408B   * * *
208.67.177.8    960B    0.233100 (truncated)

...

208.67.177.7    1252B   0.214902 (truncated)
---------------------------------
PMTU walking summary:
================================================
Name Server     Keys    Small   Largest Optimal
IP              fit?    Buffer  Buffer  Buffer
------------------------------------------------
208.67.177.139  yes     1981    4096    1981
208.67.177.8    no      1252    0       0
208.67.177.138  yes     1981    4096    1981
208.67.177.7    no      1252    0       0

Eric
é

---

From: [email-masked]
Date: Wed, 19 Aug 2009 17:33:34 +0000

> Respond publicly or privately to me, I'll summarize the responses.

	a-e will occur.  maybe f.
	a handful of us are currently running studies to get percentages.

--bill