Subject: stopping amplification

From: Edward Lewis <[email-masked]>
Date: Tue, 28 Mar 2006 11:48:03 -0500

I've been urged to send out this proposal, which even I think is rather goofy.

Problem statement:

Recently, there has been a lot of hub-bub about the use of recursive 
servers to amplify traffic sent in a DDoS attack.

Problem Analysis:

There are three thorns in folks' side about this.

1) The attacks use spoofed source addresses.  This ain't something 
that can be fixed in the DNS protocol, BCP38(+/-3) be, uh, hanged.

2) The attacks make use of servers that will answer to arbitrarily 
sourced packets with lots of data.  Currently only recursive servers 
can really to this, sooner or later any DNSSEC zone server will 
qualify for this duty.

3) The attacks make use of something called amplification, 
specifically the ability to send a little message in and get a lot of 
message out.

Item 2 is something that is rather operations-ish.  But is bodes not 
well for protocol developments in the future.  Currently the only 
effective amplification that DNS affords is via recursive servers 
caching large data sets.  In the future, authoritative servers will 
do so too, via DNSSEC and other means.  (The pinata du jour on this 
one is DNSSEC.)  (Ooh, English, Spanish, and French.)

Item 3 is something that can be dealt with within the protocol.  And 
this is what my goofy idea addresses.

Potential Solution:

The crux of the goofy idea is that the only way to prevent DNS from 
being an agent of amplification is to make the queries (almost) as 
big as or bigger than the responses.  Via padding, of course.

That would enable this: I could require that such a query be padded 
up to the size in the EDNS0 field before I'll deal with it?  I know 
it's goofy to send the extra, useless bytes, but what's more 
important?   Smaller messages or stopping the "baddies" out there?

The proposal is to add an optional padding field to EDNS0.5 to allow 
for padding.  A server operator *could* then have a policy that says: 
answer up to the lesser of the EDNS0 size field and the size of the 
query.  Or, if the query is smaller than the EDNS0 size field, answer 
REFUSED.

Obligatory disclaimer:

Yes, this is goofy.  But, if we want to stamp out amplification, how 
else can it be done?  I am willing to accept that "we don't intend to 
stamp out amplification." Or, "is this a long term solution to a 
short term problem?"

PS - Yes, April 1 is this Saturday.  Take that for what it's worth.
-- 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Edward Lewis                                                +1-571-434-5468
NeuStar

Nothin' more exciting than going to the printer to watch the toner drain...

--
to unsubscribe send a message to [email-masked] with
the word 'unsubscribe' in a single line as the message text body.
archive: <http://ops.ietf.org/lists/namedroppers/>

---

From: bert hubert <[email-masked]>
Date: Tue, 28 Mar 2006 20:02:45 +0200

> I've been urged to send out this proposal, which even I think is rather 

Exceptionally so. Apologies for not criticising this more deeply, but on a
general note I'd like to draw attention to the 'amplification' that regular
recursing nameservers perform.

The PowerDNS recursor takes a lot of effort to prevent queries that have
failed recently, and would probably fail again, and to direct queries to
nameservers that do respond.

Yet despite all this throttling effort, which routinely prevents up to 40%
of queries in production, PowerDNS is only a packet-attenuator of 50-80%,
iow, for each 100 queries, 20 to 50 packets are sent to the internet.

See all this in gory detail on the graphs here:
http://adsl-xs4all.ds9a.nl/rrd/ . A small dependency-light version of the
recursor is available on http://svn.powerdns.com/pdns-recursor.tar.bz2

I wonder how well other recursors do, and might perhaps not be performing
any useful role at all in attenuating packets.

	Bert.

-- 
http://www.PowerDNS.com      Open source, database driven DNS Software 
http://netherlabs.nl              Open and Closed source services

---

From: Paul Vixie <[email-masked]>
Date: Tue, 28 Mar 2006 18:34:35 +0000

# Problem Analysis:
# 
# There are three thorns in folks' side about this.
# 
# 1) The attacks use spoofed source addresses.  This ain't something 
# that can be fixed in the DNS protocol, BCP38(+/-3) be, uh, hanged.

agreed.

# 2) The attacks make use of servers that will answer to arbitrarily 
# sourced packets with lots of data.  Currently only recursive servers 
# can really to this, sooner or later any DNSSEC zone server will 
# qualify for this duty.

not agreed.  authority servers are also willing to answer arbitrarily sourced
packets with lots of data.  and when we close all the open recursive servers,
you can bet that the attacks will change to "pick 50,000 .COM names, find out
who their nameservers are, issue spoofed source query streams for
WWW.<domain>.COM".  recursion isn't the problem.  and when DNSSEC makes it
worse, it'll also be worse for authority responses.

# 3) The attacks make use of something called amplification, 
# specifically the ability to send a little message in and get a lot of 
# message out.

not agreed.  the most valuable feature (to an attacker) of the current
setup is that syntactically valid DNS datagrams get sent to UDP/53 on the
victim servers.  they can't be firewalled or filtered at line rate inside
the victim's ISP since they look just like the packets that OUGHT to be
delivered.  (this distinguishes them from ICMP attacks, which can be
rate limited or filtered far from the servers being attacked.)  so,
amplification isn't the problem.  amplification is just "gravy."

# Item 2 is something that is rather operations-ish.  But is bodes not 
# well for protocol developments in the future.  Currently the only 
# effective amplification that DNS affords is via recursive servers 
# caching large data sets.  In the future, authoritative servers will 
# do so too, via DNSSEC and other means.  (The pinata du jour on this 
# one is DNSSEC.)  (Ooh, English, Spanish, and French.)

since "dig @ns.watson.ibm.com. www.ibm.com a" sends a 29-octet query (plus
UDP and IP headers) soliciting a 130-octet response (plus UDP/IP headers),
i really think we should stop saying that you need DNSSEC to get amplification
from an authority server.

since there are enough authority servers out there, and enough bots available
to stream spoofed-source queries to them, to overload pretty much any victim's
pipe, i think we should also stop saying that amplification is even necessary.

# Item 3 is something that can be dealt with within the protocol.  And 
# this is what my goofy idea addresses.
# 
# Potential Solution:
# 
# The crux of the goofy idea is that the only way to prevent DNS from 
# being an agent of amplification is to make the queries (almost) as 
# big as or bigger than the responses.  Via padding, of course.

as you said, "yes, this is goofy."  but not for the reasons you noted.  what's
wrong with this idea is that it solves only the problems we're not having,
while leaving in place all the weaknesses that make today's DNS dangerous and
fragile.

# Obligatory disclaimer:
# 
# Yes, this is goofy.  But, if we want to stamp out amplification, how 
# else can it be done?  I am willing to accept that "we don't intend to 
# stamp out amplification." Or, "is this a long term solution to a 
# short term problem?"
# 
# PS - Yes, April 1 is this Saturday.  Take that for what it's worth.

since recursion and amplification aren't necessary to the success of this
attack, i think we had better focus on reflection and spoofing.  reflection
as in open recursive nameservers.  let's force the attackers into using
authority servers for this, and then let's de-mix the mode of the authority
servers so that there's no "authority and recursion in the same server image"
so that authority servers can safely discard any UDP/53 packet from any other
authority server (and from any known-open known-abused recursive nameserver.)

---

From: [email-masked]
Date: Tue, 28 Mar 2006 17:11:12 +0000

> that can be fixed in the DNS protocol, BCP38(+/-3) be, uh, hanged.

	the real problem.. :)

> can really to this, sooner or later any DNSSEC zone server will 

	other applications do this too.

> specifically the ability to send a little message in and get a lot of 

	kind of like http, eh?
 
> big as or bigger than the responses.  Via padding, of course.

	so, we give the bad boys an -EXTRA- communications channel
	as an effective deterent?
	what could you do with those extra padding bits?

--bill

---

From: Edward Lewis <[email-masked]>
Date: Tue, 28 Mar 2006 13:34:52 -0500

>>  that can be fixed in the DNS protocol, BCP38(+/-3) be, uh, hanged.

Yes, then we don't need to worry that DNS amplifies.  I have no 
problem with that.

>	other applications do this too.

As in "but, ma, the other kids were throwing rocks too."

>	kind of like http, eh?

Kind of, but no.  HTTP uses stream based transport, which is not 
suitable for the spoofed-packet-DNS-amplified attacks.

>>  this is what my goofy idea addresses.

The only reason why I bothered to pursue this is that it would be 
good if protocols "took care of their own laundry."  They don't have 
to, but it would be good.

>	what could you do with those extra padding bits?

No, we aren't.  What the padding is doesn't matter.  It's not like I 
can't hide secret messages in images or mail.

---

From: Edward Lewis <[email-masked]>
Date: Tue, 28 Mar 2006 13:40:54 -0500

So, it looks like DNS amplification isn't the problem to solve. 
(Based on a sample set of 2.)  Do we branch into another set of 
questions, or live contentedly with status quo?

>any useful role at all in attenuating packets.

Okay - then let's address that.

Is the resolver's algorithm something of concern to the 
interoperability of the protocol's implementations?  (I ask because 
this is my gating function for what I think is a candidate for DNSEXT 
WG.  Mind you - "my gating" - a non-authortiative measure.)

Or should DNSOP make some judgement on the footprint of damage by a 
less than optimal resolver algorithm?
-- 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Edward Lewis                                                +1-571-434-5468
NeuStar

---

From: Edward Lewis <[email-masked]>
Date: Tue, 28 Mar 2006 14:00:46 -0500

Ahh, yes, we chatted about this.  Port 53 packets do have special 
status in fire wall rules.

>authority servers for this,

But, one other person (I wish I could recall whom) said that it isn't 
even reflection that is the problem.  If I could recall whom, I'd 
probably recall the reason.

What is the difference between me spoofing the address of an 
authoritative server and port 53 and me sending a spoofed query from 
the intended victim to the authoritative server on port 53?  (I'm 
asking for clarification, not as a challenge, although it might sound 
that way.)  Let's assume that the query:response size is about 
1:1.negligible, to write reflection off.

>                             and then let's de-mix the mode of the authority

I don't get this as being part of the problem.  So, to clarify, all me to ask:

How does a receiving server know whether the incoming packet is from 
another authority server?  As an aside - I can send queries from port 
53 on my laptop, all I need to be is the privileged user. 
Conversely, can't (on BIND say) I tell the server to use a different 
port for sourcing packets?
-- 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Edward Lewis                                                +1-571-434-5468
NeuStar

---

From: Roy Arends <[email-masked]>
Date: Wed, 29 Mar 2006 00:26:59 +0200

> authority server (and from any known-open known-abused recursive 
nameserver.)

I agree mostly. 

The resolver has the ability to use tcp. The spoofed source does not have 
that ability. Hence the reflector is able to build a tcp session between 
itself and the authoritative server. This is a significant part of the 
amplification. Sending requests with spoofed source addresses directly to 
the server does not have this effect. Also caching is neglectable since 
(in this paticular attack) the requests carry a random string as part of a 
name which gets synthesized by the server due to a wildcard.

Reflection and amplification (resulting in TCP) really is the biggest part 
of the problem. 

Ignoring requests with the RD bit set (as in dropping it on the floor, not 
sending 'refused' back) in the authoritative server should be a 
configurable policy.

Roy

---

From: bert hubert <[email-masked]>
Date: Wed, 29 Mar 2006 00:50:31 +0200

> The resolver has the ability to use tcp. The spoofed source does not have 

This has been tried. http://www.irbs.net/internet/nanog/0507/0186.html

WorldNic quit doing that pretty soon - I bet it didn't work, too many
clients behind firewalls that were stupidly configured.

	bert

---

From: Peter Koch <[email-masked]>
Date: Wed, 29 Mar 2006 01:00:39 +0200

> sending 'refused' back) in the authoritative server should be a 

Have a look at the (legitimate) queries arriving at your servers that have
the RD bit set. These are non-negligible. In fact, I'm not aware of any document
that actually says an iterating resolver (aka recursive server) SHOULD/MUST only
send queries with RD==0. Might be a nice idea.
The RD bit is insufficient evidence that something's wrong.

And again: the "open" part in "open recursive name server" is evenly important.
Even with recursion disabled, such systems still can be abused.

-Peter

---

From: Roy Arends <[email-masked]>
Date: Wed, 29 Mar 2006 01:03:53 +0200

> clients behind firewalls that were stupidly configured.

This has nothing to do with 'what has been tried'. Furthermore the reason 
why "it didn't work" might have something to do with broken resolvers as 
well.

Roy

---

From: Roy Arends <[email-masked]>
Date: Wed, 29 Mar 2006 01:12:04 +0200

Peter Koch wrote on 03/29/2006 01:00:39 AM:

> the RD bit set. 

I have. 

> These are non-negligible.

They were. my servers are authoritative only. It turned out that the only 
requests coming in with RD bit set where my own debug attempt using DiG 
which sets RD by default. 

> In fact, I'm not aware of 
> any document
> send queries with RD==0. 

rfc 1035:

RD              Recursion Desired - this bit may be set in a query and
                is copied into the response.  If RD is set, it directs
                the name server to pursue the query recursively.
                Recursive query support is optional.

This 'optional' is ambiguous enough to interpret as 'ignore'. In fact, 
IIRC, some authoritative implementations do this by default. 

> Might be a nice idea.

It'll teach'em though, gnarf gnarf. :)
 
> And again: the "open" part in "open recursive name server" is 
evenlyimportant.
> Even with recursion disabled, such systems still can be abused.

Absolutely !

Roy

---

From: Peter Koch <[email-masked]>
Date: Wed, 29 Mar 2006 01:28:41 +0200

> which sets RD by default. 

i'll try to back this with some 'fresh' figures the next days.

> This 'optional' is ambiguous enough to interpret as 'ignore'. In fact, 

Come on, apply the robustness principle, not martial law. Support being
optional means that the targeted server may not honor the RD bit (which is
what we are talking about closing recursors) but is no excuse for dropping
the query on the floor (at least not during the 'normal' resolution process).

-Peter

---

From: Roy Arends <[email-masked]>
Date: Wed, 29 Mar 2006 01:53:38 +0200

> 
> Come on, apply the robustness principle, not martial law.

I was. I'm being liberal in what I accept, but very, VERY consertive in 
what I send back.

> Support being

I'm acting as the devils' advocate here. 

Recursive nameservers talk to authoritative nameservers. They clear the RD 
bit since they intent to do recursion themselves. If a request does not 
come from a recursive server, and the authoritative server is not willing 
to do recursion, who is doing the recursion ? Anykey, one may imho explain 
'drops the request' as a result of 'Recursive query support is optional'.

It has been implemented this way.

Roy

---

From: Ben Laurie <[email-masked]>
Date: Wed, 29 Mar 2006 15:15:25 +0100

> I was. I'm being liberal in what I accept, but very, VERY consertive in 

I should note that it is not considered a very good idea to be liberal
in what you accept these days: it has a tendency to cause interesting
security problems.

Cheers,

Ben.

-- 
http://www.apache-ssl.org/ben.html           http://www.links.org/

"There is no limit to what a man can do or how far he can go if he
doesn't mind who gets the credit." - Robert Woodruff