Subject: [dnsext] DNS over SCTP problems, part 2, practice

From: William Allen Simpson <[email-masked]>
Date: Mon, 10 Aug 2009 13:57:38 -0400

In my previous thread, "DNS over SCTP problems, part 1, theory", some of
the most egregious security problems are covered.  This thread is for some
additional operational difficulties.

SCTP is an overly complicated protocol, with different kinds of TLV
parameters, some of which are called "chunks" instead.  It probably
would have been better to only have chunks.

But DNS packets already have their own variable length parameters, so
there's 4 or more layers of variable length checking, and overlapping
layers of error returns.

It would be easier and better to add cookie extensions to EDNS0 (or TCP).

===

SCTP has a "State Cookie" that is of unspecific, variable length.

    "An implementation SHOULD make the cookie as small as possible to
    ensure interoperability." [Pages 50-51, section 3.3.11; 61,
    section 5.1.3.]

How small?  What's the minimum size?  How about the maximum size?  The
specification doesn't specify....

Obviously, a 32-bit cookie would be a waste of space and time.  But
apparently that's allowed.  Even a zero bit cookie is allowed.

There has been recent discussion about whether to upgrade DNSsec from
SHA-1 (160-bits) to SHA-256.  One of the issues is the change in size.

Yet this is peanuts compared to a specification that from the outset
failed to provide for expected sizes.  Upgrading a SCTP implementation
could cause the DNS to fail without warning.  The error messages are not
sufficient to deterministically indicate that it's the length of the
"State Cookie" causing the problem.

===

SCTP has some facilities for multi-homing.  One might think this would
support anycast DNS servers.  One might be wrong.

The "State Cookie" is dependent on a secret.  Certainly, such a secret
would not be very secure passed in the clear between anycast systems, nor
could a deterministically generated "secret" remain secret for long.  Some
form of confidentiality protocol would be required.  Moreover, it would not
be as easy to ensure that truly random secrets time-out synchronously.

There's no provision for falling back to a specific system, nor for error
messages that relate to different systems handling the subsequent traffic.

===

SCTP has many options for control and data chunks for streams (useless for
DNS), yet is missing well-known TCP options that make RTT calculation and
congestion control easier.

Others have already discussed the underlying assumption that connections
are long-term.

Operationally, this is not a good fit for DNS.

--
to unsubscribe send a message to [email-masked] with
the word 'unsubscribe' in a single line as the message text body.
archive: <http://ops.ietf.org/lists/namedroppers/>

---

From: Doug Otis <[email-masked]>
Date: Mon, 10 Aug 2009 12:26:06 -0700

> It would be easier and better to add cookie extensions to EDNS0 (or TCP).

Additional DNS exchanges based upon EDNS0 options are likely inserted by 
the application or the API which makes control over congestion avoidance 
questionable and the API more complex.

> SCTP has a "State Cookie" that is of unspecific, variable length.

SCTP cookie information is exchanged during association initialization. 
  DNS data would not be exchanged by the server until receipt of cookie 
ack, just as there would not be any during initialization of TCP 
connections.

> SHA-1 (160-bits) to SHA-256.  One of the issues is the change in size.

SCTP can fragment and reassemble packets that exceed association PMTUs. 
  SCTP headers afford a means to directly place data to avoid copying 
and head of queue blocking.  A win over TCP.

> Upgrading a SCTP implementation could cause the DNS to fail without
> warning.

When SCTP is not supported, the fallback would be to UDP and then TCP.

> The error messages are not sufficient to deterministically indicate
 > that it's the length of the "State Cookie" causing the problem.

See above.

> support anycast DNS servers.  One might be wrong.

SCTP would be affected by anycast in a similar manner to that of TCP.

> be as easy to ensure that truly random secrets time-out synchronously.

Concerns about SCTP cookies or error handling appear exaggerated.  SCTP 
cookies mitigate source spoofing which is accomplished without secrets 
and hashes.  A spoofed packet would still need to guess the value for 4 
x 32 bit variables plus the TSN.  Distributing zones to any cast servers 
could easily include a salt added to a shared secret.

> messages that relate to different systems handling the subsequent traffic.

SCTP does not need to support multi-homing.  DNS already provides 
alternative server mechanisms.

> congestion control easier.

Review sctpConstants.h for a better idea of the application layer. 
Verification Tags and TSN permit the tracking of specific exchanges, and 
their response.

> Others have already discussed the underlying assumption that connections

When queries are made in bursts (perhaps while obtaining various server 
resolutions related to a specific domain), being able to combine several 
closely spaced transactions into a common association will significantly 
reduce the overall latency.  Heartbeat rates can be adjusted, and 
associations can end at any time.  Persistence of associations might be 
dictated by use, and resource availability.

> Operationally, this is not a good fit for DNS.

SCTP does offer a good fit.  Use of SCTP by DNS will also help SCTP 
support for use with HTTP.  At some distant point in time, SOHO routers 
might even be able to handle multi-homing with IPv4. :^)

-Doug

---

From: bert hubert <[email-masked]>
Date: Mon, 10 Aug 2009 22:14:39 +0200

> SCTP is an overly complicated protocol, with different kinds of TLV

And perhaps to drive in the final nail, here are some of my earlier findings:

 Underneath the hood, the kernel is dealing with the exact same
 challenge as select() or poll() however. So you just moved the
 problem. In addition, it appears that except for only using a single
 fd, there is nothing 'lightweight' about an SCTP association.

 I measure around 8 kilobytes per association. This would mean your
 millions of associations mentioned above require tens of gigabytes of
 ram on a commonly used operating system.

 SCTP is not remotely like UDP.

http://www.ops.ietf.org/lists/namedroppers/namedroppers.2009/msg01220.html

> Operationally, this is not a good fit for DNS.

   Bert

---

From: William Allen Simpson <[email-masked]>
Date: Mon, 10 Aug 2009 16:50:40 -0400

> questionable and the API more complex.
Goodness gracious, I'd not design it that way -- why would anybody?

> SCTP cookie information is exchanged during association initialization. 

You can assume we've read the specification; especially replying to a
message with page citations to the RFC.

> ack, just as there would not be any during initialization of TCP 
And this is relevant, how?

> SCTP can fragment and reassemble packets that exceed association PMTUs. 

You can assume we've read the specification.

>  SCTP headers afford a means to directly place data to avoid copying and 
> head of queue blocking.  A win over TCP.
> 
And this is relevant, how?

I'm not sure it's always true, as the checksum is in the header.  And
there's little control of packet alignment crossing user-system
boundaries and task switching.

But every TCP stack I've implemented (2) has attempted to provide this
feature in the general case of routing between interfaces.

> When SCTP is not supported, the fallback would be to UDP and then TCP.
Actually, since SCTP would only be the fallback from UDP, that would only
leave TCP.

After you've tried UDP (a 2 RTT timeout), and SCTP (minimum 4 RTT timeout,
but probably with exponential backoff), why exactly are we falling back to
TCP?  Won't most applications have already retried another server?

>  > that it's the length of the "State Cookie" causing the problem.
What above?  We need deterministic error messages to indicate that the
server has to be taken out of the queue, and log it.

> SCTP would be affected by anycast in a similar manner to that of TCP.
No, it's much worse.  Anycast TCP is affected by loss of the connection.
SCTP has the same problem, *and* an additional cookie problem.

> could easily include a salt added to a shared secret.
At this point, it's obvious to me that I'm speaking to somebody without a
basic understanding of cookies; the concept *requires* secrets and hashes.

Some years ago, the first time that I'd looked at SCTP, it seemed to me
that cookies were just tacked on, after the original authors were told
that those random 32-bit variables weren't cryptologically significant.

Now, we've come full circle.

> alternative server mechanisms.
???  It's listed as a feature?

> Review sctpConstants.h for a better idea of the application layer.

What page in the RFC?

Or how does that relate to options that aren't in the RFC?

> Verification Tags and TSN permit the tracking of specific exchanges, and 
???  Random bits?

> dictated by use, and resource availability.
Nothing in any operating system I'm familiar with works that way.

And you might Google "I hate keep-alives"

   Craig Partridge Vice-Chairman, I Hate Keep-Alives Association

   [For those not in the know, Phil Karn is chair of the IHKAA, ...

> might even be able to handle multi-homing with IPv4. :^)
Proof by assertion is not good argument.

Nor is this list about HTTP.  (HTTP over SCTP, shuddering at the thought.)

SOHO routers need to pass EDNS today, not "some distant point in time"....

---

From: Nicholas Weaver <[email-masked]>
Date: Mon, 10 Aug 2009 14:21:42 -0700

Stupid question:

This is intended primarily as a fallback position for those EDNS0  
capable resolvers who find themselves behind a middlebox which blocks  
either EDNS0 responses, responses >512B or fragmented responses,  
correct?

What suggestion is there that for such a middlebox will allow SCTP  
unmolested without needing it to be tunneled over UDP?

---

From: [email-masked]
Date: Tue, 11 Aug 2009 09:13:12 +0100

> SOHO routers need to pass EDNS today, not "some distant point in 
time"....

Amen  to that!

Ray

-- 
Ray Bellis, MA(Oxon) MIET
Senior Researcher in Advanced Projects, Nominet
e: [email-masked], t: +44 1865 332211

---

From: Doug Otis <[email-masked]>
Date: Tue, 11 Aug 2009 10:13:01 -0700

On 8/10/09 1:50 PM, William Allen Simpson wrote:
> Doug Otis wrote:
>> Additional DNS exchanges based upon EDNS0 options are likely inserted
>> by the application or the API which makes control over congestion
>> avoidance questionable and the API more complex.
>

This viewed DNS as an application using network resources.  An EDNS0 
page option modifies DNS queries, and requires reassembly of partial DNS 
responses.

Unless a transport is used or developed, a shim layer will be needed 
between the UDP API and the existing DNS (application).  This shim layer 
will introduce new error states and additional delays.

> And this is relevant, how?

The concern about potential cookie size problems with SCTP have not been 
found in practice since there is no data contention.
,---
| Yet this is peanuts compared to a specification that from the outset
| failed to provide for expected sizes.  Upgrading a SCTP implementation
| could cause the DNS to fail without warning.  The error messages are not
| sufficient to deterministically indicate that it's the length of the
| "State Cookie" causing the problem.
'---

SCTP offers a range of choices in how server resources are protected. 
Protections required by the server can be implemented in SCTP without 
impacting interchange.  SCTP provides protections not available with 
standard TCP.

An anti-SYN flood strategy, used in TCP, algorithmically constrains 
initial SYN values to avoid retaining initial values.  This uses a 24 
bit hash result of a server secret combined with the IP addresses and 
ports of the client and server combined with 5 bits of time and 8 
different MSS selections.  This TCP modification represents a weak 
strategy in comparison to what is normally offered with SCTP.  This TCP 
modification does not support TCP options, and limits MSS values.

> And this is relevant, how?

When partial pages of DNS responses are returned in separate UDP 
packets, redundant information is still needed to link response chains, 
where a new state must track response reassembly.  SCTP solves these 
issues, and defines data locations even for out-of-sequence packets, 
allowing direct placement as packets are received.  This is not a 
property found with TCP.

> feature in the general case of routing between interfaces.

Middle boxes prevent an assurance with respect to data alignment within 
TCP byte streams.  SCTP has never been a byte stream and does not suffer 
this problem.

> Actually, since SCTP would only be the fallback from UDP, that would only

The goal would be to have SCTP support all DNS traffic and have it 
attempted first.  Delays caused by timeouts due to blocked UDP responses 
are avoided, as are spoofed reflected attacks.  SCTP also better ensures 
responses are protected from spoofed poisoning.

This protection also avoids issuing large responses toward spoofed 
addresses, either as a set of pages or as large packets.

> TCP? Won't most applications have already retried another server?

UDP and TCP would support legacy clients.  Clients that implement SCTP 
would accommodate separate timeouts and likely track non-support. 
SCTP's ability to manage persistent associations would reduce overall 
setup overhead and latency.

> We need deterministic error messages to indicate that the

See RFC 4460.  The errata corrected typos and error information.  New 
error types can be defined as it becomes needed in practice, as one has 
been.  Errors due to large cookies have not been a problem.

> SCTP has the same problem, *and* an additional cookie problem.

Each server can implement its own resource protection strategy.  Only 
when resource commitments is stored in cookies would there be a need for 
additional considerations, such as hashing info with secrets.  Even 2 
additional 32 bit values, combined with a secure hash and a server 
secret offers protections well beyond what is available with TCP.

> Now, we've come full circle.

Perhaps you could explain the cookie risk related to use with DNS, since 
it is not apparent.  When cookies are used to retain resource 
commitments, additional considerations are required.  SCTP does not 
define the scheme used.  Perhaps you could write a BCP to address the 
concerns when cookies are used to retain resources.  After all, SCTP has 
not precluded any server protection strategy.

> ??? It's listed as a feature?

Unfortunately, to minimize problems caused by SOHO routers for DNS, SCTP 
multi-homing should be avoided with IPv4.  While SCTP supports 
multi-homing, it would not be appropriate for DNS with IPv4 due to 
limited support on SOHO equipment. Again, multi-homing is not needed.

>> Review sctpConstants.h for a better idea of the application layer.

http://tools.ietf.org/html/draft-ietf-tsvwg-sctpsocket-19

> Or how does that relate to options that aren't in the RFC?

This was in regard to information available to applications with what 
might be regarded a reference implementation.

>> Verification Tags and TSN permit the tracking of specific exchanges,
>> and their response.
>>
> ??? Random bits?

These randomly generated identifiers drive state within the stack.

>> When queries are made in bursts (perhaps while obtaining various
>> server resolutions related to a specific domain), being able to
>> combine several closely spaced transactions into a common association
>> will significantly reduce the overall latency. Heartbeat rates can be
>> adjusted, and associations can end at any time. Persistence of
>> associations might be dictated by use, and resource availability.
>>

Good news. See page 49 of

http://tools.ietf.org/html/draft-ietf-tsvwg-sctpsocket-19

section 8.1.8. Automatic Close of Associations (SCTP_AUTOCLOSE)

SCTP has automated this type of use. :^)

> Nor is this list about HTTP. (HTTP over SCTP, shuddering at the thought.)

DNS is not the only service needing to become more robust against DDoS.

> SOHO routers need to pass EDNS today, not "some distant point in time"....

The multi-homing feature of SCTP is not required to support EDNS.  At 
some point, it might become suitable.  Avoiding the issuing of large 
responses to spoofed addresses, or even streams of partial responses to 
spoofed addresses represents network congestion concerns mitigated 
through connections or associations.

Rather than developing a transport layer within DNS, DNS can instead 
benefit from the SCTP features aimed at latency sensitive services.  It 
remains doubtful an EDNS0 page feature will adequately mitigate 
reflected spoof address attacks, or properly deal with DNS induced 
congestion when abused.  Such an effort of extending UDP should be 
viewed on par with development of a new transport.

Investigating SCTP with DNS should not detract from other approaches 
being considered, but SCTP still warrants review and testing as a 
possible long term solution.

-Doug

---

From: William Allen Simpson <[email-masked]>
Date: Tue, 11 Aug 2009 14:44:59 -0400

> will introduce new error states and additional delays.
I'm tired of strawman arguments.  There is no "EDNS0 page option".  After
my previous query, somebody kindly pointed me at the earlier discussion,
where my mail handler showed me that I'd read the first 2 posts in the
thread and ignored it thereafter.

> found in practice since there is no data contention.

???  Are you saying it hasn't been found in practice because it hasn't
been implemented?  Or that you have no data?

> impacting interchange.  SCTP provides protections not available with 
Your answer to a specific technical problem reads like a propaganda
brochure; content free and hand-waving.

> allowing direct placement as packets are received.  This is not a 
What partial pages?  What data locations?  Are you talking about a
particular implementation of SCTP and/or TCP?  Certainly, there's
nothing in the SCTP specification itself....

And it appears that you misused the term of art, head of queue blocking.
It has nothing to do with TCP or transport (it's about switching queues
of packets).  Or some other group has misappropriated the terminology that
we've long used in the network routing field.

> TCP byte streams.  SCTP has never been a byte stream and does not suffer 
What middle boxes?  I've yet to see a box between my user application
space and my kernel network space, or between two routable interfaces.

Enough hand-waving.  Specify the manufacturer and the network.  Show
your work.

> attempted first.

I think we've just arrived at over my dead body....

[remainder of message elided]

We in the networking and security world have been waiting for widespread
deployment of a workable DNSsec for many, many years.  It finally seems
within reach, and that got me interested here again.

It should work through many/most IPv4 SOHO boxen.  SCTP does not.

Looks to me like a badly designed solution looking for a problem.

---

From: Doug Otis <[email-masked]>
Date: Tue, 11 Aug 2009 19:32:16 -0700

On 8/11/09 11:44 AM, William Allen Simpson wrote:
>> Unless a transport is used or developed, a shim layer will be
>> needed between the UDP API and the existing DNS (application). This
>> shim layer will introduce new error states and additional delays.
>
> I'm tired of strawman arguments. There is no "EDNS0 page option".
> After my previous query, somebody kindly pointed me at the earlier
> discussion, where my mail handler showed me that I'd read the first 2
> posts in the thread and ignored it thereafter.

Okay, this leaves fragmented UDP that may not work in many cases, where
even trying might lead to DDoS concerns, or TCP with its higher
associated overhead, when compared to persistent SCTP.

>> The concern about potential cookie size problems with SCTP have not
>> been found in practice since there is no data contention.
>
> ??? Are you saying it hasn't been found in practice because it
> hasn't been implemented? Or that you have no data?

There are documented concerns within the specification, while there have
also been years of use without this being an issue. Exceeding the PMTU,
in this case, is not likely due to the size of the cookie state (which
is conveyed to the client), but due to an inordinate number of
multi-home parameter lists options. A blundering application would then
see a timeout when the server does not receive a fragmented packet.
Defining error values exchanged within the transport would be
paradoxical, since the transport failed to initialize. It seems easier
to be careful and pay attention to the API variables and chunk sizes.

IMHO, initial plans at using SCTP should not employ the multi-homing
feature.

>> SCTP offers a range of choices in how server resources are
>> protected. Protections required by the server can be implemented in
>> SCTP without impacting interchange. SCTP provides protections not
>> available with standard TCP.
>>

Sorry. You are right. I should have been more specific. Section 5.1.3 of
RFC 4690 provides a good overview of what is expected to be in the
cookie state. Do you see anything wrong with this section?

> nothing in the SCTP specification itself....

This was in response to the suggestion I thought you endorsed
regarding the EDNS0 page mode.

> And it appears that you misused the term of art, head of queue
> blocking. It has nothing to do with TCP or transport (it's about
> switching queues of packets). Or some other group has misappropriated
> the terminology that we've long used in the network routing field.

Sorry, the blocking caused by TCP's byte stream dependence upon ordered
delivery. What is a good term for this? SCTP offers unordered delivery
of framed data chunks.

>>  attempted first.  Delays caused by timeouts due to blocked UDP
>> responses are avoided, as are spoofed reflected attacks.  SCTP also
>> better ensures responses are protected from spoofed poisoning.
>>

You provided a good review of problems facing SCTP. IMHO, these
are not fatal, nor does everything need to transition to SCTP
immediately and work for everyone. How would you view SCTP if it proved
to provide lower latency, and greater reliability for enterprise and ISP
uses. What would you say if SCTP avoid risks related to the high
levels of amplification created by jumbo packet UDP responses?

> We in the networking and security world have been waiting for
> widespread deployment of a workable DNSsec for many, many years. It
> finally seems within reach, and that got me interested here again.
>

While I understand this view, having an alternative ready when problems
occur does not seem to be a bad approach. There remains the fallbacks
where those working over broken legacy equipment might suffer some
degraded service.

> Looks to me like a badly designed solution looking for a problem.

While it is not perfect, persistent associations and the ability to
frame separate DNS messages offers unappreciated advantages and quick
integration with DNS. Perhaps you are now viewing the solution as using
UDP jumbo frames. IMHO, this would be a problem looking for victims. :^(

-Doug

---

From: Paul Vixie <[email-masked]>
Date: Wed, 12 Aug 2009 06:31:06 +0000

> From: Nicholas Weaver <[email-masked]>
> Date: Mon, 10 Aug 2009 14:21:42 -0700
> 
> Stupid question:
> 
> This is intended primarily as a fallback position for those EDNS0 capable
> resolvers who find themselves behind a middlebox which blocks either
> EDNS0 responses, responses >512B or fragmented responses, correct?

incorrect.  whatever transport we add should be the first one tried, with
existing transports pushed down (just as edns pushed down non-edns.)

> unmolested without needing it to be tunneled over UDP?

none.  but when it doesn't work, the failure is unambiguous, which is good.

---

From: Olafur Gudmundsson <[email-masked]>
Date: Wed, 12 Aug 2009 11:49:25 -0400

>better ensures responses are protected from spoofed poisoning.

<no-hat>
Who's goal?
IMHO SCTP is being proposed as an alternative to TCP not UDP.

         Olafur

---

From: Michael Graff <[email-masked]>
Date: Wed, 12 Aug 2009 11:42:08 -0500

> IMHO SCTP is being proposed as an alternative to TCP not UDP.

Additionally, who has done the study to see which middleware boxes and
firewalls will allow SCTP though?

Really, are we deciding that it is more likely that these broken things
will pass unknown IP protocols when they won't pass fragments?

- --Michael

---

From: Eric Osterweil <[email-masked]>
Date: Wed, 12 Aug 2009 12:19:30 -0600

On Aug 12, 2009, at 10:42 AM, Michael Graff wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>

fwiw, Michael sounds exactly right to me.  I think the SCTP discussion  
could very likely cause more unforeseen problems than anyone is  
anticipating.  It's a very dramatic change that involves a huge number  
of unknown quantities for DNS.

Eric
é

---

From: Douglas Otis <[email-masked]>
Date: Wed, 12 Aug 2009 12:12:23 -0700

On 8/12/09 8:49 AM, Olafur Gudmundsson wrote:
>>> Actually, since SCTP would only be the fallback from UDP, that would
>>> only leave TCP.
>>

In general, concerns related to UDP EDNS0@4096 are in regard to network 
amplification, timeouts due to transit failures, and perhaps increased 
poisoning risks.  When a path does not support requisite PMTUs, there 
could be issues related to overwhelming reassembly during DoS events or 
increased glue exposure, since fragments are easier to spoof.

Advantages envisioned for SCTP over TCP is an ability to automatically 
manage persistent associations effectively, which minimizes 
initialization and PMTU overheads.  This transport can also efficiently 
handle multiple DNS messages without significant change to DNS.

SCTP is not expected to initially provide comprehensive solutions for 
everyone. By attempting SCTP first, customers of the providers that 
offer this transport can then improve service latency and security from 
out-of-path poisoning, while also better protecting their networks from 
congestion.  SCTP also support TLS.  Caching non-support of SCTP should
allow reversion to legacy behaviors that immediately attempt UDP, and 
fallback to TCP.

While SCTP may not have been implemented equally in every case, it 
should compare well with TCP in respect to initial exchanges.  When 
TCP's sequence number contains a secret hash validation, this also means 
TCP options can not be accepted and where server secrets therefore 
impact SCTP equally.

The multi-homing feature of SCTP should not be used with DNS in general 
deployments. This could lead to vulnerabilities within environments 
where IP addresses are reassigned for mobility, for example.  Proper 
support for multi-homing suggests one-way hashes of tie-tags should be 
exchanged within the Cookie State, instead of actual tie-tags.

Tie-tags are mandated by SCTP to resolve race conditions, but the RFC, 
as yet, does not recommend obscuring tie-tags, nor offer a method for 
doing so.  Multi-homing also mandates common port assignments which is 
problematic within IPv4.  For general deployment of SCTP, multi-homing 
should not be used.  In effect, this means calling bind() before sendmsg().

BTW. This should eliminate William's concern related to INIT or INIT-ACK 
exceeding PMTUs. The SCTP socket API does not appear to specify an 
error, which makes William's complaint valid, but this should not impact 
DNS.

-Doug