Subject: [dnsext] draft-ietf-dnsext-dns-tcp-requirements-00

From: Alfred Hönes <[email-masked]>
Date: Wed, 7 Oct 2009 14:29:56 +0200 (MESZ)

Ray, all,

I have looked at draft-ietf-dnsext-dns-tcp-requirements-00
and have a few suggestions for improvement, and would like to
contribute to the discussion of open topics.

Following a linear walk-through of the draft:

(1)  Front matter -- nit

I'd prefer to list the Updated RFCs in ascending RFC number order:

 Updates: 1123, 1035 (if approved)                        October 6, 2009
---
 Updates: 1035, 1123 (if approved)                        October 6, 2009

(2)  Section 1, 1st para -- discussion / improvement

   Most DNS [RFC1035] transactions take place over the UDP [RFC0792]
   protocol.  The TCP [RFC0793] protocol is used for zone transfers and
|  is supported by some implementations for the transfer of other
   packets which exceed the protocol's original 512 byte packet-size
   limit.

"some implementations" (in the 3rd line) seems to be too vague
and pessimistic.  According to recent reports, it looks like only
a few implementations do not support TCP for DNS transport.

So perhaps "many implementations" or even "most implementations"
would be more appropriate -- keep in mind the sentence refers to
*implementations*, not counts of installations of authoritative
DNS servers with TCP _enabled_ by their adminitrators.

(3)  Section 3, 2nd para -- nit

I suggest to balance the use of commas, by inserting another one:

   However, whilst RFC 1123 predates the current RFC 2119 terminology
|  document it uses exactly the same text:
---
   However, whilst RFC 1123 predates the current RFC 2119 terminology
|  document, it uses exactly the same text:
           ^

(4)  Section 3, 8th para -- nit

Once again, I suggest to place another comma, and also to add
a pair of  parentheses for clarity and readability:

|  Since the original core specifications for DNS were written the
|  Extension Mechanisms for DNS EDNS0 [RFC2671] have been introduced.
   [...]
---                                                           v
|  Since the original core specifications for DNS were written, the
|  Extension Mechanisms for DNS (EDNS0 [RFC2671]) have been introduced.
   [...]                        ^               ^

(5)  Section 5 -- important DISCUSS

Do you / we really want to update RFC 793 (TCP) and RFC 1122 as well,
in a significant way?
That's what the current text attempts without saying so.
I fear a lot of guys in TCPM wouldn't appreciate that very much!

The TCP connection state machine (RFC 793, pg. 23 ff.) specifies
that the party first performing an active CLOSE for a TCP
connection has eventually to transition through the TIME_WAIT
state, awaiting a timeout of twice the Maximum Segment Lifetime
(MSL) of the Internet.  (For the sake of exactness: "first" more
precisely means "before having itself received a FIN segment".)
MSL is defined as 2 minutes, confirmed in RFC 1122.

This is a very important feature for security purposes, and obeyance
to that long timeout has become even more important after the
introduction of wireless link technologies with their inherent
risk of uncontrolled long packet delays, e.g. during a handover.

Thus, the party performing the active CLOSE carries the burden to
keep TCP connection state for 4 more minutes.

Therefore, it does not significantly reduce the state load of a
DNS server if it actively closes a TCP connection after a very
short period (as recommended by the current text in the penultimate
paragraph of Section 5), under normal operating conditions.
It generally only makes sense for a TCP server to close connections
idle for a period of time comparable to, or exceeding 2*MSL (4 min),
in order to have a massive effect on the amount of connection state
to be held.
Notwithstanding that, emergency behavior (for periods of ongoing
connection state DoS attacks) contrary to the TCP RFCs will perhaps
be sanctioned even by some purists.

However, I'd like to draw your attention to the new TCP User Timeout
Option (UTO, RFC 5482) as a means to signal the idle timeout strategy
intended by the communicating TCP endpoints.  Such indication carried
from the DNS client to the server could provide good hints for an
advanced connection management scheme in a DNS server.
Using the UTO, the server can be aware of the time the client wants
to hold an idle connection open, and vice versa. For instance,
if the server signals a short UTO and the client does not accept
that, such client not closing the idle connection after that time
span might become regarded by the server as 'hostile', and the
connection becoming a high priority candidate of server-side CLOSE.

It should be explored whether the use of TCP UTO should be
recommended for new/updated DNS implementations.
As an incentive for client-side support of TCP UTO, DNS servers could
offer preferential treatment to connections with negotiated UTO,
with regard to the server-side connection state cleanup strategy.

Also, it might be worth giving a few hints on appropriate performance
'knobs': suppressing the Nagle algorithm, setting PSH in TCP segments,
to accommodate the message oriented TCP payload (as opposed to the
byte stream paradigm inherent in TCP), thus expediting sending and
receive processing, respectively, based on DNS packet boundaries.

(6)  Section 6

a)
You are asking for the specs.  Here are two snippets:

RFC 1035, Section 4.1.1 (page 26):

|  ID     A 16 bit identifier assigned by the program that
|         generates any kind of query.  This identifier is copied
|         the corresponding reply and can be used by the requester
                                             ^^^^^^^^^^^^^^^^^^^^^
|         to match up replies to outstanding queries.
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RFC 1035, Section 4.2.1 (page 32):

|                    [...].  Queries or their responses may be reordered
|  by the network, or by processing in name servers, so resolvers should
|  not depend on them being returned in order.

This appears in the section on UDP transport, but this sentence is
clearly not dedicated to UDP (which is explicitely mentioned in all
sentences above and afterwards there specific to UDP) and it
indeed  _is_ applicable to any transport.

b)
Most DNS clients already support parallel queries over UDP.
The responses can arrive in any order in this case as well
(see above).

So why should "streamed" (pseudo-parallel) queries over a TCP
connection pose any problem, if the client follows the specs
for correlating responses to open queries in STD 13, STD 3,
RFC 2181, and (last but not least) RFC 5452 ?

Thus, any resolver supporting parallel queries to the same
server over UDP should not have any problems to perform
query "streaming" over a TCP connection as well.
I consider the ability to correlate responses to open queries
a MUST for resolvers, independent of transport.

The DNS server code also should not have trouble in any case;
if it does not support parallel query processing, it simply
will not read again from the TCP (or UDP) socket before having
sent the answer to the last query read.  If the server does
support parallel query processing, it will do that in independent
execution streams, and any obligation to sequence responses would
make the processing unnecessarily complicated; I do not expect
that any server implementation does perform such ordering of
responses, parallel to the order of queries received.

Note: The axfr-clarify draft closes an underspecification in
STD 13, requesting the standard use of the ID field for the
multi-packet AXFR responses as well, and thus even assures the
ability (if desired by the DNS client) to perform standard DNS
queries interleaved with an ongoing AXFR transaction.

Kind regards,
  Alfred.

-- 

+------------------------+--------------------------------------------+
| TR-Sys Alfred Hoenes   |  Alfred Hoenes   Dipl.-Math., Dipl.-Phys.  |
| Gerlinger Strasse 12   |  Phone: (+49)7156/9635-0, Fax: -18         |
| D-71254  Ditzingen     |  E-Mail:  [email-masked]                     |
+------------------------+--------------------------------------------+

---

From: Alex Bligh <[email-masked]>
Date: Wed, 07 Oct 2009 14:17:23 +0100

> MSL is defined as 2 minutes, confirmed in RFC 1122.

I think there is some talking at cross purposes here.

RFC793 specifies that TIME_WAIT at 2xMSL occurs *after* the connection
close has begun (i.e. once at least one packet with FIN set has
been transmitted).

RFC1025 says:
>    - If the server needs to close a dormant connection to reclaim
>      resources, it should wait until the connection has been idle
>      for a period on the order of two minutes.  In particular, the
>      server should allow the SOA and AXFR request sequence (which
>      begins a refresh operation) to be made on a single connection.
>      Since the server would be unable to answer queries anyway, a
>      unilateral close or reset may be used instead of a graceful
>      close.

This is talking about waiting 2 minutes *before* the initiation of
the close, i.e. before sending the first packet with a FIN bit.
So there are (at least) 2 delays, the RFC1025 2 minute delay,
then the RFC793 2xMSL delay. If I have understood this right,
Roy's draft proposes removing the first of these, but not the
second, so doesn't affect RFC793/1122. Or have I read this wrong?

I /think/ this is orthogonal to UTO (at least in the absence of
keepalives).

-- 
Alex Bligh

---

From: Alfred Hönes <[email-masked]>
Date: Wed, 7 Oct 2009 16:19:02 +0200 (MESZ)

Thanks Alex for your response.

>> connection has eventually to transition through the TIME-WAIT
> RFC793 specifies that TIME-WAIT at 2xMSL occurs *after* the connection

More precisely: Once the first FIN has been sent, it has been ACKed
by the peer, and the peer's FIN has also been received (and ACKed).
FYI: These secondary events can occur in different order; the ACK for
the first FIN can arrive before or after the peer's FIN, and it can
be piggy-backed to it in the same TCP segment; the intermediate TCP
states needed to account for that are FIN-WAIT-1 and FIN-WAIT-2,
and CLOSING, respectively.

> RFC1025 says:

Did you mean RFC 1035 ?   :-)

> So there are (at least) 2 delays, the RFC1025 2 minute delay,

(again, 1035, isn't it?)

> then the RFC793 2xMSL delay. If I have understood this right,

Yep!

> second, so doesn't affect RFC793/1122. Or have I read this wrong?

This way, you only are able to reduce the connection-related resource
consumption by roughly 33% (6 min --> 4 min 4 sec; maybe a bit more,
if the stack is able to free partial resources for connections in
TIME-WAIT state).  That alone is not worth the change, IMHO.

The 2 minutes User Timeout are of the same order of magnitude as
2*MSL -- a balanced design, as you can see.

BTW: Network problems causing retries for the FIN/ACK handshake,
an irresponsive client, or a purposely misbehaving client can add
to the waiting period, by causing the server to be stalled in the
abovementioned intermediate states for another full User Timeout each.
(Similar delays can happen at the server for client-initiated CLOSE,
in the intermediate states CLOSE-WAIT and LAST-ACK.)

The draft quotes tuned HTTP servers as examples (3rd-to-last para
in Section 5), and reportedly, highly specialized HTTP servers also
aggressively reduce the effective MSL value assumed in order to
obtain a significant benefit from quickly closing conections
considered idle.  This behavior is highly controversial, but this
kind of 'TCP tuning' is nevertheless supported by major OS vendors
and documented in their Performance Tuning Guides.

>From the reference to the HTTP server example I concluded that
the spirit of the draft was to sublimely also propose to adopt
that behavior for DNS servers, and my introducing question aimed
at challenging that assumption.

The background material should help the reader to better understand
the context, and I see it did so (for you).  :-)

> I /think/ this is orthogonal to UTO (at least in the absence of

It *is* orthogonal, but as sketched, I envision that the use of
TCP UTO might allow to implement better resource management
strategies for high-load DNS servers -- better both for the server
and its non-hostile DNS clients -- without a stringent need to
change the User Timeout recommendation of 2min from RFC 1034.
That certainly will need much discussion to be worked out in detail.

Since the draft is not entitled "TCP Transport for DNS Mandatory"
but "DNS Transport over TCP", a discussion of UTO (and/or other
relevant topics -- I already have mentioned Nagle suppression / PSH --
should be within scope of the draft.

Kind regards,
  Alfred.

---

From: Alex Bligh <[email-masked]>
Date: Wed, 07 Oct 2009 16:08:21 +0100

--On 7 October 2009 16:19:02 +0200 "Alfred H?nes" <[email-masked]> wrote:

> TIME-WAIT state).

Note that it is quite possible starting the close is sufficient
to free a pile of resources (e.g. SNDBUF/RCVBUF, which I /think/
you no longer need after going into FINWAIT1/FINWAIT2, i.e.
for that 2xMSL period), though my TCP stack knowledge is no
doubt far rustier than yours.

Subject to that, I agree, absent of any changes to TCP usage, but ...

>  That alone is not worth the change, IMHO.

... the changes to TCP that would need to go with it are themselves outside
the remit of this w/g, so I think we are limited to a reference to other
RFCs (e.g. UTO) that might help. I see no harm in removing the requirement
to wait 2 minutes before starting to close the connection, and (perhaps)
pointing out that further freeing up resources is going to require
something else to be done to the TCP stack.

> the context, and I see it did so (for you).  :-)

Yes (& thanks :-). I think the draft should at least be clarified as to
what its intent is here. If your point was that the draft should not
tacitly endorse behaviour that is outside the specific behaviour
described (i.e. removing the 2 minute timeout), I agree; it should
either be explicit (and considered here), or not be in there.

> should be within scope of the draft.

Yes to your second para above; I agree that considerations (like UTO) that
change TCP parameters / how TCP are used are in scope. But in relation to
your first para, even if UTO was recommended and used, if the 1034
requirement for not initiating a close until 2 minutes of idle session has
elapsed, then I don't think you gain much do you?

-- 
Alex Bligh

---

From: [email-masked]
Date: Wed, 7 Oct 2009 16:04:50 +0100

> at challenging that assumption.

If only I were that clever.

No, the point of the example was to show that _application level_ two 
minute timeouts are very excessive.

TBH I had forgotten about the TCP MSL limits, although I shall now read up 
on how the high performance web servers are tweaking MSL to achieve this 
sort of throughput :)

cheers,

Ray

-- 
Ray Bellis, MA(Oxon) MIET
Senior Researcher in Advanced Projects, Nominet
e: [email-masked], t: +44 1865 332211

---

From: Douglas Otis <[email-masked]>
Date: Thu, 08 Oct 2009 18:36:33 -0700

On 10/7/09 5:29 AM, Alfred � wrote:
> DNS servers with TCP _enabled_ by their administrators.

Perhaps the important perspective to keep would be the number of 
implementations that actively employ the TCP protocol for query traffic. 
  Being able to enable TCP does not mean an implementation is robust 
enough to support TCP without having it interfere with the UDP service.

Perhaps it would be better to exchange the word "supported" with 
"employed" instead.

> I fear a lot of guys in TCPM wouldn't appreciate that very much!

Agreed.

> be sanctioned even by some purists.

So to reduce TCP security, one only needs to cause excessive loads on 
the server?  Perhaps the easiest way to bring down DNS over TCP would be 
to request large answers and never acknowledge their receipt. In-flight 
buffers would thereby increase the amount of per connection data 
expected of the server.  Dumping buffers during excessive loads would 
likely aggravate the problem to further reduce the service stability.

> connection becoming a high priority candidate of server-side CLOSE.

This option runs somewhat afoul of the use of SYN cookies (RFC4987). 
This is described in section 3 where UTO information is not recorded, 
and the connection needs to repeat the option once a connection is 
established.  How many implementations provide this ability?

> recommended for new/updated DNS implementations.

It would seem RFC4987 would represent a more important recommendation. 
Holding in-flight data, even for shorter periods, still leaves the 
server exposed, even with abbreviated timeouts.

> receive processing, respectively, based on DNS packet boundaries.

Don't expect message packets will remain aligned with packets even when 
using the PSH technique.  Perhaps one of the questions that should be 
answered would be how should DNS servers handle unacknowledged TCP data?
Packet alignment does not help in answering this problem, as it would be 
a mistake to assume a packet offers a message boundary.

> |         the corresponding reply and can be used by the requester

Perhaps an implementation might depend upon port-tuples to multiplex 
conversations.  This could be easily done with UDP, but less so for 
persistent TCP connections returning unordered responses. Dealing with 
unordered responses should be a requirement however.

-Doug