Subject: Truncation interoperability problems

From: "D. J. Bernstein" <[email-masked]>
Date: 10 Feb 2000 04:38:02 -0000

RFC 1035 says that UDP messages ``are restricted to 512 bytes ....
Longer messages are truncated and the TC bit is set.''

The obvious interpretation of ``truncated'' here is ``truncated to 512
bytes.'' In my new cache, when I have a message longer than 512 bytes to
send by UDP, I truncate it to 512 bytes and set the TC bit.

Unfortunately, Squid, a popular client, dies if it receives a UDP packet
truncated in the middle of an answer record. Dying is much more serious
than merely dropping truncated packets, as most clients do.

BIND doesn't trigger this Squid bug, because BIND truncates messages
between records. In this case, Squid uses a record set that's almost
always incomplete; this behavior is still wrong but users don't notice.

I am considering the following solution for my cache: when the answer
section goes past the 512-byte mark, truncate it immediately after the
query, and set the TC bit. From a correct client's point of view, all
record sets are potentially incomplete in this situation, so the packet
has to be discarded in any case. Eliminating the useless records also
prevents confused clients from using incomplete record sets.

Does BIND's truncation at <512 bytes comply with RFC 1035? Does
truncation immediately after the query comply with RFC 1035? Are there
any uses of truncated packets for which this would cause problems?

DNSEXT should issue a clarification here, to point out any relevant
interoperability issues and make clear what has to be done.

---Dan

to unsubscribe send a message to [email-masked] with
the word 'unsubscribe' in a single line as the message text body.

---

From: Rob Austein <[email-masked]>
Date: Thu, 10 Feb 2000 03:30:05 -0500

In theory it is possible for a packet with a truncated answer section
to be marginally useful if the first RR in the answer section is
intact and happens to be a CNAME.

---

From: Robert Elz <[email-masked]>
Date: Thu, 10 Feb 2000 22:09:26 +1100

Date:        10 Feb 2000 04:38:02 -0000
    From:        "D. J. Bernstein" <[email-masked]>
    Message-ID:  <[email-masked]>

  | RFC 1035 says that UDP messages ``are restricted to 512 bytes ....
  | Longer messages are truncated and the TC bit is set.''

  | The obvious interpretation of ``truncated'' here is ``truncated to 512
  | bytes.''

I don't think that's obvious at all, though it is clearly one of the
possible interpretations.   I guess the question is what is the point of
sending a partial RR?  It shouldn't do any harm to a properly implemented
resolver, but it cannot possibly do any good either, the best that can happen
is for the resolver to ignore that partial record (or just treat the whole
answer as a request to try again using TCP).   The only benefit would appear
to be a trivial simplifying of the packet building code in the server, the
cost (aside from violating "be conservative in what you send") is that
poorly written resolvers are much more likely to fail, and those extra
few bytes of meaningless overhead being transported around.

  | In my new cache, when I have a message longer than 512 bytes to
  | send by UDP, I truncate it to 512 bytes and set the TC bit.

Sounds like lazy coding to me.

  | Unfortunately, Squid, a popular client, dies if it receives a UDP packet
  | truncated in the middle of an answer record. Dying is much more serious
  | than merely dropping truncated packets, as most clients do.

That's not good.   I assume you have sent a bug report to the squid people?

  | BIND doesn't trigger this Squid bug, because BIND truncates messages
  | between records. In this case, Squid uses a record set that's almost
  | always incomplete; this behavior is still wrong but users don't notice.

No, not wrong, for an application, what squid does there is just fine.
Caches (DNS caches, squid being a web cache starts to overload the term...)
aren't allowed to remember partial RRsets - but there is no reason at all
for a client to fail to use the data it has been able to obtain.

There is no basic difference between getting back one (or two) A records
and using that one (or two) than getting back a dozen (or more) and ignoring
all but the first one returned, which is what many applications do with
the data they get back.

  | I am considering the following solution for my cache: when the answer
  | section goes past the 512-byte mark, truncate it immediately after the
  | query, and set the TC bit.

That would be legal as well, though less useful to those clients (like
squid apparently) that simply want any data as quickly as possible, and
don't care whether or not they have all of the data.

  | From a correct client's point of view, all
  | record sets are potentially incomplete in this situation,

All record sets in the section that was in progress where the truncation
occurred, yes.   It is entirely possible for there to be a complete answer,
or answer and authority section, but truncation after that (in the additional
data).  In that case, everything in the sections that are complete is
intact, and completely useable.

  | so the packet has to be discarded in any case.

No, not at all, some of the data might be complete and useable.  Even for
the parts that are not, it is for the client to decide whether the amount that
it received is enough to be useful or not.   Only DNS caches are prohibited
from using partial RRsets - because they have no idea whether the client they
would later return the partial set to needs the whole set, or only a part.

  | Eliminating the useless records also
  | prevents confused clients from using incomplete record sets.

Perfectly reasonable clients can use partial RRsets.   You don't need
to protect them from themselves.

  | Does BIND's truncation at <512 bytes comply with RFC 1035?

Yes.

  | Does truncation immediately after the query comply with RFC 1035?

Yes.

  | Are there
  | any uses of truncated packets for which this would cause problems?

Yes, as above (especially if you are going to drop sections that were complete
because of a later section which does not completely fit).

If you really feel you must protect clients from using partial RRsets,
drop just the one (entire) RRset that would not fit in the reply.  If
that means that you are left with just the question, and no answers at
all, then so be it.  Usually it won't.

  | DNSEXT should issue a clarification here, to point out any relevant
  | interoperability issues and make clear what has to be done.

This does something that sounds as if it could have usefully been covered
in 2181, had anyone thought of it then, and perhaps should be in 2181bis
or in a similar doc.

kre

---

From: Edward Lewis <[email-masked]>
Date: Thu, 10 Feb 2000 08:39:50 -0500

At 11:38 PM -0500 2/9/00, D. J. Bernstein wrote:
>bytes.''

I disagree with your interpretation, therefore I do not consider your
interpretation to be "obvious."

IMHO, a resolver, upon seeing the TC bit set to indicate truncation should
initiate a TCP request to the responding server to get a complete answer.
Trying to make use of a truncated reply is an effort of questionable
payoff.  As the resolver had just gotten an answer from a server to which
TCP is to be initiated, the resolver already knows the IP address of a
server containing the answer.  Barring a failure within the next few
seconds, the TCP interaction will have the complete answer in the
resolver's hands within a few seconds.

If the resolver thinks that a few seconds is too long to wait, the resolver
must be in the position to judge whether the data in hand is all that is
needed.  This sounds like a decision that a general purpose resolver would
be hard pressed to make.  E.g., let's say an A record set for a name
results in a truncated answer because the domain name has so many
addresses, each address only reachable by some subset of the network.
Using round-robining of answers, it would be hard to predict whether the
one A record is included or truncated,  imagine what computation the
resolver would have to undertake to find the right answer and whether it is
present.  It would be more economical to just initiate the TCP connection
and pass the answer to the application.

Recall that a TCP delivered response contains the data in the UDP message
plus the missing records, so there is no saving of time in partially
processing the included data.

As far as a resolver crashing because of a record being malformed, or the
record counts being incorrectly sent, this is something that should be
remedied in the resolver.  No software that crashes is correct.  As far as
the resolver's attempt to parse the body of the message even though the TC
bit is set, I would consider this unwise.  I do not mean to disparage the
reputation of the author/authors of the resolver, but I would recommend
correcting the resolver to avoid crashes on malformed data and to quite
processing a response if there is something anomolous in the header, such
as the TC bit.

As far as whether a server whould truncate on a 512 byte count boundary or
on an RR or RR set boundary, I would vote for the RR set boundary.  This is
a bit more computationally intensive than on a strict 512 byte boundary.
The principle I apply here in this decision is "be conservative in what you
send, liberal in what you parse."

PS - Truncating on a byte count boundary will be less deterministic once
the EDNS feature allowing larger UDP segments becomes operational.

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=--=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Edward Lewis                                                NAI Labs
Phone: +1 443-259-2352                      Email: [email-masked]

"Trying is the first step to failure." - Homer Simpson
"No! Try not. Do... or do not. There is no try." - Yoda
"It takes years of training to know when to do nothing" - Dogbert 1/21/00

Opinions expressed are property of my evil twin, not my employer.

---

From: Edward Lewis <[email-masked]>
Date: Thu, 10 Feb 2000 08:58:05 -0500

I should have consulted 1035 before sending the previous message.  No where
does the RFC 1035 specify how a message is to be truncated, except for this
passage in section 6.2:

#When a response is so long that truncation is required, the truncation
#should start at the end of the response and work forward in the
#datagram.  Thus if there is any data for the authority section, the
#answer section is guaranteed to be unique.

This only demands that the message is shaved from the bottom.  Nothing is
said about the granularity of the shaving.  (Note that this says "should"
and not "must," however, RFC 2119 had not yet been available to define the
severity of a "should.")

Note that other RFCs do describe an RR set based algorithm for truncating
data.  For example, see RFC 2535, section 4.2.  Compare the first and
second clauses.

I would consider the truncation you described as legal.  As for the rest of
my previous message, it stands, i.e., I would not interpret the truncation
as being on the 512 byte boundary, I don't agree that the interpretaion you
offer is obvious, and that the resolver you cite is buggy.

---

From: "D. J. Bernstein" <[email-masked]>
Date: 10 Feb 2000 21:40:27 -0000

Robert Elz writes:
  [ removing a set from the answer section removes the whole section? ]
> Usually it won't.

Where do you get that idea?

Squid sends A queries and PTR queries, not * queries. In theory there
could be CNAMEs, but there aren't any in my random sample of truncated
packets. Anyway, CNAME records are useless for most clients.

The big exception is MTAs. MTAs currently care about CNAMEs---although
this practice is going away. To work around BIND's lameness problems,
MTAs use * lookups to check for CNAMEs. In this case, however, my cache
stops the answer after one cached record set anyway.

---Dan