Subject: ANY queries, recursing nameserver?

From: bert hubert <[email-masked]>
Date: Tue, 28 Mar 2006 20:46:39 +0200

People,

I've trawled RFC 1034, 1035 and 2181, yet I can't find it written down on
what a recursing nameserver should do on receiving an ANY query (aka
QTYPE=*).

http://www.faqts.com/knowledge_base/view.phtml/aid/30221 lists nonsense I
think - step 4 of the algorithm is never reached if RD is turned on.

'Mr DNS' (why I think was Cricket), said in 1997:

   Actually, a T_ANY lookup isn't guaranteed to return the TXT records,
   unless you're querying the authoritative name server directly. On another
   name server, T_ANY will only return whatever the server already knows
   about the domain name.

I'd love to believe this but it doesn't sound all too useful to me, but it
sure makes coding simple.

Anybody have an authoritative (haha) reference?

Thanks.

-- 
http://www.PowerDNS.com      Open source, database driven DNS Software 
http://netherlabs.nl              Open and Closed source services

--
to unsubscribe send a message to [email-masked] with
the word 'unsubscribe' in a single line as the message text body.
archive: <http://ops.ietf.org/lists/namedroppers/>

---

From: Edward Lewis <[email-masked]>
Date: Tue, 28 Mar 2006 14:07:23 -0500

>Anybody have an authoritative (haha) reference?

I'll nominate section 5.3.3. of rfc 1034:

# 5.3.3. Algorithm
#
# The top level algorithm has four steps:
#
#    1. See if the answer is in local information, and if so return
#       it to the client.

(Boom. Done. To OSI with the rest of the algorithm.)

5.3.3. is "pointed to" in 4.3.2:

4.3.2. Algorithm

# The actual algorithm used by the name server will depend on the local OS
# and data structures used to store RRs.  The following algorithm assumes
# that the RRs are organized in several tree structures, one for each
# zone, and another for the cache:
#
#   1. Set or clear the value of recursion available in the response
#      depending on whether the name server is willing to provide
#      recursive service.  If recursive service is available and
#      requested via the RD bit in the query, go to step 5,
#      otherwise step 2.
...
#   5. Using the local resolver or a copy of its algorithm (see
#      resolver section of this memo) to answer the query.  Store
#      the results, including any intermediate CNAMEs, in the answer
#     section of the response.

and: # 5. RESOLVERS
(to show that 5.3.3 is resolvers.)

-- 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Edward Lewis                                                +1-571-434-5468
NeuStar

Nothin' more exciting than going to the printer to watch the toner drain...

---

From: bert hubert <[email-masked]>
Date: Tue, 28 Mar 2006 21:24:00 +0200

> #    1. See if the answer is in local information, and if so return

Bit of solipsism here I think - this basically says the answer to an ANY
query is the answer.

> #     section of the response.

More of the same here. 

It might simply be that the original implementations copped out - it
requires extra work to "properly" support the ANY query because to do so you
need to cache the ANY answer as a block, with the TTD at now + min(ttls).
You also need to store it separately.

Also, you'd probably quickly be lying because if a record was added later
on, the ANY query would only return it once the first entry of the previous
ANY answer timed out.

Oh well - I'll implement the logic that PowerDNS does a full ANY query if
nothing is in the cache and if something is, just return what we have. 

I counted the number of ANY queries in a recent tcpdump, they are very rare
and appear to be only used by DNSRBL lookups.

Thanks.

---

From: Edward Lewis <[email-masked]>
Date: Tue, 28 Mar 2006 14:33:09 -0500

>Bit of solipsism here I think - this basically says the answer to an ANY

I suppose "See if an answer" would have been better.  But then again, 
I don't know the difference between its and it's, and between 
descendant and descendent.

>on, the ANY query would only return it once the first entry of the previous

The DNS was not intended to deal in temporal logic.  "Coherency" 
means ask any server now and get a consistent answer, nothing about 
asking two servers in time sequence.

In fact, you can ask a cache an "ANY" and then a specific for a 
record set it is missing.  The subsequent ANY ought to return what 
hasn't times out plus the specific on.

>nothing is in the cache and if something is, just return what we have.

Sounds like what I'd expect a server to do.

>and appear to be only used by DNSRBL lookups.

T_ANY is at best a debugging tool.  It has been used in the past to 
get mail records I think, but really, T_ANY is just for debugging and 
others trying to abuse the service.

IMHO.

---

From: Mark Andrews <[email-masked]>
Date: Wed, 29 Mar 2006 09:49:26 +1100

> others trying to abuse the service.

	T_ANY was used because old versions of named would return
	SERVFAIL if there wasn't a record of the requested type and
	named had detected a error while loading.  Some MTA's thought
	they needed to query for CNAME which almost always didn't
	exist and as people were really bad about checking their
	logs for errors this would just break email.  By querying
	for "*" you avoided this problem.

	named now hard fails on any zone file error which make it
	harder to ignore errors.

	Mark
--
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]

---

From: Kevin Darcy <[email-masked]>
Date: Wed, 29 Mar 2006 00:04:46 -0500

Edward Lewis wrote:

> I suppose "See if an answer" would have been better. But then again, I 
> don't know the difference between its and it's, and between descendant 
> and descendent. 

I agree. "The" answer is the full set of RRsets, from an authoritative 
source, which may require recursion. "An" answer is any of the RRsets 
that might happen to be laying around in some particular cache at some 
particular time. The distinction between the definite and indefinite 
article is critical, IMO. Almost as critical as the distinction between 
"its" and "it's" :-).

> The DNS was not intended to deal in temporal logic. "Coherency" means 
> ask any server now and get a consistent answer, nothing about asking 
> two servers in time sequence.
>
> In fact, you can ask a cache an "ANY" and then a specific for a record 
> set it is missing. The subsequent ANY ought to return what hasn't 
> times out plus the specific on.
>

I'd have to disagree. A resolver which is providing recursion should 
recurse for "the" answer. If the client didn't want recursion, i.e. if 
they only wanted "an" answer from their closest resolver, they wouldn't 
have set RD in the first place. So it's not appropriate to return a 
recursion-less answer for QTYPE=* while at the same time providing 
recursion-full answers to more specific QTYPEs of the same QNAME. And 
it's *doubly* inappropriate, to set RA in a response where the resolver 
declined to perform the recursion requested because of its 
discriminatory treatment of QTYPE=*.

> T_ANY is at best a debugging tool. It has been used in the past to get 
> mail records I think, but really, T_ANY is just for debugging and 
Um, RD=0 is the debugging tool. QTYPE=* could IMO be somewhat useful, in 
limited situations, if it weren't so crippled by poor/questionable 
resolver-implementation choices. Whether the abuse potential of an 
unfettered QTYPE=* would outweigh its usefulness, is a debatable point...

- Kevin

---

From: "Olaf M. Kolkman" <[email-masked]>
Date: Wed, 29 Mar 2006 09:44:15 +0200

Scripture reading time :-)....

> (to show that 5.3.3 is resolvers.)

Why did you skip 4?

#   4. Start matching down in the cache.  If QNAME is found in the
#      cache, copy all RRs attached to it that match QTYPE into the
#      answer section.  If there was no delegation from
#      authoritative data, look for the best one from the cache, and
#      put it in the authority section.  Go to step 6.

That tells us to first look at the cache and return anything that is  
there if there is anything at all. Only if there is nothing there we  
proceed to step 5.

> nothing is in the cache and if something is, just return what we have.
is spot on.

Or am I missing something.

--Olaf

-----------------------------------------------------------
Olaf M. Kolkman
NLnet Labs
http://www.nlnetlabs.nl/

---

From: bert hubert <[email-masked]>
Date: Wed, 29 Mar 2006 09:54:39 +0200

> >#      requested via the RD bit in the query, go to step 5,

Because of step 1.

---

From: Paul Vixie <[email-masked]>
Date: Wed, 29 Mar 2006 07:57:32 +0000

# That tells us to first look at the cache and return anything that is  there
# if there is anything at all. Only if there is nothing there we  proceed to
# step 5.
# 
# In other words, Bert's approach...is spot on.
# 
# Or am I missing something.

as a matter of practical consideration, even if the rules were to be changed
or clarified in a way that matched the common sense interpretation (which is
that if RD=1 and the content-in-case didn't come about due to QTYPE=*, one
should go fetch QTYPE=* from the authority server and update the cache thusly
before generating the response), there are so many servers who treat QTYPE=*
as a maintainance mode query ("tell me what's in the cache but don't change
it just because i'm asking you this question") that the "new rule" would be
pretty much vapid.  i'm in possession of at least one recursive nameserver
who will return current-cache-content unless that would be empty in which 
case a QTYPE=* query is issued toward an authority server.  all we know for
certain is, initiators of QTYPE=* queries toward caching forwarders are
expecting the unpredictable.  that won't change no matter what the scriptures
turn out to have intended.

---

From: "Olaf M. Kolkman" <[email-masked]>
Date: Wed, 29 Mar 2006 10:00:43 +0200

>>> #      requested via the RD bit in the query, go to step 5,

Doohhh...

--Olaf

---

From: Josh Littlefield <[email-masked]>
Date: Wed, 29 Mar 2006 10:17:37 -0500

> turn out to have intended.

I don't think this is accidental.  The example in 6.2.2 of RFC 1034 
(second and third packet diagrams) clearly shows a non-authoritative 
server responding with whatever is in cache.

The resolver algorithm in 5.3.3, which is referenced by step 5 in 4.3.2 
begins with:

    1. See if the answer is in local information, and if so return
       it to the client.

This is probably open to interpretation with respect to QTYPE=ANY 
queries, but the QTYPE matching described in 3.7.1 says QTYPE * (ANY) 
"matches all RR types".  It doesn't say anything about being guaranteed 
to produce all RRs for a name, just that it matches all RRs it is 
compared against.

RFC 1035 also has text that implies QTYPE=* returns whatever is in 
cache.  Section 7.1 (Transforming a user request into a query) says,

   "Where possible, the QTYPE and QCLASS should correspond to a single 
type and a single class, because this makes the use of cached data much 
simpler.  The reason for this is that the presence of data of one type 
in a cache doesn't confirm the existence or non-existence of data of 
other types, hence the only way to be sure is to consult an 
authoritative source"

It may not seem useful or intuitive to someone who wishes to retrieve 
all RRs for a name without going to the authoritative source, but I 
think the behavior of a caching server returning what's in cache when 
queried for QTYPE=* is a valid interpretation of the standard.

  -josh

-- 
=====================================================================
Josh Littlefield                                  Cisco Systems, Inc.
[email-masked]                             1414 Massachusetts Avenue
tel: 978-936-1379  fax: 978-936-2226       Boxborough, MA  01719-2205

---

From: Paul Vixie <[email-masked]>
Date: Wed, 29 Mar 2006 15:45:22 +0000

# I don't think this is accidental.

that's not always as straightforwardly obvious as just "read the examples".

# It may not seem useful or intuitive to someone who wishes to retrieve all
# RRs for a name without going to the authoritative source, but I think the
# behavior of a caching server returning what's in cache when queried for
# QTYPE=* is a valid interpretation of the standard.

that's why the last failed EDNS1 proposal (written in august 2002) had, in
addition to multiple QNAME support, a bit called RRD ("recursion really
desired").  i still think this should be explored.  time to repost the draft?

---

From: Ólafur Guðmundsson<[email-masked]>
Date: Wed, 29 Mar 2006 12:34:00 -0500

At 00:04 29/03/2006, Kevin Darcy wrote:
>>I suppose "See if an answer" would have been better. But then 
>>again, I don't know the difference between its and it's, and 
>>between descendant and descendent.
>
>I agree. "The" answer is the full set of RRsets, from an 
>authoritative source, which may require recursion. "An" answer is 
>any of the RRsets that might happen to be laying around in some 
>particular cache at some particular time. The distinction between 
>the definite and indefinite article is critical, IMO. Almost as 
>critical as the distinction between "its" and "it's" :-).

ANY != ALL
As this is ANY query the "An answer" is sufficient.

Paul's RDD bit proposal was to address this. It is also possible
to add a new query type (ALL) to differentiate between the two behaviors.

Not sure either effort is worth the deployment cost.
A simple FAQ/BCP that says if you want "The" ALL answer,
query authorative servers directly.

         Olafur

---

From: Edward Lewis <[email-masked]>
Date: Wed, 29 Mar 2006 14:10:37 -0500

Going back to the start of the thread...

Does anyone want a T_ALL?

The original post seemed more like a request for clarification than a 
request to do T_ALL.
-- 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Edward Lewis                                                +1-571-434-5468
NeuStar

---

From: bert hubert <[email-masked]>
Date: Wed, 29 Mar 2006 21:32:15 +0200

> Does anyone want a T_ALL?

I wouldn't say no to it, but what most clients actually want is probably
more along the lines of T_ADDR. I can't think of a use case where you'd just
want to have *everything*.

It would need to be an EDNS thing though.

Generally, I think there are bigger fish to fry in the DNS world.

My own favorite, and one that I think DNS operators are actually waiting
for, is the zone provisioning protocol as described in:
http://www.ops.ietf.org/lists/namedroppers/namedroppers.2003/msg00565.html

And in this message to dnsop, which appears not to have arrived or hasn't
been archived:

I've been pondering writing a 'zone provisioning protocol', from
http://wiki.powerdns.com/projects/trac/wiki/TodoList :

 The big idea waiting to happen is the Zone Provisioning Protocol which
 allows 'dns peers' to exchange zones, or force resyncs (regardless of SOA
 serial numbers). This might live outside of PowerDNS

Would anybody be interested in this?

My idea would be to keep this as simple as possible, but still provide for
zones to be provisioned and deprovisioned by different parties, ie, you get
to deprovision only the zones you've previously provisioned etc.

It is highly probable that this protocol will not be on port 53, nor use DNS
messages. It would only talk about DNS.

Let me know.

---

From: Mark Andrews <[email-masked]>
Date: Thu, 30 Mar 2006 09:14:15 +1100

> messages. It would only talk about DNS.

	It would be interesting.  Whatever it is would need to be
	extensible and handle vendor specific extensions.
--
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]

---

From: Paul Vixie <[email-masked]>
Date: Thu, 30 Mar 2006 02:05:07 +0000

# > I've been pondering writing a 'zone provisioning protocol', from
# > http://wiki.powerdns.com/projects/trac/wiki/TodoList :
# > 
# >  The big idea waiting to happen is the Zone Provisioning Protocol which
# >  allows 'dns peers' to exchange zones, or force resyncs (regardless of SOA
# >  serial numbers). This might live outside of PowerDNS
# > 
# > Would anybody be interested in this?

i have a proposal for this, vendor independent, prototyped as some perl
scripts.  i'll see if the academic paper i submitted has a URL yet.

---

From: Kevin Darcy <[email-masked]>
Date: Wed, 29 Mar 2006 22:57:17 -0500

Josh Littlefield wrote:

>> # # In other words, Bert's approach...is spot on.
>> # # Or am I missing something.
>>
>> who will return current-cache-content unless that would be empty in 
>> which case a QTYPE=* query is issued toward an authority server.  all 
>> we know for

No, I think that's a misreading. The intro to the examples section 
clearly states that the example queries are RD=0 unless otherwise 
specified. So the example you cite is putatively an RD=0 query, which is 
apples and oranges from what we're discussing in this thread.

> The resolver algorithm in 5.3.3, which is referenced by step 5 in 
> 4.3.2 begins with:
>
> "matches all RR types".  It doesn't say anything about being 
> guaranteed to produce all RRs for a name, just that it matches all RRs 
> it is compared against. 

I think you're misreading a matching rule as a step in, or a statement 
about the resolution algorithm. Saying that X matches Y doesn't 
_ipso_facto_ imply anything about how the Y data is fetched, where it's 
fetched from, what effect the RD flag has, how much work should be 
expended trying to get the Y answer, etc. It's simply saying that 
*if*and/or*when*and/or*once*the*data*becomes*available* (by whatever 
means, maybe it just materialized out of thin air), then a match either 
occurs or doesn't occur. It's more declaritive than procedural. By 
reading 5.3.3 as a statement about the resolution algorithm, I think 
you're imparting far too much passivity to that algorithm in the 
(arbitrarily) special case of QTYPE=*. The *general* resolution 
algorithm, in simplistic terms, is to recurse for the answer that the 
client requested, when recursion was requested and the resolver is 
willing (as defined by its administrator's conscious policy decisions) 
and able to provide it and "the answer" (the definition of which can and 
has been quibbled over) is not already available in cache. Why make a 
special case for QTYPE=* queries in the resolution algorithm, just 
because of the way the matching rule was described in a different part 
of the RFC set?

> type and a single class, because this makes the use of cached data 
> much simpler.  The reason for this is that the presence of data of one 
> type in a cache doesn't confirm the existence or non-existence of data 
> of other types, hence the only way to be sure is to consult an 
> authoritative source" 

Actually, I read it exactly the opposite way, i.e. as an operational 
recommendation to users/applications to *minimize* their use of QTYPE=* 
and/or QCLASS=* queries, since such queries *force* recursive resolvers 
to go out and fetch fresh data from authoritative sources in order to be 
"sure" that the answers returned are complete. The motivation for this 
paragraph seems to be that overuse of QTYPE=* and/or QCLASS=* might lead 
to performance/scalability problems.

                                                            - Kevin

---

From: bert hubert <[email-masked]>
Date: Thu, 30 Mar 2006 08:40:58 +0200

> scripts.  i'll see if the academic paper i submitted has a URL yet.

That would be very cool, especially if we can whip it up old-school
'implement first, spec later' style, and keep things simple.
	
	Bert