]> Negative Caching of DNS Resolution Failures Verisign
12061 Bluemont Way Reston VA 20190 United States of America +1 703 948-3200 dwessels@verisign.com https://verisign.com
Verisign
12061 Bluemont Way Reston VA 20190 United States of America +1 703 948-3200 wicarroll@verisign.com https://verisign.com
Verisign
12061 Bluemont Way Reston VA 20190 United States of America +1 703 948-3200 mthomas@verisign.com https://verisign.com
ops dnsop DNS Negative Caching In the DNS, resolvers employ caching to reduce both latency for end users and load on authoritative name servers. The process of resolution may result in one of three types of responses: (1) a response containing the requested data, (2) a response indicating the requested data does not exist, or (3) a non-response due to a resolution failure in which the resolver does not receive any useful information regarding the data's existence. This document concerns itself only with the third type. RFC 2308 specifies requirements for DNS negative caching. There, caching of TYPE 2 responses is mandatory and caching of TYPE 3 responses is optional. This document updates RFC 2308 to require negative caching for DNS resolution failures. RFC 4035 allows DNSSEC validation failure caching. This document updates RFC 4035 to require caching for DNSSEC validation failures. RFC 4697 prohibits aggressive requerying for NS records at a failed zone's parent zone. This document updates RFC 4697 to expand this requirement to all query types and to all ancestor zones.
Introduction Caching has always been a fundamental component of DNS resolution on the Internet. For example, states:
The sheer size of the database and frequency of updates suggest that it must be maintained in a distributed manner, with local caching to improve performance.
The early DNS RFCs (, , , and ) primarily discuss caching in the context of what calls "positive responses", that is, when the response includes the requested data. In this case, a TTL is associated with each Resource Record (RR) in the response. Resolvers can cache and reuse the data until the TTL expires. describes negative response caching, but notes it is optional and only talks about name errors (NXDOMAIN). This is the origin of using the SOA MINIMUM field as a negative caching TTL. updated to specify new requirements for DNS negative caching, including making it mandatory for caching resolvers to cache name error (NXDOMAIN) and no data (NODATA) responses when an SOA record is available to provide a TTL. further specified optional negative caching for two DNS resolution failure cases: server failure and dead/unreachable servers. This document updates to require negative caching of all DNS resolution failures and provides additional examples of resolution failures, to require caching for DNSSEC validation failures, as well as to expand the scope of prohibiting aggressive requerying for NS records at a failed zone's parent zone to all query types and to all ancestor zones.
Motivation Operators of DNS services have known for some time that recursive resolvers become more aggressive when they experience resolution failures. A number of different anecdotes, experiments, and incidents support this claim. In December 2009, a secondary server for a number of in-addr.arpa subdomains saw its traffic suddenly double, and queries of type DNSKEY in particular increase by approximately two orders of magnitude, coinciding with a DNSSEC key rollover by the zone operator . This predated a signed root zone, and an operating system vendor was providing non-root trust anchors to the recursive resolver, which became out of date following the rollover. Unable to validate responses for the affected in-addr.arpa zones, recursive resolvers aggressively retried their queries. In 2016, the Internet infrastructure company Dyn experienced a large attack that impacted many high-profile customers. As documented in a technical presentation detailing the attack (see ), Dyn staff wrote:
At this point we are now experiencing botnet attack traffic and what is best classified as a "retry storm" Looking at certain large recursive platforms > 10x normal volume
In 2018, the root zone Key Signing Key (KSK) was rolled over . Throughout the rollover period, the root servers experienced a significant increase in DNSKEY queries. Before the rollover, a.root-servers.net and j.root-servers.net together received about 15 million DNSKEY queries per day. At the end of the revocation period, they received 1.2 billion per day: an 80x increase. Removal of the revoked key from the zone caused DNSKEY queries to drop to post-rollover but pre-revoke levels, indicating there is still a population of recursive resolvers using the previous root trust anchor and aggressively retrying DNSKEY queries. In 2021, Verisign researchers used botnet query traffic to demonstrate that certain large public recursive DNS services exhibit very high query rates when all authoritative name servers for a zone return refused (REFUSED) or server failure (SERVFAIL) responses (see ). When the authoritative servers were configured normally, query rates for a single botnet domain averaged approximately 50 queries per second. However, with the servers configured to return SERVFAIL, the query rate increased to 60,000 per second. Furthermore, increases were also observed at the root and Top-Level Domain (TLD) levels, even though delegations at those levels were unchanged and continued operating normally. Later that same year, on October 4, Facebook experienced a widespread and well-publicized outage . During the 6-hour outage, none of Facebook's authoritative name servers were reachable and did not respond to queries. Recursive name servers attempting to resolve Facebook domains experienced timeouts. During this time, query traffic on the .COM/.NET infrastructure increased from 7,000 to 900,000 queries per second .
Related Work describes negative caching for four types of DNS queries and responses: name errors, no data, server failures, and dead/unreachable servers. It places the strongest requirements on negative caching for name errors and no data responses, while server failures and dead servers are left as optional. is a Best Current Practice that documents observed resolution misbehaviors. It describes a number of situations that can lead to excessive queries from recursive resolvers, including requerying for delegation data, lame servers, responses blocked by firewalls, and records with zero TTL. makes a number of recommendations, varying from "SHOULD" to "MUST". describes "The DNS thundering herd problem" as a situation arising when cached data expires at the same time for a large number of users. Although that document is not focused on negative caching, it does describe the benefits of combining multiple identical queries to upstream name servers. That is, when a recursive resolver receives multiple queries for the same name, class, and type that cannot be answered from cached data, it should combine or join them into a single upstream query rather than emit repeated identical upstream queries. , "", includes a section that describes the phenomenon known as "Birthday Attacks". Here, again, the problem arises when a recursive resolver emits multiple identical upstream queries. Multiple outstanding queries make it easier for an attacker to guess and correctly match some of the DNS message parameters, such as the port number and ID field. This situation is further exacerbated in the case of timeout-based resolution failures. Of course, DNSSEC is a suitable defense to spoofing attacks. describes "". This permits a recursive resolver to return possibly stale data when it is unable to refresh cached, expired data. It introduces the idea of a failure recheck timer and says:
Attempts to refresh from non-responsive or otherwise failing authoritative nameservers are recommended to be done no more frequently than every 30 seconds.
Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.
DNS transport:
In this document, "DNS transport" means a protocol used to transport DNS messages between a client and a server. This includes "classic DNS" transports, i.e., DNS-over-UDP and DNS-over-TCP , as well as newer encrypted DNS transports, such as DNS-over-TLS , DNS-over-HTTPS , DNS-over-QUIC , and similar communication of DNS messages using other protocols. Note: at the time of writing, not all DNS transports are standardized for all types of servers but may become standardized in the future.
Conditions That Lead to DNS Resolution Failures A DNS resolution failure occurs when none of the servers available to a resolver client provide any useful response data for a particular query name, type, and class. A response is considered useful when it provides either the requested data, a referral to a descendant zone, or an indication that no data exists at the given name. It is common for resolvers to have multiple servers from which to choose for a particular query. For example, in the case of stub-to-recursive, the stub resolver may be configured with multiple recursive resolver addresses. In the case of recursive-to-authoritative, a given zone usually has more than one name server (NS record), each of which can have multiple IP addresses and multiple DNS transports. Nothing in this document prevents a resolver from retrying a query at a different server or the same server over a different DNS transport. In the case of timeouts, a resolver can retry the same server and DNS transport a limited number of times. If any one of the available servers provides a useful response, then it is not considered a resolution failure. However, if none of the servers for a given query tuple <name, type, class> provide a useful response, the result is a resolution failure. Note that NXDOMAIN and NOERROR/NODATA responses are not conditions for resolution failure. In these cases, the server is providing a useful response, indicating either that a name does not exist or that no data of the requested type exists at the name. These negative responses can be cached as described in . The remainder of this section describes a number of different conditions that can lead to resolution failure. This section is not exhaustive. Additional conditions may be expected to cause similar resolution failures.
SERVFAIL Responses Server failure is defined in as: "The name server was unable to process this query due to a problem with the name server." A server failure is signaled by setting the RCODE field to SERVFAIL. Authoritative servers return SERVFAIL when they don't have any valid data for a zone. For example, a secondary server has been configured to serve a particular zone but is unable to retrieve or refresh the zone data from the primary server. Recursive servers return SERVFAIL in response to a number of different conditions, including many described below. Although the extended DNS errors method exists "primarily to extend SERVFAIL to provide additional information," it "does not change the processing of RCODEs" . This document operates at the level of resolution failure and does not concern particular causes.
REFUSED Responses A name server returns a message with the RCODE field set to REFUSED when it refuses to process the query, e.g., for policy or other reasons . Authoritative servers generally return REFUSED when processing a query for which they are not authoritative. For example, a server that is configured to be authoritative for only the example.net zone may return REFUSED in response to a query for example.com. Recursive servers generally return REFUSED for query sources that do not match configured access control lists. For example, a server that is configured to allow queries from only 2001:db8:1::/48 may return REFUSED in response to a query from 2001:db8:5::1.
Timeouts and Unreachable Servers A timeout occurs when a resolver fails to receive any response from a server within a reasonable amount of time. Additionally, a DNS transport may more quickly indicate lack of reachability in a way that wouldn't be considered a timeout: for example, an ICMP port unreachable message, a TCP "connection refused" error, or a TLS handshake failure. refers to these conditions collectively as "dead / unreachable servers". Note that resolver implementations may have two types of timeouts: a smaller timeout that might trigger a query retry and a larger timeout after which the server is considered unresponsive. discusses the requirements for resolvers when retrying queries. Timeouts can present a particular problem for negative caching, depending on how the resolver handles multiple outstanding queries for the same <query name, type, class> tuple. For example, consider a very popular website in a zone whose name servers are all unresponsive. A recursive resolver might receive tens or hundreds of queries per second for that website. If the recursive server implementation joins these outstanding queries together, then it only sends one recursive-to-authoritative query for the numerous pending stub-to-recursive queries. However, if the implementation does not join outstanding queries together, then it sends one recursive-to-authoritative query for each stub-to-recursive query. If the incoming query rate is high and the timeout is large, this might result in hundreds or thousands of recursive-to-authoritative queries while waiting for an authoritative server to time out. A recursive resolver that does not join outstanding queries together is more susceptible to Birthday Attacks (), especially when those queries result in timeouts.
Delegation Loops A delegation loop, or cycle, can occur when one domain utilizes name servers in a second domain, and the second domain uses name servers in the first. For example: In this example, no names under foo.example or example.com can be resolved because of the delegation loop. Note that a delegation loop may involve more than two domains. A resolver that does not detect delegation loops may generate DDoS-levels of attack traffic to authoritative name servers, as documented in the TsuNAME vulnerability .
Alias Loops An alias loop, or cycle, can occur when one CNAME or DNAME RR refers to a second name, which, in turn, is specified as an alias for the first. For example: The need to detect CNAME loops has been known since at least , which states in Section :
Of course, by the robustness principle, domain software should not fail when presented with CNAME chains or loops; CNAME chains should be followed and CNAME loops signalled as an error.
DNSSEC Validation Failures For zones that are signed with DNSSEC, a resolution failure can occur when a security-aware resolver believes it should be able to establish a chain of trust for an RRset but is unable to do so, possibly after trying multiple authoritative name servers. DNSSEC validation failures may be due to signature mismatch, missing DNSKEY RRs, problems with denial-of-existence records, clock skew, or other reasons. already discusses the requirements and reasons for caching validation failures. of this document strengthens those requirements.
FORMERR Responses A name server returns a message with the RCODE field set to FORMERR when it is unable to interpret the query . FORMERR responses are often associated with problems processing Extension Mechanisms for DNS (EDNS(0)) . Authoritative servers may return FORMERR when they do not implement EDNS(0), or when EDNS(0) option fields are malformed, but not for unknown EDNS(0) options. Upon receipt of a FORMERR response, some recursive clients will retry their queries without EDNS(0), while others will not. Nonetheless, resolution failures from FORMERR responses are rare.
Requirements for Caching DNS Resolution Failures
Retries and Timeouts A resolver MUST NOT retry a given query to a server address over a given DNS transport more than twice (i.e., three queries in total) before considering the server address unresponsive over that DNS transport for that query. A resolver MAY retry a given query over a different DNS transport to the same server if it has reason to believe the DNS transport is available for that server and is compatible with the resolver's security policies. This document does not place any requirements on how long an implementation should wait before retrying a query (aka a timeout value), which may be implementation or configuration dependent. It is generally expected that typical timeout values range from 3 to 30 seconds.
Caching Resolvers MUST implement a cache for resolution failures. The purpose of this cache is to eliminate repeated upstream queries that cannot be resolved. When an incoming query matches a cached resolution failure, the resolver MUST NOT send any corresponding outgoing queries until after the cache entries expire. Implementation details for such a cache are not specified in this document. The implementation might cache different resolution failure conditions differently. For example, DNSSEC validation failures might be cached according to the queried name, class, and type, whereas unresponsive servers might be cached only according to the server's IP address. Developers should document their implementation choices so that operators know what behaviors to expect when resolution failures are cached. Resolvers MUST cache resolution failures for at least 1 second. Resolvers MAY cache different types of resolution failures for different (i.e., longer) amounts of time. Consistent with , resolution failures MUST NOT be cached for longer than 5 minutes. The minimum cache duration SHOULD be configurable by the operator. A longer cache duration for resolution failures will reduce the processing burden from repeated queries but may also increase the time to recover from transitory issues. Resolvers SHOULD employ an exponential or linear backoff algorithm to increase the cache duration for persistent resolution failures. For example, the initial time for negatively caching a resolution failure might be set to 5 seconds and increased after each retry that results in another resolution failure, up to a configurable maximum, not to exceed the 5-minute upper limit. Notwithstanding the above, resolvers SHOULD implement measures to mitigate resource exhaustion attacks on the failed resolution cache. That is, the resolver should limit the amount of memory and/or processing time devoted to this cache.
Requerying Delegation Information identifies circumstances in which:
...every name server in a zone's NS RRSet is unreachable (e.g., during a network outage), unavailable (e.g., the name server process is not running on the server host), or misconfigured (e.g., the name server is not authoritative for the given zone, also known as "lame").
It prohibits unnecessary "aggressive requerying" to the parent of a non-responsive zone by sending NS queries. The problem of aggressive requerying to parent zones is not limited to queries of type NS. This document updates the requirement from to apply more generally:
Upon encountering a zone whose name servers are all non-responsive, a resolver MUST cache the resolution failure. Furthermore, the resolver MUST limit queries to the non-responsive zone's parent zone (and to other ancestor zones) just as it would limit subsequent queries to the non-responsive zone.
DNSSEC Validation Failures states:
To prevent such unnecessary DNS traffic, security-aware resolvers MAY cache data with invalid signatures, with some restrictions.
This document updates with the following, stronger, requirement:
To prevent such unnecessary DNS traffic, security-aware resolvers MUST cache DNSSEC validation failures, with some restrictions.
One of the restrictions mentioned in is to use a small TTL when caching data that fails DNSSEC validation. This is, in part, because the provided TTL cannot be trusted. The advice from herein can be used as guidance on TTLs for caching DNSSEC validation failures.
IANA Considerations This document has no IANA actions.
Security Considerations As noted in , an attacker might attempt a resource exhaustion attack by sending queries for a large number of names and/or types that result in resolution failure. Resolvers SHOULD implement measures to protect themselves and bound the amount of memory devoted to caching resolution failures. A cache poisoning attack (see ) resulting in denial of service may be possible because failure messages cannot be signed. An attacker might generate queries and send forged failure messages, causing the resolver to cease sending queries to the authoritative name server (see for a similar "data corruption attack" and Section 5.2 of for a "DNSDoS attack"). However, this would require continued spoofing throughout the backoff period and repeated attacks due to the 5-minute cache limit. As in , this attack's effects would be "localized and of limited duration".
Privacy Considerations This specification has no impact on user privacy.
References Normative References Informative References TuDoor Attack: Systematically Exploring and Exploiting Logic Vulnerabilities in DNS Response Pre-processing with Malformed Packets IEEE Symposium on Security and Privacy (SP) Botnet Traffic Observed at Various Levels of the DNS Hierarchy More details about the October 4 outage Observations on Resolver Behavior During DNS Outages Verisign TsuNAME: exploiting misconfiguration and vulnerability to DDoS DNS IMC '21: Proceedings of the 21st ACM Internet Measurement Conference, Pages 398-418 Roll Over and Die? Dyn, DDoS, and DNS Roll, Roll, Roll Your Root: A Comprehensive Analysis of the First Ever DNSSEC Root KSK Rollover IMC '19: Proceedings of the Internet Measurement Conference, Pages 1-14
Acknowledgments The authors wish to thank , , , , , , , , , and other members of the DNSOP Working Group for their feedback and contributions.