DNS query duplication

The tendency for free resources to be exploited to the point of over-consumption and even exhaustion is a well-known economic and social concept, frequently described with the term the tragedy of the commons. Because use of the resource costs each individual user nothing, there is no financial incentive for any user to moderate their consumption of the common resource. Over-consumption inevitably ensues!

In the DNS, name resolution space queries are free. To what extent do we see over-querying by recursive resolvers in the Domain Name System (DNS)? There is certainly some motivation for a recursive resolver to query enthusiastically, given that a strategy of what one could term over-querying can work around slow or unresponsive nameservers, or repair UDP datagram loss.

Over-querying describes a behaviour where a DNS resolution system does not wait for a local timer to expire to conclude that an original UDP-based query has been lost. Rather, it repeats the query well before a reasonable timeout interval has elapsed. The apparent thinking behind this rapid-fire over-querying behaviour is that in the event of query loss, the subsequent query may be successful. The response to this subsequent query will arrive back at the resolver soon after the response to the original query would've arrived, saving the resolver (and the end user) from waiting for a query timeout interval. If both queries are successful, then the resolver will simply discard the responses to the later query.

As queries are free, the cost to the authoritative server in generating the answer twice is borne by the authoritative server and not the recursive resolver. Such a behaviour can improve the responsiveness and resilience of a recursive resolver, and the cost of generating these additional responses is met by others. In other words, if there is no cost to the resolver in this behaviour, then there is no motivation to adopt a more conservative query stance!

The question here is: To what extent are DNS resolvers in today's Internet excessive in generating duplicate queries?

We'll use the APNIC Labs measurement data generated on a single day, 1 June 2026, for this study. This system uses an online ad presentation system to seed measurement tasks on a collection of users' browsers. The system generates 35 million advertisement (ad) presentations for each 24 hours, spread over users from all parts of the Internet. Each individual measurement ad presentation contains up to 15 different component URL fetches (for different measurement tasks), where each URL contains a unique DNS name to resolve. A unique label ensures that no DNS caching is used to resolve these names. All end user DNS resolution requests are seen as queries at the measurement DNS authoritative servers and not answered by intermediate DNS caches held by resolvers.

There are various forms of DNS behaviours that are configured into the APNIC measurement set, some of which are deliberately intended to generate additional queries. For example, a DNS name where the authoritative nameserver always returns a response code. Here, we are looking at just those DNS query names that are served by a dual-stack nameserver that always promptly responds to A and AAAA query types. UDP is not a reliable transport protocol, and some datagram loss is expected. It would be reasonable to anticipate an overall query loss rate of 2% or lower in most circumstances. We expect to see a comparable query duplication rate when looking at a random sample set of DNS data from this experiment.

The measurement experiment directs users located in each of these regions to a DNS server that is physically located in the same region as the user. We use a division of the Internet into six major areas, as shown in Table 1.

Table 1 - APNIC Lab's measurement regions.

The dataset that was gathered during this 24-hour period is shown in Table 2. We are distinguishing between an original query and successive duplicate queries, where a duplicate is defined as a query that has the same query name and query type.

Table 2 - Query duplication measurement results.

The average for the full set of data for this day shows a query duplication rate of 38% of all seen query name/query type couplets. Users in the (Indian Subcontinent) region have a far higher query duplication rate of 61%, and the (China/ Hong Kong) region shows a 47% query duplication rate. The (North America) has a query name duplication rate of 28%, and (Europe) has a 27% query name duplication rate. This data indicates that there is a more prevalent DNS resolution behaviour that duplicates DNS queries in the Indian and Chinese regions.

Of course, average values can be misleading, and average values can be skewed by a small number of DNS resolution environments performing many duplicate queries. The highest number of duplicate queries was 8,771 in this 24-hour observation period, observed at the eu server in Frankfurt. The overall distribution of the number of duplicates of each of the queries that are part of this measurement is shown in Figure 1.

90% of all duplicate query sets for a single query name / query type couplet occur three or fewer times, and 58% of duplicates are observed as a single duplicate of the original query. Just 1% of duplicate query sets are sets with ten or more duplicate queries. The most common form of duplicate query behaviour is where the original DNS query is duplicated just once.

A DNS resolver using a UDP-based query will conventionally perform a repeat query if a UDP timer expires, as it is attempting to recover from the situation where the sender is assuming that the UDP query or response datagram has been dropped. A DNS recursive resolver will also perform a re-query when the earlier DNS response has been cached and the response's cache time to live (TTL) has expired. We can discern between these two resolver behaviours by looking at the elapsed time between the original query and the duplicate query.

Figure 2 shows the duplicate query profile over the 24-hour period, looking at the elapsed time between the original query and the duplicate query.

As well as an extremely large collection of duplicates in the period immediately following the initial query, there are visible local peaks in duplicate queries at periods of 6, 12, and 18 hours after the initial query, and a second repeat pattern at 8 and 16 hours. The TTL value of DNS responses in this measurement is set to 60 seconds, so these far longer periods associated with duplicate queries are likely to be some form of resolver implementation behaviour, attempting to perform a refresh of earlier cached data that was retrieved hours earlier.

Figure 3 looks at the first 300 seconds between the initial query and the duplicate query, using a unit of queries per second. In this figure, there are local peaks at 60, 120, 180, and 240 seconds, which are likely to be related to TTL expiration in the resolver's cache. The cache refresh is supposed to be triggered by an incoming query for the cached query name and query type.

However, the nature of this measurement is that each DNS query name is uniquely generated for a single use by a single user. There are not supposed to be duplicate queries, and there are not supposed to be query-triggered cache refresh queries. It is still the case that the number of these longer-period duplicates is far smaller than the number of duplicate queries that occur in the first second.

Figure 4 looks at the query duplication profile in the first second following the initial query.

There are local peaks at 320ms, 750ms, and 800ms, as well as a 1-second peak.

45% of all duplicate queries occur in the first 10ms following the initial query (Figure 5). This indicates that the major component of the duplicate DNS query behaviour we are observing is not due to re-querying following UDP-related timeouts in the part of the sender. This is the result of some other resolver behaviour mode that is observed at the authoritative server as a rapid-fire couplet of queries for the same query name and query type (Figure 6).

An interpretation of this behaviour is that the resolver could be attempting to improve the performance and resilience of the DNS query process by sending the two queries back-to-back using some element of difference that improves the overall resilience of the process. For example, a resolver could generate queries to two different authoritative nameservers at the same time, and it could then act upon the first nameserver to respond. Or it could use IPv4 and IPv6 transports. Or the resolver could both forward the query to an open recursive resolver and attempt to resolve the name itself.

However, we only have one authoritative nameserver in this experiment, so using multiple authoritative nameservers in some form of resolution race is not an option here. We can measure the diversity of resolver IP addresses generating the queries in the first 100ms after the initial query. Is the query being passed to multiple recursive resolvers to set up a race between resolvers to resolve a name in an effort to optimize resolution performance and resolution resilience? Or, are there large volumes of a degenerate form of query duplication where the same recursive resolver simply duplicates the query and sends it to the same authoritative nameserver in a back-to-back manner?

Of these queries, we observed 56,498,331 using a different IP address than the initial query, which is 82% of these duplicate queries, and 12,091,477 using the same IP address as the initial query.

We observed that 48% of the duplicating query sets are IPv4 only, 12% are IPv6 only, and the remaining 40% have both IPv4 and IPv6 queries. It's possible to understand the motivation of a strategy of performing a rapid query duplication by asking on both IPv4 and IPv6. It's not exactly friendly to the network or to the authoritative server, as it gratuitously doubles the authoritative server's query load, but from a strictly selfish perspective, it can add a level of resilience and fallback performance to the resolution operation.

In terms of the networks that host these recursive resolvers, 3,572,156 query sets (5% of the total) have directed their queries to multiple recursive resolvers that are in multiple networks (different origin Autonomous Systems (ASes), while 41% use resolvers in just one network, using the same network as the end client. 11,167,864 query sets (16% of the total) use a known open resolver. Again, this rapid splaying of queries to multiple recursive resolvers can be self-justified as a measure intended to improve the resilience and performance of a local DNS resolution system, while it exacts a cost in increased query loads on the authoritative servers, as it doubles the query load.

The DNS resolution behaviour that is challenging to justify is the rapid query duplication being performed by the same resolver IP address to the same authoritative server. This behaviour appears to be quite commonplace, with 6,117,444 query sets from a total of 44,810,600 query sets (14%) that were observed to have this form of single resolver query duplication.

Which networks host these duplicating resolvers? Table 3 lists the 25 networks that host resolvers with the highest total of same-resolver rapid query duplication levels within each network.

Table 3 - Query self-duplication rates, top 25 networks.

The open resolvers, operated by Google and Cloudflare, are listed here, but with overall duplication rates of 0% and 3%, this level of query duplication is relatively minor. Of greater concern are resolvers located in Reliance Joi and Bharti Airtel in India, Viettel and VNPT in Viet Nam, and Telekomunikasi Selular in Indonesia. The high query duplication rates seen in the DNS resolution systems used by users served by Deutsche Telekom in Germany, Softbank in Japan, Amazon O2, and Bell Canada are also curious. These are all very high-volume DNS resolution environments.

The top 25 individual resolvers that are generating these rapid self-replicated queries are shown in Table 4.

Table 4 - Query self-duplication rates, top 25 resolver IP addresses.

Most of the resolvers in this table do not perform this rapid query duplication in every case, although the resolvers used in Fibercom Pakistan, One Hungary, and Optimum Telecom Algeria exhibit this behaviour in more than 90% of observed cases.

It's not clear what is causing this DNS query behaviour. It's unlikely that this is the result of a deliberate configuration choice, so a more likely explanation is that this is anomalous behaviour associated with the front end of a DNS resolverfarm. Such DNS resolver farms are commonly used by access ISPs, as they allow a large-scale DNS resolution system to be implemented across multiple DNS engines, while simplifying the end clients' configuration by pushing out a small set of DNS resolver IP addresses that reference the load-balancing front-end machinery of the farm.

One explanation of this observed duplication behaviour is that incoming DNS queries, which are normally passed to just one instance of a local DNS resolver engine in the farm, are being passed to multiple resolver engines, possibly by accident, or because of some transient overload condition or similar unintentional behaviour. It is unlikely that this failure will be visible to either the end user or the network operator, as the failure is not a case of inability to resolve a DNS name, or to provide an incorrect response, but a case where more queries than are normally required are being generated. If this is an intermittent behaviour for some DNS resolution systems, as appears to be the case in the collected data, then detection of this behaviour is highly unlikely.

It's often said of the DNS that it's a system that simply could not work in any usable manner were it not for caching. Not only does caching suppress DNS query volumes and improve DNS response times, but caching can also mask over many forms of otherwise anomalous resolver behaviour.

We are amplifying this behaviour in our measurement experiment by using unique DNS labels, bypassing local DNS caches. This rapid duplication of a query that arrives at an authoritative nameserver would appear only at intervals bounded by the TTLs using the DNS data, and it is unlikely that this would form a significant portion of normal DNS traffic volumes coming from users, and as such, it's a DNS resolution behaviour that is somewhat unlikely to be detected.

The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.