Faster DNS Reverse Lookup for a Large Batch of IP Addresses

Question

For analytics purposes, I would like to perform reverse DNS queries for large batches of IP addresses. The "big" meaning is at least tens of thousands per hour. I am looking for ways to increase processing speed, i.e. Reduce processing time per batch.

Wrapping the asynchronous version of Dns.GetHostEntry into expected tasks has already helped a lot (compared to consecutive requests), which has led to appox bandwidth. 100-200 IPs / second:

static async Task DoReverseDnsLookups()
{
    // in reality, thousands of IPs
    var ips = new[] { "173.194.121.9", "173.252.110.27", "98.138.253.109" }; 
    var hosts = new Dictionary<string, string>();

    var tasks =
        ips.Select(
            ip =>
                Task.Factory.FromAsync(Dns.BeginGetHostEntry,
                    (Func<IAsyncResult, IPHostEntry>) Dns.EndGetHostEntry, 
                    ip, null)
                    .ContinueWith(t => 
                    hosts[ip] = ((t.Exception == null) && (t.Result != null)) 
                               ? t.Result.HostName : null));

    var start = DateTime.UtcNow;
    await Task.WhenAll(tasks);
    var end = DateTime.UtcNow;

    Console.WriteLine("Resolved {0} IPs in {1}, that {2}/sec.", 
      ips.Count(), end - start, 
      ips.Count() / (end - start).TotalSeconds);
}

Any ideas for further improving your processing speed?

For example, is there a way to send a packet of IP addresses to a DNS server?

, , / - , " m .

Answer 1

• , TPL Dataflow ActionBlock , . ActionBlock MaxDegreeOfParallelism TPL , , :

var block = new ActionBlock<string>(
    async ip => 
    { 
        try
        {
            var host = (await Dns.GetHostEntryAsync(ip)).HostName;
            if (!string.IsNullOrWhitespace(host))
            {
                hosts[ip] = host;
            }
        }
        catch
        {
            return;
        }
    },
    new ExecutionDataflowBlockOptions { MaxDegreeOfParallelism = 5000});

• , ip .

• .net Dns, DNS (, LLMNR), . DNS-, , ARSoft.Tools.Net.

---

P.S: :

• GetHostEntryAsync FromAsync
• , ConcurrentDictionary.

Answer 2

, , :

• , . , .
• DNS- , . 4 , , Windows. , , , , cahce.
• , Task Parallelism, DNS- . , , DNS-, .

, .