All geek questions in one place

Python and Twisted DNS Server - Asynchronous Issues

I am trying to create a database-driven DNS server (in particular, to process only MX records and push everything else up) in Twisted using Python 2.7. The code below works (in terms of getting the result), but does not work asynchronously. Instead, any DNS queries entering the system block the entire program from any other queries until a response is received to the first. We need to scale this, and at the moment we cannot understand where we did wrong. If someone has a working example to share or see a problem, we will be infinitely grateful.

import settings import db from twisted.names import dns, server, client, cache from twisted.application import service, internet from twisted.internet import defer class DNSResolver(client.Resolver): def __init__(self, servers): client.Resolver.__init__(self, servers=servers) @defer.inlineCallbacks def _lookup_mx_records(self, hostname, timeout): # Check the DB to see if we handle this domain. mx_results = yield db.get_domain_mx_record_list(hostname) if mx_results: defer.returnValue( [([dns.RRHeader(hostname, dns.MX, dns.IN, settings.DNS_TTL, dns.Record_MX(priority, forward, settings.DNS_TTL)) for forward, priority in mx_results]), (), ()]) # If the hostname isn't in the DB, we forward # to our upstream DNS provider (8.8.8.8). else: i = yield self._lookup(hostname, dns.IN, dns.MX, timeout) defer.returnValue(i) def lookupMailExchange(self, name, timeout=None): """ The twisted function which is called when an MX record lookup is requested. :param name: The domain name being queried for (eg example.org). :param timeout: Time in seconds to wait for the query response. (optional, default: None) :return: A DNS response for the record query. """ return self._lookup_mx_records(name, timeout) # App name, UID, GID to run as. (root/root for port 53 bind) application = service.Application('db_driven_dns', 1, 1) # Set the secondary resolver db_dns_resolver = DNSResolver(settings.DNS_NAMESERVERS) # Create the protocol handlers f = server.DNSServerFactory(caches=[cache.CacheResolver()], clients=[db_dns_resolver]) p = dns.DNSDatagramProtocol(f) f.noisy = p.noisy = False # Register as a tcp and udp service ret = service.MultiService() PORT=53 for (klass, arg) in [(internet.TCPServer, f), (internet.UDPServer, p)]: s = klass(PORT, arg) s.setServiceParent(ret) # Run all of the above as a twistd application ret.setServiceParent(service.IServiceCollection(application))

EDIT NO. 1

blakev suggested that I cannot use the generator correctly (which is certainly possible). But if I simplify this a bit so as not to even use the database, I still cannot process more than one DNS query at a time. To test this, I disabled the class. The following is my entire executable test file. Even in this heavily stripped-down version of my server, Twisted does not accept any requests until the first one appears.

 import sys import logging from twisted.names import dns, server, client, cache from twisted.application import service, internet from twisted.internet import defer class DNSResolver(client.Resolver): def __init__(self, servers): client.Resolver.__init__(self, servers=servers) def lookupMailExchange(self, name, timeout=None): """ The twisted function which is called when an MX record lookup is requested. :param name: The domain name being queried for (eg example.org). :param timeout: Time in seconds to wait for the query response. (optional, default: None) :return: A DNS response for the record query. """ logging.critical("Query for " + name) return defer.succeed([ (dns.RRHeader(name, dns.MX, dns.IN, 600, dns.Record_MX(1, "10.0.0.9", 600)),), (), () ]) # App name, UID, GID to run as. (root/root for port 53 bind) application = service.Application('db_driven_dns', 1, 1) # Set the secondary resolver db_dns_resolver = DNSResolver( [("8.8.8.8", 53), ("8.8.4.4", 53)] ) # Create the protocol handlers f = server.DNSServerFactory(caches=[cache.CacheResolver()], clients=[db_dns_resolver]) p = dns.DNSDatagramProtocol(f) f.noisy = p.noisy = False # Register as a tcp and udp service ret = service.MultiService() PORT=53 for (klass, arg) in [(internet.TCPServer, f), (internet.UDPServer, p)]: s = klass(PORT, arg) s.setServiceParent(ret) # Run all of the above as a twistd application ret.setServiceParent(service.IServiceCollection(application)) # If called directly, instruct the user to run it through twistd if __name__ == '__main__': print "Usage: sudo twistd -y %s (background) OR sudo twistd -noy %s (foreground)" % (sys.argv[0], sys.argv[0])
+4
source share
1 answer

Matte

I tried your last example and it works great. I think you can check it wrong.

In your subsequent comments, you talk about using the time.sleep (5) method in the search method to simulate a slow response.

You cannot do this. He will block the reactor. If you want to simulate a delay, use the .callLater reactor to start delayed

eg,

 def lookupMailExchange(self, name, timeout=None): d = defer.Deferred() self._reactor.callLater( 5, d.callback, [(dns.RRHeader(name, dns.MX, dns.IN, 600, dns.Record_MX(1, "mail.example.com", 600)),), (), ()] ) return d

Here is how I tested:

 time bash -c 'for n in "google.com" "yahoo.com"; do dig -p 10053 @127.0.0.1 "$n" MX +short +tries=1 +notcp +time=10 & done; wait'

And the result shows that both answers returned after 5 seconds

 1 10.0.0.9. 1 10.0.0.9. real 0m5.019s user 0m0.015s sys 0m0.013s

Likewise, you must make sure that calls to your database are not blocked:

Some other points:

+2
source

Source: https://habr.com/ru/post/1500181/

More articles:


All Articles