Permanent gaierror "Temporary glitch in name resolution" after running for several hours

Question

I have a long python script running with an upstart. This script makes quite a few requests. At first, everything worked well, but after a few hours, I begin to constantly receive the following error for each request:

File "/opt/a/a-env/local/lib/python2.7/site-packages/atom/client.py", line 119, in request File "/opt/a/a-env/local/lib/python2.7/site-packages/atom/http_core.py", line 420, in request File "/opt/a/a-env/local/lib/python2.7/site-packages/atom/http_core.py", line 489, in _http_request File "/usr/lib/python2.7/httplib.py", line 931, in endheaders File "/usr/lib/python2.7/httplib.py", line 794, in _send_output File "/usr/lib/python2.7/httplib.py", line 756, in send File "/usr/lib/python2.7/httplib.py", line 1134, in connect File "/usr/lib/python2.7/socket.py", line 553, in create_connection gaierror: [Errno -3] Temporary failure in name resolution 

This is not a name resolution or DNS issue because simply restarting the application fixes this problem.

I tried both Python 2.6 2.7 and the same situation arises.

I am running Linux 2.6.35-30-virtual #61-Ubuntu SMP Tue Oct 11 18:26:36 UTC 2011 x86_64 GNU/Linux

There were several reports with this problem, but none of them had a direct explanation or solution:

• https://github.com/ggstuart/greenview/issues/4
• Permanent "Temporary glitch in name resolution" after running for several hours

Answer 1

I think this happens when you get into the "too many open files" state. The next time this happens, try to see how many file descriptors the script has opened.

Answer 2

The main reason is that /etc/resolv.conf is only read when python starts up. To force a permission table update, you can run this (on Linux):

 import ctypes libc = ctypes.cdll.LoadLibrary('libc.so.6') res_init = libc.__res_init res_init()