Using httplib2 in python 3 right? (Problems with timeout)

Question

Hey first post, I'm really stuck on httplib2. I read about this with diveintopython3.org, but it doesn't say anything about the timeout function. I am looking at the documentation, but the only thing I see is the ability to put an int timeout, but no units are indicated (seconds? Milliseconds? What is the default value if not?) This is what I have (I also have code to check that the answer is and try again, but he never tried more than once)

h = httplib2.Http('.cache', timeout=None) for url in list: response, content = h.request(url) more stuff... 

This way the Http object stays around until some arbitrary time, but I load a ton of pages from the same server, and after a while it freezes when I receive the page. Errors do not occur, the thing just hangs on the page. So I try:

 h = httplib2.Http('.cache', timeout=None) for url in list: try: response, content = h.request(url) except: h = httplib2.Http('.cache', timeout=None) more stuff... 

But then it recreates another Http object each time (it follows the path “except”). I don’t understand how to keep receiving with the same object until it expires and I will do another one. Also, is there a way to set a timeout on an individual request?

Thanks for the help!

Answer 1

Set the timeout to 1 and you will pretty quickly find out if it means one millisecond or one second.

I don't know what your try / except should solve, if it hangs on h.request (url) in one case, it should hang in another.

If memory runs out in this code, then httplib2 does not receive garbage collection properly. Perhaps you have circular links (although this doesn't look like the one above), or it could be a bug in httlib2.

Answer 2

Due to an error, httplib2 measured the timeout in seconds , multiplied by 2 to version 0.7.5 (2012-08-28).