Is there a way to globally override request timeout?

Question

We have a query library utility littered throughout our project. We recently encountered an error in one of our destinations, where he froze an average transaction and decided to simply close the connection.

Naturally, our application followed this example.

Is there an environment variable or some other way to set a timeout? Even if it is significant (say, 30 seconds), it should be enough to stop all work from stopping due to one service. If possible, it should be global so that I do not have to find every use of it, and so that people can not forget to add it in the future.

Answer 1

Unfortunately, looking at the code, there is no way to set the global default value. I was a little surprised by this, as I expected this to be a fairly common case. If you run a request to add a function, let me know (for example, in the comments on this post).

The reason for this is because methods like get(...) , post(...) , etc., are just thin wrappers over the Session.request(...) ( requests.get(...) creates a new one-time session for only one request). This method takes a timeout as an argument and does not check the Session internals for a value if there is no timeout argument, so you should always put it there manually, like 2ps suggested in its answer .

Sources:

• requests/__init__.py - import the API into the scope of the package to provide the requests.get(...) -like utilities
• requests.api - API module imported to the item above; use single shot sessions
• requests.sessions - Session implementation
  • line 337 starts the Session class
  • line 463 runs the request(...) method
  • line 526 actually uses the timeout parameter
  • line 534 start get(...) method

PS. See this pull request . Disclaimer: it mine.

Answer 2

Two ways to do this. This is due to some dirty monkey patches. Tested on python 3.6.

Put this somewhere in a file, such as main.py, init .py or urls.py, and make sure it gets called.

Option 1: fix the query method

 import requests def request_patch(slf, *args, **kwargs): print("Fix called") timeout = kwargs.pop('timeout', 2) return slf.request_orig(*args, **kwargs, timeout=timeout) setattr(requests.sessions.Session, 'request_orig', requests.sessions.Session.request) requests.sessions.Session.request = request_patch 

Option 2: Run the Session Class

 import requests class SessionTimeoutFix(requests.Session): def request(self, *args, **kwargs): print("Fix called") timeout = kwargs.pop('timeout', 2) return super().request(*args, **kwargs, timeout=timeout) requests.sessions.Session = SessionTimeoutFix 

Answer 3

Instead, you can inherit the request.Session class and rewrite the request function, for example, like this:

 HTTP_TIMEOUT = 30 class TimeoutRequestsSession(requests.Session): def request(self, *args, **kwargs): kwargs.setdefault('timeout', HTTP_TIMEOUT) return super(TimeoutRequestsSession, self).request(*args, **kwargs) session = TimeoutRequestsSession() session.get('https://www.google.com') # connection timeout is default 30s 

Answer 4

You can do this in one call using the timeout parameter:

 requests.get('http://www.google.com', timeout=10) 

The timeout parameter specifies the number of seconds or a tuple (connection timeout, read timeout). You can learn more about this here .

If you want to make this global, the easiest way is to reorganize all the calls into a wrapper class and make sure everyone uses the wrapper class to call this API endpoint. I quickly checked the request code and did not see the use of global override for the timeout, since the default behavior is to wait for data.

Answer 5

Why not?:

  for method in ("get", "options", "head", "post", "put", "patch", "delete"): setattr( session, method, functools.partial(getattr(session, method), timeout=timeout), )