Python requests - user agent reevaluated

Question

I have

logindata = { 'username': 'me', 'password': 'blbla' } payload = {'from':'me', 'lang':'en', 'url':csv_url} headers = { 'User-Agent': 'Mozilla/5.0' } api_url = 'http://dev.mypage.com/admin/app/import/' with requests.Session() as s: s.post(api_url, data=json.dumps(logindata), headers=headers) print s.headers # An authorised request. r = s.get(api_url, params=payload, headers=headers) 

They reject me, but this is because of the ban 403. And I printed the headlines, I get:

 ..'User-Agent': 'python-requests/2.2.1 CPython/2.7.5 Windows/7'.. 

Why is my 'User-Agent': 'Mozilla/5.0' making excess profits? What am I missing here?

Answer 1

headers not stored inside the session this way.

You need to either explicitly pass them each time you make a request, or install s.headers once:

 with requests.Session() as s: s.headers = {'User-Agent': 'Mozilla/5.0'} 

You can verify that the correct headers were sent using the response.request.headers check:

 with requests.Session() as s: s.headers = {'User-Agent': 'Mozilla/5.0'} r = s.post(api_url, data=json.dumps(logindata)) print(r.request.headers) 

Also look at how the Session class is implemented - each time you make a request it combines request.headers with the headers that you set for the session object:

 headers=merge_setting(request.headers, self.headers, dict_class=CaseInsensitiveDict), 

Answer 2

If you want the session to use specific headers for all requests, you need to set these headers in the session explicitly:

 with requests.Session() as s: s.headers.update(headers) s.post(api_url, data=json.dumps(logindata)) # An authorised request. r = s.get(api_url, params=payload) 

The string s.headers.update(headers) adds your dictionary to the session headers.

Sessions never copy information from reuse requests to other requests. Only information from responses (in particular cookies) is captured for reuse.

For more information, see requests Documentation of session objects :

Sessions can also be used to provide default data to query methods. This is done by providing data to the properties of the Session object.