How do you view the request headers used by mechanization?

Question

I am trying to submit some data to a form programmatically. I have a small problem when the server "does not like" that I am sending it. Disappointing, there are no error messages or anything that could help diagnose the problem, everything that does it spins me to the same page that I started on when I clicked br.submit() .

When I click the submit button manually in the browser, the summary page shows a little "success!". message. Such a message does not appear when sending through a script. In addition, no changes are actually sent to the server. This is rather strange, and for the first time I came across this behavior.

Delving into Mechanize documents, he suggests that with these strange, hard-to-diagnose problems, it is best to copy the request headers that are actually sent by the browser.

My question is, how can I see request headers when I call br.submit() ?

Code example

 location = 'http://ww.mysite.com' br = mechanize.Browser() cj = mechanize.LWPCookieJar() br.set_cookiejar(cj) username = MY_USER_NAME password = MY_PASSWORD br.addheaders.append(('Authorization', 'Basic %s' % base64.encodestring('%s:%s' % (username, password)))) br.open(location) br.select_form(nr=0) br['text'] = 'MY JUNK TO SUBMIT' #Text field. Can put anything br['DropDown1'] = ['4'] #This is a dropdown of integer values br['DropDown2'] = ['3'] #Also a dropdown of ints br.submit() 

How do I know which headers are sent when the form is submitted?

Answer 1

You ask how to see which headers are sent by your browser or mechanization?

---

Browser

Like other commentators, you can check the headers sent by browsers using a plugin like Firebug (Firefox), developer tools (IE 'F12' , Chrome Developer Tools and Opera Dragonfly ), etc.

---

Mechanize

Using mechanization, you can get a copy of the headers sent using

 import mechanize br = mechanize.Browser() br.open("http://stackoverflow.com") request = br.request request.header_items() 

What gives in this case

 [('Host', 'stackoverflow.com'), ('User-agent', 'Python-urllib/2.7')] 

---

Other / One Click

As always for debugging or if nothing is provided, you can use Wireshark to check which headers are sent over the network. Tip : use a filter like (http.request.uri == "http://stackoverflow.com/")