Check IP address used for Python / Scrapy + ProxyMesh request

Question

I started using ProxyMesh with Scrapy. It is assumed that ProxyMesh provides a pool of IP addresses on a single port. The sample code code below is repeated in a loop, the middleware is enabled, and overall it works great. Can I track (and if so, how?) What IP address is used for each specific request?

request = scrapy.Request(producturl, self.parse_product_info) request.meta['proxy'] = 'http://uk.proxymesh.com:xxx' yield request 

I found similar entries in SOF, but did not address this specific issue.

Answer 1

As stated in the comments, the information goes to the response headers, just check:

 def parse_response(self, response): print response.headers 

You should see the X-Proxymesh-Ip header with the proxy assigned.

Another alternative would be to use crawlera , which offers even more features (like headers, sessions, and cookie handling) and better documentation.