Loading Python file from url using query library

Question

I want to upload a file to a url. The file I want to download is not located on my computer, but I have a file url. I want to download it using a query library. So, I want to do something like this:

url = 'http://httpbin.org/post' files = {'file': open('report.xls', 'rb')} r = requests.post(url, files=files) 

But the only difference is that the report.xls file comes from some URL that is not on my computer.

Answer 1

The only way to do this is to load the body of the URL so that you can download it.

The problem is that the form that takes file expects the body of the file in HTTP POST. Someone can write a form that uses a URL and makes a selection ... but it will be a different form and request than the one that takes the file (or maybe the same form with an additional file and optional URL )

You do not need to download it and save to a file, of course. You can simply load it into memory:

 urlsrc = 'http://example.com/source' rsrc = requests.get(urlsrc) urldst = 'http://example.com/dest' rdst = requests.post(urldst, files={'file': rsrc.content}) 

Of course, in some cases, you can always forward the file name or some other headers, such as Content-Type . Or, for huge files, you can transfer streams from one server to another without downloading, and then download the entire file at the same time. You will have to do such things manually, but almost everything is easy with requests and is well explained in the docs. *

---

* Well, this last example is not so simple ... you should get raw socket packets from requests and read and write , and make sure that you are not at a dead end, and so on ...

Answer 2

The documentation that may suit you is an example . A file-like object can be used as a stream for a POST request. Combine this with the stream response for your GET (passing stream=True ) or one of the other parameters here .

This allows you to perform POST from another GET without having to buffer the entire payload locally. In the worst case scenario, you may have to write a class similar to a file as “glue code” that allows you to pass an object to POST glue, which in turn reads the GET response.

(This is similar to a documented method using Node.js request .)