Wget: How to specify both options --directory-prefix AND --output-document

Question

When I use only -P or -O with wget only, everything works as advertised.

 $: wget -P "test" http://www.google.com/intl/en_com/images/srpr/logo3w.png Saving to: `test/logo3w.png' 

.

 $: wget -O "google.png" http://www.google.com/intl/en_com/images/srpr/logo3w.png 2012-01-23 21:47:33 (1.20 MB/s) - `google.png' saved [7007/7007] 

However, combining the two reasons wget ignores -P .

 $: wget -P "test" -O "google.png" http://www.google.com/intl/en_com/images/srpr/logo3w.png 2012-01-23 21:47:51 (5.87 MB/s) - `google.png' saved [7007/7007] 

I set a variable for the directory (generated by the last fragment of the URL) and the file name (generated by the count loop), so http://www.google.com/aaa/bbb/ccc gives file = /directory/filename , or for item 1, /ccc/000.jpg

Substituting this into the code:
Popen(['wget', '-O', file, theImg], stdout=PIPE, stderr=STDOUT)
wget silently fails (at each iteration of the loop).

When I turn on -d debugging and register -a log.log , each iteration prints DEBUG output created by Wget 1.13.4 on darwin10.8.0.

When I delete -O and file , the operation is fine.

My question is: Is there a way A) Indicate as -P AND -O in wget (preferred), or
B) Insert a string in -O containing / -characters that do not crash?

Any help would be appreciated.

Answer 1

You should just pass dir/000.jpg to -O from wget :

 import subprocess import os.path subprocess.Popen(['wget', '-O', os.path.join(directory, filename), theImg]) 

It is not entirely clear from your question whether you are already doing something similar to this, but if you were and still have not succeeded, I can think of two reasons:

• The -O argument contains the leading / , which makes wget fail, because it does not have permission to accidentally create directories in / (root).

• The directory in which you specify wget to write does not exist. You can verify that it exists by creating it first using os.mkdir in the Python standard library.

You can also try removing the stdout= and stderr= arguments from the Popen call Popen that you can see errors directly or print them using Python.

Answer 2

Documentation wget.download (..):

 def download(url, out=None, bar=bar_adaptive): """High level function, which downloads URL into tmp file in current directory and then renames it to filename autodetected from either URL or HTTP headers. :param bar: function to track download progress (visualize etc.) :param out: output filename or directory :return: filename where URL is downloaded to """ ... 

Use the following call to upload a file to a specific directory (already existing) with a custom file name:

 wget.download(url, path_to_output_file) 

If you want the function call to abstract the creation of the directory, if it does not already exist, use:

 urllib.urlretrieve(url, path_to_output_file) 

Answer 3

This line of code from @Jaydev works brilliantly:

wget.download(url, path_to_output_file)