What is the correct method for doing curls from a subprocess?

Question

I tried calling curl from subprocess to load images, but kept getting curl (error code 2 .., which from the document refers to CURL_FAILED_INIT ). I do not use urllib , because in the end I will execute the script using subprocess . Below is a snippet of code

 import subprocess import multiprocessing def worker(fname, k): f = open(fname, 'r') i = 0 for imgurl in f: try: op = subprocess.call(['curl', '-O', imgurl], shell=False) except: print 'problem downloading image - ', imgurl def main(): flist = [] flist.append(sys.argv[1]) flist.append(sys.argv[2]) ... for k in range(1): p = multiprocessing.Process(target=worker, args=(flist[k],k)) p.start() 

O / P:

curl: try 'curl --help' or 'curl - manual' for more information

2

curl: try 'curl --help' or 'curl - manual' for more information

2

....

Answer 1

If you want to run a shell command, a subprocess is the way to go. Because it can run a shell command in its own process, using multiprocessing is redundant at best. Multiprocessing comes in handy when you want to run the function of your python program in a separate process. It seems you are going to run a shell command, not a python function.

I am not familiar with curl . If you want to get standard output from curl , use subprocess.Popen() . subprocess.call() returns the program return code, not stdout .

See http://docs.python.org/release/3.2/library/subprocess.html

Sort of:

 subp = subprocess.Popen(['curl', '-O', imgurl], stdout=subprocess.PIPE, stderr=subprocess.PIPE) curlstdout, curlstderr = subp.communicate() op = str(curlstdout) 

maybe closer. Not familiar with curl , as I said, so your program may be different.