Python popen grep

Question

I would like Popen to do:

grep -i --line-buffered "grave" data/*.txt 

When launched from the shell, this gives me the desired result. If I start, in the same directory where I am testing grep , python repl and following the instructions from the docs , I get what should be the appropriate argument list for submitting Popen with:

 ['grep', '-i', '--line-buffered', 'grave', 'data/*.txt'] 

The result of p = subprocess.Popen(args) is

 grep: data/*.txt: No such file or directory 

and if I try p = subprocess.Popen(args, shell=True) , I get:

 Usage: grep [OPTION]... PATTERN [FILE]... Try `grep --help' for more information. 

Any help on how to complete the required process? I am on MacOS Lion.

Answer 1

If you type * in bash, the shell expands it to the files in this directory before executing the command. Python Popen does not do this, so what you do when you call Popen, as grep says, is a file named *.txt in the data directory, and not all .txt files in the data directory. This file does not exist and you will receive the expected error.

To solve this problem, you can tell python to run the command through the shell by passing shell=True to Popen:

 subprocess.Popen('grep -i --line-buffered grave data/*.txt', shell=True) 

What translates to:

 subprocess.Popen(['/bin/sh', '-c', 'grep -i --line-buffered "grave" data/*.txt']) 

As explained in the Popen documentation .

Here you should use a string instead of a list because you want to execute /bin/sh -c "grep -i --line-buffered "grave" data/*.txt" (NB quotes around the command, making it the only argument to sh ) If you use a list, this command runs: /bin/sh -c grep -i --line-buffered "grave" data/*.txt , which gives you the result of a simple grep run.

Answer 2

The problem is that the shell is doing file globbing: data / * for you. txt

You will need to do this yourself, for example, using the glob .

 import glob cmd_line = ['grep', '-i', '--line-buffered', 'grave'] + glob.glob('data/*.txt')