Subprocess.call () is ignored when using shell = True w / list

Question

I am trying to get the subprocess.call python method to accept some args commands through a list (consisting of a sequence of strings) as indicated in the python documentation. To study this behavior before inserting it into the actual script, I opened IPython, executed several commands with various combinations of shell settings and args commands and got the following behavior:

In [3]: subprocess.call(['ls', '-%sl' %'a']) total 320 drwxr-xr-x 20 Kohaugustine staff 680 Oct 15 16:55 . drwxr-xr-x 5 Kohaugustine staff 170 Sep 12 17:16 .. -rwxr-xr-x 1 Kohaugustine staff 8544 Oct 15 16:55 a.out -rwxr-xr-x 1 Kohaugustine staff 8544 Oct 3 10:28 ex1-6 -rw-r--r--@ 1 Kohaugustine staff 204 Oct 3 10:28 ex1-6.c -rwxr-xr-x 1 Kohaugustine staff 8496 Oct 3 10:15 ex1-7 -rw-r--r--@ 1 Kohaugustine staff 71 Oct 3 10:15 ex1-7.c -rwxr-xr-x 1 Kohaugustine staff 8496 Sep 12 16:22 hello -rw-r--r--@ 1 Kohaugustine staff 58 Sep 12 16:27 hello.c -rwxr-xr-x 1 Kohaugustine staff 8496 Sep 12 16:24 hello.o -rwxr-xr-x 1 Kohaugustine staff 8496 Sep 12 16:24 hello_1.o -rwxr-xr-x 1 Kohaugustine staff 8496 Sep 12 16:27 hello_2.o -rwxr-xr-x 1 Kohaugustine staff 8496 Sep 12 16:27 hello_3.o -rwxr-xr-x 1 Kohaugustine staff 8544 Oct 15 16:55 lesson_1-5 -rw-r--r--@ 1 Kohaugustine staff 185 Sep 28 10:35 lesson_1-5.c -rwxr-xr-x 1 Kohaugustine staff 8496 Sep 21 10:06 temperature.o -rw-r--r--@ 1 Kohaugustine staff 406 Sep 21 09:54 temperature_ex1-3.c -rw-r--r--@ 1 Kohaugustine staff 582 Sep 21 10:06 temperature_ex1-4.c -rw-r--r--@ 1 Kohaugustine staff 178 Sep 23 17:21 temperature_ex1-5.c -rwxr-xr-x 1 Kohaugustine staff 8496 Sep 23 17:21 temperature_ex1-5.o Out[3]: 0 In [4]: subprocess.call(['ls', '-%sl' %'a'], shell=True) a.out ex1-7 hello.c hello_2.o lesson_1-5.c temperature_ex1-4.c ex1-6 ex1-7.c hello.o hello_3.o temperature.o temperature_ex1-5.c ex1-6.c hello hello_1.o lesson_1-5 temperature_ex1-3.c temperature_ex1-5.o Out[4]: 0 In [6]: subprocess.call(['ls', '-al']) total 320 drwxr-xr-x 20 Kohaugustine staff 680 Oct 15 16:55 . drwxr-xr-x 5 Kohaugustine staff 170 Sep 12 17:16 .. -rwxr-xr-x 1 Kohaugustine staff 8544 Oct 15 16:55 a.out -rwxr-xr-x 1 Kohaugustine staff 8544 Oct 3 10:28 ex1-6 -rw-r--r--@ 1 Kohaugustine staff 204 Oct 3 10:28 ex1-6.c -rwxr-xr-x 1 Kohaugustine staff 8496 Oct 3 10:15 ex1-7 -rw-r--r--@ 1 Kohaugustine staff 71 Oct 3 10:15 ex1-7.c -rwxr-xr-x 1 Kohaugustine staff 8496 Sep 12 16:22 hello -rw-r--r--@ 1 Kohaugustine staff 58 Sep 12 16:27 hello.c -rwxr-xr-x 1 Kohaugustine staff 8496 Sep 12 16:24 hello.o -rwxr-xr-x 1 Kohaugustine staff 8496 Sep 12 16:24 hello_1.o -rwxr-xr-x 1 Kohaugustine staff 8496 Sep 12 16:27 hello_2.o -rwxr-xr-x 1 Kohaugustine staff 8496 Sep 12 16:27 hello_3.o -rwxr-xr-x 1 Kohaugustine staff 8544 Oct 15 16:55 lesson_1-5 -rw-r--r--@ 1 Kohaugustine staff 185 Sep 28 10:35 lesson_1-5.c -rwxr-xr-x 1 Kohaugustine staff 8496 Sep 21 10:06 temperature.o -rw-r--r--@ 1 Kohaugustine staff 406 Sep 21 09:54 temperature_ex1-3.c -rw-r--r--@ 1 Kohaugustine staff 582 Sep 21 10:06 temperature_ex1-4.c -rw-r--r--@ 1 Kohaugustine staff 178 Sep 23 17:21 temperature_ex1-5.c -rwxr-xr-x 1 Kohaugustine staff 8496 Sep 23 17:21 temperature_ex1-5.o Out[6]: 0 In [7]: subprocess.call(['ls', '-al'], shell = True) a.out ex1-7 hello.c hello_2.o lesson_1-5.c temperature_ex1-4.c ex1-6 ex1-7.c hello.o hello_3.o temperature.o temperature_ex1-5.c ex1-6.c hello hello_1.o lesson_1-5 temperature_ex1-3.c temperature_ex1-5.o Out[7]: 0 

It seems that whenever shell = True, the result seems the same as:

 In [9]: subprocess.call(['ls']) a.out ex1-7 hello.c hello_2.o lesson_1-5.c temperature_ex1-4.c ex1-6 ex1-7.c hello.o hello_3.o temperature.o temperature_ex1-5.c ex1-6.c hello hello_1.o lesson_1-5 temperature_ex1-3.c temperature_ex1-5.o Out[9]: 0 

I am puzzled; what happened to the '-a' option when i set shell = True? Didn't the shell read it? I read the Docs and that he says that when shell = True, it means that my specified command will be executed through the shell, so this should mean that ls -a has been sent to the shell and acts as a shell. Then why the behavior in [4] and [7]? Also, pydocs does not explain this directly (although it says that the subprocess will NOT do when we set shell = False); What does this mean when we enable shell = False? Is a new process spawned in the OS without actually having a shell?

Also, in case it may seem inconvenient that I use the format string in [3] and [4], its because in my actual script where I will use subprocess.call, I will have to rely on these lines format to replace them in the corresponding command options. I can’t hardcode some command line options. Using a clean line for args is out of the question, because in my script there will be a method that should perform operations with lists in commands. I don't know if there could be a better way to do this, but if it really helps, if anyone can offer something else.

Many thanks!

Answer 1

When shell is set to True, the first argument is added to ["/bin/sh", "-c"] . If this argument is a list, the resulting list

 ["/bin/sh", "-c", "ls", "-al"] 

That is, only ls used as the argument to the -c option, not ls -al . -al used as the first argument to the shell itself, not ls .

When using shell=True you usually just want to pass a single line and let the shell split it according to the normal rules for splitting shell words.

 # Produces ["/bin/sh", "-c", "ls -al"] subprocess.call("ls -al", shell=True) 

In your case, it does not see how you need to use shell=True at all.

Answer 2

When you use shell=True with a list, additional arguments are passed to the shell itself, and not to the command running in the shell. Then they can be assigned from the shell script (passed as argv[0] ) as $0 , $1 , etc.

The simplest answer is “don't do this”: if you want to pass a list, do not use shell=True ; if you want to pass a string always use shell=True .

---

However, you can form your team in such a way as to read these arguments. The following is an example that violates my rule above - a command that you could not implement [*] without shell=True (and /bin/sh provided by bash), since it relies on the built-in version of bash printf (which supports %q as an extension ):

 subprocess.call([ "printf '%q\\n' \"$0\" \" $@ \"", 'these strings are\r\n', '"shell escaped" in the output from this command', "so that they can *safely* be run through eval" ], shell=True) 

---

[*] - Not quite right; one could use /bin/bash as argv[0] with shell=False to implement this more reliably, since then it would no longer depend on your /bin/sh being bash .