Write Error: Broken Pipe

I need to run the tool about 300 directories. Each run takes from 1 minute to 30 minutes or more. So, I wrote a python script that has a loop to run the tool across all directories one by one.

my python script has code like:

for directory in directories: os.popen('runtool_exec ' + directory) 

But when I run the python script, I get the following error messages:

 .. tail: write error: Broken pipe date: write error: Broken pipe .. 

All I do is log in to the remote server using ssh, where the tools, python script and theme directories are stored. When I individually run the tool from the command line using the command:

 runtool_exec directory 

It works great. A β€œbroken pipe” error only appears when I run using a python script. Any idea of ​​a workaround?

+4
source share
2 answers

The errors you see are related to the fact that you use os.popen() to run the command, which means that the channel opens and connects to the stdout command. Whenever a command (or anything that it executes without redirecting stdout ) wants to write to stdout , it will try to write to the pipe. But you are not holding the phone, since you are not assigning the result of os.popen() anywhere, therefore it is cleared by Python and closed. Thus, processes that try to write to it collide with a broken pipe and cause the error you see. ( os.popen() does not redirect stderr to the channel, so you see errors anyway.)

Another problem in your os.popen() call is that you are not checking to make sure that the directory does not contain characters specific to your shell. If the catalog contains a quote, for example, or *, strange things may arise. Using os.popen() like this is pretty bad.

Instead, you should use subprocess.Popen() and explicitly specify what you want to do with the output of the process (both stdout and stderr.) You pass subprocess.Popen() list of arguments (including what you want to run) instead of one lines, and she generally avoids the shell, does not need common sense - check your lines. If you really want to ignore the conclusion, you would do it with something like:

 devnull = open(os.devnull, 'w') for directory in directories: subprocess.Popen(['runtool_exec', directory], stdout=devnull) 

although I would highly recommend at least doing some rudimentary checks on the result of the subprocess. Try checking to see if the process terminated prematurely or with an error code.

+9
source

In case this is useful to someone else, the same problem could be the result of using pipe (|) in a shell command. See here for a solution to fooobar.com/questions/427848 / ...

0
source

Source: https://habr.com/ru/post/1307964/


All Articles