Inter-process communication without FIFO

Question

Inside the BASH script, we can have several processes running in the background that interact using named pipes, FIFOs registered in the file system. An example of this might be:

#!/bin/bash mkfifo FIFO # BG process 1 while :; do echo x; done & >FIFO # BG process 2 while :; do read; done & <FIFO exit 

I wonder if it is possible to make the same relationship between background script processes without using FIFO in the file system, possibly with some kind of file descriptor redirection.

Answer 1

Here is an example that starts two subprocesses implemented as functions of the same shell script ... One subprocess generates numbers 1 ... 5 (sleeps between fingerprints), the second one from a fixed filedescriptor (5, to which STDOUT of the first FD is redirected ), multiplied by 2 and prints again. The main process redirects the STDOUT of this second process to another fixed filedescriptor (6), and then to read from it in a loop.

It works basically the same as in C code with fd pairs created by the pipe (2) system call. To understand what is going on, run the script under strace -f!

Bash Version 4.2.24 (1) runs on Ubuntu / x86.

 [ubuntu /home/chris] $ bash --version GNU bash, version 4.2.24(1)-release (i686-pc-linux-gnu) Copyright (C) 2011 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software; you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. 

Script output:

 [ubuntu /home/chris] $ ./read_from_fd.sh Got number 2. Got number 4. Got number 6. Got number 8. Got number 10. 

Source:

 #!/bin/bash # Generate data, output to STDOUT. generate_five_numbers() { for n in `seq 5` ; do echo $n sleep 2 done } # Read data from FD#5, multiply by two, output to STDOUT. multiply_number_from_fd5_by_two() { while read n <&5 ; do echo "$(( $n * 2 ))" done } # choose your FD number wisely ;-) # run generator with its output dup'ed to FD #5 exec 5< <( generate_five_numbers ) # run multiplyier (reading from fd 5) with output dup'ed to FD #6 exec 6< <( multiply_number_from_fd5_by_two ) # read numbers from fd 6 while read n <&6 ; do echo "Got number $n." done 

Process tree during operation:

 ──read_from_fd.sh(8118)─┬─read_from_fd.sh(8119)───sleep(8123) └─read_from_fd.sh(8120) 

Answer 2

Bash 4 has coprocessors .

You can also use anonymous named pipes, aka process substitution in Bash 2, 3 or 4.

Answer 3

You can use nc (aka netcat ), which allows you to connect standard script streams to a network socket. Of course, it also works on localhost, so you can use it for IPC between scripts. The bonus is the ability to have scripts running on different hosts, which is impossible with FIFO (this is normal, perhaps in NFS, but it would be rather cumbersome to configure if you no longer have NFS in place).

Answer 4

I just want to point out that ugly hacks did not want to be born that way.

The part that receives the data:

 node -e "require('net').createServer(function(s){s.pipe(process.stdout)}).listen(1337)" 

The part that sends data:

 echo "write clean code they said" > /dev/tcp/localhost/1337 echo "it will pay off they said" > /dev/tcp/localhost/1337 

---

It even works in MSysGit Bash for Windows, to my surprise.

Answer 5

Have you considered using signals? If you only need to trigger an event (without passing arguments), using kill and trap works fine (be careful with the semantics, although, for example, use SIGUSR1).

You may need to rework the logic, as in the example below:

 subprocess_finished() { np=$( jobs -p | wc -l ) } start_processing() { myfile="$1" # DO SOMETHING HERE!! kill -SIGUSR1 $2 } CPUS=$( lscpu | grep "^CPU(s):" | rev | cut -f 1 -d ' ' | rev ) POLLPERIOD=5 # 5s between each poll np=0 trap subprocess_finished SIGUSR1 for myfile in * do start_processing "$myfile" $$ & np=$( jobs -p | wc -l ) echo "$( date +'%Y-%m-%d %H:%M:%S' ) [$!] Starting #$np on $CPUS: $myfile" if [ $np -eq $CPUS ] then # Wait for one CPU to be free trap subprocess_finished SIGUSR1 while [ $np -eq $CPUS ] do sleep $POLLPERIOD done fi done done # wait for the last subprocesses while [ ! -z "$( jobs -rp )" ] do sleep $POLLPERIOD done