Process change in grep missing expected results

Question

Suppose I have a program that outputs:

abcd l33t 1234 

which I will simulate with printf 'abcd\nl33t\n1234\n' . I would like to give this conclusion to two programs at the same time. My idea would be to replace the process by means of tee . Suppose I want to provide a copy of grep output:

 printf 'abcd\nl33t\n1234\n' | tee >(grep '[az]' >&2) | grep '[0-9]' 

I get the following with Bash 4.1.2 (Linux, CentOS 6.5), which is good:

 l33t 1234 abcd l33t 

But if process substitution is not redirected to stderr (i.e. without >&2 ), like this:

 printf 'abcd\nl33t\n1234\n' | tee >(grep '[az]') | grep '[0-9]' 

Then I get:

 l33t 1234 l33t 

A peculiar stdout expression from process substitution (first grep) is used by the process after the pipe (second grep). Except that the second grep is already reading things on its own, so I assume that it should not take things into account from the first grep. If I'm not mistaken (which I certainly have).

What am I missing?

Answer 1

Explanation

Regarding the command line, replacing a process is just a way to create a special file name. (See also docs .) Thus, the second pipeline looks like this:

 printf 'abcd\nl33t\n1234\n' | tee /dev/fd/nn | grep '[0-9]' 

where nn is the descriptor file number. The full output of printf goes to /dev/fd/nn , and also goes to grep '[0-9]' . Therefore, only numerical values ​​are printed.

As for the process inside >() , it inherits stdout of its parent. In this case, this stdout is inside the pipe. Therefore, the output of grep '[az]' passes through the pipeline in the same way as the standard output of tee . As a result, the pipeline as a whole skips only lines containing numbers.

When you write instead of stderr ( >&2 ), you bypass the last stage of the pipeline. Therefore, the output of grep '[az]' on stderr is sent to the terminal.

Fix

To fix this without using stderr, you can use a different alias for your screen. For instance:.

 printf 'abcd\nl33t\n1234\n' | tee >(grep '[az]' >/dev/tty ) | grep '[0-9]' # ^^^^^^^^^ 

which gives me a way out

 l33t 1234 abcd l33t 

Testing this

To sort this, I ran echo >(ps) . The ps process was a child of the bash process executing the pipeline.

I also ran

 printf 'abcd\nl33t\n1234\n' | tee >(grep '[az]') 

without | grep '[0-9]' | grep '[0-9]' at the end. In my system, I see

 abcd <--- the output of the tee l33t ditto 1234 ditto abcd <-- the output of the grep '[az]' l33t ditto 

All five lines are in grep '[0-9]' .

Answer 2

After tee you have two threads

 abcd l33t 1234 

The first grep ( >(grep '[az]' >&2 ) filters out

 abcd l33t 

and prints the result to its (!!!) stderr - which is still connected to your terminal ...

So another simple demo:

 printf 'abcd\nl33t\n1234\n' | tee >(grep '[az]' >&2) | grep '[0-9]' 

it prints

 l33t 1234 abcd l33t 

now add wc -l

 printf 'abcd\nl33t\n1234\n' | tee >(grep '[az]' >&2) | grep '[0-9]' | wc -l 

and you will get

 abcd l33t 2 

where you see clearly:

 abcd l33t 

is the stderr of the first grep , but the second stdout grep is redirected to wc and prints

 2 

Now another test:

 printf 'abcd\nl33t\n1234\n' | tee >(grep '[az]' ) | cat - 

Output

 abcd l33t 1234 abcd l33t 

eg. two lines from grep and full input from print

quantity:

 printf 'abcd\nl33t\n1234\n' | tee >(grep '[az]' ) | cat - | wc -l 

Output

 5