Debugger for unix pipe commands

Question

As I build the * nix piped commands, I find that I want to see the result of one step in order to verify the correctness before building the next step, but I do not want to restart each step. Does anyone know of a program that will help with this? It will support the conclusion of the last stage automatically for use at any new stages. I usually do this by sending the result of each command to a temporary file (i.e. Tee or running each command one at a time), but it would be nice if the program handled this.

I present something like a tabbed interface where each tab is labeled with each pipe command, and selecting the tab shows the output (at least one hundred lines) of applying this command to the previous result.

Answer 1

Use 'tee' to copy the intermediate results to some file and then transfer them to the next stage of the channel, for example:

cat /var/log/syslog | tee /tmp/syslog.out | grep something | tee /tmp/grep.out | sed 's/foo/bar/g' | tee /tmp/sed.out | cat >>/var/log/syslog.cleaned 

Answer 2

You can also use channels if you need bidirectional communication (i.e. with netcat):

 mknod backpipe p nc -l -p 80 0<backpipe | tee -a inflow | nc localhost 81 | tee -a outflow 1>backpipe 

( through )

Answer 3

tee (1) is your friend. It sends its input to both the specified file and stdout.

Hold it between your pipes. For instance:

 ls | tee /tmp/out1 | sort | tee /tmp/out2 | sed 's/foo/bar/g' 

Answer 4

There is also a "pv" command - available in debian / ubuntu repeaters, which shows the throughput of your pipes.

Example from the man page: Transferring a file from another process and transferring the expected size to pv:

  cat file | pv -s 12345 | nc -w 1 somewhere.com 3000