Shell: capture output files in variables

Question

Commands like openssl have arguments like -out <file> for the output files. I would like to capture the contents of these output files in shell variables for use in other commands without creating temporary files. For example, to create a self-signed certificate, you can use:

 openssl req -new -newkey rsa:2048 -subj / -days 365 -nodes -x509 -keyout KEYFILE -out CERTFILE 2>/dev/null 

In the closest way, I have to capture both output files in order to echo them to stdout via process overrides, but this is not ideal, because you still have to parse them separately.

 openssl req -new -newkey rsa:2048 -subj / -days 365 -nodes -x509 -keyout >(key=$(cat -); echo $key) -out >(cert=$(cat -); echo $cert) 2>/dev/null 

Is there a clean way to capture the contents of output files in shell variables?

Answer 1

Most modern shells now support / dev / stdout as the file name to redirect to stdout. This is good enough for single file solutions, but for two output files you need to go with a "process replacement".

  eval "$ (openssl req -new -newkey rsa: 2048 -subj / -days 365 -nodes -x509 -keyout> (echo" keyout = '$ (cat)' ") -out> (echo out =" '$ ( cat) '"))" 

It uses process substitution to direct each "file" to a separate process that prints to assign calculated values. All of this is then passed to eval to complete the actual tasks.

Save stderr output to display pop-up error messages. It is useful to record it during trouble.

Edit: incorporate into Charles Duffy's good paranoia:

  flockf = "$ (mktemp -t tmp.lock. $$. XXXXXX)" ||  exit $ ?;

 eval "$ (openssl req -new -newkey rsa: 2048 -subj / -days 365 -nodes -x509 \
         -keyout> (set -x; 99> "$ flockf" && \
                 flock -x "$ flockf" printf "keyout =% q" "$ (cat)";  ) \ 
         -out> (set -x; 99> "$ flockf" && \
                 flock -x "$ flockf" printf "out =% q" "$ (cat)";  ) \ 
         ) ";
 rm -f "$ flockf" 

Answer 2

An extension to Gilbert's answer , providing additional paranoia:

 eval "$( openssl req -new -newkey rsa:2048 -subj / -days 365 -nodes -x509 \ -keyout >(printf 'keyout=%q\n' "$(</dev/stdin)") \ -out >(printf 'out=%q\n' "$(</dev/stdin)") )" 

(Note that this is not appropriate if your data contains NULs that bash cannot store in its own shell variable, in which case you will want to assign the contents to your variables in base64 -encoded form).

Unlike echo "keyout='$(cat)'" , printf 'keyout=%q\n' "$(cat)" ensures that even malicious content cannot be eval uated by shell like wildcard, command redirection or other content other than letter data.

---

To explain why this is necessary, take a simplified case:

 write_to_two_files() { printf 'one\n' >"$1"; printf 'two\n' >"$2"; } write_to_two_files >(echo "one='$(cat)'") >(echo "two='$(cat)'") 

... we get an output similar to (but without special order):

 two='two' one='one' 

... which, when eval ed, sets two variables:

 $ eval "$(write_to_two_files >(echo "one='$(cat)'") >(echo "two='$(cat)'"))" $ declare -p one two declare -- one="one" declare -- two="two" 

However, let's say that our program behaves differently:

 ## Demonstrate why eval'ing content created by echoing data is dangerous write_to_two_files() { printf "'%s'\n" '$(touch /tmp/i-pwned-your-box)' >"$1" echo "two" >"$2" } eval "$(write_to_two_files >(echo "one='$(cat)'") >(echo "two='$(cat)'"))" ls -l /tmp/i-pwned-your-box 

Instead of simply assigning the result to a variable, we rated it as code.

---

If you are interested in ensuring that the two print operations are performed at different times (preventing them from mixing), it is useful to add an additional lock. This is due to a temporary file, but does not write your material to disk (avoiding it is the most compelling reason to prevent the temporary use of files):

 lockfile=$(mktemp -t output.lck.XXXXXX) eval "$( openssl req -new -newkey rsa:2048 -subj / -days 365 -nodes -x509 \ -keyout >(in=$(cat); exec 99>"$lockfile" && flock -x 99 && printf 'keyout=%q\n' "$in") \ -out >(in=$(cat); exec 99>"$lockfile" && flock -x 99 && printf 'out=%q\n' "$in") )" 

Please note that we only block writing, not reading, so we can’t get into the conditions of the race (i.e. where openssl does not finish writing to file-A, since it is blocked when writing to file-B, which can never terminate because the subshell on the read side of file-A records contains a lock).