Forcing bash to expand variables in a line loaded from a file

Question

I am trying to figure out how to make bash (force?) Expand the variables in a line (which was loaded from a file).

I have a file called "something.txt" with the contents:

hello $FOO world 

Then i ran

 export FOO=42 echo $(cat something.txt) 

this returns:

  hello $FOO world 

It did not extend $ FOO, although a variable was set. I cannot eval or the source of the file - since it will try to execute it (it is not executable as it is), I just want the string with the variables to be interpolated).

Any ideas?

Answer 1

I came across what I think is the answer to this question: envsubst command.

 envsubst < something.txt 

If it is not already available on your distribution, it is located in

GNU gettext package.

@Rockallite - I wrote a small script wrapper to take care of the "\ $" problem.

(By the way, there is an envsubst “feature” explained in https://unix.stackexchange.com/a/294400/7088 for expanding only certain variables in the input, but I agree that avoiding exceptions is much more convenient.)

Here is my script:

 #! /bin/bash ## -*-Shell-Script-*- CmdName=${0##*/} Usage="usage: $CmdName runs envsubst, but allows '\$' to keep variables from being expanded. With option -sl '\$' keeps the back-slash. Default is to replace '\$' with '$' " if [[ $1 = -h ]] ;then echo -e >&2 "$Usage" ; exit 1 ;fi if [[ $1 = -sl ]] ;then sl='\' ; shift ;fi sed 's/\\\$/\${EnVsUbDolR}/g' | EnVsUbDolR=$sl\$ envsubst "$@" 

Answer 2

Many answers that use eval and echo look like work, but break into different things, such as several lines trying to avoid shell metacharacters, go into a template that is not intended to extend bash, etc.

I had the same problem, and I wrote this shell function, which, as far as I can tell, handles everything correctly. It will still only suppress trailing newlines from the template due to bash command substitution rules, but I never found this to be a problem if everything else remains intact.

 apply_shell_expansion() { declare file="$1" declare data=$(< "$file") declare delimiter="__apply_shell_expansion_delimiter__" declare command="cat <<$delimiter"$'\n'"$data"$'\n'"$delimiter" eval "$command" } 

For example, you can use it like this: parameters.cfg , which is really a shell script that just sets the variables, and template.txt , which is a template that uses these variables:

 . parameters.cfg printf "%s\n" "$(apply_shell_expansion template.txt)" > result.txt 

In practice, I use this as a kind of lightweight template system.

Answer 3

you can try

 echo $(eval echo $(cat something.txt)) 

Answer 4

You do not want to print each line, you want to evaluate it so that Bash can perform variable replacement.

 FOO=42 while read; do eval echo "$REPLY" done < something.txt 

See the help eval or Bash manual for more information.

Answer 5

Another approach (which seems icky, but I put it anyway):

Write the contents of the something.txt file to the temp file with the echo statement wrapped around it:

 something=$(cat something.txt) echo "echo \"" > temp.out echo "$something" >> temp.out echo "\"" >> temp.out 

then return it back to the variable:

 RESULT=$(source temp.out) 

and $ RESULT will be expanded. But that seems wrong!

Answer 6

If you want variable references to be extended (a goal that I had for myself), you can do the following.

 contents="$(cat something.txt)" echo $(eval echo \"$contents\") 

(The highlighted quotes around $ content are key here)

Answer 7

Next solution:

• allows you to replace variables that are defined

• leaves unchanged variables that are not defined. This is especially useful for automated deployments.

• supports the replacement of variables in the following formats:

  ${var_NAME}

  $var_NAME

• reports which variables are not defined in the environment and returns an error code for such cases

 TARGET_FILE=someFile.txt; ERR_CNT=0; for VARNAME in $(grep -P -o -e '\$[\{]?(\w+)*[\}]?' ${TARGET_FILE} | sort -u); do VAR_VALUE=${!VARNAME}; VARNAME2=$(echo $VARNAME| sed -e 's|^\${||g' -e 's|}$||g' -e 's|^\$||g' ); VAR_VALUE2=${!VARNAME2}; if [ "xxx" = "xxx$VAR_VALUE2" ]; then echo "$VARNAME is undefined "; ERR_CNT=$((ERR_CNT+1)); else echo "replacing $VARNAME with $VAR_VALUE2" ; sed -i "s|$VARNAME|$VAR_VALUE2|g" ${TARGET_FILE}; fi done if [ ${ERR_CNT} -gt 0 ]; then echo "Found $ERR_CNT undefined environment variables"; exit 1 fi