I want to generate a SQOOP command that is added to some variable, for example CUSTOM_PARAMS. I defined a variable in the file: sayhi.cfg
The variable has some single quotes, as well as "orc".
cat hi.cfg
CUSTOM_PARAMS="--query select * from blah..blah where \$CONDITIONS --split-by blah --create-hcatalog-table --hcatalog-external-table --hcatalog-storage-stanza stored as 'orc' --compress --compression-codec 'snappy' --fields-terminated-by '|' -m 5 --target-dir /any/dir"
When I go to a file on the command line and do an echo, it gives me exactly what is written there, which looks right.
source hi.cfg
echo "$CUSTOM_PARAMS"
--query select * from blah..blah where $CONDITIONS --split-by blah --create-hcatalog-table --hcatalog-external-table --hcatalog-storage-stanza stored as 'orc' --compress --compression-codec 'snappy' --fields-terminated-by '|' -m 5 --target-dir /any/dir
But when I call this from a shell script, as shown below:
cat hi.sh
echo "Generating Sqoop Command"
source $HOME/hi.cfg
echo "${CUSTOM_PARAMS}"
SQOOP_COMMAND="SQOOP statement : sqoop import blah blah "$CUSTOM_PARAMS""
echo $SQOOP_COMMAND
The * character in a variable is treated as a command:
sh hi.sh
Generating Sqoop Command
--query select * from blah..blah where $CONDITIONS --split-by blah --create-hcatalog-table --hcatalog-external-table --hcatalog-storage-stanza stored as 'orc' --compress --compression-codec 'snappy' --fields-terminated-by '|' -m 5 --target-dir /any/dir
SQOOP statement : sqoop import blah blah --query select 0 00 000000_0 000073_0 000103_0 02 09.txt 1 from blah..blah where $CONDITIONS --split-by blah --create-hcatalog-table --hcatalog-external-table --hcatalog-storage-stanza stored as 'orc' --compress --compression-codec 'snappy' --fields-terminated-by '|' -m 5 --target-dir /any/dir
I need to run the SQOOP statement later in the script and tried a couple of options, but didn't help. I also tried \*, but did not help, it outputs:
Generating Sqoop Command
--query select \* from blah..blah where $CONDITIONS --split-by blah --create-hcatalog-table --hcatalog-external-table --hcatalog-storage-stanza stored as 'orc' --compress --compression-codec 'snappy' --fields-terminated-by '|' -m 5 --target-dir /any/dir
SQOOP statement : sqoop import blah blah --query select \* from blah..blah where $CONDITIONS --split-by blah --create-hcatalog-table --hcatalog-external-table --hcatalog-storage-stanza stored as 'orc' --compress --compression-codec 'snappy' --fields-terminated-by '|' -m 5 --target-dir /any/dir`