Redirecting output is usually a function of the shell you are working with, and given how useful it is, I would be amazed if you found an error in it :-)
Are you sure you are not redirecting with awk , not a shell?
What happens when you do:
echo 'hello' | awk '{print}' >qq.tmp
Update:
If this is your code, as stated, this is because the $a is not expanded by your shell script, as the team awk is in single quotes.
for a in $(find $OUPUT_DIR/ -maxdepth 1 -mindepth 1 -type d -printf "%P\n") do echo $a is a directory awk -F, '{ if ($10 == '"$a"') print $0 }' $OUPUT_DIR/CDRNOutput_${CDR_DATE}.csv > $OUPUT_DIR/$a/CDR-${CDR_DATE}.csv done
What I mean is to pass specific awk values using the -v , something like (in your case):
awk -F, -va=$a '{ if ($10==a) print $0 }' ...
Variables then become first-class awk citizens, without worrying about who makes the extension.
Further update:
I stood behind my original advice. There is something definitely related to the chosen method.
I have a directory in my home directory called XpVm (among others), and I created a CDRNOutput_X.csv file containing one line:
1,2,3,4,5,6,7,8,9,XpVm,11
When I execute:
for a in $(find . -maxdepth 1 -mindepth 1 -type d -printf "%P\n" | grep -v '^\.') do echo $a is a directory awk -F, '{ if ($10 == '"$a"') { print $0 } else { print "NO"; } }' ./CDRNOutput_X.csv done
(I deleted the directories starting with . , Since they were causing another problem), I get this output:
workspace is a directory NO Documents is a directory NO XpVm is a directory NO Downloads is a directory NO
which is clearly not what is expected. However, when I use the -v for awk , as I originally suggested, the command:
for a in $(find . -maxdepth 1 -mindepth 1 -type d -printf "%P\n" | grep -v '^\.') do echo $a is a directory awk -F, -va=$a '{ if ($10 == a) { print $0 } else { print "NO" } }' ./CDRNOutput_X.csv done
(the only difference is changing to a ), I get:
workspace is a directory NO Documents is a directory NO XpVm is a directory 1,2,3,4,5,6,7,8,9,XpVm,11 Downloads is a directory NO
what is right.
Final update (hopefully):
I think the problem is resolved. Now I'm on a different machine (so the directory names are just tmp and tmp2 ), and when I run the original script:
for a in $(find . -maxdepth 1 -mindepth 1 -type d -printf "%P\n" | grep -v '^\.') do echo $a is a directory awk -F, '{ if ($10 == '"$a"') { print $0 } else { print "NO"; } }' ./CDRNOutput_X.csv done
with the modified CDRNOutput_X.csv containing tmp instead of XpVm , I get:
tmp is a directory NO tmp2 is a directory NO
This is because the if treated as awk like:
if ($10 == tmp) {
(without the quotes as the quotes are actually located outside the line awk , used to surround the directory name). This will check $10 for equality with the awk variable named tmp , not the actual string "tmp" . You need to make sure the quotes are inside the awk script, for example:
if ($10 == "tmp") {
and you can do it with the following script (only the if line has changed):
#!/bin/bash for a in $(find . -maxdepth 1 -mindepth 1 -type d -printf "%P\n" | grep -v '^\.') do echo $a is a directory awk -F, '{ if ($10 == "'"$a"'") { print $0 } else { print "NO"; } }' ./CDRNOutput_X.csv done
Note that double quotes are duplicated. I still kept double quotes around $a if someone committed the terrible crime of creating a file with a space in it :-)
Running this script calls:
tmp is a directory 1,2,3,4,5,6,7,8,9,tmp,11 tmp2 is a directory NO
what I think, what you were striving for.
So, the result is that if you don't want to use awk variables, you can just change the awk line:
'{ if ($10 == '"$a"') print $0 }'
in
'{ if ($10 == "'"$a"'") print $0 }'
and it should work fine.