Reusing or repeating input as arguments in a command (an example is given)

Question

I will explain this problem with an example. Let's say I want to create a list with two columns. Column 1 contains the file name, and column 2 contains the md5 sum of this file.

This can be done using the following script:

for FILE in `ls -p | grep -v "/"`; do printf "%s %s\n" "$FILE" `md5 -q "$FILE"`; done; 

Is it possible to do this on a single line using pipes? I tried using different combinations of sed , xargs and printf , but I think I'm missing something. Here is one of my attempts:

 ls -p | grep -v "/" | xargs -I FILE printf "%s %s\n" FILE `md5 -q FILE` 

In this attempt, the FILE inside the reverse steps is not replaced, which is not surprising.

Is this where I should just use a multi-line script? Since there is no logic or control flow, I feel that a single liner should be possible and that maybe I am not using my tools properly, or I have not found a suitable tool.

Apologies for the title, I have no idea what to call this question.

Answer 1

You can use md5sum because I don't have md5 :)

 ls -p | grep -v / | xargs md5sum | awk '{print $2,$1}' 

and this can be a safer, more reliable way:

 find -maxdepth 1 -type f -exec md5sum {} \; | awk '{s=$2; $2=$1; $1=s;}1' 

this should work with md5:

 find -maxdepth 1 -type f -exec md5sum {} \; | sed 's/[^(]*(\([^)]*\)) =/\1/' 

Answer 2

What about:

 ls *.cc | xargs -I FILE md5 FILE | sed 's/^MD5 (\(.*\)) =/\1/' 

Or if you are concerned about spaces in your files (which I don’t think you should allow yourself anyway),

 find . -name "*.cc" -maxdepth 1 | xargs -I FILE md5 FILE | sed 's/^MD5 (\(.*\)) =/\1/' 

Answer 3

 ls -p | grep -v "/" | xargs -I FILE sh -c 'printf "%s %s\n" $0 `md5 -q $0`' FILE 

Your code did not work because the backlinks are handled by the original shell before passing the xargs arguments. You need to protect them from being processed by enclosing them in quotation marks, and then passing this argument to sh .

However, this seems like a bad way to do it. It does not work for files with spaces in names, and grep is a dumb way to skip directories.

 for FILE in * do if [ -f "$FILE" ] then printf "%s %s\n" "$FILE" `md5 -q "$FILE"` done 

Answer 4

To take care of problems with filenames with strange characters in the name that require escaping, the EXCEPT " character or file names with md5 sums and spaces as part of their names:

 find . -maxdepth 1 -type f -print0 |xargs -0 -n1 -I{} bash -c 'echo -n "{} " ; cat "{}" |md5sum' | sed 's/ -$//' 

Using cat makes the filename for md5sum standard input ( - ), which we remove from sed . However, no complex regular expressions are required to reorder the columns, since we use echo -n to print the file name as the first output column.

Example:

 samveen@precise :/tmp/tmp.JpmDNkhcRG$ touch "ab'c" samveen@precise :/tmp/tmp.JpmDNkhcRG$ touch "ab d41d8cd98f00b204e9800998ecf8427e c" samveen@precise :/tmp/tmp.JpmDNkhcRG$ touch "ab c" samveen@precise :/tmp/tmp.JpmDNkhcRG$ touch "abc" samveen@precise :/tmp/tmp.JpmDNkhcRG$ touch "ab > c" samveen@precise :/tmp/tmp.JpmDNkhcRG$ touch "ab\!c" samveen@precise :/tmp/tmp.JpmDNkhcRG$ find . -maxdepth 1 -type f -print0 |xargs -0 -n1 -I{} bash -c 'echo -n "{} " ; cat "{}" |md5sum'| sed 's/ -$//' ./ab'c d41d8cd98f00b204e9800998ecf8427e ./ab d41d8cd98f00b204e9800998ecf8427e c d41d8cd98f00b204e9800998ecf8427e ./ab c d41d8cd98f00b204e9800998ecf8427e ./abc d41d8cd98f00b204e9800998ecf8427e ./ab c d41d8cd98f00b204e9800998ecf8427e ./ab\!c d41d8cd98f00b204e9800998ecf8427e