BASH output column formatting

Question

First message. HELLO WORLD. Working on my first script that just checks if the list of my websites is online, and then returns the HTTP code and the amount of time it takes to return this to another file on my desktop.

- THIS script WILL WORK ON MAC OSX -

I would like to modify my script so that it formats its output into 3 neat columns.

currently

#!/bin/bash file="/Users/USER12/Desktop/url-list.txt" printf "" > /Users/USER12/Desktop/url-results.txt while read line do printf "$line" >> /Users/USER12/Desktop/url-results.txt printf "\t\t\t\t" >> /Users/USER12/Desktop/url-results.txt curl -o /dev/null --silent --head --write-out '%{http_code} %{time_total}' "$line" >> /Users/USER12/Desktop/url-results.txt printf "\n" >> /Users/USER12/Desktop/url-results.txt done <"$file" 

which is displayed in the following format

 google.com 200 0.389 facebook.com 200 0.511 abnormallyLongDomain.com 200 0.786 

but I would like to format into neat aligned columns for easy reading

 DOMAIN_NAME HTTP_CODE RESPONSE_TIME google.com 200 0.389 facebook.com 200 0.511 abnormallyLongDomain.com 200 0.486 

Thanks for helping everyone!

Answer 1

column very nice. You, however, already use printf , which gives you great control over the output format. Using printf functions also simplifies the code:

 #!/bin/bash file="/Users/USER12/Desktop/url-list.txt" log="/Users/USER12/Desktop/url-results.txt" fmt="%-25s%-12s%-12s\n" printf "$fmt" DOMAIN_NAME HTTP_CODE RESPONSE_TIME > "$log" while read line do read code time < <(curl -o /dev/null --silent --head --write-out '%{http_code} %{time_total}' "$line") printf "$fmt" "$line" "$code" "$time" >> "$log" done <"$file" 

With the above format, the output is as follows:

 DOMAIN_NAME HTTP_CODE RESPONSE_TIME google.com 301 0.305 facebook.com 301 0.415 abnormallyLongDomain.com 000 0.000 

You can fine-tune the output format, such as distance or alignment, by changing the fmt variable in the script.

Further refinement

The above code opens and closes the log file with each cycle. This can be avoided, as suggested by Charles Duffy, simply by using exec to redirect stdout to the log file before the first printf statement:

 #!/bin/bash file="/Users/USER12/Desktop/url-list.txt" exec >"/Users/USER12/Desktop/url-results.txt" fmt="%-25s%-12s%-12s\n" printf "$fmt" DOMAIN_NAME HTTP_CODE RESPONSE_TIME while read line do read code time < <(curl -o /dev/null --silent --head --write-out '%{http_code} %{time_total}' "$line") printf "$fmt" "$line" "$code" "$time" done <"$file" 

Alternatively, as Hepner suggests, print statements can be grouped:

 #!/bin/bash file="/Users/USER12/Desktop/url-list.txt" fmt="%-25s%-12s%-12s\n" { printf "$fmt" DOMAIN_NAME HTTP_CODE RESPONSE_TIME while read line do read code time < <(curl -o /dev/null --silent --head --write-out '%{http_code} %{time_total}' "$line") printf "$fmt" "$line" "$code" "$time" done <"$file" } >"/Users/USER12/Desktop/url-results.txt" 

The advantage of grouping is that after the group, stdout is automatically restored to its normal value.

Answer 2

Shortened a little

 #!/bin/bash file="./url.txt" fmt="%s\t%s\t%s\n" ( printf "$fmt" "DOMAIN_NAME" "HTTP_CODE" "RESPONSE_TIME" while read -r line do printf "$fmt" "$line" $(curl -o /dev/null --silent --head --write-out '%{http_code} %{time_total}' "$line") done <"$file" ) | column -t > ./out.txt 

You don’t need to redirect each printf , but you can wrap part of your script in (...) and run it in a subnet to redirect its output. Print each field separated by one tab and use the column command to format it.

In any case, it’s usually best not to put the file names (or headers) in the script and reduce them to

 #!/bin/bash while read -r line do printf "%s\t%s\t%s\n" "$line" $(curl -o /dev/null --silent --head --write-out '%{http_code} %{time_total}' "$line") done | column -t 

and use it like:

 myscript.sh < url-list.txt >result.txt 

this allows you to use your script in pipes, for example:

 something_produces_urls | myscript.sh | grep 200 > somewhere.txt