All geek questions in one place

How to manage log documents inside a shell script?

I have a rather long bash script that calls quite a few external commands ( git clone , wget , apt-get and others) that print a lot of material to standard output.

I want the script to have several options for details, so it prints all of the external commands, the summary version (for example, "Installing dependencies ...", "Compilation ...", etc.) or nothing. But how can I do this without cluttering up all the code?

I thought about possible solutions for this: you need to create a wrapper function that runs external commands and prints what is needed for standard output, depending on the parameters specified at the beginning. It seems easier to implement, but it means adding a lot of extra interference to the code.

Another solution is to send all the output to a couple of external files, and when analyzing the arguments at the beginning of the script, run tail -f in this file, if specified. It would be very easy to implement, but it seems to me that it's quite complicated, and I am concerned about its impact on performance.

Which one is better? I am also open to other solutions.

+5
source share
4 answers

You already have the cleanest idea in your question (wrapper function), but you seem to think it will be dirty. I would suggest you reconsider. It might look like this (not necessarily a complete solution, just to give you a basic idea):

 #!/bin/bash # Argument 1 : Logging level for that command # Arguments 2... : Command to execute # Output suppressed if command level >= current logging level log() { if (($1 >= logging_level)) then "${@:2}" >/dev/null 2>&1 else "${@:2}" fi } logging_level=2 log 1 command1 and its args log 2 command2 and its args log 3 command4 and its args

You can organize any required redirection (with file descriptors, if you want) for processing in the wrapper function, so that the rest of the script remains readable and does not have redirects and conditions depending on the selected logging level.

+4
source

Improving @Fred's idea is a bit more, we could create a small registration library this way:

 declare -A _log_levels=([FATAL]=0 [ERROR]=1 [WARN]=2 [INFO]=3 [DEBUG]=4 [VERBOSE]=5) declare -i _log_level=3 set_log_level() { level="${1:-INFO}" _log_level="${_log_levels[$level]}" } log_execute() { level=${1:-INFO} if (( $1 >= ${_log_levels[$level]} )); then "${@:2}" >/dev/null else "${@:2}" fi } log_fatal() { (( _log_level >= ${_log_levels[FATAL]} )) && echo "$(date) FATAL $*"; } log_error() { (( _log_level >= ${_log_levels[ERROR]} )) && echo "$(date) ERROR $*"; } log_info() { (( _log_level >= ${_log_levels[INFO]} )) && echo "$(date) INFO $*"; } log_debug() { (( _log_level >= ${_log_levels[DEBUG]} )) && echo "$(date) DEBUG $*"; } log_verbose() { (( _log_level >= ${_log_levels[VERBOSE]} )) && echo "$(date) VERBOSE $*"; } # functions for logging command output log_debug_file() { (( _log_level >= ${_log_levels[DEBUG]} )) && [[ -f $1 ]] && echo "=== command output start ===" && cat "$1" && echo "=== command output end ==="; } log_verbose_file() { (( _log_level >= ${_log_levels[VERBOSE]} )) && [[ -f $1 ]] && echo "=== command output start ===" && cat "$1" && echo "=== command output end ==="; }

Let's say that the above source is in a library file called logging_lib.sh, we could use it in a regular shell script as follows:

 #!/bin/bash source /path/to/lib/logging_lib.sh set_log_level DEBUG log_info "Starting the script..." # method 1 of controlling a command output based on log level log_execute INFO date # method 2 of controlling the output based on log level date &> date.out log_debug_file date.out log_debug "This is a debug statement" ... log_error "This is an error" ... log_fatal "This is a fatal error" ... log_verbose "This is a verbose log!"

Result:

 Fri Feb 24 06:48:18 UTC 2017 INFO Starting the script... Fri Feb 24 06:48:18 UTC 2017 === command output start === Fri Feb 24 06:48:18 UTC 2017 === command output end === Fri Feb 24 06:48:18 UTC 2017 DEBUG This is a debug statement Fri Feb 24 06:48:18 UTC 2017 ERROR This is an error Fri Feb 24 06:48:18 UTC 2017 FATAL This is a fatal error

As we can see, log_verbose did not produce any output, since the log level is on DEBUG, one level below VERBOSE. However, log_debug_file date.out did indeed output the result, as well as log_execute INFO , as the log level is set to DEBUG, which is = INFO.

Using this as a base, we could also write shells if we need even finer tuning:

 git_wrapper() { # run git command and print the output based on log level }

With their help, the script could be strengthened to accept the argument --log-level level , which can determine the verbosity of the log with which it should work.

If anyone is interested to know why some variables are named with leading underscores in the code above, see this post:

+4
source

Solution 1. Consider using additional file descriptors. Redirect required file descriptors to STDOUT or / dev / null depending on the verbosity chosen. Redirect the output of each statement in the script to a file descriptor corresponding to its importance. Take a look at https://unix.stackexchange.com/a/218355 .

+1
source

Solution 2

Set $ ​​required_verbosity and send the STDOUT of each statement in the script to the script helper with two parameters, something like this:

expression | logger actual_verbosity $ required_verbosity

In the logger script, the echo is STDIN in STDOUT (or the log file, whatever) if $ actual_verbosity> = $ required_verbosity.

+1
source

Source: https://habr.com/ru/post/1264597/

More articles:


All Articles