Scripting code coverage tools

Question

Is there any way to check for shell script coverage? I have a project with a lot of shell scripts, and you need to ensure that static analysis can be done to cover shell scripts. Is there an available tool?

Answer 1

I seriously doubt that there can be any static code analysis performed in shell scripts, especially because shell scripts must call external programs and rely on what these external programs return, and there are a myriad of external programs and states the external environment. It looks like a static code analysis problem that relies heavily on the eval like mechanism, but shell scripts are all about eval-style programming.

However, there are some general pointers that may be useful for "checking" correctly, covering code, and documenting shell scripts, as the main languages ​​do:

• You can always run the script with the -x option (AKA xtrace ) - it will output a trace similar to stderr:

   + log_end_msg 0 + [ -z 0 ] + retval=0 + log_end_msg_pre 0 + : + log_use_fancy_output + TPUT=/usr/bin/tput + EXPR=/usr/bin/expr + [ -t 1 ] + [ xxterm != x ] + [ xxterm != xdumb ] 

• Bash allows you to redirect this stream to an external FD (using BASH_XTRACEFD ) - it is much easier to parse in practice.

• This is not trivial, but you can write a program that finds the appropriate code fragments executed using the xxtrace output and gives you a fancy report on the code coverage - like what was called, how many times, which parts of the code didn't execute at all and therefore did not have testing coverage.

• Actually there is a wonderful tool called shcov that uses this process, although it is somewhat simplified and doesn’t work "handle all possible cases perfectly (especially when we are talking about long and complex lines)

• And last but not least, there is a minimalistic shelldoc project (akin to javadoc ) that helps to generate documentation based on comments in shell scripts. Yes, this is a shameless plugin :)

Answer 2

I don’t think that COTS tools are available for testing, regardless of the scripting language, and there are many.

Another poster suggested a special approach that might work with some tools: get them to dump some trace data and try to match this with the actual code to get your coverage. He says that this is a kind of work ... that the problem is with most hueristics.

Another approach that could be used to create test coverage for your favorite scripting language is covered in my white paper on a general approach for creating tools for testing objects using software transformations. My company is building a line of such tools for more popular languages ​​in this way.

Answer 3

You can try shcov , a Python-based tool licensed by GPL v2. He was probably left by the author, but he creates HTML-based graphical reports and seemed (in my limited testing) to be fairly accurate in terms of coverage analysis.

Answer 4

I wrote a tool that can do coverage for shell scripts, the name shAge means shell script, the project is located here

To make coverage if any shell scripts do the following

• First download the shAge.jar file
• Make sure you have jdk1.6 65 update.

• run a program, for example

  java -jar shAge.jar hello.sh

• the contents in the shell script will be executed and finally a report will be generated

• The report will be disclosed with a file name in the format .html hello.sh.html
• Here is an example output