Print JUnit error count in Ant

Question

<target name="test" depends="compile-test"> <junit failureProperty="test.failure"> <classpath refid="classpath.test" /> <formatter type="brief" usefile="false" /> <batchtest> <fileset dir="${tst-dir}" includes="**/Test*.class" /> </batchtest> </junit> <fail message="test failed" if="test.failure" /> </target> 

I want to print how many test cases:

• failed
• Error
• transmitted

making changes only to the build.xml file. How can i do this?

Answer 1

You can use the junitreport task to collect test results.

If you need to print totals in an assembly file, you can use a filter chain to extract information from the generated report.

Maybe (should?) Be an easier way to do this, but I haven't seen it.

 <target> <junit failureProperty="test.failure"> <classpath refid="classpath.test" /> <!-- use XML formatter and let it output to file --> <formatter type="xml" /> <!-- specify output dir for test result files --> <batchtest todir="tmp/results"> <fileset dir="${tst-dir}" includes="**/Test*.class" /> </batchtest> </junit> <!-- generate report with junitreport --> <junitreport todir="tmp"> <fileset dir="tmp/results" /> <report todir="tmp/report" /> </junitreport> <!-- concat the report through a filter chain to extract what you want --> <concat> <fileset file="tmp/report/overview-summary.html" /> <filterchain> <linecontainsregexp> <regexp pattern='title="Display all tests"' /> </linecontainsregexp> <tokenfilter> <replaceregex pattern='&lt;td&gt;&lt;a href="all-tests.html" title="Display all tests"&gt;(\d+)&lt;/a&gt;&lt;/td&gt;&lt;td&gt;&lt;a href="alltests-fails.html" title="Display all failures"&gt;(\d+)&lt;/a&gt;&lt;/td&gt;&lt;td&gt;&lt;a href="alltests-errors.html" title="Display all errors"&gt;(\d+).*$' replace="Run: \1, Failed: \2, Errors: \3" /> </tokenfilter> </filterchain> </concat> <fail message="test failed" if="test.failure" /> </target> 

The output will look something like this:

 Buildfile: C:\\test\unit_test.xml test: [junit] Test MyUnitTest FAILED [junit] Test MyUnitTest2 FAILED [junitreport] Processing C:\\test\tmp\TESTS-TestSuites.xml to C:\DOCUME~1\xxx\LOCALS~1\Temp\1\null1075123857 [junitreport] Loading stylesheet jar:file:/C:/eclipse/eclipse-jee-ganymede-SR2-win32/eclipse/plugins/org.apache.ant_1.7.0.v200803061910/lib/ant-junit.jar!/org/apache/tools/ant/taskdefs/optional/junit/xsl/junit-frames.xsl [junitreport] Transform time: 906ms [junitreport] Deleting: C:\DOCUME~1\xxx\LOCALS~1\Temp\1\null1075123857 [concat] Run: 8, Failed: 4, Errors: 1 BUILD FAILED C:\test\unit_test.xml:32: test failed Total time: 1 second 

If you are performing a large number of tests, you will now have the overhead of extracting the report generation.

Answer 2

The exact same answer as sudocode, but with this line for analyzing the report (works for RSA 8.5.1 / JUnit 4.11), and I was not allowed to post this piece of code as a comment in sudocodes code - and I am not allowed to comment .. .):

 <replaceregex pattern='&lt;td&gt;&lt;a title="Display all tests" href="all-tests.html"&gt;(\d+)&lt;/a&gt;&lt;/td&gt;&lt;td&gt;&lt;a title="Display all failures" href="alltests-fails.html"&gt;(\d+)&lt;/a&gt;&lt;/td&gt;&lt;td&gt;&lt;a title="Display all errors" href="alltests-errors.html"&gt;(\d+).*$' replace="Run: \1, Failed: \2, Errors: \3" /> 

Thanks sudocode!

Answer 3

As an alternative to the sudokod approach: you can set the printsummary attribute in junit-task . This will print a summary after each test class. This is not a general summary.

  <junit failureProperty="test.failure" printsummary="yes"> <classpath refid="classpath.test" /> <formatter type="brief" usefile="false" /> <batchtest> <fileset dir="${tst-dir}" includes="**/Test*.class" /> </batchtest> </junit> 

Answer 4

Try this attribute in your junit task:

 printsummary="yes" 

For a smooth javadoc-like html report, change your formatter to:

 <formatter type="xml" /> 

and then create reports for the purpose that causes this:

 <junitreport> <fileset dir="${report.dir}/tmp"> <include name="TEST-*.xml" /> </fileset> <report format="frames" styledir="${junitxslt.dir}" todir="${report.dir}/html" /> </junitreport> 

Answer 5

You can use junitreport to create a federated XML report from a set of test runs.

Then, to generate a text summary, you can create an XSLT and use ant XSLT to format the file. This will produce an output file, but you can use ant to read this and repeat it in the console.

XSLT should use something line by line to count test boxes, errors, and crashes.

  count(//testsuites/testcase) count(//testsuites/testcase/error) count(//testsuites/testcase/error) 

(If you really want to modify your ant build file, you can generate XSLT in a temporary folder during build and delete it later.)

Answer 6

Try inserting Jsoup into a custom task and use the task in your assembly to extract the necessary data from the summary-summary.html.

My code snippet below is

 import org.apache.tools.ant.BuildException; import org.apache.tools.ant.Task; import org.jsoup.Jsoup; import org.jsoup.nodes.Document; public class ExtractJunitSummaryTask extends Task { protected String overviewSummaryHtml; protected String outProperty; public void execute() throws BuildException { StringBuffer sb = new StringBuffer(); // TODO: read and put overviewSummaryHtml file content into a String Document doc = Jsoup.parse(sb.toString()); String allTests = doc.getElementsByAttributeValueContaining("href", "all-tests").text(); String allFailures = doc.getElementsByAttributeValueContaining("href", "alltests-fails").text(); String allErrors = doc.getElementsByAttributeValueContaining("href", "alltests-errors").text(); String allSkippedTests = doc.getElementsByAttributeValueContaining("href", "alltests-skipped").text();