Get the number of passed and failed tests with their name from the command line

Question

I am writing a Python script that will run mvn testin different folders, and I want to get the number of passed tests and failed tests and their names from the script.

Now I just managed to start the process and get its output

proc = subprocess.run(["mvn", "test", "-f", PROJ_PATH])
print(proc.stdout)

output:

   Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.715 s
[INFO] Finished at: 2016-10-12T18:59:11+03:00
[INFO] Final Memory: 10M/212M
[INFO] ------------------------------------------------------------------------

I know that I can use regexp and try to parse the output, but there may be some more suitable ways to enable Maven from Python or Bash.

Answer 1

There are several solutions, but no direct one ... they will all include some parsing (XML or text file).

XML reports

, , . Surefire XML target/surefire-reports. XML, . XML XSD .

XML ( ) target/surefire-reports TEST-${testClass}.xml, ${testClass} . my.test.MyTest :

<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="my.test.MyTest" tests="4" errors="1" skipped="1" failures="1">
  <properties> <!-- omitted, contains system properties for the test --> </properties>
  <testcase name="test" classname="my.test.MyTest" time="0.096">
    <failure></failure>
  </testcase>
  <testcase name="test2" classname="my.test.MyTest" time="0.001">
    <error></error>
  </testcase>
  <testcase name="test3" classname="my.test.MyTest" time="0.002"/>
  <testcase name="test4" classname="my.test.MyTest" time="0">
    <skipped/>
  </testcase>
</testsuite>

( , ). , <testsuite> , 4 , 1, 1 1; 1 . , <testcase> name, . 4 :

• (, , ): <testcase> <failure>.
• ( , ): <testcase> <error>.
• skip: <testcase> <skipped>.
• success: <testcase> .

, classname ( ) name ( ).

Surefire plain reportFormat

<plugin>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.19.1</version>
  <configuration>
    <reportFormat>plain</reportFormat>
  </configuration>
</plugin>

:

Running my.test.MyTest
Tests run: 4, Failures: 1, Errors: 1, Skipped: 1, Time elapsed: 0.169 sec <<< FAILURE! - in my.test.MyTest
test(my.test.MyTest)  Time elapsed: 0.112 sec  <<< FAILURE!
java.lang.AssertionError
    at my.test.MyTest.test(my.test.MyTest.java:16)

test2(my.test.MyTest)  Time elapsed: 0.001 sec  <<< ERROR!
java.lang.IllegalArgumentException: Exception
    at my.test.MyTest.test2(my.test.MyTest.java:21)

test3(my.test.MyTest)  Time elapsed: 0.002 sec
test4(my.test.MyTest) skipped

, (.*)\((.*)\)\s+(?|(skipped)|Time elapsed:.*<<< (.*)): 1, 2 3 ; 3 , .