Cucumbers and junit have the wrong number of tests in jenkins (true fire)

Question

When I try to use Cucumber and Junit together through Maven (surefire), then the report has the wrong number of tests. I have only 250 tests, but Jenkins shows me 1,200 tests. So when I researched it, I could only find the link, which is the surefire plugin problem.

https://github.com/cucumber/cucumber-jvm/issues/322

How to fix it?

Answer 1

I could only create a hack with confidence. if you want, you can use:

• you need to create a Junit report from Cucumber to some folder (1) 'oucumber'
• copy the assurance reports (excluding the Cucmber report created with the assurance) to a certain (2) "surefire-reports-fixed" folder where the cucumber generated the report.
• copy cucumber junit-report from (1) to (2)
• jenkins should use the (2) 'surefire-reports-fixed' folder

Examples of changes in Maven:

<plugin> <plugin> <artifactId>maven-resources-plugin</artifactId> <version>2.6</version> <executions> <execution> <id>change-bad-cucumber-test-file</id> <!-- here the phase you need --> <phase>prepare-package</phase> <goals> <goal>copy-resources</goal> </goals> <configuration> <outputDirectory>${basedir}/target/surefire-reports-fixed</outputDirectory> <resources> <resource> <directory>${basedir}/target/cucumber</directory> </resource> <resource> <directory>${basedir}/target/surefire-reports</directory> <excludes> <exclude>**/*CucumberTest.*</exclude> </excludes> </resource> </resources> </configuration> </execution> </executions> </plugin> 

changes in CucumberTest.java:

 @RunWith(Cucumber.class) @Cucumber.Options(format = { "pretty", "junit:target/cucumber/TEST-packageHereToClass.CucumberTest.xml", "json:target/cucumber.json" }) public class CucumberTest { ... } 

in jenkins set the correct folder for tests for (2) "surefire-reports-fixed- <" /

Answer 2

The following links seem relevant.

Discussion of the main problem of definitions of steps considered as tests:

https://github.com/cucumber/cucumber-jvm/issues/577

A separate issue, which in the comments discusses, that a fix for moving from counting steps as tests to counting scripts as junit tests will be available, but is related to Gherkin 3 development efforts:

https://github.com/cucumber/cucumber-jvm/issues/263

Answer 3

The following worked for me-. I added the cucumber-junit report format to @cucumbeOptions, i.e. - "junit: target / surefire-reports / TEST-TestSuite.xml" and then when I ran it in eclipse (also in VSO), Junit reports were generated in the TEST-TestSuite.xml file with the correct amount.

Code below ->

  @CucumberOptions( features= {"src/test/resources/Features"}, plugin = {"pretty:STDOUT","html:Reports/cucumber-pretty","junit:target/surefire-reports/TEST-TestSuite.xml", "com.cucumber.listener.ExtentCucumberFormatter:Reports/Reporting/Quixxi_Report.html"}, monochrome = true, dryRun=false )