I have a set of tests that run in continuous integration using a parallel test. After deployment, the environment is a little unstable, so when the package is completed, failed tests are restarted (the environment is stable again at this point).
Basically, there is only one jenkins build, and it will do this:
task :jenkins1_with_rerun do selenium_successful = system "bundle exec parallel_cucumber features/web/ -o \"-p jenkins1\" -n 3" p 'start rerun' rerun_successful = run_rake_task("features:jenkins_rerun") unless selenium_successful || rerun_successful raise 'Cucumber tests failed' end end
The first execution launches a package with parallel tests. Here is the profile of the cucumber:
jenkins1: <%= std_opts %> HEADLESS=true --tags @part1 --tags ~@wip --format junit --out log/junit --format ParallelTests::Cucumber::FailuresLogger --out parallel_cucumber_failures.log
Upon completion. The second run is executed, starting the failed tests recorded in the file. The profile of the cucumber for this implementation is as follows:
jenkins_rerun: ENVIRONMENT=master SCREENSHOT=true HEADLESS=true --format pretty --no-source --format junit --out log/junit_rerun @parallel_cucumber_failures.log
Now. Everything works. The only problem is that I'm using jenkins junit report to plot failed and successful tests. I may have reports like this:
------------------- Tests --- KO --- OK
First performance --- 75 ---- 10 ---- 65
Relay ------------- 10 ------- 0 ---- 10
This is 100% green because all problems were caused by instability after deployment. So I want the jenkins junit report to say that 75 tests, 75 OK, 0 KO, or something similar, were running.
Currently, junit’s first run report speaks of 75 tests, I have 10 knockouts; and the second unit says that out of 10 tests there are 0 knockouts.
What could be a good solution? Can I mix the results of both Junit reports?
I also agree to display in jenkins as junit reports, each with a graph. But I think jenkins only allows you to display one junit report report.