I have JUnit 5 @TestFactory that generates a stream of test cases for the class, and when I run it, IntelliJ tells me that all the tests pass. However, when I re-run the tests in Netbeans, I see that some of the tests actually failed due to exceptions. The number of "passed tests" is the same, but it seems that any failed tests just disappear in IntelliJ, and are not reported as errors.
The proof is that the tests seem to end in exit code 255.
In the images below you can see how IntelliJ reports only the passed tests (22,307), while NetBeans reports are transmitted both (22,307) and unsuccessful (4).
IntelliJ IDEA:
Netbeans
I am running JUnit version 5.0.0-M3 and IntelliJ IDEA version 2016.3.5.
Edit 2017-03-10: So far, I have come to the conclusion that the tests are being executed, since I can set a breakpoint just before the exception is thrown, and go through how JUnit will catch it. As far as I can tell, everything looks right going through the JUnit code, so I have no idea why IntelliJ doesn't pick it. If I run the surefire:test
target manually, I can see all the failures in the generated XML files.
source share