Jenkins reports successful testing, although there are errors caused by exceptions?

Question

I have SeleniumWebdriver / TestNG / Maven / Java continuous integration tests that run every time after deployment. Sometimes there is no element in the user interface, and tests throw an exception (which later gets into the code because I turned off the browser in the catch statement), so the assembly is marked as successful.

The strange thing is: I also had failures in tests caused by exceptions, and the build was still considered successful.

How can I configure the maven pom.xml file or jenkins assembly so that it can flag every test that throws an exception, FAILURE?

EDIT: after receiving the robjohncox response, I now have one more thing I need to do:

How can I reset the error again?

} catch (Exception e) { // TODO Auto-generated catch block e.printStackTrace(); quit(driver); sendMail(); } 

Is this done so?

  throw e; 

Answer 1

I think the problem is that you caught the exception in the code. Since you are handling the exception, it does not apply to your test runner, and therefore the testing tool does not know that the exception was thrown.

After detecting an exception and disabling the browser, you should throw an exception again, and then test failures should be reported using your test environment. The code will look something like this:

 public void myTestCase() { try { // Do the testing } catch(Exception ex) { // Turn off the browser throw ex } }