Doctest error with zero exit code

Question

In my test code, my doctest failed, but the script exits with a null return value, which causes the CI to pass, which is not intended.

Is this the correct behavior of the doctest module?

My script ends with:

if __name__ == '__main__': import doctest doctest.testmod() 

The output looks like this:

 ********************************************************************** File "test/test.py", line 7, in __main__ Failed example: f(1,0) Expected: ----- type: <type 'exceptions.ZeroDivisionError'> value: integer division or modulo by zero x ----- Got: ----- type: <type 'exceptions.ZeroDivisionError'> value: integer division or modulo by zero ----- ********************************************************************** 1 items had failures: 1 of 1 in __main__ ***Test Failed*** 1 failures. tux@iPad :~/programming/exception-notifier(fix-travis)(0)$ echo $? 0 

Answer 1

I find that using doctest.testmod(raise_on_error=True) will throw an exception when the test fails, which will cause the script to exit with non-zero code.

Python doc here :

The optional raise_on_error argument is false by default. If so, an exception occurs on the first failure or an unexpected exception in the example. This allows fault tolerance to be delayed. The default behavior is to continue executing the examples.

Answer 2

@fossilet is responsible for correctly breaking down assemblies that fail to test but throw an exception before doctest can write anything to the console. This makes your logs much less useful for identifying problems.

The alternative is to call

 sys.exit(doctest.testmod()[0]) 

This makes the process exit code equal to the number of failed tests. Your CI tool should interpret nonzero exit codes as failures. But the doctrine will still go to the console.