Python pretest for coverage failure exceptions

Question

I recently came across a simple but unpleasant mistake. I had a list, and I wanted to find the smallest member in it. I used Python's built-in min (). Everything worked fine until in a strange scenario the list was empty (due to a strange user input that I could not expect). My application crashed using ValueError (BTW - not documented in white papers).

I have very extensive unit tests and I regularly check coverage to avoid such surprises. I also use Pylint (everything is integrated in PyDev) and I never ignore warnings, but I could not catch this error before my users did.

Is there anything I can change in my methodology to avoid such runtime errors? (which would be caught at compile time in Java / C #?).

I am looking for something more than porting my code with a big try. What else can I do? How many other functions in Python hide unpleasant surprises?

Answer 1

The problem is that the wrong external input crashed your program. The solution is an exhaustive unit test of possible input scripts at the borders of your code. You say that your unit tests are "extensive", but you obviously did not test this feature. Code coverage is a useful tool, but it is important to remember that coverage code does not match its rigorous testing. Careful testing is a combination of coverage use cases as well as lines of code.

The methodology I use is to trust internal callers, but never trust external callers or inputs. Therefore, I am clearly not a unit test for an empty list in any code outside the first function that receives an external input. But this input function should be exhaustive .

In this case, I believe that the exception of the library - reasonable behavior - it makes no sense to request min empty list. The library cannot legally set a value for you, such as 0, since you can deal with negative numbers, for example.

I think that an empty list should never reach the code that min asks for - it should be identified at the input, and either raise an exception there, or set it to 0, if this works for you or something else, it’s something that works for you.

Answer 2

Even in Java / C #, the exception class, RuntimeError is not marked and will not be detected by the compiler (therefore they are called RuntimeError not CompileError).

In python, some exceptions, such as KeyboardInterrupt, are especially hairy, as they can be raised at almost any arbitrary point in the program.

I am looking for something more than porting my code with a big try.

Nothing but this, please. It is much better to let exceptions access the user and stop the program, rather than skipping the error silently (Zen of Python).

Unlike Java, Python does not require all Exceptions to be caught, because the requirement that all Exceptions be caught makes it too easy for programmers to ignore the Exception (by writing an empty exception handler).

Just relax, let the mistake stop; let the user report this to you so that you can fix it. Another alternative is that you go to the debugger within forty-two hours because the client data is corrupted everywhere because of the empty mandatory exception handler.

So, what you need to change in your methodology, think that the exception is bad; they are not beautiful, but they are better than alternatives.

Answer 3

You could use randomized testing:

 #!/usr/bin/env python import random from peckcheck import TestCase, an_int, main def a_seq(generator): return lambda size: [generator(size) for _ in xrange(random.randrange(size))] class TestMin(TestCase): def testInputNoThrow(self, x=a_seq(an_int)): min(x) if __name__=="__main__": main() 

To install peckcheck , type:

 $ pip install http://github.com/downloads/zed/peckcheck/peckcheck-0.1.v2.6.tar.gz 

Or just grub peckcheck.py

Answer 4

I do not know a direct answer to your question; I would also like it if Pilint warned of such possibilities. My general practice, given that empty lists cause problems in all situations, is to check the truth lists before using them; eg:

 val = min(vals) if vals else 0 

In many cases, this is “free,” since you often need to check for None . It can also pay off with special empty lists to avoid, i.e. Start a new thread, process, or database transaction to process null items.