Warnings from the perspective of the caller (aka Python equivalent of Perp carp)?

Question

Short version:

Is there a way to achieve the same effect in Python that is achieved with the Perl Carp::carp utility?

Long version (for those who are not familiar with Carp::carp ):

Suppose we implement some library API function (i.e. it is intended for use by other programmers in their code), say spam , and suppose that spam contains some code to verify the validity of the arguments passed to it. Of course, this code should throw an exception if any problem is found with these arguments. Let's say we want the related error message and trace to be as useful as possible for someone debugging some client code.

Ideally, the last trace line generated by this raised exception should indicate a “violation code,” namely, a line in the client code where spam is called with invalid arguments.

Unfortunately, this is not what will happen, at least by default, using Python. Instead, the last trace line will be mentioned somewhere inside the library’s internal code, where the exception was actually raise 'd, which would be rather obscure for the intended audience of this particular trace.

Example:

 # spam.py (library code) def spam(ham, eggs): ''' Do something stupid with ham and eggs. At least one of ham and eggs must be True. ''' _validate_spam_args(ham, eggs) return ham == eggs def _validate_spam_args(ham, eggs): if not (ham or eggs): raise ValueError('if we had ham ' 'we could have ham and eggs ' '(if we had eggs)') # client.py (client code) from spam import spam x = spam(False, False) 

When we run client.py , we get:

 % python client.py Traceback (most recent call last): File "client.py", line 3, in <module> x = spam(False, False) File "/home/jones/spam.py", line 7, in spam _validate_spam_args(ham, eggs) File "/home/jones/spam.py", line 12, in _validate_spam_args raise ValueError('if we had ham ' ValueError: if we had ham we could have ham and eggs (if we had eggs) 

whereas we want to be closer:

 % python client.py Traceback (most recent call last): File "client.py", line 3, in <module> x = spam(False, False) ValueError: if we had ham we could have ham and eggs (if we had eggs) 

... with a violation code ( x = spam(False, False) ) as the last line of the trace.

We need some way to report the error “from the perspective of the caller” (this is what Carp::carp allows you to do in Perl).

EDIT: To be clear, this question is not about LBYL and EAFP, nor prerequisites, nor programming. I'm sorry if I gave this wrong impression. This question is about how to create a trace, starting from several (one, two) levels up the call stack.

EDIT2: The Python traceback module is the obvious place to look for the Python equivalent of Perl Carp::carp , but after studying it for some time, I could not find a way to use it for what I want to do. FWIW, Perl Carp::carp allows you to fine-tune the source frame for tracing by exposing the global (hence dynamically limited) variable $Carp::CarpLevel . Library functions other than APIs that can carp -out, local -ize and increment this variable when writing (for example, local $Carp::CarpLevel += 1; ). I don't see anything even remotely like this Python traceback module. So, if I didn't miss anything, any solution using Python traceback would have to take a completely different approach ...

Answer 1

This is really a convention issue, exception handling in python is intended to be used to a large extent (ask for forgiveness, not ask for permission). And given that you work in a different language space, you want to follow these conventions - i.e. / you really want to tell the developers where the exception site is. But if you really need to do this ...

Using the validation module

checking the module will do almost everything you need to restore a beautiful version of the carp that works without worrying about decorators (see below). According to the comments in this answer , it may happen that this approach is broken down into pythons other than cpython

 # revised carp.py import sys import inspect def carp( msg ): # grab the current call stack, and remove the stuff we don't want stack = inspect.stack() stack = stack[1:] caller_func = stack[0][1] caller_line = stack[0][2] sys.stderr.write('%s at %s line %d\n' % (msg, caller_func, caller_line)) for idx, frame in enumerate(stack[1:]): # The frame, one up from `frame` upframe = stack[idx] upframe_record = upframe[0] upframe_func = upframe[3] upframe_module = inspect.getmodule(upframe_record).__name__ # The stuff we need from the current frame frame_file = frame[1] frame_line = frame[2] sys.stderr.write( '\t%s.%s ' % (upframe_module, upframe_func) ) sys.stderr.write( 'called at %s line %d\n' % (frame_file, frame_line) ) # Exit, circumventing (most) exception handling sys.exit(1) 

What for the following example:

  1 import carp 2 3 def f(): 4 carp.carp( 'carpmsg' ) 5 6 def g(): 7 f() 8 9 g() 

Outputs:

 msg at main.py line 4 __main__.f called at main.py line 7 __main__.g called at main.py line 9 

Using Traceback

This was an original approach.

The carp equivalent can also be written in python by manipulating trace objects, see the documentation in the trace module . The main task in this case is to enter an exception code and trace trace. It is worth noting that the code in this section is very fragile.

 # carp.py import sys import traceback ''' carp.py - partial emulation of the concept of perls Carp::carp ''' class CarpError(Exception): def __init__(self, value): self.value = value def __str__(self): return repr(self.value) def carpmain( fun ): def impl(): try: fun() except CarpError as ex: _, _, tb = sys.exc_info() items = traceback.extract_tb(tb)[:-1] filename, lineno, funcname, line = items[-1] print '%s at %s line %d' % (ex.value, filename, lineno) for item in items[1:]: filename, lineno, funcname, line = item print '\t%s called at %s line %d' % (funcname, filename, lineno) return impl def carp( value ): raise CarpError( value ) 

which can be called using the following basic process:

 import carp def g(): carp.carp( 'pmsg' ) def f(): g() @carp.carpmain def main(): f() main() 

The output of which:

 msg at foo.py line 4 main called at foo.py line 12 f called at foo.py line 7 g called at foo.py line 4 

Perl Reference Example

For completeness, both solutions proposed in this answer were debugged by comparing the results with this equivalent perl example:

  1 use strict; 2 use warnings; 3 use Carp; 4 5 sub f { 6 Carp::carp("msg"); 7 } 8 9 sub g { 10 f(); 11 } 12 13 g(); 

Which has an output:

 msg at foo.pl line 6 main::f() called at foo.pl line 10 main::g() called at foo.pl line 13 

Answer 2

You can use try..except in the top-level API function ( foo ) to throw another exception:

 class FooError(Exception): pass def foo(): try: bar() except ZeroDivisionError: raise FooError() def bar(): baz() def baz(): 1/0 foo() 

Thus, when an API user calls foo and an exception is thrown, all they see is a FooError , not an internal ZeroDivisionError .

Answer 3

What you want to do is known as the preconditions installation function , and in Python there is no language support. Python is also not as much hacked as perl (if, possibly, if you use PyPy), so it cannot be completely added.

Thus, the PyContracts module seems to do this relatively smoothly, using function decorators and string-based precondition specifications. I didn’t use the module myself, but it looks like it can make you something closer than you want. Here is the first example on his information page:

 @contract def my_function(a : 'int,>0', b : 'list[N],N>0') -> 'list[N]': # Requires b to be a nonempty list, and the return # value to have the same length. ...