Exception handling in the Python Behave Testing framework

Question

I was thinking of switching from nose to behave for testing (mocha / chay etc. spoiled me). So far so good, but I can’t figure out how to test exceptions:

@then("It throws a KeyError exception") def step_impl(context): try: konfigure.load_env_mapping("baz", context.configs) except KeyError, e: assert (e.message == "No baz configuration found") 

With a nose, I can annotate the test with

 @raises(KeyError) 

I cannot find anything like this in behavior (not in the source, not in the examples, not here). Of course, it would be great if you could specify the exceptions that can be selected in the outline of the script.

Has anyone been on this path?

Answer 1

I am new to BDD, but, as a rule, the idea would be that the test document that the client leads can expect rather than implementation steps. Therefore, I expect the canonical way to test this would be as follows:

 When I try to load config baz Then it throws a KeyError with message "No baz configuration found" 

With steps defined as:

 @when('...') def step(context): try: # do some loading here context.exc = None except Exception, e: context.exc = e @then('it throws a {type} with message "{msg}"') def step(context, type, msg): assert isinstance(context.exc, eval(type)), "Invalid exception - expected " + type assert context.exc.message == msg, "Invalid message - expected " + msg 

If this is a generic template, you can simply write your own decorator:

 def catch_all(func): def wrapper(context, *args, **kwargs): try: func(context, *args, **kwargs) context.exc = None except Exception, e: context.exc = e return wrapper @when('... ...') @catch_all def step(context): # do some loading here - same as before 

Answer 2

This try / catch approach works by Barry, but I see some problems:

• Adding an attempt / with the exception of your steps means that the errors will be hidden.
• Adding an additional decorator is inelegant. I would like my decorator to be modified by @where

My suggestion is to

• has an exception pending error output
• in try / catch, raise if error was not expected
• in after_scenario, raise the error if the expected error is not found.
• use modified data / when / then everywhere

code:

  def given(regexp): return _wrapped_step(behave.given, regexp) #pylint: disable=no-member def then(regexp): return _wrapped_step(behave.then, regexp) #pylint: disable=no-member def when(regexp): return _wrapped_step(behave.when, regexp) #pylint: disable=no-member def _wrapped_step(step_function, regexp): def wrapper(func): """ This corresponds to, for step_function=given @given(regexp) @accept_expected_exception def a_given_step_function(context, ... """ return step_function(regexp)(_accept_expected_exception(func)) return wrapper def _accept_expected_exception(func): """ If an error is expected, check if it matches the error. Otherwise raise it again. """ def wrapper(context, *args, **kwargs): try: func(context, *args, **kwargs) except Exception, e: #pylint: disable=W0703 expected_fail = context.expected_fail # Reset expected fail, only try matching once. context.expected_fail = None if expected_fail: expected_fail.assert_exception(e) else: raise return wrapper class ErrorExpected(object): def __init__(self, message): self.message = message def get_message_from_exception(self, exception): return str(exception) def assert_exception(self, exception): actual_msg = self.get_message_from_exception(exception) assert self.message == actual_msg, self.failmessage(exception) def failmessage(self, exception): msg = "Not getting expected error: {0}\nInstead got{1}" msg = msg.format(self.message, self.get_message_from_exception(exception)) return msg @given('the next step shall fail with') def expect_fail(context): if context.expected_fail: msg = 'Already expecting failure:\n {0}'.format(context.expected_fail.message) context.expected_fail = None util.show_gherkin_error(msg) context.expected_fail = ErrorExpected(context.text) 

I import the changed data / then / when instead of behaving and add environment.py to my environment to trigger the context. The expected failure before the script and checking it after:

  def after_scenario(context, scenario): if context.expected_fail: msg = "Expected failure not found: %s" % (context.expected_fail.message) util.show_gherkin_error(msg) 

Answer 3

The try / except method you suggest is really right, because it shows how you really use the code in real life. However, there is a reason why you do not quite like it. This leads to ugly problems with things like:

 Scenario: correct password accepted Given that I have a correct password When I attempt to log in Then I should get a prompt Scenario: correct password accepted Given that I have a correct password When I attempt to log in Then I should get an exception 

If I write the definition of the step without try / in addition, the second scenario will fail. If I write it using try / except, then in the first scenario, the risk will hide the exception, especially if the exception occurs after the invitation has already been printed.

Instead, these scripts should, IMHO, be written as something like

 Scenario: correct password accepted Given that I have a correct password When I log in Then I should get a prompt Scenario: correct password accepted Given that I have a correct password When I try to log in Then I should get an exception 

In step "I log in" you should not use try; “I'm trying to get in” comes up neatly to try to give away the fact that there may not be success.

Then the question arises of reusing code between two almost, but not exactly identical steps. We probably do not want to have two functions that are both inputs. Besides the simple general function that you call, you can also do something like this at the end of your steps file.

 @when(u'{who} try to {what}') def step_impl(context): try: context.exception=None except Exception as e: context.exception=e 

This will automatically convert all the steps containing the word "try" into steps with the same name, but with an attempt to delete, and then protect them with try / except.

There are a few questions about when you really need to deal with exceptions in BDD, as they are not visible to the user. This is not part of the answer to this question, although I have put them in a separate publication .

Answer 4

Conducting behavior is irrelevant. Therefore, he does not provide a solution for this. There are already enough Python packages that solve this problem.

CM. ALSO: behave.example: select a claim mapping library