ExpectedFailure counts as an error, not as accepted

Question

I use expectedFailure because there is an error that I want to write that I cannot fix right now, but I want to return to it in the future. My understanding of expectedFailure is that it will consider the test passed, but the summary says that there were x the number of expected failures (similar to how it works with missing tets).

However, when I run my test suite, I get the following:

 $ ./manage.py test eav.QueryTest Creating test database for alias 'default'... .EE ====================================================================== ERROR: test_q_object_with_exclude (eav.tests.managers.QueryTest) ---------------------------------------------------------------------- _ExpectedFailure ====================================================================== ERROR: test_q_objects_unioned (eav.tests.managers.QueryTest) ---------------------------------------------------------------------- _ExpectedFailure ---------------------------------------------------------------------- Ran 3 tests in 1.095s FAILED (errors=2) Destroying test database for alias 'default'... 

I'm not sure if this is due to the Django test runner or something that I am doing wrong.

 @unittest.expectedFailure def test_q_object_with_exclude(self): # Everyone except Bob q_set = eav_m.Process.objects.exclude( Q(eav__details__city__contains='Y')) self.assertEqual(q_set.count(), 4) 

Answer 1

Your understanding of expectedFailure is correct. Your problem is that these tests do not fail, they throw an exception that does not match the failure.

The decorator you are looking for is skip .

Answer 2

No, the expected failure will not help you write negative tests, use self.assertNotEqual for this. expectedFailure is used to indicate a test, which, as you know, is broken, and you confirm this fact. Do not miss this by stating that your confirmation is not valid.

Answer 3

You should take a look at the SO question because this is the wrong behavior. A test with the expectedFailure decorator should not be considered a failure if the test fails.