C # Unit Testing warns instead of failing

Question

When using Assert (...), if the logical test fails, the unit test is canceled and the rest of the unit test does not start. Is there a way to make the logic test fail, but just provide a warning or something else and still run the rest of the unit test?

Context example: I have a test that creates some students, teachers and classes, creates relationships, and then puts them into a database. Then, some SSIS packages are run in this database, which take the existing data and convert it to another database schema in another database. Then the test should check the new database for certain things, for example, for the correct number of rows, actions, etc.

Obviously, other tests are erasures and modes, but they all follow the same structure - they create data in the source db, run SSIS packets, check the data in the target db.

Answer 1

It looks like you are trying to test too many things in one test.

If the precondition is not met, then apparently the rest of the test will also fail. I would rather finish the test as soon as I find out that this is not what I expect.

Unit testing concepts red fail, green pass. I know that MSTest also allows yellow, but it is not going to do what you want. You can do Assert.Inconclusive to get a yellow light. I used this when I was working on a code base that had a lot of integration tests based on data from a specific database. Instead of failing the test, I began to make the results inconclusive. The code may have worked very well, but the data was missing. And there was no reason to believe that the data would always be there (they were not good IMO tests).

Answer 2

From what you explained in the question, this is more of a reception test (than a unit test). Unit testing frameworks are designed for quick failure. That's why Assert behaves the way it does (and that's good).

Returning to your problem: you should take a look at using an acceptance test testing platform like Fitnesse, which will support what you want and show me the steps that failed, but continue to the end.
However, if you MUST use the unit testing framework, use the collection parameter / parameter to model this behavior. eg

• Maintain List<string> in test
• add a descriptive error message for each failed step
• At the end of the test, assert that the collecting variable is empty.

Answer 3

If you use Gallio / MbUnit , you can use Assert.Multiple to achieve what you want. It captures failed statements, but does not stop the test immediately. All unsuccessful statements are collected and reported later at the end of the test.

 [Test] public void MultipleAssertSample() { Assert.Multiple(() => { Assert.Fail("Boum!"); Assert.Fail("Paf!"); Assert.Fail("Crash!"); }); } 

The test in the above example obviously fails, but what arises is that the three failures are shown in the test report. Execution does not stop on the first failure.

Answer 4

Unit testing of black and white - either you pass the tests or not, or you break the logic or not, or your data in the database is correct or not (although the unit of testing with the database is no longer unit testing in itself).

What are you going to do with the warning? Is it a pass or a failure? If it passes, then what is the point of unit testing in this case? If it fails .. well .. it just won't work.

I suggest spending a little time figuring out what should be tested and how it should be tested. "Unit testing" is a cliché term used by many for very different things.

Answer 5

I had a similar problem when I wanted to get a more meaningful failure report. I compared collections and used the wrong number of elements - I don’t know what the real reason for the failure was. Unfortunately, I ended up writing the comparison code manually - therefore, checking all the conditions and then making a single at the end with a good error message.

Answer 6

I know that your question was asked a few years ago. But recently (around 2017 or 2018) NUNIT 3 supports warnings. You can embed the [boolean] test in Assert.Warning in the same way as Assert.Fail. But instead of a single Assert line failing the entire test, running the test will record a warning and continue the test.

Read about it here: https://github.com/nunit/docs/wiki/Warnings

It behaves like Multiple (listed by @Yann Trevin above, and Multiple is also available in NUnit 3.0). An excellent difference, however, is in integration tests, which show the flexibility of using stand-alone Assert.Warning commands. Contrast with the band. Claims in a multiple instance. After completing multiple confirmation, the test may not continue.

Integration tests, especially those that can work for several hours to possibly check how well all microservices work together, require a restart. In addition, if you have several teams (external, internal, external, infernal and eternal) and time zones that fix the code almost all the time, it can be difficult to get a new product to run a comprehensive integration test. the path to the end of his workflow when all parts are placed together. (note. It is important to bring together teams with sufficient knowledge in the subject area and at least with sufficient knowledge in the field of software development to form reliable “contracts” about what each API will be and to manage it well. This should help eliminate inconsistencies implied above.)

Simple black and white testing, passing / not passed, is absolutely suitable for unit testing.

But as systems become more abstract, multi-level with respect to service after service, agent by agent, the ability to recognize the reliability and reliability of systems becomes more and more important. We already know that small blocks of code will work as intended; Unit tests and code coverage tell us this. But when they all have to work on top of someone else’s infrastructure (AWS, Azure, Google Cloud), unit testing is not good enough.

Knowing how many times the service had to retry, how much the service cost, will the system comply with SLA under certain loads? This is what integration tests can help figure out using the Assert type you asked about @dnatoli.

Given the number of years since your question, you are almost certainly already an expert.