Should what happens in the unit test be displayed as it starts?

Question

Since I code my unit tests, I usually find that I am inserting the following lines:

Console.WriteLine("Starting InteropApplication, with runInBackground set to true..."); try { InteropApplication application = new InteropApplication(true); application.Start(); Console.WriteLine("Application started correctly"); } catch(Exception e) { Assert.Fail(string.Format("InteropApplication failed to start: {0}", e.ToString())); } //test code continues ... 

All my tests are almost the same. They display information on why they failed, or they display information on what they are doing. I did not have any formal methods of how unit tests should be encoded. Should they display information about what they are doing? Or should the tests be quiet and not show any information at all about what they are doing and display error messages?

NOTE. Language is C #, but I don't care about a specific answer to the language.

Answer 1

I'm not sure why you will do this - if your unit test is named well, you already know what it does. If it fails, you know which test failed (and which statement failed). If this does not help, you know that it succeeded.

It seems completely subjective, but to me it seems completely redundant information that just adds noise.

Answer 2

I personally would recommend giving only errors and a summary of the number of test runs and how many have passed. However, this is a completely subjective view. Show what suits your needs.

Answer 3

I recommend against this - I think unit testing should work on the philosophy of Unix tools - don't say anything when everything goes well. I find that building tests to give meaningful information when they fail is the best - this way you get a good short result when everything works, and it's easy to see what went wrong when problems arise - errors are not lost so that scroll blindness.

Answer 4

I would actually suggest against him (although not belligerently). It connects the user interface of your tests with the test implementation (what if the tests are run through the GUI viewer?). As an alternative, I would suggest one of the following:

• I am not familiar with NUnit, but PyUnit allows you to add a test description, and when tests are run with a detailed version, this description is printed. I would look at the NUnit documentation to make sure that this is what you can do.
  
• Extend the TestCase class that you inherit to add the function from which you are calling the log, which is trying to run the test. Thus, different implementations can handle messages differently.

Answer 5

I would say that you should output everything that suits you, but too much can lead you out of the test runner. BTW, your sample code hardly looks like a unit test , most likely an integration / system test.

Answer 6

I like to buffer a detailed log (about 20 lines or so), but I do not show it until I get to some error. When an error occurs, it is nice to have some context.

OTOH unit tests should be small pieces of unrelated code with specific input and output requirements. In most cases, displaying the input that caused the error (i.e., Incorrect output) is enough to trace the problem back to its roots.

Answer 7

This may be too specific for the language, but when I write NUnit tests, I try to do this, only I use the System.Diagnostics.Trace library instead of the console, so the information is displayed only if I decide to monitor the trace.

Answer 8

You do not need to, if the tests work silently, then there was no error. As a rule, there is no reason for the tests to give any result, in addition, if the test fails. If it works, then it is launched by the specified test runner that passed the test, i.e. He is green". Running a test (along with many tests with console output) through a test runner in the IDE, you will send the console log to messages that no one cares about.

The testing you wrote is not a unit test, but is more like an integration / system test, since you seem to be using the application as a whole. A unit test will test the public method in the class, preferably keeping the class as isolated as possible.

Answer 9

Using kinda console I / O ignores the whole purpose of a unit testing system. You can also copy the entire test manually. If you use a unit testing system, your tests should be very malleable, tied to minimal capabilities.

Answer 10

Displaying information may be useful; if you are trying to find out why the test failed, it may be useful to see more than just a stack trace, and what happened before the program reached the point where it failed.

However, in the “normal” case when everything succeeds, these messages are an unnecessary mess that distracts from what you are really trying to do, i.e. looking at an overview of which tests were successful and failed.

I would suggest redirecting your debug messages to a log file. You can do this by writing all your log message code to call the special "Print Log" function, or if you are writing a console program, you should be able to redirect stdout to another file (I know you can do it like on Unix, so on Windows). This way you get a high level overview, but the details are there if you need them.

Answer 11

I would not put extra Try / Catch sentences in Unit Tests. First of all, the expected exception in the unit test will already throw a Failure error. This is the default behavior of NUnit. In essence, a test harness wraps every call to your test functions with this code. Also, just using e.ToString () to show what happened, I believe that you are losing a lot of information. By default, I believe that NUnit will display not only the type of exception, but also the call stack, which I don’t think you see with your method.

Secondly, there are times when it is necessary. For example, you can use the [ExpectedException] attribute to actually tell when this will happen. Just make sure that when testing Asserts related to exceptions (e.g. Asserting count count> 0, etc.) you put a good description as an argument to assert. This is useful.

Everything else is not required at all. If the tests of your device are so large that you start to write WriteLines with what “step” of the test you are in, this is usually a sign that your test should really be broken down into several smaller tests. In other words, you are not performing a unit test, but rather an integration test.

Answer 12

Have you seen the xUnit style of unit test frameworks?
See the Ron Jeffries website for a fairly large list.

One of the principles of this framework is that during a test run they produce little or no output, and only really an indicator of success at the end. In case of failures, a more detailed conclusion of the cause of the failure can be obtained.
The reason for this mode is that, while everything is in order, you do not want to be disturbed by the additional output, and, of course, if there is a failure, you do not want to miss it due to the noise of another output.

Answer 13

Well, you only need to know when the test failed and why it failed. You do not need to know what happens if, for example, you do not have a cycle, and you want to know exactly where the test died in the cycle.

Answer 14

I think you do a lot more work for yourself. Testing either fails, but there should be an exception to the rule, and you must enable the runner unit test handle and throw an exception. What you do adds cruft, the exception logged by the test runner will tell you the same thing.

Answer 15

The only time I show what happens is if there is some aspect of it that will be easier to test manually. For example, if you have code that takes a little time to run and may get stuck in an infinite loop, you can print the message so often to indicate that it is still progressing.

Always ensure that fault messages are clearly allocated from another output.

Answer 16

You could write a test method as follows. It's up to your code nose which test style you prefer. I prefer not to write additional try-catch and Console.WriteLines.

 public void TestApplicationStart() { InteropApplication application = new InteropApplication(true); application.Start(); } 

The test framework I worked with interprets any unhandled (and unexpected) exception as a failed test.

Think about the time you took on the gold plate of this test and how many more meaningful tests you could write from now on.