Code Modulation Testing

Question

At my workplace, we have a C # .net solution containing about 50 projects and about 2000 unit tests. After changing the code, it is required that we run all the tests in the solution before pushing our changes to the build server. Running all tests may take about 10-15 minutes. I thought: hey, maybe there might be some process that will analyze all the code changes that I made, and then decide to run only the appropriate testing methods. If such an analysis is possible, then instead of running 2000 tests, the computer will run only 15 or 100 tests, then we are talking about a very good increase in productivity and fewer headaches. Anyone have any ideas on how to implement this, or if such software is available?

Answer 1

Our C # Testing Tool for testing can do this.

• It gradually changes only files that have changed since the last coverage run.

• It can be used to track test coverage for each unit test, and which files are involved. When a file changes, it compares it with the previous version; if a method participates in some unit tests that have previously changed, it identifies those unit tests. You should organize this process to match test runs with coverage data and re-run the identified test units, but this is a small matter of scripting. The reason for this is to allow the coverage testing tool to work with arbitrary testing structures.

You can do this on the build server to save time. More importantly, you can give each developer this so that he can only run the tests that are needed to verify the code that he changes before he checks the changes.

Answer 2

It is very difficult to say what depends on what is in a multi-project environment. Perfectly:

• Your unit tests will be true unit tests, and 2000 of them should run in 20 seconds.
• Your longer tests will be configured to run continuously on your build server, so you will still get a notification very quickly that you messed up things if you missed something, but you could personally afford to be a little more selectively run only those tests which, in your opinion, influenced your code changes before committing.

Answer 3

Visual Studio 2010 Premium and Ultimate have the Test Impact tool, which does just that.

Answer 4

In our environment, we use NCrunch http://www.ncrunch.net/

He builds projects and runs unit tests in the background using multiple threads. It nicely informs you of failed unit tests.

NCrunch does pretty much what you need. Take a look.