Visual Studio 2012 MSTest vs. NUnit Pros and Cons

Question

We must decide which technology to use for our unit testing. We are currently using Visual Studio 2010 and are not happy with the MSTest that came with this. It works poorly, works poorly (for example, the output directory of the test output is not recognized correctly) and has several problems when trying to test assemblies in 32-bit and 64-bit versions. Worse, MSTest does not have a good impedance match for our Jenkins build system. Therefore, we thought about switching to NUnit. However, no one on our team has enough impact on NUnit. In addition, we will soon move on to Visual Studio 2012.

I need to know the pros and cons of the latest version of Visual Studio 2012 MSTest vs Nunit . Since most of the stack overflow articles are related to older versions of VS, they are not related to us. I think that since 2010 Microsoft has significantly improved MSTest. Please provide an objective comparison with the detailed technical problems you encountered in both technologies ( only new versions )

Answer 1

I am currently using both MSTest and NUnit. IMHO NUnit is still the best solution. If you have Visual Studio 2012 Premium edition, then this is really good, except for the fact that you cannot combine tests. I like that it is integrated into Visual Studio, but the lack of grouping and the ability to run a subset of tests without manual selection is a huge problem.

Coverage analysis is also pretty neat in Premium. It is fast and gives you what you need fast. This is a premium feature.

Since there are still no functions in MSTest (even remote functions since vs2010), I would still recommend using NUnit for unit tests. Benefits include test grouping by namespace, the ability to add annotations to test scripts (running the same test several times with different parameters), and it works well with Opencover and a report generator for analyzing coverage. The main argument cited is that it is not integrated as MSTest, so it really depends on how important it is to you, whether it is con.

Answer 2

@Biranchi: now it doesn't matter which unit test framework you use in Visual Studio 2012 (and up). See my blog post here, continued to the one you are referring to. http://blogs.msdn.com/b/visualstudioalm/archive/2012/11/20/part-2-using-traits-with-different-test-frameworks-in-the-unit-test-explorer.aspx

You can even mix and match tests from different frameworks, you can even do it up to the method level!
This means that you can even move legacy code from one to another without any adverse side effects.

Also see this how to use Nuget to install the NUnit adapter in a solution, freeing the developer to install it. http://blogs.msdn.com/b/visualstudioalm/archive/2013/06/11/part-3-unit-testing-with-traits-and-code-coverage-in-visual-studio-2012-using- the-tfs-build-and-the-new-nuget-adapter-approach.aspx

@Sriwantha: MSTest is a simpler structure than NUnit. NUnit (as well as XUnit) give you more flexibility, which also reduces the amount of code to write. One example: if you use categories (and you should), MSTest requires that the category beautify each method. NUnit allows you to decorate a class that takes effect for all methods of this class. NUnit also allows the use of strongly typed categories

public class Integration : Category {} 

This is enough to declare a category that you can use instead

 Category("Integration"); 

where you risk spelling mistakes.

NUnit has much better support for data-driven tests. NUnit also supports theories

to name a few.

Answer 3

VS2012 allows you to classify tests into groups if you have Update 1 or later: http://msdn.microsoft.com/en-us/library/hh270865.aspx#BKMK_Grouping_and_filtering_the_test_list

Answer 4

Have you looked at VS Feature features? http://blogs.msdn.com/b/visualstudioalm/archive/2012/11/09/how-to-manage-unit-tests-in-visual-studio-2012-update-1-part-1-using- traits-in-the-unit-test-explorer.aspx

Grouping is much better in mstest (2012 update 1) compared to nunit.

Answer 5

One more thing to add here. It seems that the MSTEST engine does not work with TFS Build in certain scenarios. If you use the TFS assembly, it will not report missed tests (marked with the Ignore attribute). It will only include Assert.Inconclusive. If this is a problem, you should use NUnit instead of MSTest.