I have a Visual Studio 2012 solution using the .NET 4.5 platform. The rest of our development system is TFS2010. Recently, some of our unit tests fail only on the build server. After going through several rounds of debugging, I was able to determine that the cause of the failed tests was most likely due to the fact that the MSTest version is used on the build server. Tested tests pass when starting from VS2012 on my local computer and when starting from VS2012 on the actual build server.
I followed the steps described in the following articles (I did not find any useful articles directly related to VS2012 and TFS2010) without success: Launching the VS2010 UnitTests project from Team Team and VS2010, TFS 2008 and Unit Tests
On the build server, I now have the following:
- Installed VS2010 Ultimate SP1
- Installed version of VS2012
- Installed SDK.NET 4.5 package
- The Microsoft.TeamFoundation.Build.targets file has been changed to "C: \ Program Files (86) \ MSBuild \ Microsoft \ VisualStudio \ TeamBuild \" and "C: \ Program Files \ MSBuild \ Microsoft \ VisualStudio \ TeamBuild \", therefore the AssemblyFile attribute build tasks TestToolsTask points to "C: \ Program Files (x86) \ Microsoft Visual Studio 11.0 \ Common7 \ IDE \ PrivateAssemblies \ Microsoft.TeamFoundation.Build.ProcessComponents.dll"
I have 2 build definitions that I tested:
- In the first build definition, I left the ToolPath “MS Test Launch” action empty, so it uses "C: \ Program Files (x86) \ Microsoft Visual Studio 10.0 \ Common7 \ IDE"
- In another build definition, I changed the ToolPath "Run MS Test" parameter to "C: \ Program Files (x86) \ Microsoft Visual Studio 11.0 \ Common7 \ IDE"
Both build definitions look correctly built and use the following MSBuild.exe file: C: \ Windows \ Microsoft.NET \ Framework \ v4.0.30319 \ MSBuild.exe
The first build error launches the MStest VS2010 version with any errors, but 5 of the approx. 2030 tests fail. These tests do not crash when starting from Visual Studaio 2012. Command line example (actual paths and server names removed):
C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe /nologo /usestderr /searchpathroot:"d:\Builds\xxxxx\Binaries" /resultsfileroot:"d:\Builds\xxxxx\TestResults" /testcontainer:"d:\Builds\xxxxx.dll" /testcontainer:"d:\Builds\xxxxx.dll" /publish:"http://xxxxx /publishbuild:"vstfs:///Build/Build/xxxxx" /teamproject:"xxxxx" /platform:"x86" /flavor:"Debug"
During the second build definition, MSTest does not start with the output below. If I run this command from the command line on the build server without the switches causing the Invalid switch error, then MSTest starts and the corresponding tests are passed:
The MSTestActivity was invoked without a value for Platform or Flavor. The values x86 and Debug were used. C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\MSTest.exe /nologo /usestderr /searchpathroot:"d:\Builds\xxxxx\Binaries" /resultsfileroot:"d:\Builds\xxxxx\TestResults" /testcontainer:"d:\Builds\xxxxx.dll" /publish:"http://xxxxx" /publishbuild:"vstfs:///Build/Build/xxxxx" /teamproject:"xxxxx" /platform:"x86" /flavor:"Debug" Invalid switch "/publish". Invalid switch "/publishbuild". Invalid switch "/teamproject". Invalid switch "/platform". Invalid switch "/flavor". For switch syntax, type "MSTest /help"
I did many runs of these collections, but I could not solve the problem. Is there anything else that I am missing that needs to be done either on the build server or in the build definition?
Due to the actual tests that do not match, they have in common that they compare the objects in the lists, but, as mentioned, they pass when starting from VS2012.