How can I get MSTest.exe to return 0, even if the test failed?

Question

We have an assembly that runs on TeamCity and contains several different test projects. Each of them starts one after another, and we use MSTest.exe from Rakefile to organize everything this way:

desc 'Run Unit Tests' mstest :data_test => [:build_database_tests] do |mstest| puts build_header("Data Tests") mstest.command = msTestCommand mstest.parameters = [ "/resultsfile:dTest.trx", "/detail:errormessage", "/detail:description", "/usestderr" ] mstest.assemblies "../../../Database/DataTests/bin/Release/DataTests.dll" end 

This creates a good TRX file that TeamCity can analyze and everything that it does for a clean tab of a clean report. However, MSTest.exe returns 1 if the test fails, which the rake interprets as a failure, and the rest of the tests do not start.

We would like to suppress this behavior; we want the build to continue to work when the test fails, so that we can see that other tests can also fail. How can we make Rake ignore the return code and / or cause MSTest.exe not to return an error code if the test fails?

Answer 1

MSTest itself will run ALL test suites, even if the previous failure. Your problem is with the interpretation that rake makes. Most, if not all, assembly languages ​​(ANT, MSBuild, Gradle, etc.) have a "Continue Error" behavior.

Performing a quick google search for "rake continue on error" I found a couple of links:

http://www.rake.build/fascicles/004-ignore-failed-tasks.html Continue rake after failure

The main point I got is that you can use sh to repeat a series of tasks and ignore crashes so ALL tasks are performed.