Assert.Inconclusive () call in async unit test reported as crash

Question

I have a series of unit tests that connect to an Azure Storage emulator. In the installer, my code checks to see if there is something listening on the emulator port, and if the StorageNotAvailable flag is not set.

In each of my tests, I have code ...

 if ( StorageNotAvailable ) Assert.Inconclusive( "Storage emulator is not available" ) // where storage emulator is available, continue as normal 

As expected, when the test returns void , it correctly reports in the Tester Explorer as "Non-Convertible".

When some asynchronous methods are used in the test, and the signature [TestMethod] returns Task , then the test is reported in TestExplorer as "Failed" instead of "Inconclusive".

How can I get the async method for a report as Inconclusive?

EDIT

Some additional details may be in order. Here are some sample tests that I put together to demonstrate the problem I am seeing.

 [TestMethod] public void MyTestMethod() { Assert.Inconclusive( "I am inconclusive" ); } [TestMethod] public async Task MyTestMethodAsync() { Assert.Inconclusive( "I am an error" ); } 

Some environment details may also be in order:

• Windows 10 x64 1703 Build 15063.608
• Visual Studio Enterprise 2017 15.3.5
• .NET 4.7.02046
• VS Extensions That May Be Relevant
  • Microsoft Visual Studio Platform
  • MSTest V2 Create Unit Test Extension
  • IntelliTest MSTest V2 Extension
  • MSTest V2 Templates
• Links to projects that may be relevant.
  • Microsoft.VisualStudio.TestPlatform.TestFramework v14.0.0.0
  • Microsoft.VisualStudio.TestPlatform.TestFramework.Extensions v14.0.0.0
• NuGet projects that may be relevant
  • MSTest.TestAdapter v1.1.18
  • MSTest.TestFramework v1.1.18
• The goal of the project is the .NET Framework v4.7.

Answer 1

Assert.Inconclusive raises a special kind of exception that will cause the task to catch this exception. Since the Task library and async are not aware of them, we cannot blame them for complaints. The Task frame will complete the exception in the AggregateException, which I suspect is being reported. This was a good guess, but it turned out that the code looking for AssetInconclusiveException compared the excited instance with the implementation of MstestV1, not MsTestV2.

But I believe that this should be considered as an error in the MsTest v2 runner, which should check all failed tasks and look at the exception that caused them to fail.

Behavior is a known behavior at the moment, and I just published a PR to fix it . Pull Request Merged, now just wait while the next Nuget build is running.

---

This hotfix has been merged and released in the latest 1.2.0 package .

Answer 2

This is caused by a confirmed error in the MsTest.Framework code - issue No. 249 on GitHub tracks the issue and a possible solution:

Asynchronous tests that use the Assert.Inconclusive report as Error

https://github.com/Microsoft/testfx/issues/249