Why can't rdmd run all unit tests?

Question

Unfortunately, I cannot reproduce this behavior in a minimal working example, so this may be too vague. However, I can at least indicate what does not cause behavior.

I have a module D containing several classes with unit tests, with a structure similar to:

import std.stdio; class c1{ int c1func(int blah){ return blah; } unittest{ writeln("Testing c1 func."); stdout.flush(); } } class c2(T:c3) : c1{ private static int c2func(int blah){ return blah; } unittest{ writeln("Testing c2 func."); stdout.flush(); } int c2fun2(int blah){ T tmp = new T(); return tmp.c3fun(blah); } } class c3{ abstract int c3fun(int blah); } class c4 : c3{ override int c3fun(int blah){ return blah; } } unittest{ writeln("Testing class c1."); stdout.flush(); c1 myc2 = new c2!(c4)(); } 

My expectation is that the form call:

 rdmd --main -unittest tmp.d 

will produce output

 Testing c1 func. Testing class c1. Testing c2 func. 

and it (c2 func unit tests does not start until the instance is created).

However, in my similar, but much longer module D, unit tests are not performed to match the c2 class of patterns (I put the print statements identical to those indicated here on all my unit tests).

This is a little worrisome, as I have actually relied for some time on the correctness of these other unit tests. In addition, rdmd is happy to accept syntax errors in unit test code, even though this code does not run.

So here is what I can rule out:

• Unit tests are read by rdmd.
• rdmd actually runs against the latest version of the file.
• rdmd seems to be happy to run unit tests to implement template classes, even with abstract class specializations for its arguments, if these template classes are actually created.
• The functions in my template class (akin to c2 in the sample code) are actually called during the main unit test block below, so the template class was definitely created.
• rdmd itself is not to blame, since compiling using dmd and the same flags also prevents these unit tests.

Any ideas on what might be wrong here? I would provide MWE if I could, but every version of what I wrote works fine!

Answer 1

This seems to be due to rdmd ordering unit tests, which is not what I expected.

It seems that RDMD does not call unit tests for an instance of the template class, not at the time the instance was created, but instead right after the unit test completes, in which the class instance is completed.

Here's the MWE:

 import std.stdio; class c1{ int c1func(int blah){ return blah; } unittest{ writeln("Testing c1 func."); stdout.flush(); } } class c2(T:c3) : c1{ private static int c2func(int blah){ return blah; } unittest{ writeln("Testing c2 func."); stdout.flush(); } int c2fun2(int blah){ T tmp = new T(); return tmp.c3fun(blah); } } class c3{ abstract int c3fun(int blah); } class c4 : c3{ override int c3fun(int blah){ return blah; } } unittest{ writeln("Testing class c1."); stdout.flush(); c1 myc2 = new c2!(c4)(); assert(1==0); } 

In this case, the unit test will fail at the bottom (due to assert (1 == 0)), and so the unit class template tests will never be executed.

This poses a serious problem when using template classes along with unit tests. If your template class fails, somehow fails, which leads to the unit test where it was created to fail, then its own unit tests are never executed. In the event of a silent failure (in my code with errors in the template class, a silent exit was called), this will manifest itself, since the tests of the template class are never executed.

Summary : DMD usually just runs unit tests in the order in which they appear. However, since template classes do not exist without some completion of the template, unit tests within the template class do not run in the order in which they are displayed. Instead, they start (in order) immediately after the first unit test, where an instance of the template class is created.

Workaround : Adding a separate unit test directly below the template class that simply creates the instance, with all the types you want to test, will run the unit tests in the correct order. However, a seemingly more effective policy would be to run those unit tests in front of the test block in which the instance is created!