Catch.hpp unit testing: How to dynamically create test cases?

I am using CATCH v1.1 build 14 to unit test my C ++ code.

As part of the testing, I would like to check the outputs of several modules in my code. There is no fixed number of modules; more modules can be added at any time. However, the code for testing each module is identical. Therefore, I think it would be ideal to put the test code in a for loop. In fact, using catch.hpp , I confirmed that I can dynamically create sections in a test case, where each section corresponds to a module. I can do this by including the SECTION macro in a for loop, for example:

 #include "catch.hpp" #include <vector> #include <string> #include "myHeader.h" TEST_CASE("Module testing", "[module]") { myNamespace::myManagerClass manager; std::vector<std::string> modList; size_t n; modList = manager.getModules(); for (n = 0; n < modList.size(); n++) { SECTION(modList[n].c_str()) { REQUIRE(/*insert testing code here*/); } } } 

(This is not a complete working example, but you get the idea.)

Here is my dilemma. I would like to test the modules myself, so if one module fails, it will continue testing the other modules, rather than interrupting the test. However, how CATCH works, it cancels the entire test case if one REQUIRE fails. For this reason, I would like to create a separate test case for each module, and not just a separate section. I tried to put the for loop outside the TEST_CASE macro, but this code does not compile (as I expected):

 #include "catch.hpp" #include <vector> #include <string> #include "myHeader.h" myNamespace::myManagerClass manager; std::vector<std::string> modList; size_t n; modList = manager.getModules(); for (n = 0; n < modList.size(); n++) { TEST_CASE("Module testing", "[module]") { SECTION(modList[n].c_str()) { REQUIRE(/*insert testing code here*/); } } } 

Perhaps this can be done by writing my own main() , but I don’t see how to do it exactly. (Would I put my TEST_CASE code directly in main() ? What if I want to save the TEST_CASE code in another file? Will it also affect my other, more standard test cases?)

I can also use CHECK macros instead of REQUIRE macros to avoid interrupting the test case when the module crashes, but then I get the opposite problem: it tries to continue the test on the module, which was supposed to fail early. If I could just put each module in my test case, this should give me perfect behavior.

Is there an easy way to dynamically create test cases in CATCH? If so, can you give me an example of how to do this? I read the CATCH documentation and searched the Internet, but I could not find any directions on how to do this.

+5
source share
2 answers

It seems that Catch could switch to property-based testing, which I hope will allow you to dynamically create test cases. In the meantime, here is what I did.

I created a .cpp file with one TEST_CASE for one module and a global variable for the module name. (Yes, I know that global variables are evil, so I try and use it as a last resort):

module_unit_test.cpp :

 #include "catch.hpp" #include <string> #include "myHeader.h" extern const std::string g_ModuleName; // global variable: module name TEST_CASE("Module testing", "[module]") { myNamespace::myManagerClass manager; myNamespace::myModuleClass *pModule; SECTION(g_ModuleName.c_str()) { pModule = manager.createModule(g_ModuleName.c_str()); REQUIRE(pModule != 0); /*insert more testing code here*/ } } 

Then I create an executable file that will run this test on a single module specified on the command line. (I tried to loop through Catch::Session().run() below, but Catch does not allow it to run more than once.) The object file from the code below module_test.cpp and from the unit test code above module_unit_test.cpp are linked when creating the executable file.

module_test.cpp :

 #define CATCH_CONFIG_RUNNER #include "catch.hpp" #include <string> #include <cstdio> std::string g_ModuleName; // global variable: module name int main(int argc, char* argv[]) { // Make sure the user specified a module name. if (argc < 2) { std::cout << argv[0] << " <module name> <Catch options>" << std::endl; return 1; } size_t n; char* catch_argv[argc-1]; int result; // Modify the input arguments for the Catch Session. // (Remove the module name, which is only used by this program.) catch_argv[0] = argv[0]; for (n = 2; n < argc; n++) { catch_argv[n-1] = argv[n]; } // Set the value of the global variable. g_ModuleName = argv[1]; // Run the test with the modified command line arguments. result = Catch::Session().run(argc-1, catch_argv); return result; } 

Then I execute a loop in a separate executable (not tied to the object files from the above code):

module_test_all.cpp :

 #include <cstdlib> #include <vector> #include <string> #include "myHeader.h" int main(int argc, char* argv[]) { std::string commandStr; int result, status = 0; myNamespace::myManagerClass manager; std::vector<std::string> modList; size_t m, n; // Scan for modules. modList = manager.getModules(); // Loop through the module list. for (n = 0; n < modList.size(); n++) { // Build the command line. commandStr = "module_test " + modList[n]; for (m = 1; m < argc; m++) { commandStr += " "; commandStr += argv[m]; } // Do a system call to the first executable. result = system(commandStr.c_str()); // If a test fails, I keep track of the status but continue // looping so all the modules get tested. status = status ? status : result; } return status; } 

Yes, this is ugly, but I confirmed that it works.

0
source

There is a way to achieve what you are looking for, but I would notice that you are doing it wrong: -

Unit tests are designed to test each device, i.e. You are writing a component and a test to verify the correct behavior of this component. If you later decide to modify one component in some way, you will update the corresponding test.

If you combine all your tests for all your components into the same file, it becomes much more difficult to isolate a device that behaves differently.

If you want to exclude component testing because it is essentially the same for all your components, you can do one of the following:

1. Extract the general tests into a separate header file

You can # determine the name of the type of component you want to test, and then include the header file with all the tests in it:

 // CommonTests.th #include "catch.hpp" TEST_CASE("Object Can be instantiated", "[ctor]") { REQUIRE_NOTHROW(COMPONENT component); } // SimpleComponent.t.cpp #define COMPONENT SimpleComponent #include "CommonTests.th" 

This is easy to do, but it has a drawback - when you run a test runner, you will have repeated tests (by name), so you can run all tests or by tags.

You can solve this by strict component name and pre-adding it to the name of the test case.

** 2. Call general tests, parameterizing the component **

Put your general tests in a separate file and call the general test methods directly:

 // CommonTests.th void RunCommonTests(ComponentInterface& itf); // CommonTests.t.cpp void RunCommonTests(ComponentInterface& itf) { REQUIRE(itf.answerToLifeUniverseAndEverything() == 42); } // SimpleComponent.t.cpp #include "SimpleComponent.h" #include "CommonTest.th" #include "catch.hpp" TEST_CASE("SimpleComponent is default-constructible", "[ctor]") { REQUIRE_NOTHROW(SimpleComponent sc); } TEST_CASE("SimpleComponent behaves like common components", "[common]") { SimpleComponent sc; RunCommonTests(sc); } 
+1
source

Source: https://habr.com/ru/post/1244614/


All Articles