Python unittest: run multiple statements in a loop without crashing at first, but continue

Question

Scenario: one of my test cases runs a shell program with several input files and a specific output. I would like to test different options for these I / O, and each of these options is saved in its own folder, i.e. the folder structure

/testA
/testA/inputX
/testA/inputY
/testA/expected.out
/testB
/testB/inputX
/testB/inputY
/testB/expected.out
/testC/
...

Here is my code to run this test:

def test_folders(self):
    tfolders = glob.iglob(os.environ['testdir'] + '/test_*')                
    for testpath in tfolders:
        testname = os.path.basename(testpath)
        self.assertTrue(self.runTest(testname))

testname in this case is "testX". It runs an external program with inputsin this folder and compares it with expected.outin the same folder.

Problem: As soon as it hits a test that fails, the test stops and receives:

Could not find or read expected output file: C:/PROJECTS/active/CMDR-Test/data/test_freeTransactional/expected.out
======================================================================
FAIL: test_folders (cmdr-test.TestValidTypes)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\PROJECTS\active\CMDR-Test\cmdr-test.py", line 52, in test_folders
    self.assertTrue(self.runTest(testname))
AssertionError: False is not true

: , ? ( unittest )

Answer 1

- ?

def test_folders(self):
    tfolders = glob.iglob(os.environ['testdir'] + '/test_*')    

    failures = []    
    for testpath in tfolders:
        testname = os.path.basename(testpath)
        if not self.runTest(testname):
            failures.append[testname]

    self.assertEqual([], failures)