I have a PHPUnit test suite that is currently causing a fatal error due to a class definition that was not found. This, ultimately, is a failure of the testing code itself and a failure of the developer before confirming the code before executing the code.
However, such things happen from time to time, and it would be great if, in the event of a fatal error (regardless of who is ultimately responsible), the test is simply flagged as a failure, and the rest of the Suite test is still running.
I read about the --process-isolation switch, and as far as I can tell, it should take care of that. Since each test is performed in a separate process, if the child dies due to a fatal error, the parent can continue to work. In fact, this is explicitly stated in this answer to a similar question: https://stackoverflow.com/a/212960/2128 which shows the exact type of output that I would like to see myself.
However, I seem to get the same result regardless of whether I use the --process-isolation flag:
WITHOUT process isolation
[ kogi@phagocyte ~]$ /usr/bin/phpunit --colors --verbose --coverage-html "target/coverage" ~/app/zend/tests/application/ PHP Fatal error: Class 'Rmd_Database_OldObject' not found in /home/kogi/app/zend/private/models/translate/Poll.php on line 9 PHP Stack trace: PHP 1. {main}() /usr/bin/phpunit:0 PHP 2. PHPUnit_TextUI_Command::main() /usr/bin/phpunit:46 PHP 3. PHPUnit_TextUI_Command->run() /usr/share/pear/PHPUnit/TextUI/Command.php:130 PHP 4. PHPUnit_Runner_BaseTestRunner->getTest() /usr/share/pear/PHPUnit/TextUI/Command.php:150 PHP 5. PHPUnit_Framework_TestSuite->addTestFiles() /usr/share/pear/PHPUnit/Runner/BaseTestRunner.php:96 PHP 6. PHPUnit_Framework_TestSuite->addTestFile() /usr/share/pear/PHPUnit/Framework/TestSuite.php:419 PHP 7. PHPUnit_Util_Fileloader::checkAndLoad() /usr/share/pear/PHPUnit/Framework/TestSuite.php:358 PHP 8. PHPUnit_Util_Fileloader::load() /usr/share/pear/PHPUnit/Util/Fileloader.php:79 PHP 9. include_once() /usr/share/pear/PHPUnit/Util/Fileloader.php:95 PHP 10. require_once() /home/kogi/app/zend/tests/application/translate/PollTest.php:11 Fatal error: Class 'Rmd_Database_OldObject' not found in /home/kogi/app/zend/private/models/translate/Poll.php on line 9 Call Stack: 0.0003 91584 1. {main}() /usr/bin/phpunit:0 0.0076 612672 2. PHPUnit_TextUI_Command::main() /usr/bin/phpunit:46 0.0076 613744 3. PHPUnit_TextUI_Command->run() /usr/share/pear/PHPUnit/TextUI/Command.php:130 0.0246 1249464 4. PHPUnit_Runner_BaseTestRunner->getTest() /usr/share/pear/PHPUnit/TextUI/Command.php:150 0.0706 1626680 5. PHPUnit_Framework_TestSuite->addTestFiles() /usr/share/pear/PHPUnit/Runner/BaseTestRunner.php:96 0.1691 8053584 6. PHPUnit_Framework_TestSuite->addTestFile() /usr/share/pear/PHPUnit/Framework/TestSuite.php:419 0.1693 8057320 7. PHPUnit_Util_Fileloader::checkAndLoad() /usr/share/pear/PHPUnit/Framework/TestSuite.php:358 0.1694 8057664 8. PHPUnit_Util_Fileloader::load() /usr/share/pear/PHPUnit/Util/Fileloader.php:79 0.1711 8240600 9. include_once('/home/kogi/app/zend/tests/application/translate/PollTest.php') /usr/share/pear/PHPUnit/Util/Fileloader.php:95 0.1805 9187768 10. require_once('/home/kogi/app/zend/private/models/translate/Poll.php') /home/kogi/app/zend/tests/application/translate/PollTest.php:11
With process isolation
[ kogi@phagocyte ~]$ /usr/bin/phpunit --colors --verbose --coverage-html "target/coverage" --process-isolation ~/app/zend/tests/application/ PHP Fatal error: Class 'Rmd_Database_OldObject' not found in /home/kogi/app/zend/private/models/translate/Poll.php on line 9 PHP Stack trace: PHP 1. {main}() /usr/bin/phpunit:0 PHP 2. PHPUnit_TextUI_Command::main() /usr/bin/phpunit:46 PHP 3. PHPUnit_TextUI_Command->run() /usr/share/pear/PHPUnit/TextUI/Command.php:130 PHP 4. PHPUnit_Runner_BaseTestRunner->getTest() /usr/share/pear/PHPUnit/TextUI/Command.php:150 PHP 5. PHPUnit_Framework_TestSuite->addTestFiles() /usr/share/pear/PHPUnit/Runner/BaseTestRunner.php:96 PHP 6. PHPUnit_Framework_TestSuite->addTestFile() /usr/share/pear/PHPUnit/Framework/TestSuite.php:419 PHP 7. PHPUnit_Util_Fileloader::checkAndLoad() /usr/share/pear/PHPUnit/Framework/TestSuite.php:358 PHP 8. PHPUnit_Util_Fileloader::load() /usr/share/pear/PHPUnit/Util/Fileloader.php:79 PHP 9. include_once() /usr/share/pear/PHPUnit/Util/Fileloader.php:95 PHP 10. require_once() /home/kogi/app/zend/tests/application/translate/PollTest.php:11 Fatal error: Class 'Rmd_Database_OldObject' not found in /home/kogi/app/zend/private/models/translate/Poll.php on line 9 Call Stack: 0.0003 91752 1. {main}() /usr/bin/phpunit:0 0.0076 612824 2. PHPUnit_TextUI_Command::main() /usr/bin/phpunit:46 0.0076 613896 3. PHPUnit_TextUI_Command->run() /usr/share/pear/PHPUnit/TextUI/Command.php:130 0.0246 1250360 4. PHPUnit_Runner_BaseTestRunner->getTest() /usr/share/pear/PHPUnit/TextUI/Command.php:150 0.0708 1627528 5. PHPUnit_Framework_TestSuite->addTestFiles() /usr/share/pear/PHPUnit/Runner/BaseTestRunner.php:96 0.1688 8054296 6. PHPUnit_Framework_TestSuite->addTestFile() /usr/share/pear/PHPUnit/Framework/TestSuite.php:419 0.1690 8057992 7. PHPUnit_Util_Fileloader::checkAndLoad() /usr/share/pear/PHPUnit/Framework/TestSuite.php:358 0.1691 8058336 8. PHPUnit_Util_Fileloader::load() /usr/share/pear/PHPUnit/Util/Fileloader.php:79 0.1707 8241296 9. include_once('/home/kogi/app/zend/tests/application/translate/PollTest.php') /usr/share/pear/PHPUnit/Util/Fileloader.php:95 0.1801 9188464 10. require_once('/home/kogi/app/zend/private/models/translate/Poll.php') /home/kogi/app/zend/tests/application/translate/PollTest.php:11
For those of us who cannot efficiently distinguish between our heads, the two outputs are literally identical (except for runtime and memory usage, which is negligible).
In both cases, a fatal error destroys the entire test suite. In this particular case, this happens in the third test, and the remaining 150 tests (in several other files / sets) are never executed.
What am I doing wrong here? Is there another way to survive the fatal error (mark the test as unsuccessful) in one test and run the remaining tests?
EDIT
I am using PHPUnit 3.6.10
EDIT
Comments on the answer to this question inspired a new ticket to the PHPUnit GitHub page: https://github.com/sebastianbergmann/phpunit/issues/545