Matlab UnitTest TestCase with additional graphs of results?

Question

When testing project objects and functions in Matlab using the matlab.unittest.TestCase (new for 2013) classes, a graph is sometimes required to visualize the actual / expected data.

I have used the following method so far, but I think this is not the best way:

classdef test1 < matlab.unittest.TestCase properties var1=3; var2=5; % sample variables graph_output = 0; % boolean end methods(Test) function testfunction(testCase) my_result = my_fun(testCase.var1, testCase.var2); testCase.verifyEqual(my_result,expected_result) if testCase.graph_output plot(my_result) end end end 

On the command line, I use test_obj=test1 and then test_obj.graph_output=1 before using run(test_obj) to plot graphs as well as to test the function.

The best way to do this is to use a separate method. I tried this by highlighting my_result in the property list, but after the test completes, Matlab seems to reinitialize my_result , making the output graph meaningless.

Does anyone know a way around this, or a better way to output test results on demand?

Answer 1

Usually, when people are interested in finding the results of a particular test, because something went wrong. This is a good opportunity to use your own diagnostics. Here is one of them that displays a link to the MATLAB command window, which displays the expected value in relation to the actual value, and also prints links that will load data from the test into the workspace.

 classdef test1 < matlab.unittest.TestCase methods(Test) function firstTest(testCase) import matlab.unittest.constraints.IsEqualTo; % Test should pass! actualValue = 1:10; expectedValue = 1:10; diagnostic = @()myPlotDiagnostic(actualValue, expectedValue); testCase.verifyThat(actualValue, IsEqualTo(expectedValue), diagnostic); end function secondTest(testCase) import matlab.unittest.constraints.IsEqualTo; % Test should fail with a diagnostic! actualValue = [1 2 3 4 12 6 7 8 9 10]; expectedValue = 1:10; diagnostic = @()myPlotDiagnostic(actualValue, expectedValue); testCase.verifyThat(actualValue, IsEqualTo(expectedValue), diagnostic); end function thirdTest(testCase) import matlab.unittest.constraints.IsEqualTo; % Test should also fail with a diagnostic! actualValue = [1 2 3 4 -12 6 7 8 9 10]; expectedValue = 1:10; diagnostic = @()myPlotDiagnostic(actualValue, expectedValue); testCase.verifyThat(actualValue, IsEqualTo(expectedValue), diagnostic); end end end function myPlotDiagnostic(actualValue, expectedValue) temporaryFile = tempname; save(temporaryFile, 'actualValue', 'expectedValue'); fprintf('<a href="matlab:plot([%s], [%s], ''*r'')">Plot Data</a>\n', num2str(expectedValue), num2str(actualValue)); fprintf('<a href="matlab:load(''%s'')">Load data into workspace</a>\n', temporaryFile); end 

Running this test will lead to conclusions that contain

• Links that will load actual and expected values ​​into the workspace
• Links that when clicked will display the following graphs:

Of course, they will be displayed, although if the test fails, but this is usually the desired behavior!

NB I prefer to use the IsEqualTo syntax IsEqualTo that the tests read (almost) like English. But this is a style decision.

Answer 2

Update: Now it is much more optimized in R2017a with the inclusion of FigureDiagnostic and ScreenshotDiagnostic . Check them out before going too far along this path!

Original answer

You can do this not only for failure conditions, but also for passing conditions with a combination of user diagnostics and DiagnosticValidationPlugin diagnostics. You can do this quickly using a function descriptor, but if this is what you often want to do for many of your tests, consider creating your own Diagnostic subclass:

 classdef PlotDiagnostic < matlab.unittest.diagnostics.Diagnostic properties Title Actual Expected end methods function diag = PlotDiagnostic(title, actual, expected) diag.Title = title; diag.Actual = actual; diag.Expected = expected; end function diagnose(diag) diag.DiagnosticResult = sprintf('Generating plot with title "%s"', diag.Title); f = figure('Title', diag.Title); ax = axes('Parent', f); plot(ax, 1:numel(diag.Actual), diag.Actual, 'r', 1:numel(diag.Expected), diag.Expected','b'); end end end 

Then you can get a test that uses this like this:

 classdef FooTest < matlab.unittest.TestCase methods(Test) function testFails(testCase) actual = 1:10; expected = fliplr(actual); testCase.verifyEqual(actual, expected, PlotDiagnostic('Title1', actual, expected)); end function testPasses(testCase) actual = 1:10; expected = actual; testCase.verifyEqual(actual, expected, PlotDiagnostic('Title2', actual, expected)); end end end 

Now that you have these tests, you will see them in a crash condition. However, you can also see them in the conditions of passage using the DiagnosticsValidationPlugin, which evaluates the diagnostics even in the conditions of passage, to ensure that the diagnostic code is an error (it would be too weak not to intercept the diagnostic information from a real failure, since there was an error into a diagnostic code that is not commonly used). It will look like this:

 >> import matlab.unittest.*; >> runner = TestRunner.withNoPlugins; >> runner.addPlugin(matlab.unittest.plugins.DiagnosticsValidationPlugin); >> suite = TestSuite.fromClass(?FooTest); >> runner.run(suite) 

Please note that with R2014a you can write your own plugin to listen to this diagnostic diagnostics instead of using DiagnosticsValidationPlugin. Indeed, in this example, we do not use this plugin with the intent to verify that the diagnosis is an error, so it would be better to write our own plugin for this specific purpose.

In addition, in R2014b, you can use the method to connect it to another watch face. If you want, you can call:

 testCase.log(Verbosity.Detailed, PlotDiagnostic('Title2', actual, expected)); 

Then you see only graphs when you use LoggingPlugin at the "Detailed" level of detail, for example.