NUnit - repeat the test case 3 times if it does not work

Question

I have few test cases for automating the website user interface.

I want to try my test case at least three times if it fails the first and second time. Thus, I want to make sure that this test case does not work sequentially.

Please let me know if we have the ability to use on NUnit. I am using C # with NUnit.

Answer 1

You can add a new attribute to nunit based on attribute repetition and rebuild the library. It is very simple.

[Test] [Repeat( 25 )] public void MyTest( ){ // your test logic here } 

Answer 2

Starting with NUnit 3.0, there is a “Retry” attribute that looks like it will do what Kumar wants.

See Retry Attribute

Answer 3

Loop over logic, count errors. Statement. (Error == 0) Boom completed.

I would advise you not to do this. Write the best test you can prove is consistent.

Answer 4

There are 4 exceptions that nunit uses to report the test result. These are: TimeoutException, AssertException, SuccessException, and IgnoreException. You can use them from your nunit testing functions. I think you could put your test code in a try-catch and catch the nunit assert exception exactly twice. But I must say that, as noted earlier, the need for intermittent failure of the test is a reason for review.

Answer 5

If you want your test case to run several times before it reports an error, you can use the Retry attribute.

 [Test] [Retry(3)] public void MyTest( ){ // your test logic here } 

Answer 6

I created a wrapper script for nunit3-console in python that runs the script, and if there are any failed tests, it will re-run the failed tests until - - --max-retries . It will also combine the new results with the original TestResult.xml .

The script is called nunit-repeat and is located at https://github.com/sashoalm/nunit-repeat/ .

Usage example:

 python nunit-repeat.py --max-retries 3 -- .\MockFailingTests\packages\NUnit.ConsoleRunner.3.10.0\tools\nunit3-console.exe .\MockFailingTests\MockFailingTests\bin\Debug\MockFailingTests.dll 

The code:

 # Script to run nunit tests and then repeat any failed tests until we run out of retries. import os import sys import subprocess import xml.etree.ElementTree as ET # Finds the corresponding test with the same name in another XML file. def findCorrespondingTest(tree2, failedTest): test2 = tree2.findall(".//test-case[@fullname='{}']".format(failedTest.get('fullname'))) if len(test2) != 1: print(failedTest.get('fullname')) for ii in test2: print(ii.get('fullname')) raise Exception("len(test2) != 1") return test2[0] # Python XML lib doesn't show you who is the parent of a given element so we add the info ourselves. def addParentInfo(tree): for child in tree: setParent(child, tree) addParentInfo(child) # We strip the parentage info so it doesn't pollute the XML file. def stripParentInfo(tree): for child in tree: child.attrib.pop('__my_parent__', 'None') stripParentInfo(child) # Gets the parent of a given element. def getParent(tree): return tree.attrib['__my_parent__'] # Sets the parent of a given element. def setParent(tree, parent): tree.attrib['__my_parent__'] = parent # Updates a parent test suite of a failed test that has passed on rerun. def updateTestSuite(tree): tree.attrib['failed'] = str(int(tree.attrib['failed']) - 1) tree.attrib['passed'] = str(int(tree.attrib['passed']) + 1) if tree.attrib['failed'] == '0': tree.attrib['result'] = 'Passed' # In-place replaces a failed test with a successful one. def replaceTest(tree1, tree2): parent = getParent(tree1) tree1.__setstate__(tree2.__getstate__()) setParent(tree1, parent) # Updates the entire chain of parent test suites. def updateParentTestSuites(testCase): suite = getParent(testCase) while suite and suite.tag == 'test-suite': updateTestSuite(suite) suite = getParent(suite) # Merges the results from a rerun into the original test. # Any failed test that has become successful upon rerun is updated. def mergeRerunResults(tree1, tree2): for failedTest in tree1.iterfind(".//test-case[@result='Failed']"): test2 = findCorrespondingTest(tree2, failedTest) if test2.attrib['result'] == 'Passed': replaceTest(failedTest, test2) updateParentTestSuites(failedTest) # Checks whether we have any failed tests. def hasFailedTests(tree): return len(tree.findall(".//test-case[@result='Failed']")) > 0 # Writes the failed tests, one per line, in testlist.txt. This file # will be passed to nunit console runner. def writeFailedTests(tree): f = open('testlist.txt', 'w') for failedTest in tree.iterfind(".//test-case[@result='Failed']"): name = failedTest.attrib['fullname'] f.write(name + '\n') # Retries all the failing tests, until all pass or we run out of retries. def retryFailedTests(args, retries): # Add the testfilter to nunit command line. args.append('--testlist') args.append('testlist.txt') # Load the test results from the first invocation. tree = ET.parse('TestResult.xml') addParentInfo(tree.getroot()) # Run the retries. while retries > 0 and hasFailedTests(tree): retries -= 1 writeFailedTests(tree) subprocess.call(args) mergeRerunResults(tree, ET.parse('TestResult.xml')) # Write the final results. stripParentInfo(tree.getroot()) tree.write('TestResult.xml') # Check if we still have failing tests. if hasFailedTests(tree): raise Exception("There are failed tests even after retrying.") # Main function. def main(): args = sys.argv # Get the number of retries. try: retries = int(args[args.index('--max-retries') + 1]) except: retries = 3 # Get the nunit command line. args = args[args.index('--')+1:] # Invoke nunit the first time. subprocess.call(args) # Retry any failed tests. retryFailedTests(args, retries) # Execute main function. main()