Driver Behavior Performing Parallel Tests of Selenium TestNG with dataprovider

Question

I want to run selenium tests in parallel at TestNg, which use @dataprovider. Ideally, tests are parallel by method (one test = one method), rather than a simple set of parallelism by the browser. I read somewhere that about 5 instances of ChromeDriver can be controlled at the same time, so I thought it should be possible. Later I plan to go to grid2. For development, I run things with the IntelliJ Idea test runner by right-clicking + running the XML configuration file.

I had problems running my tests in parallel (on grid2 and locally), so I created a sample of more or less what I want to do.

Here is my test class

package tests; import org.openqa.selenium.By; import org.openqa.selenium.Keys; import org.openqa.selenium.WebDriver; import org.openqa.selenium.WebElement; import org.openqa.selenium.chrome.ChromeDriver; import org.openqa.selenium.firefox.FirefoxDriver; import org.openqa.selenium.interactions.Actions; import org.testng.annotations.*; import java.util.concurrent.TimeUnit; import static org.testng.Assert.assertNotNull; public class ParallelTest { public static final String SEARCH_TERMS = "search-terms"; private WebDriver driver; @BeforeMethod @Parameters({"browser"}) public void beforeMethod(@Optional("chrome") String browser){ driver = getBrowser(browser); driver.manage().deleteAllCookies(); driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS); } private WebDriver getBrowser(String browser) { if(browser.equals("chrome")){ System.setProperty("webdriver.chrome.driver", "webdrivers\\chromedriver.exe"); return new ChromeDriver(); } return new FirefoxDriver(); } @AfterMethod public void afterMethod(){ driver.quit(); } @Test(description = "Check parallel selenium works.", dataProvider = SEARCH_TERMS) public void parallelSeleniumTest(String searchTerm){ driver.get("http://google.com"); WebElement search = driver.findElement(By.id("gbqfq")); new Actions(driver) .sendKeys(search, searchTerm) .sendKeys(search, Keys.ENTER) .perform(); String firstResult = driver.findElements(By.className("r")).get(0).getText(); assertNotNull(firstResult); System.out.println(firstResult); } @DataProvider(name = SEARCH_TERMS, parallel = true) public Object[][] getSearchTerms(){ return new Object[][]{ {"google"}, {"microsoft"}, {"facebook"}, {"amazon"}, {"apple"}, {"oracle"}, {"yahoo"}, {"jetbrains"}, {"intellij idea"}, {"selenium"}, {"java"}, {"testng"}, {"code"} }; } } 

I introduced some native events, as I used them heavily in my test suite.

And here is the TestNg xml configuration file

 <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd"> <suite thread-count="4" name="vfr6-ui-tests" parallel="methods"> <test name="parallel-test-firefox"> <parameter name="browser" value="firefox"/> <classes> <class name="tests.ParallelTest"/> </classes> </test> <test name="parallel-test-chrome"> <parameter name="browser" value="chrome"/> <classes> <class name="tests.ParallelTest"/> </classes> </test> </suite> 

I read that an instance of one driver per test is usually the most convenient to maintain. The problem is that the firefox test runs sequentially, while the chrome check spits out all the data points as test cases, tries to open a bunch of browser instances, then everything fails. My tests will have 10-25 or 300-500 data points (cyclic transition between customers or customers x products).

What is the best way to configure the driver, dataprovider, and test runner to achieve the best parallelism when running tests?

Answer 1

I had the same experience with dataProvider. In my case, I used the dataProvider attribute (parallel = true). There are two solutions to your problem.

• Use dataProvider in both the test class and use the factory annotation for your constructor. In the factory annotation attribute, use dataProvider = "Your data transmitter name." In testng.xml, use parallel = instances instead of parallel = methods.

  The disadvantage of the above approach is that when you get a report; maybe this is maven-surefire, testng Eclipse report or reportNG, you do not see that the parameters are passed up. To overcome this, you can use the following approach.

• Create a factory class and create a test class in the factory method, using for the loop. (Run a loop from 0.) In the test class, define a constructor that receives a parameter from the factory class. Define dataProvider in this test class, which can use the parameter (data point) obtained in the constructor. Define a BeforeMethod or BeforeClass method that can use this parameter or data point, and your test methods should have a dataProvider attribute pointing to the desired dataProvider. Again, in testng.xml use parallel = "instances".

  Also, use the try / catch block to instantiate the driver object and close the browser. This will help you avoid skipping due to a setUp tearDown method failure.

Answer 2

You really don't need to use Factory. If I were you, I would call this code in the dataprovider method:

 driver = getBrowser(browser); 

Then return the driver instances as the second argument column to the test method. Performing this method allows the dataprovider to generate browser instances. To improve this, you can instead use the builder design pattern in the form of the DriverHelper class, which can replace the getBrowser method by generating a much more specific driver configuration before passing the driver instance to the test method.

NOTE. Keep in mind that if you ever want to use Spring to load your drivers in the future, this method will not work at all. In fact, you cannot use the DataProvider at all. But if you are not using Spring, then I would say that this is the most elegant way to do this.