Can the surefire plug-in and fail-safe systematize random execution of test classes?

Question

Reliable and fail-safe plugins seem to run test classes, while tests defined in a class run in an undefined order.

To discover tests that rely on order (which we consider bad tests), we want to make the order different for each run. In the ideal case, we would have a mechanism to disable randomization or the number of seeds that repeat the order (you need the old palm-based emulator to have a seed number that gives a sequence of random tests).

Let me know if you know how to do this? If not, I think I can work in a local plug and then send it.

thanks

Peter

Answer 1

Some of the other answers link to the surefire maven documentation page, but, like most maven documents, there are no examples of how to actually specify parameters in maven XML morass. Here's how to do it with the surefire plugin:

<properties>
  <surefire.plugin.version>2.16</surefire.plugin.version>
</properties>

<build>
 <plugins>
   <plugin>
     <groupId>org.apache.maven.plugins</groupId>
     <artifactId>maven-surefire-plugin</artifactId>
     <version>${surefire.plugin.version}</version>
     <configuration>
       <runOrder>random</runOrder>
     </configuration>
   </plugin>
 </plugins>
</build>

Answer 2

Point runOrder of Surefire to "random" http://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#runOrder

Answer 3

, unit test, Surefire/Failsafe, .

, Junit ( ClassRunner):

JUnit ?

, : http://randomjunit.sourceforge.net/

Answer 4

, . Maven-Surefire-PLugin , . Maven-Failsafe-plugin , , , . , maven-surefire-plugin :

http://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#runOrder

This, of course, will be affected by the testing framework you use. In JUnit, you can only influence your order in a limited way. In TestNG, this is a completely different story, because TestNG has the ability to define dependencies, etc.

Maven-Failsafe-Plugin has the same features that affect the execution order.

http://maven.apache.org/surefire/maven-failsafe-plugin/integration-test-mojo.html#runOrder