How to generate methods in jmh tests?

Question

I am using jmh ( http://openjdk.java.net/projects/code-tools/jmh/ ) to compare some method. I also have a set of parameters that I want to use as arguments to run this method. Is it possible to generate a method for each specific parameter value (with @GenerateMicroBenchmark annotation)?

Now I use a similar implementation, but this is not so convenient, because I have to write a lot of single code manually:

interface State { int action(); void prepare(); } class Method { ...; toString() { return "State1_" + param; } } { Method[] states; curState = -1; count = 0; int[] params = {1000, 2000, 3000, 4000, 5000, 6000, 7000, 8000, 9000, 10000, 11000, 12000}; for (int param: params) { states[count++] = new Method(param); } } @Setup(Level.Invocation) public void prepareState() { if (curState != -1) { states[curState].prepare(); } } @GenerateMicroBenchmark public int param_1000() { curState = 0; return states[curState].action(); } @GenerateMicroBenchmark public int param_2000() { curState = 1; return states[curState].action(); } @GenerateMicroBenchmark public int param_3000() { curState = 2; return states[curState].action(); } ... @GenerateMicroBenchmark public int param_12000() { curState = 11; return states[curState].action(); } 

Answer 1

Tests are usually not reusable without any major breakdown. Most of the attempts that I saw when people trying to simplify their tests violated their conviction. For example, using arrays here can compensate for the results for nano-tests (for example, if action () is small). Level.Invocation is usually a bad idea, as stated in its Javadoc .

The bottom line is that since the use of the API is allowed, this does not necessarily mean that you should use it. Here is what you should do instead:

 @State(Scope.Thread) class MyBenchmark { Method[] states; @Setup public void setup() { int[] params = {1000, 2000, 3000, 4000, 5000, 6000, 7000, 8000, 9000, 10000, 11000, 12000}; int count = 0; for (int param: params) { states[count++] = new Method(param); } } int doWork(int idx) { states[idx].prepare(); return states[idx].action(); } @GenerateMicroBenchmark public int param_1000() { doWork(0); } ... @GenerateMicroBenchmark public int param_12000() { doWork(11); } } 

... or even:

 @State(Scope.Thread) class MyBenchmark { Method p1000, p2000, ..., p12000; @Setup public void setup() { p1000 = new Method(p1000); p2000 = new Method(p2000); ... p12000 = new Method(p12000); } @GenerateMicroBenchmark public int param_1000() { p1000.prepare(); return p1000.action(); } ... @GenerateMicroBenchmark public int param_12000() { p12000.prepare(); return p12000.action(); } } 

An alternative would be to accept the parameter from the outside and use the Java API to manipulate the parameter. For example:

 @State(Scope.Thread) class MyBenchmark { final int param = Integer.getInteger("param", 1000); Method m; @Setup public void setup() { m = new Method(param); } @GenerateMicroBenchmark public int work() { m.prepare(); return m.action(); } public static void main(String[] args) throws RunnerException { Options opts = new OptionsBuilder() .include(".*") .jvmArgs("-Dparam=2000") .build(); RunResult runResult = new Runner(opts).runSingle(); Result result = runResult.getPrimaryResult(); System.out.println(); System.out.println("API replied benchmark score: " + result.getScore() + " " + result.getScoreUnit() + " over " + result.getStatistics().getN() + " iterations"); } }