Run a test using a completed future

Question

I am trying to measure the performance of a particular method. I just run the tests when the method is called directly, but when the method used a terminating future with a custom executor, everything collapsed. I used a method to use a completed future to force a timeout if the method takes too much time.

@Benchmark
@BenchmarkMode(Mode.SingleShotTime)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@Threads(value = 5)
@Warmup(iterations = 20)
@Measurement(iterations = 50, timeUnit = TimeUnit.MILLISECONDS)
public String very_big_query(TestState testState) throws Exception {
    return testState.transpiler.process(testState.veryBigQuery);
}

---

@State(Scope.Thread)
public static class TestState {
    String veryBigQuery;
    Transpiler transpiler;

    @Setup(Level.Trial)
    public void doSetupTrial() throws Exception {
        veryBigQuery = "(";
        for(int i = 0; i < 99; i++) {
            veryBigQuery += String.format("java_%s OR ", i);
        }
        veryBigQuery += "java_100) AND (";
        for(int i = 100; i < 199; i++) {
            veryBigQuery += String.format("java_%s OR ", i);
        }
        veryBigQuery += String.format("java_%s)", 200);
    }

    @Setup(Level.Invocation)
    public void doSetupInvocation() throws Exception {
        random = ThreadLocalRandom.current().nextInt(0, productionQueries.size());
        randomQuery = productionQueries.get(random);
        transpiler = new Transpiler(5, 100); //number of threads in custom pool for the executor, timeout in milliseconds
    }
}

---

public String process(final String input) throws Exception {
    CompletableFuture<String> cf = CompletableFuture.supplyAsync(() -> {
        return SOME_STRING;
    }, executor);

    return cf.get(timeoutInMillis, TimeUnit.MILLISECONDS);
}

---

this.executor = Executors.newFixedThreadPool(numberOfThreads);

I get this error

JMH finished, but the forked virtual machine did not exit, threads? Waiting another 24 seconds ...

Incomplete threads:

Can someone explain to me why this is happening and how I approach him to make it work?

Answer 1

JMH, ( JMH , ) .

Level.Invocation, , ...

@TearDown.