如何在JUnit测试中运行JMH？

Question

如何在JUnit测试中运行JMH？

javajunitjmh

43

如何在现有项目中使用JUnit测试运行JMH基准测试？官方文档建议创建一个独立的项目，使用Maven shade插件，并在main方法中启动JMH。这是必要的吗？为什么推荐这样做？

- Aleksandr Dubinsky

4个回答

6

使用注解的声明式方法:

@State(Scope.Benchmark)
@Threads(1)
public class TestBenchmark {
       
    @Param({"10","100","1000"})
    public int iterations;


    @Setup(Level.Invocation)
    public void setupInvokation() throws Exception {
        // executed before each invocation of the benchmark
    }

    @Setup(Level.Iteration)
    public void setupIteration() throws Exception {
        // executed before each invocation of the iteration
    }

    @Benchmark
    @BenchmarkMode(Mode.AverageTime)
    @Fork(warmups = 1, value = 1)
    @Warmup(batchSize = -1, iterations = 3, time = 10, timeUnit = TimeUnit.MILLISECONDS)
    @Measurement(batchSize = -1, iterations = 10, time = 10, timeUnit = TimeUnit.MILLISECONDS)
    @OutputTimeUnit(TimeUnit.MILLISECONDS)
    public void test() throws Exception {
       Thread.sleep(ThreadLocalRandom.current().nextInt(0, iterations));
    }


    @Test
    public void benchmark() throws Exception {
        String[] argv = {};
        org.openjdk.jmh.Main.main(argv);
    }

}

- Cristian Florescu

仅提供代码的答案不受欢迎。这个解决方案与现有答案有何不同和/或优势？调用jmh.Main如何导致正确的测试运行？ - Aleksandr Dubinsky

1

这是另一种简化的方法。就是这样。 - Cristian Florescu

我并不是在批评你。我只是列出了你应该在帖子文本中回答的问题。发布没有解释的代码是不好的。 - Aleksandr Dubinsky

1

区别更或多或少明显！？：上面的代码提供了注释作为测试设置 - 另一个是编程方法。两者共同之处在于JUnit仅用于启动JMH。这是个人偏好 - 我更喜欢注释方法。 - cljk

1

谢谢，尽管我收到了一个关于“无法找到资源：/META-INF/BenchmarkList”的消息。 - Luke

显示剩余2条评论

0

@State(Scope.Benchmark)
@Threads(1)
@Fork(1)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
@Warmup(iterations = 5, time = 1)
@Measurement(iterations = 5, time = 1)
@BenchmarkMode(Mode.All)
public class ToBytesTest {

  public static void main(String[] args) {
    ToBytesTest test = new ToBytesTest();
    System.out.println(test.string()[0] == test.charBufferWrap()[0] && test.charBufferWrap()[0] == test.charBufferAllocate()[0]);
  }

  @Test
  public void benchmark() throws Exception {
    org.openjdk.jmh.Main.main(new String[]{ToBytesTest.class.getName()});
  }

  char[] chars = new char[]{'中', '国'};

  @Benchmark
  public byte[] string() {
    return new String(chars).getBytes(StandardCharsets.UTF_8);
  }

  @Benchmark
  public byte[] charBufferWrap() {
    return StandardCharsets.UTF_8.encode(CharBuffer.wrap(chars)).array();
  }

  @Benchmark
  public byte[] charBufferAllocate() {
    CharBuffer cb = CharBuffer.allocate(chars.length).put(chars);
    cb.flip();
    return StandardCharsets.UTF_8.encode(cb).array();
  }
}

- 欲穷三千界

1

你的回答可以通过提供更多支持信息来改进。请编辑以添加进一步的细节，例如引用或文档，以便他人可以确认你的答案是正确的。您可以在帮助中心中找到有关如何编写良好答案的更多信息。 - Community

仅提供代码的答案不受欢迎。请至少解释您的答案与其他类似答案的区别。 - Aleksandr Dubinsky

0

您可以编写自己的JUnit Runner来运行基准测试。它允许您从Eclipse IDE中运行和调试基准测试。

编写继承org.junit.runner.Runner类的类

public class BenchmarkRunner extends Runner {
  //...
}

实现构造函数和几个方法

public class BenchmarkRunner extends Runner {
   public BenchmarkRunner(Class<?> benchmarkClass) {
   }

   public Description getDescription() {
    //...
   }  

   public void run(RunNotifier notifier) {
    //...
   }
}

将该Runner添加到您的测试类中

@RunWith(BenchmarkRunner.class)  
public class CustomCollectionBenchmark {
    //...
}

我在我的博客文章中详细描述了它： https://vbochenin.github.io/running-jmh-from-eclipse

- Vlad Bochenin

当链接到您自己的网站或内容（或与您有关联的内容）时，您必须在答案中披露您的关联，以便不被视为垃圾邮件。在用户名中具有与URL相同的文本或在个人资料中提及它不被视为足够的披露根据Stack Exchange政策。 - cigien

网页内容由stack overflow 提供, 点击上面的

可以查看英文原文，
原文链接

- Aleksandr Dubinsky · Accepted Answer

我一直在使用JUnit将JMH运行在我的现有Maven项目中，没有发现任何不良影响。我无法回答为什么作者建议采用不同的做法。我没有观察到结果上的差异。JMH启动一个单独的JVM来运行基准测试以隔离它们。以下是我的操作步骤：

Add the JMH dependencies to your POM:

<dependency>
  <groupId>org.openjdk.jmh</groupId>
  <artifactId>jmh-core</artifactId>
  <version>1.21</version>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>org.openjdk.jmh</groupId>
  <artifactId>jmh-generator-annprocess</artifactId>
  <version>1.21</version>
  <scope>test</scope>
</dependency>

Note that I've placed them in scope test.

In Eclipse, you may need to configure the annotation processor manually. NetBeans handles this automatically.

Create your JUnit and JMH class. I've chosen to combine both into a single class, but that is up to you. Notice that OptionsBuilder.include is what actually determines which benchmarks will be run from your JUnit test!

import java.util.ArrayList;
import java.util.List;
import java.util.Random;
import java.util.concurrent.TimeUnit;
import org.junit.Test;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.infra.Blackhole;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.options.*;


public class TestBenchmark 
{

      @Test public void 
    launchBenchmark() throws Exception {

            Options opt = new OptionsBuilder()
                // Specify which benchmarks to run. 
                // You can be more specific if you'd like to run only one benchmark per test.
                .include(this.getClass().getName() + ".*")
                // Set the following options as needed
                .mode (Mode.AverageTime)
                .timeUnit(TimeUnit.MICROSECONDS)
                .warmupTime(TimeValue.seconds(1))
                .warmupIterations(2)
                .measurementTime(TimeValue.seconds(1))
                .measurementIterations(2)
                .threads(2)
                .forks(1)
                .shouldFailOnError(true)
                .shouldDoGC(true)
                //.jvmArgs("-XX:+UnlockDiagnosticVMOptions", "-XX:+PrintInlining")
                //.addProfiler(WinPerfAsmProfiler.class)
                .build();

            new Runner(opt).run();
        }

    // The JMH samples are the best documentation for how to use it
    // http://hg.openjdk.java.net/code-tools/jmh/file/tip/jmh-samples/src/main/java/org/openjdk/jmh/samples/
    @State (Scope.Thread)
    public static class BenchmarkState
    {
        List<Integer> list;

          @Setup (Level.Trial) public void
        initialize() {

                Random rand = new Random();

                list = new ArrayList<>();
                for (int i = 0; i < 1000; i++)
                    list.add (rand.nextInt());
            }
    }

      @Benchmark public void 
    benchmark1 (BenchmarkState state, Blackhole bh) {

            List<Integer> list = state.list;

            for (int i = 0; i < 1000; i++)
                bh.consume (list.get (i));
        }
}

JMH's annotation processor seems to not work well with compile-on-save in NetBeans. You may need to do a full Clean and Build whenever you modify the benchmarks. (Any suggestions appreciated!)

Run your launchBenchmark test and watch the results!

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running com.Foo
# JMH version: 1.21
# VM version: JDK 1.8.0_172, Java HotSpot(TM) 64-Bit Server VM, 25.172-b11
# VM invoker: /usr/lib/jvm/java-8-jdk/jre/bin/java
# VM options: <none>
# Warmup: 2 iterations, 1 s each
# Measurement: 2 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 2 threads, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: com.Foo.benchmark1

# Run progress: 0.00% complete, ETA 00:00:04
# Fork: 1 of 1
# Warmup Iteration   1: 4.258 us/op
# Warmup Iteration   2: 4.359 us/op
Iteration   1: 4.121 us/op
Iteration   2: 4.029 us/op


Result "benchmark1":
  4.075 us/op


# Run complete. Total time: 00:00:06

REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.

Benchmark                                Mode  Cnt  Score   Error  Units
Foo.benchmark1                           avgt    2  4.075          us/op
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.013 sec

Runner.run even returns RunResult objects on which you can do assertions, etc.