1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
|
# ruby/benchmark
This directory has benchmark definitions to be run with
[benchmark\_driver.gem](https://github.com/benchmark-driver/benchmark-driver).
## Normal usage
Execute `gem install benchmark_driver` and run a command like:
```bash
# Run a benchmark script with the ruby in the $PATH
benchmark-driver benchmark/app_fib.rb
# Run all benchmark scripts with multiple Ruby executables or options
benchmark-driver benchmark/*.yml -e /path/to/ruby -e '/path/to/ruby --jit'
# Or compare Ruby versions managed by rbenv
benchmark-driver benchmark/*.yml --rbenv '2.5.1;2.6.0-preview2 --jit'
# You can collect many metrics in many ways
benchmark-driver benchmark/*.yml --runner memory --output markdown
```
See also:
```console
$ benchmark-driver --help
Usage: benchmark-driver [options] [YAML|RUBY]
-r, --runner [TYPE] Specify runner type: ips, time, memory, once (default: ips)
-o, --output [TYPE] Specify output type: compare, simple, markdown, record (default: compare)
-e, --executables [EXECS] Ruby executables (e1::path1,arg1,...; e2::path2,arg2;...)
--rbenv [VERSIONS] Ruby executables in rbenv (x.x.x,arg1,...;y.y.y,arg2,...;...)
--repeat-count [NUM] Try benchmark NUM times and use the fastest result or the worst memory usage
--repeat-result [TYPE] Yield "best", "average" or "worst" result with --repeat-count (default: best)
--bundler Install and use gems specified in Gemfile
--filter [REGEXP] Filter out benchmarks with given regexp
--run-duration [SECONDS] Warmup estimates loop_count to run for this duration (default: 3)
-v, --verbose Verbose mode. Multiple -v options increase visilibity (max: 2)
```
## make benchmark
Using `make benchmark`, `make update-benchmark-driver` automatically downloads
the supported version of benchmark\_driver, and it runs benchmarks with the downloaded
benchmark\_driver.
```bash
# Run all benchmarks with the ruby in the $PATH and the built ruby
make benchmark
# Or compare with specific ruby binary
make benchmark COMPARE_RUBY="/path/to/ruby --jit"
# Run vm1 benchmarks
make benchmark ITEM=vm1
# Run some limited benchmarks in ITEM-matched files
make benchmark ITEM=vm1 OPTS=--filter=block
# You can specify the benchmark by an exact filename instead of using the default argument:
# ARGS = $$(find $(srcdir)/benchmark -maxdepth 1 -name '*$(ITEM)*.yml' -o -name '*$(ITEM)*.rb')
make benchmark ARGS=../benchmark/erb_render.yml
# You can specify any option via $OPTS
make benchmark OPTS="--help"
# With `make benchmark`, some special runner plugins are available:
# -r peak, -r size, -r total, -r utime, -r stime, -r cutime, -r cstime
make benchmark ITEM=vm2_bigarray OPTS="-r peak"
```
|
