summaryrefslogtreecommitdiff
path: root/benchmark
AgeCommit message (Collapse)Author
2018-07-10benchmark/README.md: don't recommend alias installation [ci skip]k0kubun
benchmark_driver is the official way to install benchmark_driver. benchmark-driver is just an alias for it. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63930 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-10benchmark: drop all bm_ prefix for legacy driver.rbk0kubun
benchmark/*.rb is only benchmarks now. We don't need prefixes. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63928 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-10Revert "benchmark/*.yml: convert from benchmark/bm_*.rb"k0kubun
This reverts r63900. Having single-execution benchmark as a normal Ruby script is preferred by ko1. I'm not a big fan of having inconsistent benchmark formats, but I can understand some benefits of it. common.mk: remove obsolsted benchmark-each PHONY declaration, support running Ruby scripts added by this commit. README.md: follow ARGS change git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63926 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-10benchmark: resurrect peak / size metricsk0kubun
by adding runner plugins for them. benchmark/lib/benchmark_driver/runner/peak.rb: added peak runner plugin benchmark/lib/benchmark_driver/runner/size.rb: added size runner plugin common.mk: allow using them benchmark/memory_wrapper.rb: deleted in favor of those runner plugins benchmark/README.md: document them git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63924 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-09benchmark/README.md: fix -e interface for v0.14 [ci skip]k0kubun
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63919 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-09benchmark/driver.rb: fully obsolete thisk0kubun
in favor of just using benchmark_driver.gem. common.mk: The new `make benchmark` covers the both usages for old `make benchmark` and old `make benchmark-each`. So `make benchmark-each` is dropped now. benchmark/README.md: Explain its details git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63918 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-09common.mk: upgrade benchmark_driver to v0.14k0kubun
benchmark/driver.rb: deal with breaking changes which are actually introduced for this driver. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63916 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-09file_rename.yml: allow running this on mswin/mingwk0kubun
Suggested by MSP-Grep: http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/87883 git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63914 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-09app_pentominio.yml: drop unnecessary preludek0kubun
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63913 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-08benchmark/driver.rb: make this more similark0kubun
to original benchmark-driver command. I'm going to add some runner plugins to resurrect metrics which were originally supported by benchmark/driver.rb... git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63906 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-08benchmark/README.md: [ci skip] change syntax highlightk0kubun
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63905 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-08benchmark/README.md: [ci skip] write documentationk0kubun
about this directory benchmark/driver.rb: ditto git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63903 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-08benchmark/driver.rb: drop legacy Ruby script supportk0kubun
Now all benchmarks are converted to YAMLs. common.mk: Drop obsoleted bm_* pattern git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63902 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-08benchmark/app_pentomino.yml: remove original scripts after __END__k0kubun
mistakenly left by conversion script... git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63901 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-08benchmark/*.yml: convert from benchmark/bm_*.rbk0kubun
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63900 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-08benchmark/gc: prevent GC bench from depending on normal benchmarkk0kubun
scripts. This is needed to finish converting Ruby scripts to YAMLs. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63899 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-08benchmark/vm2_struct_small_aset.yml: unwrap loop_countk0kubun
since `i` is involved in this script as well... git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63898 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-08benchmark/vm2_*.yml: fix ugly conversion errorsk0kubun
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63897 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-08benchmark/vm2_*.yml: abstract away the while loopk0kubun
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63896 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-08benchmark/vm1_ensure.yml: rollback loop_count abstractionk0kubun
for now. When measured script is really too fast, while loop substituion may return a negative benchmark result. Probably benchmark_driver.gem has rooms to be improved about this. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63895 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-08benchmark/vm1_*.yml: manual fixes for ugly conversionsk0kubun
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63894 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-08benchmark/vm1_*.yml: abstract away the while loopk0kubun
benchmark/driver.rb had removed the cost for while loop in benchmark/bm_vm1_*.rb, and benchmark_driver.gem can achieve the same thing with `loop_count`. But unfortunately current benchmark_driver.gem can't solve it only for vm1_yield.yml... git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63893 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-08benchmark: convert bm_vm2_*.rb to vm2_*.ymlk0kubun
This YAML transformation is needed to support whileloop2 time substituion by benchmark_driver.gem later. This commmit changes no benchmark behavior. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63892 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-08benchmark: convert bm_vm1_*.rb to vm1_*.ymlk0kubun
This YAML transformation is needed to support whileloop time substituion by benchmark_driver.gem later. This commmit changes no benchmark behavior. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63891 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-08benchmark/driver.rb: fix wrong multiline regexpk0kubun
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63889 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-08benchmark: introduce benchmark_driver.gemk0kubun
Makefile.in: Clone benchmark-driver repository in benchmark/benchmark-driver `make update-benchmark-driver`, like simplecov. win32/Makefile.sub: Roughly do the same thing. .gitignore: Ignore the cloned repository. common.mk: Trigger `make update-benchmark-driver` to run `make benchmark` and adjust arguments for benchmark_driver.gem. benchmark/require.yml: renamed from benchmark/bm_require.rb, benchmark/prepare_require.rb benchmark/require_thread.yml: renamed from benchmark/bm_require_thread.rb, benchmark/prepare_require_thread.rb benchmark/so_count_words.yml: renamed from benchmark/bm_so_count_words.rb, benchmark/prepare_so_count_words.rb, benchmark/wc.input.base benchmark/so_k_nucleotide.yml: renamed from benchmark/bm_so_k_nucleotide.rb, benchmark/prepare_so_k_nucleotide.rb, benchmark/make_fasta_output.rb benchmark/so_reverse_complement.yml: renamed from benchmark/bm_so_reverse_complement.rb, benchmark/prepare_so_reverse_complement.rb, benchmark/make_fasta_output.rb I'm sorry but I made some duplications between benchmark/require.yml and benchmark/require_thread.yml, and between benchmark/so_k_nucleotide.yml and benchmark/so_reverse_complement.yml. If you're not comfortable with it, please combine these YAMLs to share the same prelude. One YAML file can have multiple benchmark definitions sharing prelude. benchmark/driver.rb: Replace its core feature with benchmark_driver.gem. Some old features are gone for now, but I'll add them again later. [Misc #14902] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63888 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-08benchmark: drop legacy benchmark driversk0kubun
It seems like they are all benchmark drivers but "benchmark/driver.rb" is the latest and others are no longer used. It's confusing to have multiple drivers (and actually I used benchmark/run.rb since I didn't know I should use benchmark/driver.rb). As I'm going to support only benchmark/driver.rb features in Misc#14902, let me delete them. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63886 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-08benchmark/driver.rb: simplify LoadError handlingk0kubun
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63882 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-06-03benchmark/driver.rb: add CPU time measurement targetsnormal
"Real" time is too unstable on my systems, hopefully counting only CPU time can gain more reliable benchmark results. [ruby-core:87362] [Feature #14815] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63564 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-05-23benchmark/bm_*erb*: fix for --measure=size/peak in driver.rbnormal
benchmark/memory_wrapper.rb will Kernel#load these scripts, preventing DATA from being initialized, so use heredoc instead. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63497 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-21use `--version`.ko1
* benchmark/driver.rb: use `--version` instead of `-v` to get version information. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62515 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-15Benchmarks for Array#values_atnobu
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62412 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-24Lazy Proc allocation for block parametersko1
[Feature #14045] * insns.def (getblockparam, setblockparam): add special access instructions for block parameters. getblockparam checks VM_FRAME_FLAG_MODIFIED_BLOCK_PARAM and if it is not set this instruction creates a Proc object from a given blcok and set VM_FRAME_FLAG_MODIFIED_BLOCK_PARAM. setblockparam is similar to setlocal, but set VM_FRAME_FLAG_MODIFIED_BLOCK_PARAM. * compile.c: use get/setblockparm instead get/setlocal instructions. Note that they are used for method local block parameters (def m(&b)), not for block local method parameters (iter{|&b|). * proc.c (get_local_variable_ptr): creates Proc object for Binding#local_variable_get/set. * safe.c (safe_setter): we need to create Proc objects for postponed block parameters when $SAFE is changed. * vm_args.c (args_setup_block_parameter): used only for block local blcok parameters. * vm_args.c (vm_caller_setup_arg_block): if called with VM_CALL_ARGS_BLOCKARG_BLOCKPARAM flag then passed block values should be a block handler. * test/ruby/test_optimization.rb: add tests. * benchmark/bm_vm1_blockparam*: added. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60397 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-24file.c: apply2files releases GVLnormal
This means File.chmod, File.lchmod, File.chown, File.lchown, File.unlink, and File.utime operations on slow filesystems no longer hold up other threads. The platform-specific utime_failed changes is compile-tested using a new UTIME_EINVAL macro This hurts performance on fast filesystem, but these methods are unlikely to be performance bottlenecks and (IMHO) avoiding pathological slowdowns and stalls are more important. benchmark results: minimum results in each 3 measurements. Execution time (sec) name trunk built file_chmod 0.591 0.801 Speedup ratio: compare with the result of `trunk' (greater is better) name built file_chmod 0.737 * file.c (UTIME_EINVAL): new macro to ease compile-testing * file.c (struct apply_arg): new struct * file.c (no_gvl_apply2files): new function * file.c (apply2files): release GVL * file.c (chmod_internal): adjust for apply2files changes * file.c (lchmod_internal): ditto * file.c (chown_internal): ditto * file.c (lchown_internal): ditto * file.c (utime_failed): ditto * file.c (utime_internal): ditto * file.c (unlink_internal): ditto [ruby-core:83200] [Feature #13996] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60386 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-21driver.rb: allow executing driver.rb directlyk0kubun
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60301 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-21driver.rb: add option to specify target with rbenvk0kubun
[close GH-1724] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60299 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-04Dir.empty? releases GVLnormal
This converts all slow syscalls in the Dir.empty? implementation to release GVL. We avoid unnecessarily GVL release and reacquire for each slow call (opendir, readdir, closedir) and instead only release and acquire the GVL once in the common case. Benchmark results show a small degradation in single-threaded performance: Execution time (sec) name trunk built dir_empty_p 0.689 0.758 Speedup ratio: compare with the result of `trunk' (greater is better) name built dir_empty_p 0.909 * dir.c (rb_gc_for_fd_with_gvl): new function (nogvl_dir_empty_p): ditto (dir_s_empty_p): use new functions to release GVL * benchmark/bm_dir_empty_p.rb: new benchmark [ruby-core:83071] [Feature #13958] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60111 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-01File#rename releases GVLnormal
rename(2) requires two pathname resolution operations which can take considerable time on slow filesystems, release the GVL so operations on other threads may proceed. On fast, local filesystems, this change results in some slowdown as shown by the new benchmark. I consider the performance trade off acceptable as cases are avoided. benchmark results: minimum results in each 3 measurements. Execution time (sec) name trunk built file_rename 2.648 2.804 Speedup ratio: compare with the result of `trunk' (greater is better) name built file_rename 0.944 * file.c (no_gvl_rename): new function (rb_file_s_rename): release GVL for renames * benchmark/bm_file_rename.rb: new benchmark git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60088 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-01string.c: avoid unnecessary call of str_strlen()glass
* string.c (rb_strseq_index): refactor and avoid call of str_strlen() when offset == 0. it will improve performance of String#index and #include? * benchmark/bm_string_index.rb: benchmark for this change git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60086 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-09-29array.c: improve operations on small arraysnobu
[Feature #13884] Reduce number of memory allocations for "and", "or" and "diff" operations on small arrays Very often, arrays are used to filter parameters and to select interesting items from 2 collections and very often these collections are small enough, for example: ```ruby SAFE_COLUMNS = [:id, :title, :created_at] def columns @all_columns & SAFE_COLUMNS end ``` In this patch, I got rid of unnecessary memory allocations for small arrays when "and", "or" and "diff" operations are performed. name | HEAD | PATCH -----------------+------:+------: array_small_and | 0.615 | 0.263 array_small_diff | 0.676 | 0.282 array_small_or | 0.953 | 0.463 name | PATCH -----------------+------: array_small_and | 2.343 array_small_diff | 2.392 array_small_or | 2.056 name | HEAD | PATCH -----------------+------:+------: array_small_and | 1.429 | 1.005 array_small_diff | 1.493 | 0.878 array_small_or | 1.672 | 1.152 name | PATCH -----------------+------: array_small_and | 1.422 array_small_diff | 1.700 array_small_or | 1.452 Author: Dmitry Bochkarev <dimabochkarev@gmail.com> git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60057 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-08-04benchmarks for String#scannobu
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59494 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-05-27benchmark: Rename to bm_erb_render.rbk0kubun
from bm_app_erb_render.rb. I'm told from ko1 that bm_app_* is namespace for Ruby applications, not for ERB and we should use bm_erb_* for ERB benchmark instead. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58915 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-05-26erb.rb: Generate static string with opt_str_uminusk0kubun
to skip object allocation for static string. We can't always enable frozen_string_literal pragma because we can't freeze string literals embedded by user for backward compatibility. So we need to use fstring for each static string. Since adding ".freeze" to string literals in #content_dump is slow on compiling, I used unary "-" operator instead. benchmark/bm_app_erb_render.rb: Added rendering-only benchmark to test rendering performance on production environment. This benchmark is created to reproduce the behavior on Sinatra (Tilt). Thus it doesn't use ERB#result to skip parsing compiled code. It doesn't use ERB#def_method too to regard `title` and `content` as local variables. If we use #def_method, `title` and `content` needs to be method call. I wanted to avoid it. This patch's benchmark results is: * Before app_erb_render 1.250 app_erb 0.704 * After app_erb_render 1.066 app_erb 0.686 This patch optimizes rendering performance (app_erb_render) without spoiling (total of rendering +) compiling performance (app_erb). git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58905 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-05-08benchmark/bm_vm2_fiber_switch.rb: check for fiber performancenormal
There are currently no benchmarks for Fiber performance, I should've committed this years ago when [Feature #10341] was implemented. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58606 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-05-04benchmark/bm_vm_thread_pass_flood.rb: add commentnormal
I was about to write off this benchmark while working on GVL improvements on multi-core systems. However I noticed it exposes a weakness in my work-in-progress code when I tested on an old single CPU system. Further testing reveals setting CPU affinity ("schedtool -a 0x1" on Linux) on a modern multi-core system is enough to reproduce the problem exposed by this benchmark. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58571 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-05-04benchmark/bm_vm1_gc_wb_*.rb: fix GC.start args for invalid keywordnormal
"lazy_sweep" does not appear to have ever been a valid kwarg for GC.start, however the opposite of "lazy_sweep" appears to be "immediate_sweep". So use immediate_sweep, and flip the boolean value of each arg. I guess this only started failing with r56981 in Dec 2016 ("class.c: missing unknown_keyword_error", commit e3f0cca2f26ba44c810ac980cdff7dda129ae533) * benchmark/bm_vm1_gc_wb_ary.rb: "lazy_sweep: false" => "immediate_sweep: true" * benchmark/bm_vm1_gc_wb_ary_promoted.rb: ditto * benchmark/bm_vm1_gc_wb_obj.rb: ditto * benchmark/bm_vm1_gc_wb_obj_promoted.rb: ditto git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58565 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-05-04benchmark: new single-threaded read/write benchmark with pipenormal
This is currently for testing GVL performance in the uncontended case: IO#write and IO#read unconditionally release GVL for blocking I/O with pipe. It will also be interesting to see how this changes if we switch to M:N threading model. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58556 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-05-02more benchmarks for SizedQueue and ConditionVariablenormal
I'm working on patches to improve everything in thread_sync.c, add these to track progress. * benchmark/bm_vm_thread_sized_queue.rb: add description * benchmark/bm_vm_thread_condvar1.rb: new benchmark * benchmark/bm_vm_thread_condvar2.rb: ditto * benchmark/bm_vm_thread_sized_queue2.rb: ditto * benchmark/bm_vm_thread_sized_queue3.rb: ditto * benchmark/bm_vm_thread_sized_queue4.rb: ditto git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58535 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-04-30new benchmark for SizedQueuenormal
The performance of SizedQueue is a bit more complex than regular Queue, so it deserves a separate benchmark. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58509 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-04-14Improve performance of type conversion using to_rmrkn
* object.c: Add to_r in conv_method_tbl. * defs/id.def: add to_r. * benchmark/bm_int_quo.rb: added. * benchmark/bm_time_subsec.rb: added. [Bug #13426] [ruby-core:80665] [Fix GH-1582] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58347 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
generated by cgit v1.2.3 (git 2.25.1) at 2024-05-08 23:23:10 +0000