path: root/ruby.c
AgeCommit message (Collapse)Author
2019-06-04EOF by 2 ^D on a TTYNobuyoshi Nakada
Terminate the input from a TTY by 2 ^D at the middle of line, like as many programs, `cat`, `perl` and so on, do. By the first ^D, the line will be sent without a newline, and then EOF will be send by the next ^D.
2019-05-21Do not modify shared arrayNobuyoshi Nakada
[Bug #15821]
2019-03-30ruby.c: respect features by command linenobu
* ruby.c (process_options): feature options in command line arguments take precedence over options in RUBYOPT environment variable. [ruby-core:92052] [Bug #15738] git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-03-28[DOC] fix markups [ci skip]nobu
git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-03-17Change defaults of --jit optionsk0kubun
* --jit-min-calls: 5 -> 10000 --jit-min-calls=5 obviously can compile non hotspot. This was not a problem for MJIT-benchmarks and Optcarrot because the former has very few hot optimiziable methods and the latter is likely to trigger compilation of hotspot by its intensive calls to optimizable hotspot methods and has a very short window to allow limited compilations. In real-world applications, it has more time to compile more methods and it pressures computer's limited resources like icache. We should avoid compiling too many methods. Also compiling many methods exhausts time budget for compilation in one ruby process lifetime and delays the "JIT compaction" of Ruby 2.6. JVM is known to use 1,500 for C1 (client) compiler and 10,000 for C2 (server) compiler for -XX:CompileThreshold by default. When things are called X,000 times, requiring 10,000 can eliminate compilation of methods which are called only once in these X,000 iterations and obviously not hotspot. And in fact things like unicorn-worker-killer restarts unicorn process every 4096 requests. So I felt 10,000 is good for such an environment. * --jit-max-cache: 1000 -> 100 By the same reason stated above, we should not allow compiling many methods especially on MJIT which has a larger overhead due to poor code locality by dlopen and whose code is also likely to be bigger by just inlining many VM instructions with -O3 rather than directly generating low-level code. In JVM -XX:ReservedCodeCacheSize is 32M for reserved and 48M for maximum. --jit-max-cache=1,000 could be closer to it, but in this case MJIT's compilation is slow due to data synchronization between threads (to be improved in Ruby 2.7 though) and we do not want to delay the "JIT compaction" for a long time. So I chose a really conservative number for this, but by having method inlining in the future, wider range could be optimized even with this value. * Optcarrot --disable-gems, --benchmark Lan_Master.nes 12 attempts. No significant impact. | r67276 | r67276 --jit | after --jit | |:-------------------|:------------------|:------------------| | 50.44369263063978 | 72.87390680773056 | 73.47873485047297 | | 50.58788746124193 | 78.06820808947026 | 78.29723420171945 | | 50.77509250801378 | 80.29010348842613 | 78.94689404460769 | | 50.935361702064405 | 80.42796829926374 | 80.39539527351525 | | 51.27352672981195 | 81.98758158033202 | 81.6754198664817 | | 51.720715743242124 | 82.00118535811626 | 82.22960569251283 | | 51.89643169822524 | 82.2290091613556 | 82.5013636146388 | | 51.95895898113868 | 82.37318990939565 | 82.74002377794454 | | 52.10124886807968 | 82.48796686037502 | 83.23354941183932 | | 52.292280637519376 | 83.0265226541066 | 84.01552618012572 | | 52.51856237784144 | 83.8797360318052 | 84.8588319093393 | | 52.65076845986818 | 84.80037351256634 | 85.13577756273656 | * Railsbench `WARMUP=20000 BENCHMARK=1000 bin/bench` of It gets closer to --disable=jit. | | r67276 | r67276 | after | | | | --jit | --jit | |:----------|:-------|:-------|:-------| | req/s | 891.3 | 742.2 | 841.5 | |:----------|:-------|:-------|:-------| | 50%ile ms | 1.00 | 1.21 | 1.08 | | 66%ile ms | 1.02 | 1.24 | 1.09 | | 75%ile ms | 1.03 | 1.28 | 1.10 | | 80%ile ms | 1.03 | 1.30 | 1.11 | | 90%ile ms | 1.09 | 1.42 | 1.15 | | 95%ile ms | 1.32 | 1.65 | 1.27 | | 98%ile ms | 4.79 | 2.23 | 1.81 | | 99%ile ms | 5.68 | 7.52 | 6.64 | |100%ile ms | 6.52 | 9.69 | 8.59 | git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-01-08Defer escaping control char in error messagesnobu
* eval_error.c (print_errinfo): defer escaping control char in error messages until writing to stderr, instead of quoting at building the message. [ruby-core:90853] [Bug #15497] git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-27fix missed script_compiled events. [Bug #15471]ko1
* ruby.c (process_options): script_compiled events are missed on command line -e or specified file. this commit fix it. [Bug #15471] This patch should be backport to Ruby 2.6 branch. * vm_core.h (rb_exec_event_hook_script_compiled): introduce utility function to invoke a script_compiled event. git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-19ruby.c: do not surface MJIT to userk0kubun
In some places, both JIT and MJIT are being used, but it could be confusing for new comers. We're not explaining MJIT on NEWS file or release notes as well. So we consider MJIT as an internal term of implementation like YARV. ditto git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-10AST.of -e scriptnobu
git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-21fix a warning message.ko1
git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-20add disabling MJIT features option.ko1
* introduce new configure option `--enable-mjit` and `--disable-mjit`. Default is "enable". `--disable-mjit` disables all of MJIT features so that `ruby --jit` can't enable MJIT. This option affect a macro `USE_MJIT`. This change remove `--enable/disable-install-mjit-header` option. * introduce the `ENABLE_MJIT` variable. * use `ENABLE_MJIT` option. * internal.h: respect `USE_MJIT`. Same as other *.c, *.h. * test/ruby/test_jit.rb: check `ENABLE_MJIT` key of rbconfg.rb. git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-02Just a shebang is valid codenobu
[ruby-core:89240] [Bug #15190] git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-10version.c: separate Init_ruby_descriptionnobu
* version.c (Init_ruby_description): separate to initialize RUBY_DESCRIPTION constant according to mjit. git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-27ruby.c: taint ARGV on Windowsnobu
* ruby.c (external_str_new_cstr): strings come from the external should be tainted. [ruby-dev:50596] [Bug #14941] git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-26dladdr() is declared with non-const pointer on Solarisnobu
git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-18ruby.c: accept --disable-jit optionk0kubun
by promoting jit to feature flag. mjit.h: update comment about mjit_opts.on test_rubyoptions.rb: add test for switching JIT enablement "--jit" flag usage may be deprecated later, but not discussed yet. [Feature #14878] git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-16ruby.c: disable DidYouMean as gemnobu
* ruby.c (process_options): as DidYouMean requires Rubygems, disable the former when the latter is disabled too. git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-02set up mjit.on at initializationnobu
* ruby.c (cmdline_options_init): set up mjit.on flag by MJIT_FORCE_ENABLE in the initialization function. git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-06-11ruby.c: making hidden objectsnobu
* ruby.c (add_modules): make hidden objects by particular functions. git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-05-21ruby.c: localize a variablenobu
* ruby.c (ruby_init_loadpath_safe): moved libdir to the block where it is used. git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-05-21ruby.c: runtime_libruby_pathnobu
* ruby.c (runtime_libruby_path): hoisted out platform dependent routine to get the loaded runtime library path. cygwin_conv_path does path separator and WCHAR to UTF-8 conversions too. git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-05-10RSTRING_PTR is not guaranteed to be char*-alignedshyouhei
We need to ensure aligned memory access by allocating another memory region. git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-04-27ruby.c: moved libdirnobu
* ruby.c (ruby_init_loadpath_safe): moved libdir name inside LOAD_RELATIVE. git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-04-27ruby.c: fix compilation errornobu
* ruby.c (ruby_init_loadpath_safe): fix compilation error when ENABLE_MULTIARCH but not universal binary. git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-04-27mjit.c: prefix and archdir in initnobu
* ruby.c (ruby_init_loadpath_safe): store prefix and archlibdir paths. * mjit.c (compile_c_to_so, init_header_filename): use just one library path on Windows. git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-04-26ruby.c (ruby_init_loadpath_safe): constifynobu
git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-27ruby.c: exit by --versionnobu
* ruby.c (usage): stated exiting by `--version` option with nothing done. git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-26ruby.c: fix --verbose description in usagenobu
* ruby.c (usage): fix the description of `--verbose` option, which does not print the version number unlike `-v` option. [ruby-core:86307] [Bug #14633] git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-22ruby.c: fix typo in r62530k0kubun
git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-22version.c: show +JIT when --jit is passedk0kubun
in version output. version.h: ditto ruby.c: propagate option for it updated dependency for version.c mjit.c: overwrites the RUBY_DESCRIPTION to have +JIT when --jit is passed test/ruby/test_rubyoptions.rb: add test for them Only `ruby --jit -v` will have "+JIT", but this is intentional. This may not be convenient for debugging by ticket with `ruby -v`, but it's convenient for benchmark tools that pass options (--jit) when showing it. At least such behavior is planned for benchmark_driver.gem and this behavior is designed for it. Other benchmark tools are recommended to follow the behavior too if they show version. RUBY_DESCRIPTION might be useful for it too. The position of "+JIT" is changed from original proposal because other platforms like JRuby and TruffleRuby end it with archtecture. It's made similar to JRuby, but it's upper-cased because Matz made approval for "+JIT" in the ticket. Example: $ ruby -v ruby 2.6.0dev (2018-02-22 trunk 62529) [x86_64-linux] $ ruby --jit -v ruby 2.6.0dev (2018-02-22 trunk 62529) +JIT [x86_64-linux] After --jit is made default in the future, this output may be removed. So do not rely on this output if possible. [Feature #14462] git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-12no --jit-ccnobu
* ruby.c (setup_mjit_options): removed --jit-cc option, since mjit header is affected by generated config.h which depends on the given compiler, so it cannot work with different compilers. git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-07remove declaration of ruby_engine from internal.husa
* internal.h (ruby_engine): remove declaration of ruby_engine because it's declared at ruby/version.h. * ruby.c: include ruby/version.h for ruby_engine. git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-06ruby.c: add MJIT_FORCE_ENABLE macrok0kubun
to always enable MJIT for testing. git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-05mjit.c: determine prefix of MJIT header at runtimek0kubun
so that MJIT can work if Ruby is distributed as prebuilt binary. Now mjit_init() depends on the internal const TMP_RUBY_PREFIX which is only available after ruby_init_loadpath_safe() (L1608) and before ruby_init_prelude() (L1681). So the place of mjit_init() is moved. Removed static prefix from MJIT_HEADER_ISNTALL_DIR macro. And this removes the unused LIBRUBY_LIBDIR macro as well. win32/Makefile.sub: ditto. Patch by: Lars Kanis <> [Bug #14445] git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04mjit.c: merge MJIT infrastructurek0kubun
that allows to JIT-compile Ruby methods by generating C code and using C compiler. See the first comment of mjit.c to know what this file does. mjit.c is authored by Vladimir Makarov <>. After he invented great method JIT infrastructure for MRI as MJIT, Lars Kanis <> sent the patch to support MinGW in MJIT. In addition to merging it, I ported pthread to Windows native threads. Now this MJIT infrastructure can be compiled on Visual Studio. This commit simplifies mjit.c to decrease code at initial merge. For example, this commit does not provide multiple JIT threads support. We can resurrect them later if we really want them, but I wanted to minimize diff to make it easier to review this patch. `/tmp/_mjitXXX` file is renamed to `/tmp/_ruby_mjitXXX` because non-Ruby developers may not know the name "mjit" and the file name should make sure it's from Ruby and not from some harmful programs. TODO: it may be better to store this to some temporary directory which Ruby is already using by Tempfile, if it's not bad for performance. mjit.h: New. It has `mjit_exec` interface similar to `vm_exec`, which is for triggering MJIT. This drops interface for AOT compared to the original MJIT. define macros to let MJIT know the path of MJIT header. Probably we can refactor this to reduce the number of macros (TODO). win32/Makefile.sub: ditto. compile mjit.o and mjit_compile.o. Unlike original MJIT, this commit separates MJIT infrastructure and JIT compiler code as independent object files. As initial patch is NOT going to have ultra-fast JIT compiler, it's likely to replace JIT compiler, e.g. original MJIT's compiler or some future JIT impelementations which are not public now. inits.c: define MJIT module. This is added because `MJIT.enabled?` was necessary for testing. test/lib/zombie_hunter.rb: skip if `MJIT.enabled?`. Obviously this wouldn't work with current code when JIT is enabled. test/ruby/test_io.rb: skip this too. This would make no sense with MJIT. ruby.c: define MJIT CLI options. As major difference from original MJIT, "-j:l"/"--jit:llvm" are renamed to "--jit-cc" because I want to support not only gcc/clang but also cl.exe (Visual Studio) in the future. But it takes only "--jit-cc=gcc", "--jit-cc=clang" for now. And only long "--jit" options are allowed since some Ruby committers preferred it at Ruby developers Meeting on January, and some of options are renamed. This file also triggers to initialize MJIT thread and variables. eval.c: finalize MJIT worker thread and variables. test/ruby/test_rubyoptions.rb: fix number of CLI options for --jit. thread_pthread.c: change for pthread abstraction in MJIT. Prefix rb_ for functions which are used by other files. thread_win32.c: ditto, for Windows. Those pthread porting is one of major works that YARV-MJIT created, which is my fork of MJIT, in Feature 14235. thread.c: follow rb_ prefix changes vm.c: trigger MJIT call on VM invocation. Also trigger `mjit_mark` to avoid SEGV by race between JIT and GC of ISeq. The improvement was provided by wanabe <>. In JIT compiler I created and am going to add in my next commit, I found that having `mjit_exec` after `vm_loop_start:` is harmful because the JIT-ed function doesn't proceed other ISeqs on RESTORE_REGS of leave insn. Executing non-FINISH frame is unexpected for my JIT compiler and `exception_handler` triggers executions of such ISeqs. So `mjit_exec` here should be executed only when it directly comes from `vm_exec` call. `RubyVM::MJIT` module and `.enabled?` method is added so that we can skip some tests which don't expect JIT threads or compiler file descriptors. vm_insnhelper.h: trigger MJIT on method calls during VM execution. vm_core.h: add fields required for mjit.c. `bp` must be `cfp[6]` because rb_control_frame_struct is likely to be casted to another struct. The last position is the safest place to add the new field. vm_insnhelper.c: save initial value of cfp->ep as cfp->bp. This is an optimization which are done in both MJIT and YARV-MJIT. So this change is added in this commit. Calculating bp from ep is a little heavy work, so bp is kind of cache for it. iseq.c: notify ISeq GC to MJIT. We should know which iseq in MJIT queue is GCed to avoid SEGV. TODO: unload some GCed units in some safe way. gc.c: add hooks so that MJIT can wait GC, and vice versa. Simultaneous JIT and GC executions may cause SEGV and so we should synchronize them. cont.c: save continuation information in MJIT worker. As MJIT shouldn't unload JIT-ed code which is being used, MJIT wants to know full list of saved execution contexts for continuation and detect ISeqs in use. mjit_compile.c: added empty JIT compiler so that you can reuse this commit to build your own JIT compiler. This commit tries to compile ISeqs but all of them are considered as not supported in this commit. So you can't use JIT compiler in this commit yet while we added --jit option now. Patch author: Vladimir Makarov <>. Contributors: Takashi Kokubun <>. wanabe <>. Lars Kanis <>. Part of Feature 12589 and 14235. git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-29ruby.c: no VARIABLE_LIBPATHnobu
* ruby.c (ruby_init_loadpath_safe): removed code using fixed size path buffer. relative load path is supported only on platforms where dladdr is available, or on Windows, so !VARIABLE_LIBPATH code is not used. git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-24ruby.c (open_load_file): avoid shadowing variable for errnonormal
git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-09internal.h: remove dependecy on ruby/encoding.hnobu
git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-05make rb_iseq_new* accept rb_ast_body_t instead of NODE*mame
git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-05node.h: define rb_ast_body_t and restructure rb_ast_tmame
git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-16ruby.c: script __dir__ encodingnobu
* ruby.c (process_options): fallback to the encoding of the script name since rb_realpath_internal() cannot convert the encoding when it is ASCII-8BIT. * test/ruby/test_rubyoptions.rb (test___dir__encoding): explicitly pass environment variables for locale as they are overriden by invoke_ruby. git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-29* node.h (ast_t): renamed to `rb_ast_t`.ko1
git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-27Revert "Revert "Manage AST NODEs out of GC""mame
This re-introduces r60485. This reverts commit 5a176b75b1187cbd3861c387bde65ff66396a07c. git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-27Revert "Manage AST NODEs out of GC"mame
This reverts commit 620ba74778bfdbdc34ffbb142d49ce84a0ef58e9. git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-27Manage AST NODEs out of GCmame
NODEs in AST are no longer objects managed by GC. This change will remove the restriction imposed by the GC. For example, a NODE can use more than five words (this is my primary purpose; we want to store the position data for each NODE, for coverage library), or even a NODE can have variable length (some kinds of NODEs have unused fields). To do this, however, we need more work, since Ripper still uses T_NODE objects managed by the GC. The life time of NODEs is more obvious than other kinds of objects; they are created at parsing, and they become disused immediately after compilation. This change releases all NODEs by a few `xfree`s after compilation, so performance will be improved a bit. In extreme example, `eval("x=1;" * 10000000)` runs much faster (40 sec. -> 7.8 sec. on my machine). The most important part of this change is `ast_t` struct, which has three contents: (1) NODE buffer (malloc'ed memory), (2) a reference to the root NODE, and (3) an array that contains objects that must be marked during parsing (such as literal objects). Some functions that had received `NODE*` arguments, must now receive `ast_t*`. * node.c, node.h: defines `ast_t` struct and related operations. * gc.c, internal.h: defines `imemo_ast`. * parse.y: makes `parser_params` struct have a reference to `ast_t`. Instead of `rb_node_newnode`, use `rb_ast_newnode` to create a NODE. * iseq.c, load.c, ruby.c, template/prelude.c.tmpl: modifies some functions to handle `ast_t*` instead of `NODE*`. * test/ruby/test_gc.rb: ad-hoc fix for a failed test. The test assumes GC eden is increased at startup by NODE object creation. However, this change now create no NODE object, so GC eden is not necessarily increased. git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-24ruby.c: fix r60393nobu
* ruby.c (load_file_internal): set loop options after parsing shebang line. git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-24parse.y: rb_parser_set_optionsnobu
* parse.y (yycompile0): append top-level addenda before appending prelude nodes. * parse.y (rb_parser_set_options): set top-level addendum options before parsing. git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-21The encoding of __FILE__ and __dir__ should be sameusa
* ruby.c (process_options): convert the real path of the script to locale encoding if its encoding is not locale (maybe UTF-8) on Windows/OS X. this change makes the encoding of __dir__ to the same encoding of __FILE__ when the script name is passed from commandline. * test/ruby/test_options.rb (test___dir__encoding): test for this change. git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-19ruby.c: reject NUL in $0nobu
* ruby.c (ruby_setproctitle): raise if the argument contains NUL char. process title is a NUL-terminated string. [ruby-core:82425] [Bug #13829] git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-09-20ruby.c: show help messages of --dumpsonots
git-svn-id: svn+ssh:// b2dd03c8-39d4-4d8f-98ff-823fe69b080e