summaryrefslogtreecommitdiff
path: root/iseq.c
AgeCommit message (Collapse)Author
2025-07-20merge revision(s) b42afa1dbcbb91e89852b3b3bc72484d7f0a5528, ↵nagachika
67b91e780798b80038dbfb39a06831918a75259f, f1f0cc14cc7d3f9be35b203e5583f9224b1e2387, 543e3a1896ae2fe3b5b954f6497d261ab5663a15, ed2806117a0b76e4439ce1a061fae21d9e116d69, 46e4c8673747de96838d2c5dec37446d23d99d88: [Backport #21497] Suppress gcc 15 unterminated-string-initialization warnings Drop an ignored attribute MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit GCC 13.3.0 (Ubuntu 24.04) emits the following warning: ../symbol.c: In function ‘rb_id_attrset’: ../symbol.c:175:9: warning: ‘nonstring’ attribute ignored on objects of type ‘const char[][8]’ [-Wattributes] 175 | RBIMPL_ATTR_NONSTRING() static const char id_types[][8] = { | ^~~~~~~~~~~~~~~~~~~~~ Separate `__has_attribute` from `defined(__has_attribute)` Fix Visual C warnings: ``` regenc.h(121): warning C4067: unexpected tokens following preprocessor directive - expected a newline ``` Cast up `int` instruction code to `VALUE` Fix Visual C warnings: ``` iseq.c(3793): warning C4312: 'type cast': conversion from 'int' to 'void *' of greater size iseq.c(3794): warning C4312: 'type cast': conversion from 'int' to 'void *' of greater size ``` Do not let files depend on a phony target Detect `clock_gettime` and `clock_getres` for winpthreads
2025-03-16merge revision(s) 08b3a45bc97c835b4677bf76dbce68fd51d81897: [Backport #21180]nagachika
Push a real iseq in rb_vm_push_frame_fname() Previously, vm_make_env_each() (used during proc creation and for the debug inspector C API) picked up the non-GC-allocated iseq that rb_vm_push_frame_fname() creates, which led to a SEGV when the GC tried to mark the non GC object. Put a real iseq imemo instead. Speed should be about the same since the old code also did a imemo allocation and a malloc allocation. Real iseq allows ironing out the special-casing of dummy frames in rb_execution_context_mark() and rb_execution_context_update(). A check is added to RubyVM::ISeq#eval, though, to stop attempts to run dummy iseqs. [Bug #21180] Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
2025-01-14merge revision(s) f65a6c090c229de1665af49f2e51fc1d6397ab72: [Backport #20921]Takashi Kokubun
Fix use-after-free in constant cache [Bug #20921] When we create a cache entry for a constant, the following sequence of events could happen: - vm_track_constant_cache is called to insert a constant cache. - In vm_track_constant_cache, we first look up the ST table for the ID of the constant. Assume the ST table exists because another iseq also holds a cache entry for this ID. - We then insert into this ST table with the iseq_inline_constant_cache. - However, while inserting into this ST table, it allocates memory, which could trigger a GC. Assume that it does trigger a GC. - The GC frees the one and only other iseq that holds a cache entry for this ID. - In remove_from_constant_cache, it will appear that the ST table is now empty because there are no more iseq with cache entries for this ID, so we free the ST table. - We complete GC and continue our st_insert. However, this ST table has been freed so we now have a use-after-free. This issue is very hard to reproduce, because it requires that the GC runs at a very specific time. However, we can make it show up by applying this patch which runs GC right before the st_insert to mimic the st_insert triggering a GC: diff --git a/vm_insnhelper.c b/vm_insnhelper.c index 3cb23f06f0..a93998136a 100644 --- a/vm_insnhelper.c +++ b/vm_insnhelper.c @@ -6338,6 +6338,10 @@ vm_track_constant_cache(ID id, void *ic) rb_id_table_insert(const_cache, id, (VALUE)ics); } + if (id == rb_intern("MyConstant")) rb_gc(); + st_insert(ics, (st_data_t) ic, (st_data_t) Qtrue); } And if we run this script: Object.const_set("MyConstant", "Hello!") my_proc = eval("-> { MyConstant }") my_proc.call my_proc = eval("-> { MyConstant }") my_proc.call We can see that ASAN outputs a use-after-free error: ==36540==ERROR: AddressSanitizer: heap-use-after-free on address 0x606000049528 at pc 0x000102f3ceac bp 0x00016d607a70 sp 0x00016d607a68 READ of size 8 at 0x606000049528 thread T0 #0 0x102f3cea8 in do_hash st.c:321 #1 0x102f3ddd0 in rb_st_insert st.c:1132 #2 0x103140700 in vm_track_constant_cache vm_insnhelper.c:6345 #3 0x1030b91d8 in vm_ic_track_const_chain vm_insnhelper.c:6356 #4 0x1030b8cf8 in rb_vm_opt_getconstant_path vm_insnhelper.c:6424 #5 0x1030bc1e0 in vm_exec_core insns.def:263 #6 0x1030b55fc in rb_vm_exec vm.c:2585 #7 0x1030fe0ac in rb_iseq_eval_main vm.c:2851 #8 0x102a82588 in rb_ec_exec_node eval.c:281 #9 0x102a81fe0 in ruby_run_node eval.c:319 #10 0x1027f3db4 in rb_main main.c:43 #11 0x1027f3bd4 in main main.c:68 #12 0x183900270 (<unknown module>) 0x606000049528 is located 8 bytes inside of 56-byte region [0x606000049520,0x606000049558) freed by thread T0 here: #0 0x104174d40 in free+0x98 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x54d40) #1 0x102ada89c in rb_gc_impl_free default.c:8183 #2 0x102ada7dc in ruby_sized_xfree gc.c:4507 #3 0x102ac4d34 in ruby_xfree gc.c:4518 #4 0x102f3cb34 in rb_st_free_table st.c:663 #5 0x102bd52d8 in remove_from_constant_cache iseq.c:119 #6 0x102bbe2cc in iseq_clear_ic_references iseq.c:153 #7 0x102bbd2a0 in rb_iseq_free iseq.c:166 #8 0x102b32ed0 in rb_imemo_free imemo.c:564 #9 0x102ac4b44 in rb_gc_obj_free gc.c:1407 #10 0x102af4290 in gc_sweep_plane default.c:3546 #11 0x102af3bdc in gc_sweep_page default.c:3634 #12 0x102aeb140 in gc_sweep_step default.c:3906 #13 0x102aeadf0 in gc_sweep_rest default.c:3978 #14 0x102ae4714 in gc_sweep default.c:4155 #15 0x102af8474 in gc_start default.c:6484 #16 0x102afbe30 in garbage_collect default.c:6363 #17 0x102ad37f0 in rb_gc_impl_start default.c:6816 #18 0x102ad3634 in rb_gc gc.c:3624 #19 0x1031406ec in vm_track_constant_cache vm_insnhelper.c:6342 #20 0x1030b91d8 in vm_ic_track_const_chain vm_insnhelper.c:6356 #21 0x1030b8cf8 in rb_vm_opt_getconstant_path vm_insnhelper.c:6424 #22 0x1030bc1e0 in vm_exec_core insns.def:263 #23 0x1030b55fc in rb_vm_exec vm.c:2585 #24 0x1030fe0ac in rb_iseq_eval_main vm.c:2851 #25 0x102a82588 in rb_ec_exec_node eval.c:281 #26 0x102a81fe0 in ruby_run_node eval.c:319 #27 0x1027f3db4 in rb_main main.c:43 #28 0x1027f3bd4 in main main.c:68 #29 0x183900270 (<unknown module>) previously allocated by thread T0 here: #0 0x104174c04 in malloc+0x94 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x54c04) #1 0x102ada0ec in rb_gc_impl_malloc default.c:8198 #2 0x102acee44 in ruby_xmalloc gc.c:4438 #3 0x102f3c85c in rb_st_init_table_with_size st.c:571 #4 0x102f3c900 in rb_st_init_table st.c:600 #5 0x102f3c920 in rb_st_init_numtable st.c:608 #6 0x103140698 in vm_track_constant_cache vm_insnhelper.c:6337 #7 0x1030b91d8 in vm_ic_track_const_chain vm_insnhelper.c:6356 #8 0x1030b8cf8 in rb_vm_opt_getconstant_path vm_insnhelper.c:6424 #9 0x1030bc1e0 in vm_exec_core insns.def:263 #10 0x1030b55fc in rb_vm_exec vm.c:2585 #11 0x1030fe0ac in rb_iseq_eval_main vm.c:2851 #12 0x102a82588 in rb_ec_exec_node eval.c:281 #13 0x102a81fe0 in ruby_run_node eval.c:319 #14 0x1027f3db4 in rb_main main.c:43 #15 0x1027f3bd4 in main main.c:68 #16 0x183900270 (<unknown module>) This commit fixes this bug by adding a inserting_constant_cache_id field to the VM, which stores the ID that is currently being inserted and, in remove_from_constant_cache, we don't free the ST table for ID equal to this one. Co-Authored-By: Alan Wu <alanwu@ruby-lang.org>
2023-12-21Avoid underflow of rb_yjit_live_iseq_countJohn Hawthorn
This value is only incremented when rb_iseq_translate_threaded_code is called, which doesn't happen for iseqs which result in a syntax error. This is easy to hit by running a debug build with RUBY_FREE_AT_EXIT=1, but any build and options could underflow this value by running enough evals.
2023-12-17Adjust styles [ci skip]Nobuyoshi Nakada
2023-12-15Introduce --parser runtime flagHParker
Introduce runtime flag for specifying the parser, ``` ruby --parser=prism ``` also update the description: ``` $ ruby --parser=prism --version ruby 3.3.0dev (2023-12-08T04:47:14Z add-parser-runtime.. 0616384c9f) +PRISM [x86_64-darwin23] ``` [Bug #20044]
2023-12-15[PRISM] Fix `compile_prism` when src is a fileeileencodes
`compile_prism` can take a source and file (and other arguments) or a file as the source. `compile` checks if the source is a file and if it is converts it. `compile_prism` is now doing the same thing. On the Ruby side `compile` handles a file [here](https://github.com/ruby/ruby/blob/master/iseq.c#L1159-L1162). Before: ``` "********* Ruby *************" == disasm: #<ISeq:<compiled>@<compiled>:1 (1,0)-(26,21)> local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1]) [ 1] name@0 0000 putstring "Prism" ( 25)[Li] 0002 setlocal name@0, 0 0005 putself ( 26)[Li] 0006 putobject "hello, " 0008 getlocal name@0, 0 0011 dup 0012 objtostring <calldata!mid:to_s, argc:0, FCALL|ARGS_SIMPLE> 0014 anytostring 0015 concatstrings 2 0017 send <calldata!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, nil 0020 leave hello, Prism "********* PRISM *************" ./test.rb:13:in `compile_prism': wrong argument type File (expected String) (TypeError) from ./test.rb:13:in `<main>' make: *** [run] Error 1 ``` After: ``` "********* Ruby *************" == disasm: #<ISeq:<compiled>@<compiled>:1 (1,0)-(26,21)> local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1]) [ 1] name@0 0000 putstring "Prism" ( 25)[Li] 0002 setlocal name@0, 0 0005 putself ( 26)[Li] 0006 putobject "hello, " 0008 getlocal name@0, 0 0011 dup 0012 objtostring <calldata!mid:to_s, argc:0, FCALL|ARGS_SIMPLE> 0014 anytostring 0015 concatstrings 2 0017 send <calldata!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, nil 0020 leave "********* PRISM *************" == disasm: #<ISeq:<compiled>@test_code.rb:24 (24,0)-(25,21)> local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1]) [ 1] name@0 0000 putstring "Prism" ( 24)[Li] 0002 setlocal name@0, 0 0005 putself ( 25)[Li] 0006 putobject "hello, " 0008 getlocal name@0, 0 0011 dup 0012 objtostring <calldata!mid:to_s, argc:0, FCALL|ARGS_SIMPLE> 0014 anytostring 0015 concatstrings 2 0017 send <calldata!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, nil 0020 leave ( 24) ``` Fixes ruby/prism#1609
2023-12-07Free everything at shutdownAdam Hess
when the RUBY_FREE_ON_SHUTDOWN environment variable is set, manually free memory at shutdown. Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org> Co-authored-by: Peter Zhu <peter@peterzhu.ca>
2023-12-06Adjust styles [ci skip]Nobuyoshi Nakada
2023-12-06Revert "allow enabling Prism via flag or env var"HParker
This reverts commit 9b76c7fc89460ed8e9be40e4037c1d68395c0f6d.
2023-12-06Adjust styles [ci skip]Nobuyoshi Nakada
2023-12-05allow enabling Prism via flag or env varHParker
Enable Prism using either --prism ruby --prism test.rb or via env var RUBY_PRISM=1 ruby test.rb
2023-12-02Pin instruction storagePeter Zhu
The operands in each instruction needs to be pinned because if auto-compaction runs in iseq_set_sequence, then the objects could exist on the generated_iseq buffer, which would not be reference updated which can lead to T_MOVED (and subsequently T_NONE) objects on the iseq.
2023-11-20RubyVM::InstructionSequence.compile_file_prismKevin Newton
* Provide a new API compile_file_prism which mirrors compile_file but uses prism to parse/compile. * Provide the ability to run test-all with RUBY_ISEQ_DUMP_DEBUG set to "prism". If it is, we'll use the new compile_file_prism API to load iseqs during the test run.
2023-11-15Escape and quote non-local variable namesNobuyoshi Nakada
2023-11-03[prism] Update to use new options APIsKevin Newton
2023-10-25[PRISM] Move scope_node itself to CRuby, create prism_compile.hJemma Issroff
2023-10-25[PRISM] Move pm_scope_node_init to prism_compile.cJemma Issroff
pm_scope_node_init is only used for CRuby, so should not live in the ruby/prism repo. We will merge the changes here first so they're not breaking, and will then remove from ruby/prism
2023-10-18Address PR commentsJemma Issroff
2023-10-18Put line change backJemma Issroff
2023-10-18Temporarily removed location code on scope nodesJemma Issroff
2023-10-18Remove pm_compile_context_t, move the context onto ScopeNodeJemma Issroff
We changed ScopeNodes to point to their parent (previous) ScopeNodes. Accordingly, we can remove pm_compile_context_t, and store all necessary context in ScopeNodes, allowing us to access locals from outer scopes.
2023-10-18YJIT: Add a live ISeq counter Alan Wu
It's an estimator for application size and could be used as a compilation heuristic later. Co-authored-by: Maxime Chevalier-Boisvert <maxime.chevalierboisvert@shopify.com> Co-authored-by: Takashi Kokubun <takashikkbn@gmail.com>
2023-10-02[ci skip] More docs for InstructionSequence.compileMatt Valentine-House
This commit documents that you can also pass a `File` object to `RubyVM::InstructionSequence.compile`, instead of a string, and this will behave in a similar way to `RubyVM::InstructionSequence.compile_file` e.g. ``` ❯ ./ruby -e "puts RubyVM::InstructionSequence.compile(File.open('test.rb')).disasm" == disasm: #<ISeq:<compiled>@<compiled>:1 (1,0)-(2,21)> local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1]) [ 1] name@0 0000 putstring "Ruby" ( 1)[Li] 0002 setlocal_WC_0 name@0 0004 putself ( 2)[Li] 0005 putobject "Hello, " 0007 getlocal_WC_0 name@0 0009 dup 0010 objtostring <calldata!mid:to_s, argc:0, FCALL|ARGS_SIMPLE> 0012 anytostring 0013 concatstrings 2 0015 opt_send_without_block <calldata!mid:puts, argc:1, FCALL|ARGS_SIMPLE> 0017 leave ~/git/ruby master* ≡ ⇡ ❯ ./ruby -e "puts RubyVM::InstructionSequence.compile(File.open('test.rb').read).disasm" == disasm: #<ISeq:<compiled>@<compiled>:1 (1,0)-(2,21)> local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1]) [ 1] name@0 0000 putstring "Ruby" ( 1)[Li] 0002 setlocal_WC_0 name@0 0004 putself ( 2)[Li] 0005 putobject "Hello, " 0007 getlocal_WC_0 name@0 0009 dup 0010 objtostring <calldata!mid:to_s, argc:0, FCALL|ARGS_SIMPLE> 0012 anytostring 0013 concatstrings 2 0015 opt_send_without_block <calldata!mid:puts, argc:1, FCALL|ARGS_SIMPLE> 0017 leave ``` This is explicitly allowed by this code path in `rb_iseq_compile_with_option` so we should document it. ``` if (RB_TYPE_P(src, T_FILE)) { parse = rb_parser_compile_file_path; } else { parse = rb_parser_compile_string_path; StringValue(src); } ```
2023-09-27Rename YARP symbols to prismKevin Newton
2023-09-26fix iseq kwargs table and original_iseq leaksAdam Hess
[bug #19903] Co-authored-by: Peter Zhu <peter@peterzhu.ca>
2023-09-01Copy compile options from AST directly without intermediate HashNobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/8349
2023-09-01Copy `rb_compile_option_t` only if neededNobuyoshi Nakada
Use `COMPILE_OPTION_DEFAULT` if nothing to change. Notes: Merged: https://github.com/ruby/ruby/pull/8349
2023-09-01Use macro argument not the variable directlyNobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/8349
2023-08-30Update YARP APIs to handle uint8_tKevin Newton
2023-08-29Compile more YARP node types (#8322)Jemma Issroff
* Add several more node simple types to YARP's compiler: Nodes include: DefinedNode, EmbeddedStatementsNode, LocalVariableReadNode, LocalVariableWriteNode, MultiWriteNode, OptionalParameterNode, SplatNode, YieldNode * Add AssocSplatNode, RangeNode * Add RangeNode, other helpers for future nodes * Add ArrayNode, HashNode, static literal helpers * Add branch conditionals * Add IfNode, UnlessNode * Add ScopeNode * NEW_ISEQ and NEW_CHILD_ISEQ implemented for YARP * Add nodes that depend on ScopeNode * Addressed PR comments
2023-08-29[YARP] Compile basic types (#8311)Jemma Issroff
* Add a compile_context arg to yp_compile_node The compile_context will allow us to pass around the parser, and the constants and lookup table (to be used in future commits). * Compile yp_program_node_t and yp_statements_node_t Add the compilation for program and statements node so that we can successfully compile an empty program with YARP. * Helper functions for parsing numbers, strings, and symbols * Compile basic numeric / boolean node types in YARP * Compile StringNode and SymbolNodes in YARP * Compile several basic node types in YARP * Added error return for missing node Notes: Merged-By: jemmaissroff
2023-08-28Add yarp/yarp_compiler.c (#8042)Jemma Issroff
* Add yarp/yarp_compiler.c as stencil for compiling YARP This commit adds yarp/yarp_compiler.c, and changes the sync script to ensure that yarp/yarp_compiler.c will not get overwritten * [Misc #119772] Create and expose RubyVM::InstructionSequence.compile_yarp This commit creates the stencil for a compile_yarp function, which we will continue to fill out. It allows us to check the output of compiled YARP code against compiled code without using YARP. Notes: Merged-By: jemmaissroff
2023-08-25Move SCRIPT_LINES__ away from parse.yNobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/8289
2023-08-01support `rescue` event for TracePointKoichi Sasada
fix [Feature #19572] Notes: Merged: https://github.com/ruby/ruby/pull/8150
2023-07-28`cc->cme` should not be marked.Ruby
cc is callcache. cc->klass (klass) should not be marked because if the klass is free'ed, the cc->klass will be cleared by `vm_cc_invalidate()`. cc->cme (cme) should not be marked because if cc is invalidated when cme is free'ed. - klass marks cme if klass uses cme. - caller classe's ccs->cme marks cc->cme. - if cc is invalidated (klass doesn't refer the cc), cc is invalidated by `vm_cc_invalidate()` and cc->cme is not be accessed. - On the multi-Ractors, cme will be collected with global GC so that it is safe if GC is not interleaving while accessing cc and cme. fix [Bug #19436] ```ruby 10_000.times{|i| # p i if (i%1_000) == 0 str = "x" * 1_000_000 def str.foo = nil eval "def call#{i}(s) = s.foo" send "call#{i}", str } ``` Without this patch: ``` real 1m5.639s user 0m6.637s sys 0m58.292s ``` and with this patch: ``` real 0m2.045s user 0m1.627s sys 0m0.164s ``` Notes: Merged: https://github.com/ruby/ruby/pull/8120
2023-07-27Clean up OPT_STACK_CACHING (#8132)Takashi Kokubun
Notes: Merged-By: k0kubun <takashikkbn@gmail.com>
2023-06-30Don't check for null pointer in calls to freePeter Zhu
According to the C99 specification section 7.20.3.2 paragraph 2: > If ptr is a null pointer, no action occurs. So we do not need to check that the pointer is a null pointer. Notes: Merged: https://github.com/ruby/ruby/pull/8004
2023-06-17Replace parser & node compile_option from Hash to bit fieldyui-knk
This commit reduces dependency to CRuby object. Notes: Merged: https://github.com/ruby/ruby/pull/7950
2023-06-12[Feature #19719] Universal Parseryui-knk
Introduce Universal Parser mode for the parser. This commit includes these changes: * Introduce `UNIVERSAL_PARSER` macro. All of CRuby related functions are passed via `struct rb_parser_config_struct` when this macro is enabled. * Add CI task with 'cppflags=-DUNIVERSAL_PARSER' for ubuntu. Notes: Merged: https://github.com/ruby/ruby/pull/7927
2023-05-23Move `ruby_node_name` to node.c and rename prefix of the functionyui-knk
Notes: Merged: https://github.com/ruby/ruby/pull/7844
2023-04-11Move `catch_except_p` to `compile_data`eileencodes
The `catch_except_p` flag is used for communicating between parent and child iseq's that a throw instruction was emitted. So for example if a child iseq has a throw in it and the parent wants to catch the throw, we use this flag to communicate to the parent iseq that a throw instruction was emitted. This flag is only useful at compile time, it only impacts the compilation process so it seems to be fine to move it from the iseq body to the compile_data struct. Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org> Notes: Merged: https://github.com/ruby/ruby/pull/7652
2023-04-01Remove unused VM_CALL_BLOCKISEQ flagTakashi Kokubun
2023-03-23`vm_call_single_noarg_inline_builtin`Koichi Sasada
If the iseq only contains `opt_invokebuiltin_delegate_leave` insn and the builtin-function (bf) is inline-able, the caller doesn't need to build a method frame. `vm_call_single_noarg_inline_builtin` is fast path for such cases. Notes: Merged: https://github.com/ruby/ruby/pull/7486
2023-03-07Remove obsoleted functions in rjit.cTakashi Kokubun
2023-03-06s/mjit/rjit/Takashi Kokubun
Notes: Merged: https://github.com/ruby/ruby/pull/7462
2023-03-06s/MJIT/RJIT/Takashi Kokubun
Notes: Merged: https://github.com/ruby/ruby/pull/7462
2023-03-06Stop exporting symbols for MJITTakashi Kokubun
Notes: Merged: https://github.com/ruby/ruby/pull/7459
2023-03-05Store MJIT blocks on each ISEQTakashi Kokubun
Notes: Merged: https://github.com/ruby/ruby/pull/7448
2023-03-05Invalidate everything on GC.compactTakashi Kokubun
Notes: Merged: https://github.com/ruby/ruby/pull/7448
generated by cgit v1.2.3 (git 2.25.1) at 2026-01-18 02:19:44 +0000