summaryrefslogtreecommitdiff
path: root/prism_compile.c
AgeCommit message (Collapse)Author
4 daysAdd pushtoarray insn to fix segfault with forwarding + splatRandy Stauner
Example insns diff for `def x = [3]; def a(...) = b(*x, 2, 3, ...)` == disasm: #<ISeq:a@-e:1 (1,13)-(1,42)> local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1]) [ 1] "..."@0 0000 putself ( 1)[Ca] 0000 putself 0000 opt_send_without_block <calldata!mid:x, argc:0, FCALL|VCALL|ARGS_SIMPLE> 0000 splatarray true 0000 putobject 2 0000 putobject 3 +0000 pushtoarray 2 0000 getlocal_WC_0 "..."@0 0000 sendforward <calldata!mid:b, argc:1, ARGS_SPLAT|ARGS_SPLAT_MUT|FCALL|FORWARDING>, nil 0000 leave [Re] This matches the insns produced by parse.y
2025-12-30[Bug #21784] Fix the Proc#source_location start_column for stabby lambdasBenoit Daloze
* Consistent with plain `blocks` and `for` blocks and methods where the source_location covers their entire definition. * Matches the documentation which mentions "where the definition starts/ends". * Partially reverts d357d50f0a74409446f4cccec78593373f5adf2f which was a workaround to be compatible with parse.y.
2025-12-12Binding#implicit_parameters, etc. support the implicit "it" parameterYusuke Endoh
[Bug #21049]
2025-11-17Remove alternation pattern matching handling from the prism compilerEarlopain
Since https://github.com/ruby/ruby/pull/15212 these are proper syntax errors, so no need to handle this explicitly anymore. Also updated the example in the docs for this
2025-10-27Correctly compile splats in for-loop index in prismEarlopain
Fixes [Bug #21648] This is a followup to https://github.com/ruby/ruby/pull/13597. The added test passed but didn't emit the same instructions. This also handles bare splats and aligns instructions for all cases
2025-10-23use `SET_SHAREABLE`Koichi Sasada
to adopt strict shareable rule. * (basically) shareable objects only refer shareable objects * (exception) shareable objects can refere unshareable objects but should not leak reference to unshareable objects to Ruby world
2025-10-11For prism parser, do not update $_ from STDINKevin Newton
Fixes [Bug #21635]
2025-10-01Interpolation with only string literals must not be frozenEarlopain
Basically a redo of https://github.com/ruby/ruby/commit/a1403fb7cbd1fe0df97c932be9814c86081783dc but respecting the frozen string literal magic comment Fixes [Bug #21187]
2025-09-24Ractor.shareable_procKoichi Sasada
call-seq: Ractor.sharable_proc(self: nil){} -> sharable proc It returns shareable Proc object. The Proc object is shareable and the self in a block will be replaced with the value passed via `self:` keyword. In a shareable Proc, the outer variables should * (1) refer shareable objects * (2) be not be overwritten ```ruby a = 42 Ractor.shareable_proc{ p a } #=> OK b = 43 Ractor.shareable_proc{ p b; b = 44 } #=> Ractor::IsolationError because 'b' is reassigned in the block. c = 44 Ractor.shareable_proc{ p c } #=> Ractor::IsolationError because 'c' will be reassigned outside of the block. c = 45 d = 45 d = 46 if cond Ractor.shareable_proc{ p d } #=> Ractor::IsolationError because 'd' was reassigned outside of the block. ``` The last `d`'s case can be relaxed in a future version. The above check will be done in a static analysis at compile time, so the reflection feature such as `Binding#local_varaible_set` can not be detected. ```ruby e = 42 shpr = Ractor.shareable_proc{ p e } #=> OK binding.local_variable_set(:e, 43) shpr.call #=> 42 (returns captured timing value) ``` Ractor.sharaeble_lambda is also introduced. [Feature #21550] [Feature #21557]
2025-09-13Fix prism error messages with multibyte truncationKevin Newton
When a line is going to be displayed in an error message that contains multibyte characters, we need to respect the encoding of the source and truncate only at a character boundary, as opposed to a raw byte boundary. Fixes [Bug #21528]
2025-09-13Fill in lead num for blocks with `it`Kevin Newton
Fixes [Bug #21256] Co-Authored-By: Earlopain <14981592+Earlopain@users.noreply.github.com>
2025-09-13Use API version for syntax version instead of program versionNobuyoshi Nakada
2025-09-12Explicitly use a ruby version for prism to parse the code as (#14523)Earlopain
Prism can parse multiple versions of ruby. Because of that branch release managers are ok with simply bumping prism to its latest version. However, if no version is specified, it will parse as the latest known version, which can be ahead of the maintenance branch. So we need to explicitly pass a version to not accidentally introduce new syntax to maintenance branches.
2025-08-26Remove `opt_aref_with` and `opt_aset_with`Aaron Patterson
When these instructions were introduced it was common to read from a hash with mutable string literals. However, these days, I think these instructions are fairly rare. I tested this with the lobsters benchmark, and saw no difference in speed. In order to be sure, I tracked down every use of this instruction in the lobsters benchmark, and there were only 4 places where it was used. Additionally, this patch fixes a case where "chilled strings" should emit a warning but they don't. ```ruby class Foo def self.[](x)= x.gsub!(/hello/, "hi") end Foo["hello world"] ``` Removing these instructions shows this warning: ``` > ./miniruby -vw test.rb ruby 3.5.0dev (2025-08-25T21:36:50Z rm-opt_aref_with dca08e286c) +PRISM [arm64-darwin24] test.rb:2: warning: literal string will be frozen in the future (run with --debug-frozen-string-literal for more information) ``` [Feature #21553]
2025-08-04When reading from stdin, put a wrapper around the IO objectAaron Patterson
The purpose of this commit is to fix Bug #21188. We need to detect when stdin has run in to an EOF case. Unfortunately we can't _call_ the eof function on IO because it will block. Here is a short script to demonstrate the issue: ```ruby x = STDIN.gets puts x puts x.eof? ``` If you run the script, then type some characters (but _NOT_ a newline), then hit Ctrl-D twice, it will print the input string. Unfortunately, calling `eof?` will try to read from STDIN again causing us to need a 3rd Ctrl-D to exit the program. Before introducing the EOF callback to Prism, the input loop looked kind of like this: ```ruby loop do str = STDIN.gets process(str) if str.nil? p :DONE end end ``` Which required 3 Ctrl-D to exit. If we naively changed it to something like this: ```ruby loop do str = STDIN.gets process(str) if STDIN.eof? p :DONE end end ``` It would still require 3 Ctrl-D because `eof?` would block. In this patch, we're wrapping the IO object, checking the buffer for a newline and length, and then using that to simulate a non-blocking eof? method. This commit wraps STDIN and emulates a non-blocking `eof` function. [Bug #21188]
2025-07-22Interpolated strings must not be frozenAaron Patterson
Strings concatenated with backslash may end up being frozen when they shouldn't be. This commit fixes the issue. It required a change upstream in Prism, but also a change to the Prism compiler in CRuby. https://github.com/ruby/prism/pull/3606 [Bug #21187]
2025-07-18Revert "[Bug #21256] Fix `it` parameter when splatting and `define_method` ↵Yusuke Endoh
is…" This reverts commit 265059603c3aa6a13f90096c71b32046a17938f3.
2025-07-17Fix compilation for forwarding params in PrismAaron Patterson
[Bug #21326]
2025-07-17Fix flipflop line numbersAaron Patterson
[ruby-core:121605]
2025-07-17[Bug #21256] Fix `it` parameter when splatting and `define_method` is usedEarlopain
It was failing to set the leads, like numblocks do, causing the result to be wrapped in an array
2025-07-17Fix linked list iteration when displaying errorsAaron Patterson
When a script has problem with the magic comment encoding, we only display that error. However, if there are other syntax errors in the file, the error linked list could contain multiple items. This lead to an inconsistency in the "size" field of the linked list, and the actual items in the linked list. In other words, the linked list had more than one item, but the size field was one. The error display routine would only allocate `size` items, but iterating the linked list would overrun the array. This commit changes the iterator to compare the current node to the "finish" node in the linked list, no longer assuming the linked list ends with NULL. [Bug #21461]
2025-06-22Avoid allocation for positional splat for literal array keyword argumentJeremy Evans
If all nodes in the array are safe, then it is safe to avoid allocation for the positional splat: ```ruby m(*a, kw: [:a]) # Safe m(*a, kw: [meth]) # Unsafe ``` This avoids an unnecessary allocation in a Rails method call. Details: https://github.com/rails/rails/pull/54949/files#r2052645431
2025-06-21Fix handling of PM_CONSTANT_PATH_NODE node in keyword arguments with ARGS_SPLATJeremy Evans
This was handled correctly in parse.y (NODE_COLON2), but not in prism. This wasn't caught earlier, because I only added tests for the optimized case and not the unoptimized case. Add tests for the unoptimized case. In code terms: ```ruby m(*a, kw: lvar::X) # Does not require allocation for *a m(*a, kw: method()::X) # Requires allocation for *a ``` This commit fixes the second case when prism is used.
2025-06-18Fix a missing write barrier to mandatory_only_iseqJohn Hawthorn
Found by wbcheck Notes: Merged: https://github.com/ruby/ruby/pull/13646
2025-06-12[Bug #21439] Fix `PM_SPLAT_NODE` compilation error in for loops (#13597)Ufuk Kayserilioglu
[Bug #21439] Fix PM_SPLAT_NODE compilation error in for loops This commit fixes a crash that occurred when using splat nodes (*) as the index variable in for loops. The error "Unexpected node type for index in for node: PM_SPLAT_NODE" was thrown because the compiler didn't know how to handle splat nodes in this context. The fix allows code like `for *x in [[1,2], [3,4]]` to compile and execute correctly, where the splat collects each sub-array. Notes: Merged-By: eileencodes <eileencodes@gmail.com>
2025-05-29Fix memory leak with invalid yield in prismPeter Zhu
[Bug #21383] The following script leaks memory: 10.times do 20_000.times do eval("class C; yield; end") rescue SyntaxError end puts `ps -o rss= -p #{$$}` end Before: 16464 25536 29424 35904 39552 44576 46736 51600 56096 59824 After: 13488 16160 18240 20528 19760 21808 21680 22272 22064 22336 Notes: Merged: https://github.com/ruby/ruby/pull/13464
2025-05-17[Bug #21313] Handle `it` in rescue and ensure blocks.Nick Dower
The following is crashing for me: ```shell ruby --yjit --yjit-call-threshold=1 -e '1.tap { raise rescue p it }' ruby: YJIT has panicked. More info to follow... thread '<unnamed>' panicked at ./yjit/src/codegen.rs:2402:14: ... ``` It seems `it` sometimes points to the wrong value: ```shell ruby -e '1.tap { raise rescue p it }' false ruby -e '1.tap { begin; raise; ensure; p it; end } rescue nil' false ``` But only when `$!` is set: ```shell ruby -e '1.tap { begin; nil; ensure; p it; end }' 1 ruby -e '1.tap { begin; nil; rescue; ensure; p it; end }' 1 ruby -e '1.tap { begin; raise; rescue; ensure; p it; end }' 1 ``` Notes: Merged: https://github.com/ruby/ruby/pull/13360
2025-04-29opt_new needs to happen after safe navigationAaron Patterson
If safe navigation instructions happen first, we get a stack inconsistency error. Notes: Merged: https://github.com/ruby/ruby/pull/13205
2025-04-29Don't support blockarg in opt_newMax Bernstein
We don't calculate the correct argc so the bookkeeping slot is something else (unexpected) instead of Qnil (expected). Notes: Merged: https://github.com/ruby/ruby/pull/13198
2025-04-25Inline Class#new.Aaron Patterson
This commit inlines instructions for Class#new. To make this work, we added a new YARV instructions, `opt_new`. `opt_new` checks whether or not the `new` method is the default allocator method. If it is, it allocates the object, and pushes the instance on the stack. If not, the instruction jumps to the "slow path" method call instructions. Old instructions: ``` > ruby --dump=insns -e'Object.new' == disasm: #<ISeq:<main>@-e:1 (1,0)-(1,10)> 0000 opt_getconstant_path <ic:0 Object> ( 1)[Li] 0002 opt_send_without_block <calldata!mid:new, argc:0, ARGS_SIMPLE> 0004 leave ``` New instructions: ``` > ./miniruby --dump=insns -e'Object.new' == disasm: #<ISeq:<main>@-e:1 (1,0)-(1,10)> 0000 opt_getconstant_path <ic:0 Object> ( 1)[Li] 0002 putnil 0003 swap 0004 opt_new <calldata!mid:new, argc:0, ARGS_SIMPLE>, 11 0007 opt_send_without_block <calldata!mid:initialize, argc:0, FCALL|ARGS_SIMPLE> 0009 jump 14 0011 opt_send_without_block <calldata!mid:new, argc:0, ARGS_SIMPLE> 0013 swap 0014 pop 0015 leave ``` This commit speeds up basic object allocation (`Foo.new`) by 60%, but classes that take keyword parameters see an even bigger benefit because no hash is allocated when instantiating the object (3x to 6x faster). Here is an example that uses `Hash.new(capacity: 0)`: ``` > hyperfine "ruby --disable-gems -e'i = 0; while i < 10_000_000; Hash.new(capacity: 0); i += 1; end'" "./ruby --disable-gems -e'i = 0; while i < 10_000_000; Hash.new(capacity: 0); i += 1; end'" Benchmark 1: ruby --disable-gems -e'i = 0; while i < 10_000_000; Hash.new(capacity: 0); i += 1; end' Time (mean ± σ): 1.082 s ± 0.004 s [User: 1.074 s, System: 0.008 s] Range (min … max): 1.076 s … 1.088 s 10 runs Benchmark 2: ./ruby --disable-gems -e'i = 0; while i < 10_000_000; Hash.new(capacity: 0); i += 1; end' Time (mean ± σ): 627.9 ms ± 3.5 ms [User: 622.7 ms, System: 4.8 ms] Range (min … max): 622.7 ms … 633.2 ms 10 runs Summary ./ruby --disable-gems -e'i = 0; while i < 10_000_000; Hash.new(capacity: 0); i += 1; end' ran 1.72 ± 0.01 times faster than ruby --disable-gems -e'i = 0; while i < 10_000_000; Hash.new(capacity: 0); i += 1; end' ``` This commit changes the backtrace for `initialize`: ``` aaron@tc ~/g/ruby (inline-new)> cat test.rb class Foo def initialize puts caller end end def hello Foo.new end hello aaron@tc ~/g/ruby (inline-new)> ruby -v test.rb ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +PRISM [arm64-darwin24] test.rb:8:in 'Class#new' test.rb:8:in 'Object#hello' test.rb:11:in '<main>' aaron@tc ~/g/ruby (inline-new)> ./miniruby -v test.rb ruby 3.5.0dev (2025-03-28T23:59:40Z inline-new c4157884e4) +PRISM [arm64-darwin24] test.rb:8:in 'Object#hello' test.rb:11:in '<main>' ``` It also increases memory usage for calls to `new` by 122 bytes: ``` aaron@tc ~/g/ruby (inline-new)> cat test.rb require "objspace" class Foo def initialize puts caller end end def hello Foo.new end puts ObjectSpace.memsize_of(RubyVM::InstructionSequence.of(method(:hello))) aaron@tc ~/g/ruby (inline-new)> make runruby RUBY_ON_BUG='gdb -x ./.gdbinit -p' ./miniruby -I./lib -I. -I.ext/common ./tool/runruby.rb --extout=.ext -- --disable-gems ./test.rb 656 aaron@tc ~/g/ruby (inline-new)> ruby -v test.rb ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +PRISM [arm64-darwin24] 544 ``` Thanks to @ko1 for coming up with this idea! Co-Authored-By: John Hawthorn <john@hawthorn.email>
2025-04-09Fix coverage measurement for negative line numbersYusuke Endoh
Fixes [Bug #21220] Co-Authored-By: Mike Bourgeous <mike@mikebourgeous.com> Co-Authored-By: Jean Boussier <jean.boussier@gmail.com> Notes: Merged: https://github.com/ruby/ruby/pull/13089
2025-04-07prism_compile.c: Avoid zero length allocationJean Boussier
The constant pool may be empty. Notes: Merged: https://github.com/ruby/ruby/pull/13068
2025-03-18Handle void expressions in defined?Kevin Newton
[Bug #21029] Notes: Merged: https://github.com/ruby/ruby/pull/12949
2025-01-15Align defined? implementations between parsers (#12584)Kevin Newton
Fixes [Bug #21043] Notes: Merged-By: kddnewton <kddnewton@gmail.com>
2025-01-14[PRISM] Handle forwarding inside evalKevin Newton
Fixes [Bug #21031] Notes: Merged: https://github.com/ruby/ruby/pull/12575
2025-01-07Correctly set node_id on iseq locationAaron Patterson
The iseq location object has a slot for node ids. parse.y was correctly populating that field but Prism was not. This commit populates the field with the ast node id for that iseq [Bug #21014] Notes: Merged: https://github.com/ruby/ruby/pull/12527
2025-01-07[Bug #21006] Fix defined_expr compilation of method call with parenth… ↵tomoya ishida
(#12518) [Bug #21006] Fix defined_expr compilation of method call with parenthesized receiver
2025-01-06Allow escaping from ensures through nextKevin Newton
Fixes [Bug #21001] Notes: Merged: https://github.com/ruby/ruby/pull/12513
2025-01-05Do not warn unused block when using forwardingKevin Newton
Fixes [Bug #21003] Notes: Merged: https://github.com/ruby/ruby/pull/12512
2024-12-26Handle defined? with call chains with blocksKevin Newton
Ensures we can handle expressions like `defined?(a {}.b)`. Notes: Merged: https://github.com/ruby/ruby/pull/12475
2024-12-23Revert "[Bug #20965] Define `it` like an ordinary argument" (#12418)Takashi Kokubun
Revert "[Bug #20965] Define `it` like an ordinary argument (#12398)" Reverts ruby/ruby#12398 as per https://bugs.ruby-lang.org/issues/20970#note-6 and https://bugs.ruby-lang.org/issues/20965#note-7. We need more time to design the intended behavior, and it's too late for Ruby 3.4. Notes: Merged-By: k0kubun <takashikkbn@gmail.com>
2024-12-20Provide Ractor support for **Kevin Newton
Fixes [Bug #20916] Notes: Merged: https://github.com/ruby/ruby/pull/12417
2024-12-20[PRISM] Treat it as a local when compiling patternsMatt Valentine-House
Fixes [Bug #20973] Notes: Merged: https://github.com/ruby/ruby/pull/12408
2024-12-20[PRISM] Fix compiling popped opt_str_uminus and opt_str_freezeKazuki Yamaguchi
Put a pop as needed. This example currently causes [BUG]: $ ruby --parser=prism -e'1.times{"".freeze;nil}' -e:1: [BUG] Stack consistency error (sp: 15, bp: 14) ruby 3.4.0dev (2024-12-20T00:48:01Z master 978df259ca) +PRISM [x86_64-linux] Notes: Merged: https://github.com/ruby/ruby/pull/12410
2024-12-18[Bug #20965] Define `it` like an ordinary argument (#12398)Nobuyoshi Nakada
Also fixes [Bug #20955] Notes: Merged-By: k0kubun <takashikkbn@gmail.com>
2024-12-17[PRISM] Recurse use_deconstructed_cache in Alternation NodesMatt Valentine-House
This fixes the behavioural difference between Prism and parse.y when evaluating the following code ```ruby 1 in [1 | [1]] ``` Fixes [Bug #20956] Notes: Merged: https://github.com/ruby/ruby/pull/12370
2024-12-15[Bug #20940] [PRISM] Support NO_COLORNobuyoshi Nakada
Also use bold/faint SGR when possible. Notes: Merged: https://github.com/ruby/ruby/pull/12329
2024-12-12Fix error messages so we don't output an extra lineAaron Patterson
Before this commit, when a file ended with a newline, the syntax error message would show an extra line after the file. For example, the error message would look like this: ``` [aaron@tc-lan-adapter ~/g/ruby (master)]$ echo "foo(" > test.rb [aaron@tc-lan-adapter ~/g/ruby (master)]$ od -c test.rb 0000000 f o o ( \n 0000005 [aaron@tc-lan-adapter ~/g/ruby (master)]$ wc -l test.rb 1 test.rb [aaron@tc-lan-adapter ~/g/ruby (master)]$ ./miniruby test.rb test.rb: test.rb:1: syntax error found (SyntaxError) > 1 | foo( | ^ unexpected end-of-input; expected a `)` to close the arguments 2 | ``` This commit fixes the "end of line" book keeping when printing an error so that there is no extra line output at the end of the file: ``` [aaron@tc-lan-adapter ~/g/ruby (fix-last-line-error)]$ echo "foo(" | ./miniruby -: -:1: syntax error found (SyntaxError) > 1 | foo( | ^ unexpected end-of-input; expected a `)` to close the arguments [aaron@tc-lan-adapter ~/g/ruby (fix-last-line-error)]$ echo -n "foo(" | ./miniruby -: -:1: syntax error found (SyntaxError) > 1 | foo( | ^ unexpected end-of-input; expected a `)` to close the arguments ``` Notice that in the above example, the error message only displays one line regardless of whether or not the file ended with a newline. [Bug #20918] [ruby-core:120035] Notes: Merged: https://github.com/ruby/ruby/pull/12324
2024-12-11Use malloc for prism string sourceJohn Hawthorn
Prism will later free this string via free rather than xfree, so we need to use malloc rather than xmalloc. Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org> Co-authored-by: Matthew Draper <matthew@trebex.net> Notes: Merged: https://github.com/ruby/ruby/pull/12312
2024-11-28`INIT_ANCHOR` no longer needed usuallyNobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/12198
generated by cgit v1.2.3 (git 2.25.1) at 2026-01-17 11:28:58 +0000