| Age | Commit message (Collapse) | Author |
|---|
|
[PATCH] YJIT: Bail out if proc would be stored above stack top
Fixes [Bug #21266].
|
|
Credits to @rwstauner for noticing this issue in GH-15533.
|
|
Previously, the chain_depth>0 version of setlocal blocks did not
update the type of the local variable in the context. This can leave
the context with stale type information and trigger or lead to miscompilation.
To trigger the issue, YJIT needs to see the same ISEQ before and after
environment escape and have tracked type info before the escape. To
trigger in ISEQs that do not send with a block, it probably requires
Kernel#binding or the use of include/ruby/debug.h APIs.
[Backport #21772]
|
|
Fixes: [Bug #21707]
[AW: rewrote comments]
Co-authored-by: Alan Wu <alanwu@ruby-lang.org>
|
|
Previously because we did a stack_push before ccall, in some cases we
could end up pushing an uninitialized value to the VM stack when
spilling regs as part of the ccall.
Co-authored-by: Luke Gruber <luke.gru@gmail.com>
|
|
Previously, YJIT returned truthy for the block given query at the top
level. That's incorrect because the top level script never receives a
block, and `yield` is a syntax error there.
Inside methods, the number of hops to get from `iseq` to
`iseq->body->local_iseq` is the same as the number of
`VM_ENV_PREV_EP(ep)` hops to get to an environment with
`VM_ENV_FLAG_LOCAL`. YJIT and the interpreter both rely on this as can
be seen in get_lvar_level(). However, this identity does not hold for
the top level frame because of `vm_set_eval_stack()`, which sets up
`TOPLEVEL_BINDING`.
Since only methods can take a block that `yield` goes to, have ISEQs
that are the child of a non-method ISEQ return falsy for the block given
query. This fixes the issue for the top level script and is an
optimization for non-method contexts such as inside `ISEQ_TYPE_CLASS`.
|
|
These `#[cfg(feature = "disasm")]` were unnecessary and we can provide
the information like ruby source location regardless of the availability
of capstone.
|
|
|
|
YJIT: Add more information to an assert message in jit_guard_known_class (#14480)
---------
Co-authored-by: Alan Wu <XrXr@users.noreply.github.com>
|
|
Backport of 7e733ca55168e3b1f10b685f6e9a52cf1deb5aff to
fix [Bug #21514].
|
|
|
|
|
|
Evident with the crash reported in [Bug #20997], the C replacement
codegen functions aren't authored to handle block arguments (nor
should they because the extra code from the complexity defeats
optimization). Filter sites with VM_CALL_ARGS_BLOCKARG.
|
|
Backport of GH-12660:
Previously, callers of forwardable ISeqs moved the stack pointer up
without writing to the stack. If there happens to be a stale value in
the area skipped over, it could crash due to "try to mark T_NONE". Also,
the uninitialized local variables were observable through `binding`.
Initialize the locals to nil.
[Bug #21021]
|
|
Previously, the code for dropping surplus arguments when yielding
into blocks erroneously attempted to drop keyword arguments when there
is in fact no surplus arguments. Fix the condition and test that
supplying the exact number of keyword arguments as require compiles
without fallback.
|
|
Notes:
Merged-By: maximecb <maximecb@ruby-lang.org>
|
|
A good amount of call sites always pass nil as block argument, but the
nil doesn't show up in the context. Put a runtime guard for those
cases to handle it. Particular relevant for the `ruby-lsp` benchmark in
`yjit-bench`. Up to a 2% speedup across headline benchmarks.
Co-authored-by: Takashi Kokubun <takashikkbn@gmail.com>
Co-authored-by: Maxime Chevalier-Boisvert <maxime.chevalierboisvert@shopify.com>
Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
Co-authored-by: Kevin Menard <kevin@nirvdrum.com>
Co-authored-by: Randy Stauner <randy.stauner@shopify.com>
Notes:
Merged-By: maximecb <maximecb@ruby-lang.org>
|
|
jit_prepare_lazy_frame_call is a complicated trick and comes with memory
overhead. Every use of the function should come with justification.
|
|
* YJIT: Spill/load argument registers to reuse blocks
* Mention the immediate function name
* Explain the context behind spill/load operations
Notes:
Merged-By: k0kubun <takashikkbn@gmail.com>
|
|
* YJIT: Generate specialized code for Symbol for objtostring
Co-authored-by: John Hawthorn <john@hawthorn.email>
* Update yjit/src/codegen.rs
---------
Co-authored-by: John Hawthorn <john@hawthorn.email>
Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
Notes:
Merged-By: maximecb <maximecb@ruby-lang.org>
|
|
Fewer allocations on boot, too.
Suggested-by: https://github.com/ruby/ruby/pull/12217
Notes:
Merged: https://github.com/ruby/ruby/pull/12220
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/12202
|
|
Notes:
Merged-By: maximecb <maximecb@ruby-lang.org>
|
|
* Add opt_duparray_send insn to skip the allocation on `#include?`
If the method isn't going to modify the array we don't need to copy it.
This avoids the allocation / array copy for things like `[:a, :b].include?(x)`.
This adds a BOP for include? and tracks redefinition for it on Array.
Co-authored-by: Andrew Novoselac <andrew.novoselac@shopify.com>
* YJIT: Implement opt_duparray_send include_p
Co-authored-by: Andrew Novoselac <andrew.novoselac@shopify.com>
* Update opt_newarray_send to support simple forms of include?(arg)
Similar to opt_duparray_send but for non-static arrays.
* YJIT: Implement opt_newarray_send include_p
---------
Co-authored-by: Andrew Novoselac <andrew.novoselac@shopify.com>
Notes:
Merged-By: maximecb <maximecb@ruby-lang.org>
|
|
It's more concise this way and since `return Some(EndBlock)` is the only
correct answer, no point repeating it everywhere.
Notes:
Merged: https://github.com/ruby/ruby/pull/12124
|
|
When CodeBlock::set_page fails (part of next_page(), see their docs for
exact conditions), it can cause gen_outlined_exit() to fail while there
is still plenty of memory available. Previously, this can have YJIT
running incomplete code due to taking the early return in
end_block_with_jump() that manifested as crashes with SIGILL.
Add and use a wrapper with error handling.
Notes:
Merged: https://github.com/ruby/ruby/pull/12124
|
|
Notes:
Merged-By: maximecb <maximecb@ruby-lang.org>
|
|
Notes:
Merged-By: maximecb <maximecb@ruby-lang.org>
|
|
* YJIT: Specialize `String#[]` (`String#slice`) with fixnum arguments
String#[] is in the top few C calls of several YJIT benchmarks:
liquid-compile rubocop mail sudoku
This speeds up these benchmarks by 1-2%.
* YJIT: Try harder to get type info for `String#[]`
In the large generated code of the mail gem the context doesn't have
the type info. In that case if we peek at the stack and add a guard
we can still apply the specialization
and it speeds up the mail benchmark by 5%.
Co-authored-by: Maxime Chevalier-Boisvert <maxime.chevalierboisvert@shopify.com>
Co-authored-by: Takashi Kokubun (k0kubun) <takashikkbn@gmail.com>
---------
Co-authored-by: Maxime Chevalier-Boisvert <maxime.chevalierboisvert@shopify.com>
Co-authored-by: Takashi Kokubun (k0kubun) <takashikkbn@gmail.com>
Notes:
Merged-By: maximecb <maximecb@ruby-lang.org>
|
|
When we run with RUBY_FREE_AT_EXIT, there's a false-positive memory leak
reported in YJIT because the METHOD_CODEGEN_TABLE is never freed. This
commit adds rb_yjit_free_at_exit that is called at shutdown when
RUBY_FREE_AT_EXIT is set.
Reported memory leak:
==699816== 1,104 bytes in 1 blocks are possibly lost in loss record 1 of 1
==699816== at 0x484680F: malloc (vg_replace_malloc.c:446)
==699816== by 0x155B3E: UnknownInlinedFun (unix.rs:14)
==699816== by 0x155B3E: UnknownInlinedFun (stats.rs:36)
==699816== by 0x155B3E: UnknownInlinedFun (stats.rs:27)
==699816== by 0x155B3E: alloc (alloc.rs:98)
==699816== by 0x155B3E: alloc_impl (alloc.rs:181)
==699816== by 0x155B3E: allocate (alloc.rs:241)
==699816== by 0x155B3E: do_alloc<alloc::alloc::Global> (alloc.rs:15)
==699816== by 0x155B3E: new_uninitialized<alloc::alloc::Global> (mod.rs:1750)
==699816== by 0x155B3E: fallible_with_capacity<alloc::alloc::Global> (mod.rs:1788)
==699816== by 0x155B3E: prepare_resize<alloc::alloc::Global> (mod.rs:2864)
==699816== by 0x155B3E: resize_inner<alloc::alloc::Global> (mod.rs:3060)
==699816== by 0x155B3E: reserve_rehash_inner<alloc::alloc::Global> (mod.rs:2950)
==699816== by 0x155B3E: hashbrown::raw::RawTable<T,A>::reserve_rehash (mod.rs:1231)
==699816== by 0x5BC39F: UnknownInlinedFun (mod.rs:1179)
==699816== by 0x5BC39F: find_or_find_insert_slot<(usize, fn(&mut yjit::codegen::JITState, &mut yjit::backend::ir::Assembler, *const yjit::cruby::autogened::rb_callinfo, *const yjit::cruby::autogened::rb_callable_method_entry_struct, core::option::Option<yjit::codegen::BlockHandler>, i32, core::option::Option<yjit::cruby::VALUE>) -> bool), alloc::alloc::Global, hashbrown::map::equivalent_key::{closure_env#0}<usize, usize, fn(&mut yjit::codegen::JITState, &mut yjit::backend::ir::Assembler, *const yjit::cruby::autogened::rb_callinfo, *const yjit::cruby::autogened::rb_callable_method_entry_struct, core::option::Option<yjit::codegen::BlockHandler>, i32, core::option::Option<yjit::cruby::VALUE>) -> bool>, hashbrown::map::make_hasher::{closure_env#0}<usize, fn(&mut yjit::codegen::JITState, &mut yjit::backend::ir::Assembler, *const yjit::cruby::autogened::rb_callinfo, *const yjit::cruby::autogened::rb_callable_method_entry_struct, core::option::Option<yjit::codegen::BlockHandler>, i32, core::option::Option<yjit::cruby::VALUE>) -> bool, std::hash::random::RandomState>> (mod.rs:1413)
==699816== by 0x5BC39F: hashbrown::map::HashMap<K,V,S,A>::insert (map.rs:1754)
==699816== by 0x57C5C6: insert<usize, fn(&mut yjit::codegen::JITState, &mut yjit::backend::ir::Assembler, *const yjit::cruby::autogened::rb_callinfo, *const yjit::cruby::autogened::rb_callable_method_entry_struct, core::option::Option<yjit::codegen::BlockHandler>, i32, core::option::Option<yjit::cruby::VALUE>) -> bool, std::hash::random::RandomState> (map.rs:1104)
==699816== by 0x57C5C6: yjit::codegen::reg_method_codegen (codegen.rs:10521)
==699816== by 0x57C295: yjit::codegen::yjit_reg_method_codegen_fns (codegen.rs:10464)
==699816== by 0x5C6B07: rb_yjit_init (yjit.rs:40)
==699816== by 0x393723: ruby_opt_init (ruby.c:1820)
==699816== by 0x393723: ruby_opt_init (ruby.c:1767)
==699816== by 0x3957D4: prism_script (ruby.c:2215)
==699816== by 0x3957D4: process_options (ruby.c:2538)
==699816== by 0x396065: ruby_process_options (ruby.c:3166)
==699816== by 0x236E56: ruby_options (eval.c:117)
==699816== by 0x15BAED: rb_main (main.c:43)
==699816== by 0x15BAED: main (main.c:62)
After this patch, there are no more memory leaks reported when running
RUBY_FREE_AT_EXIT with Valgrind on an empty Ruby script:
$ RUBY_FREE_AT_EXIT=1 valgrind --leak-check=full ruby -e ""
...
==700357== HEAP SUMMARY:
==700357== in use at exit: 0 bytes in 0 blocks
==700357== total heap usage: 36,559 allocs, 36,559 frees, 6,064,783 bytes allocated
==700357==
==700357== All heap blocks were freed -- no leaks are possible
Notes:
Merged-By: maximecb <maximecb@ruby-lang.org>
|
|
In [1], we started checking for gen_branch failures, but I made two
crucial mistakes. One, defer_compilation() had the same issue as
gen_branch() but wasn't checked. Two, returning None from a codegen
function does not throw away the block. Checking how gen_single_block()
handles codegen functions, you can see that None terminates the block
with an exit, but does not overall return an Err. This handling is fine
for unimplemented instructions, for example, but incorrect in case
gen_branch() fails. The missing branch essentially corrupts the
block; adding more code after a missing branch doesn't correct the code.
Always abandon the block when defer_compilation() or gen_branch() fails.
[1]: cb661d7d82984cdb54485ea3f4af01ac21960882
Fixup: [1]
Notes:
Merged: https://github.com/ruby/ruby/pull/12035
Merged-By: XrXr
|
|
We got some core dumps in the wild where a PendingBranch had everything
as None, leading to a panic unwrapping in PendingBranch::into_branch().
This happened while compiling a `branchif`.
It seems that the only way this can happen is when core::gen_branch()
fails, but not due to OOM. We wouldn't have reach into_branch() when
OOM, and the only way to not leave markers that would've set the
branch's start_addr to some value in gen_branch() is for set_target() to
fail, causing an early return.
Unfortunately, it's hard to tell the exact sequence of events that led
to this situation, but regardless, the dumps show us that we should
check for errors in gen_branch().
Because gen_branch() is used deep in the stack during compilation (e.g.
guard_known_class() -> jit_chain_guard() -> gen_branch()), it'd be bad
for compile speed to propagate the error everywhere, not to mention the
massive patch required. Opt for a flag checked near the end of
compilation.
Notes:
Merged: https://github.com/ruby/ruby/pull/11938
Merged-By: XrXr
|
|
Notes:
Merged-By: k0kubun <takashikkbn@gmail.com>
|
|
Type information in the context for no additional work!
This is the `if (special_object_p(obj)) return obj;` path in
rb_obj_dup() and for Numeric#dup, it's always the identity function.
Notes:
Merged: https://github.com/ruby/ruby/pull/11926
|
|
* YJIT: Fill in commented-out assertion
* YJIT: Rename yjit_reg_method() and add links in docs
Notes:
Merged-By: maximecb <maximecb@ruby-lang.org>
|
|
* Update yjit-bindgen deps
* YJIT: Allow shareable consts in multi-ractor mode
* Update yjit/src/codegen.rs
Co-authored-by: Alan Wu <XrXr@users.noreply.github.com>
---------
Co-authored-by: Alan Wu <XrXr@users.noreply.github.com>
Notes:
Merged-By: maximecb <maximecb@ruby-lang.org>
|
|
Previously, in the "Top-N most frequent C calls"
section of --yjit-stats output, we printed the class
name of the receiver, not the method owner. This meant
that calls on subclass instances that land on the same
method showed up as different entires.
Similarly, method called using an alias showed up as
different entries from other aliases.
Group by the resolved method instead.
Test program:
1.itself; [].itself; true.inspect; true.to_s
Before:
Top-4 most frequent C calls (80.0% of C calls):
1 (20.0%): Integer#itself
1 (20.0%): TrueClass#to_s
1 (20.0%): TrueClass#inspect
1 (20.0%): Array#itself
After:
Top-2 most frequent C calls (80.0% of C calls):
2 (40.0%): Kernel#itself
2 (40.0%): TrueClass#to_s
Notes:
Merged: https://github.com/ruby/ruby/pull/11913
|
|
* YJIT: Add `--yjit-compilation-log` flag to print out the compilation log at exit.
* YJIT: Add an option to enable the compilation log at runtime.
* YJIT: Fix a typo in the `IseqPayload` docs.
* YJIT: Add stubs for getting the YJIT compilation log in memory.
* YJIT: Add a compilation log based on a circular buffer to cap the log size.
* YJIT: Allow specifying either a file or directory name for the YJIT compilation log.
The compilation log will be populated as compilation events occur. If a directory is supplied, then a filename based on the PID will be used as the write target. If a file name is supplied instead, the log will be written to that file.
* YJIT: Add JIT compilation of C function substitutions to the compilation log.
* YJIT: Add compilation events to the circular buffer even if output is sent to a file.
Previously, the two modes were treated as being exclusive of one another. However, it could be beneficial to log all events to a file while also allowing for direct access of the last N events via `RubyVM::YJIT.compilation_log`.
* YJIT: Make timestamps the first element in the YJIT compilation log tuple.
* YJIT: Stream log to stderr if `--yjit-compilation-log` is supplied without an argument.
* YJIT: Eagerly compute compilation log messages to avoid hanging on to references that may GC.
* YJIT: Log all compiled blocks, not just the method entry points.
* YJIT: Remove all compilation events other than block compilation to slim down the log.
* YJIT: Replace circular buffer iterator with a consuming loop.
* YJIT: Support `--yjit-compilation-log=quiet` as a way to activate the in-memory log without printing it.
Co-authored-by: Randy Stauner <randy.stauner@shopify.com>
* YJIT: Promote the compilation log to being the one YJIT log.
Co-authored-by: Randy Stauner <randy.stauner@shopify.com>
* Update doc/yjit/yjit.md
* Update doc/yjit/yjit.md
---------
Co-authored-by: Randy Stauner <randy.stauner@shopify.com>
Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
Notes:
Merged-By: maximecb <maximecb@ruby-lang.org>
|
|
Module#name shows up as a top C method callee in lobsters so probably
common enough. It's also easy to substitute thanks to rb_mod_name()
already having no GC yield points.
klass = BasicObject
50_000_000.times { klass.name }
Benchmark 1: /.rubies/post/bin/ruby --yjit mod_name.rb
Time (mean ± σ): 1.433 s ± 0.010 s [User: 1.410 s, System: 0.010 s]
Range (min … max): 1.421 s … 1.449 s 10 runs
Benchmark 2: /.rubies/mstr/bin/ruby --yjit mod_name.rb
Time (mean ± σ): 1.491 s ± 0.012 s [User: 1.468 s, System: 0.010 s]
Range (min … max): 1.470 s … 1.511 s 10 runs
Summary
/.rubies/post/bin/ruby --yjit mod_name.rb ran
1.04 ± 0.01 times faster than /.rubies/mstr/bin/ruby --yjit mod_name.rb
|
|
* YJIT: Add --yjit-mem-size option
* Improve --help
* s/the region/this virtual memory region/
Co-authored-by: Maxime Chevalier-Boisvert <maxime.chevalierboisvert@shopify.com>
---------
Co-authored-by: Maxime Chevalier-Boisvert <maxime.chevalierboisvert@shopify.com>
Notes:
Merged-By: maximecb <maximecb@ruby-lang.org>
|
|
* Returning an iterator instead of a vec
* Avoid changing the meaning of end_page
---------
Co-authored-by: Takashi Kokubun <takashikkbn@gmail.com>
Notes:
Merged-By: k0kubun <takashikkbn@gmail.com>
|
|
If a Hash which is empty or only using literals is frozen, we detect
this as a peephole optimization and change the instructions to be
`opt_hash_freeze`.
[Feature #20684]
Co-authored-by: Jean Boussier <byroot@ruby-lang.org>
Notes:
Merged: https://github.com/ruby/ruby/pull/11406
|
|
If an Array which is empty or only using literals is frozen, we detect
this as a peephole optimization and change the instructions to be
`opt_ary_freeze`.
[Feature #20684]
Co-authored-by: Jean Boussier <byroot@ruby-lang.org>
Notes:
Merged: https://github.com/ruby/ruby/pull/11406
|
|
* YJIT: Pass method arguments using registers
* s/at_current_insn/at_compile_target/
* Implement register shuffle
Notes:
Merged-By: k0kubun <takashikkbn@gmail.com>
|
|
* YJIT: Allow tracing fallback counters
* Update yjit.md about --yjit-trace-exits=counter
Notes:
Merged-By: k0kubun <takashikkbn@gmail.com>
|
|
codepoint values. (#11032)
* Document why we need to explicitly spill registers.
* Simplify passing a byte value to `str_buf_cat`.
* YJIT: Enhance the `String#<<` method substitution to handle integer codepoint values.
* YJIT: Move runtime type check into YJIT.
Performing the check in YJIT means we can make assumptions about the type. It also improves correctness of stack traces in cases where the codepoint argument is not a String or a Fixnum.
Notes:
Merged-By: maximecb <maximecb@ruby-lang.org>
|
|
Notes:
Merged-By: maximecb <maximecb@ruby-lang.org>
|
|
Use an enum for the method arg instead of needing to add an id
that doesn't map to an actual method name.
$ ruby --dump=insns -e 'b = "x"; [v].pack("E*", buffer: b)'
before:
```
== disasm: #<ISeq:<main>@-e:1 (1,0)-(1,34)>
local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
[ 1] b@0
0000 putchilledstring "x" ( 1)[Li]
0002 setlocal_WC_0 b@0
0004 putself
0005 opt_send_without_block <calldata!mid:v, argc:0, FCALL|VCALL|ARGS_SIMPLE>
0007 newarray 1
0009 putchilledstring "E*"
0011 getlocal_WC_0 b@0
0013 opt_send_without_block <calldata!mid:pack, argc:2, kw:[#<Symbol:0x000000000023110c>], KWARG>
0015 leave
```
after:
```
== disasm: #<ISeq:<main>@-e:1 (1,0)-(1,34)>
local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
[ 1] b@0
0000 putchilledstring "x" ( 1)[Li]
0002 setlocal_WC_0 b@0
0004 putself
0005 opt_send_without_block <calldata!mid:v, argc:0, FCALL|VCALL|ARGS_SIMPLE>
0007 putchilledstring "E*"
0009 getlocal b@0, 0
0012 opt_newarray_send 3, 5
0015 leave
```
Notes:
Merged: https://github.com/ruby/ruby/pull/11249
|
|
* YJIT: Allow dev_nodebug to disasm release-mode code
* Revert "YJIT: Squash canary before falling back"
This reverts commit f05ad373d84909da7541bd6d6ace38b48eaf24a1.
The stray canary issue should have been solved by
def7023ee4a3fc6eeba9d3a34c31a5bcff315fac, alleviating this codegen
accommodation.
* s/runtime_assertions/runtime_checks/
---------
Co-authored-by: Alan Wu <XrXr@users.noreply.github.com>
Notes:
Merged-By: k0kubun <takashikkbn@gmail.com>
|
|
* YJIT: Local variable register allocation
* locals are not stack temps
* Rename RegTemps to RegMappings
* Rename RegMapping to RegOpnd
* Rename local_size to num_locals
* s/stack value/operand/
* Rename spill_temps() to spill_regs()
* Clarify when num_locals becomes None
* Mention that InsnOut uses different registers
* Rename get_reg_mapping to get_reg_opnd
* Resurrect --yjit-temp-regs capability
* Use MAX_CTX_TEMPS and MAX_CTX_LOCALS
|
