| Age | Commit message (Collapse) | Author |
|---|
|
Previously, for 8+ params we wound up clobbering the self param when
putting the last param in memory in the JIT entry point:
# ZJIT entry point: a@../test.rb:5
<snip>
ldur x0, [x19, #0x18]
# set method params: 8
ldur x1, [x21, #-0x58]
ldur x2, [x21, #-0x50]
ldur x3, [x21, #-0x48]
ldur x4, [x21, #-0x40]
ldur x5, [x21, #-0x38]
ldur x11, [x21, #-0x30]
ldur x12, [x21, #-0x28]
ldur x0, [x21, #-0x20]
stur x0, [sp, #-0x20]
bl #0x11e17018c
Doing the memcpys for parameters in memory first avoids this clobbering.
# set method params: 8
ldur x0, [x21, #-0x20]
stur x0, [sp, #-0x20]
ldur x12, [x21, #-0x28]
ldur x11, [x21, #-0x30]
ldur x5, [x21, #-0x38]
ldur x4, [x21, #-0x40]
ldur x3, [x21, #-0x48]
ldur x2, [x21, #-0x50]
ldur x1, [x21, #-0x58]
ldur x0, [x19, #0x18]
|
|
We weren't firing write barriers before when writing to imemo/env
objects. Wbcheck caught this with test/ruby/test_refinement.rb:
ruby -v: ruby 3.5.0dev (2025-07-22T17:05:58Z wbcheck 2569a80954) +ZJIT dev +PRISM +GC[wbcheck] [x86_64-linux]
WBCHECK ERROR: Missed write barrier detected!
Parent object: 0x558de9f4e6e0 (wb_protected: true)
rb_obj_info_dump: 0x0000558de9f4e6e0 T_IMEMO/<env>
Reference counts - snapshot: 3, writebarrier: 0, current: 4, missed: 1
Missing reference to: 0x558decf37c30
rb_obj_info_dump: 0x0000558decf37c30 method/UnboundMethod method
WBCHECK SUMMARY: Found 1 objects with missed write barriers (1 total violations)
|
|
Or else the following returns garbage since it loads after
moving SP. Prior bad disassembly:
def a(n1,n2,n3,n4,n5,n6,n7,n8) = n8
a(1,1,1,1,1,1,1,0)
# Block: bb0(v0, v1, v2, v3, v4, v5, v6, v7, v8)
stp x29, x30, [sp, #-0x10]!
mov x29, sp
# bump C stack pointer
sub sp, sp, #0x10
# Insn: v10 Return v8
# pop stack frame
adds x19, x19, #0x38
stur x19, [x20, #0x10]
# restore C stack pointer
add sp, sp, #0x10
mov sp, x29
ldp x29, x30, [sp], #0x10
ldur x0, [sp]
ret
|
|
Previously `no_dead_mov_from_vreg` generated:
0x0: ldur x0, [x0]
0x4: mov x0, x0
0x8: ret
Because of phase ordering. Split couldn't recognize that the no-op mov
because at that point it sees a `VReg`.
|
|
It has a bad habit of leaving a trailing space, for example for ARM
`ret`.
|
|
ZJIT: Support invalidating method redefinition
This commit adds support for the MethodRedefined invariant to be invalidated
when a method is redefined.
Changes:
- Added CME pointer to the MethodRedefined invariant in HIR
- Updated all places where MethodRedefined invariants are created to
include the CME pointer
- Added handling for MethodRedefined invariants in gen_patch_point to
call track_cme_assumption, which registers the patch point for
invalidation when rb_zjit_cme_invalidate is called
This ensures that when a method is redefined, all JIT code that
depends on that method will be properly invalidated.
|
|
Use `fixnum_from_isize` instead of `fixnum_from_usize` in
`fold_fixnum_bop` to properly handle negative values. Casting negative
`i64` to `usize` produces large unsigned values that exceed `RUBY_FIXNUM_MAX`.
|
|
This lets us ZJIT compiled functions show up in the profiles of, say,
perf, or samply.
Fix https://github.com/Shopify/ruby/issues/634
|
|
|
|
|
|
This issues writebarriers for objects added via gc_offsets or by
profiling. This may be slower than writebarrier_remember, but we would
like it to be more debuggable.
Co-authored-by: Max Bernstein <ruby@bernsteinbear.com>
Co-authored-by: Stan Lo <stan001212@gmail.com>
|
|
|
|
|
|
Fixes `TestZJIT::test_require_rubygems`. It was crashing locally due to
false collection of a live object. See
<https://alanwu.space/post/write-barrier/>.
Co-authored-by: Max Bernstein <max@bernsteinbear.com>
Co-authored-by: Takashi Kokubun <takashi.kokubun@shopify.com>
Co-authored-by: Stan Lo <stan.lo@shopify.com>
|
|
Fix https://github.com/Shopify/ruby/issues/592
|
|
This helps us rewrite more SendWithoutBlock into SendWithoutBlockDirect.
|
|
Fix https://github.com/Shopify/ruby/issues/627
|
|
|
|
Previously, my buggy optimization would turn `asm.sub(imm, reg)`
into `subs out, reg, imm` since it runs through the addition path which
relies on the commutative property. Don't do that because subtraction
does not commute. Good thing no one seems to use this form.
Also, delete the 2 regs match arm for Add because it's already covered
by the fallback arm -- both split_load_operand() and
split_shifted_immediate() are no-op when the input is a register.
Fixes: 1317377fa74 ("ZJIT: A64: Have add/sub to SP be
single-instruction")
|
|
The raw bytes didn't disassemble to the disassembly, but we missed this
since CI didn't run `make zjit-test` with the disasm feature.
Fixes: 1317377fa74 ("ZJIT: A64: Have add/sub to SP be
single-instruction")
|
|
Previously, it issued CMN, which doesn't add to the stack pointer.
|
|
* ZJIT: Profile each instruction at most num_profiles times
* Use saturating_add for num_profiles
|
|
Co-authored-by: Alan Wu <XrXr@users.noreply.github.com>
|
|
|
|
|
|
Previously, gen_param() access slots at `SP-x` for `x≥0` after subtracting from
SP, so it was accessing slots from above the top of the stack. Also, the
slots gen_entry_params() wrote to at entry point did not correspond to
the slots access inside the JIT function.
Redo the stack frame layout so that inside the function slots are at
`SP+x`. Write to those slots in the entry point by anticipating the size
of the frame.
Fixes test_spilled_method_args().
|
|
Now that params can be in memory, this particular load_into() was
panicking with "Invalid operands for LDUR" with
test_spilled_method_args() on ARM.
Since it's documented to be for register destinations let's validate it.
|
|
This is temporary until we have a unified calling convention.
|
|
|
|
This DRYs up the `asm_comment!` + `asm.ccall` combo, and makes ccalls
have a comment by default.
|
|
Print the filename, line number, and whether or not the function has
been optimized:
```
Initial HIR:
fn initialize@test.rb:4:
bb0(v0:BasicObject):
v2:Fixnum[1] = Const Value(1)
SetIvar v0, :@a, v2
Return v2
Optimized HIR:
fn initialize@test.rb:4:
bb0(v0:BasicObject):
v2:Fixnum[1] = Const Value(1)
SetIvar v0, :@a, v2
Return v2
```
|
|
Co-authored-by: Alan Wu <alansi.xingwu@shopify.com>
Co-authored-by: Stan Lo <stan.lo@shopify.com>
|
|
|
|
This is notably faster: no need to hash indices.
Before:
```
plum% samply record ~/.rubies/ruby-zjit/bin/ruby --zjit benchmarks/getivar.rb
ruby 3.5.0dev (2025-07-10T14:40:49Z master 51252ef8d7) +ZJIT dev +PRISM [arm64-darwin24]
itr: time
#1: 5311ms
#2: 49ms
#3: 49ms
#4: 48ms
```
After:
```
plum% samply record ~/.rubies/ruby-zjit/bin/ruby --zjit benchmarks/getivar.rb
ruby 3.5.0dev (2025-07-10T15:09:06Z mb-benchmark-compile 42ffd3c1ee) +ZJIT dev +PRISM [arm64-darwin24]
itr: time
#1: 1332ms
#2: 49ms
#3: 48ms
#4: 48ms
```
|
|
* ZJIT: Improve asm comments for side exits
* Use GuardType(Type) and GuardBitEquals(VALUE)
|
|
|
|
|
|
|
|
That's not the validator's responsibility; the caller can choose to
later.
|
|
This should be a minimal C-API needed to deal with Set objects. It
supports creating the sets, checking whether an element is the set,
adding and removing elements, iterating over the elements, clearing
a set, and returning the size of the set.
Co-authored-by: Nobuyoshi Nakada <nobu.nakada@gmail.com>
|
|
Previously a missed optimization for add followed by mov. While we're
at it, have Add and Sub share the same match arm in arm64_split().
|
|
|
|
Other instructions use it as an operand and #13814 especially needs it
to have an output for validation.
|
|
|
|
Co-authored-by: Max Bernstein <max@bernsteinbear.com>
|
|
This PR adds a validator based on dataflow analysis to ZJIT. It checks that all uses are dominated by a GEN-DEF prior.
See issue https://github.com/Shopify/ruby/issues/591
This is especially useful in validating optimizations don't zap away instructions that are actually needed, e.g. DCE.
Also included: a slight refactor of the DCE code to its own function, so I can reuse it.
Note: the algorithm uses the worklist algorithm rather than the iterative version for faster convergence.
Co-Authored-By: Max Bernstein <ruby@bernsteinbear.com>
|
|
* ZJIT: Avoid optimizing locals on eval
* Maintain the local state for eval
|
|
|
|
|
|
|
