summaryrefslogtreecommitdiff
path: root/yjit/src/codegen.rs
AgeCommit message (Collapse)Author
2022-12-02make flag clearing betterAaron Patterson
Notes: Merged: https://github.com/ruby/ruby/pull/6767
2022-12-02only generate wb when we really need toAaron Patterson
Notes: Merged: https://github.com/ruby/ruby/pull/6767
2022-12-02bail on compilation if the comptime receiver is frozenAaron Patterson
Notes: Merged: https://github.com/ruby/ruby/pull/6767
2022-12-02do not fire the wb when writing immediatesAaron Patterson
Notes: Merged: https://github.com/ruby/ruby/pull/6767
2022-12-02implement IV writesAaron Patterson
Notes: Merged: https://github.com/ruby/ruby/pull/6767
2022-12-02YJIT: Make case-when optimization respect === redefinition (#6846)Alan Wu
* YJIT: Make case-when optimization respect === redefinition Even when a fixnum key is in the dispatch hash, if there is a case such that its basic operations for === is redefined, we need to fall back to checking each case like the interpreter. Semantically we're always checking each case by calling === in order, it's just that this is not observable when basic operations are intact. When all the keys are fixnums, though, we can do the optimization we're doing right now. Check for this condition. * Update yjit/src/cruby_bindings.inc.rs Co-authored-by: Takashi Kokubun <takashikkbn@gmail.com> Co-authored-by: Takashi Kokubun <takashikkbn@gmail.com> Notes: Merged-By: maximecb <maximecb@ruby-lang.org>
2022-12-01YJIT: Reorder branches for Fixnum opt_case_dispatch (#6841)Takashi Kokubun
* YJIT: Reorder branches for Fixnum opt_case_dispatch Co-authored-by: Maxime Chevalier-Boisvert <maxime.chevalierboisvert@shopify.com> Co-authored-by: Alan Wu <alansi.xingwu@shopify.com> * YJIT: Don't support too large values Co-authored-by: Maxime Chevalier-Boisvert <maxime.chevalierboisvert@shopify.com> Co-authored-by: Alan Wu <alansi.xingwu@shopify.com> Notes: Merged-By: maximecb <maximecb@ruby-lang.org>
2022-11-30YJIT: Optimize rb_int_equal (#6838)Takashi Kokubun
Notes: Merged-By: maximecb <maximecb@ruby-lang.org>
2022-11-29YJIT: Skip checking interrupt_mask (#6825)Takashi Kokubun
Notes: Merged-By: maximecb <maximecb@ruby-lang.org>
2022-11-24YJIT: rename `InsnOpnd` => `YARVOpnd` (#6801)Maxime Chevalier-Boisvert
Rename InsnOpnd => YARVOpnd Make it more clear this refers to YARV insn/vm operands rather than backend IR, x86 or ARM insn operands. Notes: Merged-By: maximecb <maximecb@ruby-lang.org>
2022-11-23YJIT: Use NonNull pointer for CodePtr (#6792)Takashi Kokubun
Notes: Merged-By: maximecb <maximecb@ruby-lang.org>
2022-11-23YJIT: Stop passing target1 to gen_return_branchTakashi Kokubun
Notes: Merged: https://github.com/ruby/ruby/pull/6794
2022-11-23YJIT: Simplify code for RB_SPECIAL_CONST_P (#6795)Takashi Kokubun
Notes: Merged-By: maximecb <maximecb@ruby-lang.org>
2022-11-22YJIT: Skip padding jumps to side exits on Arm (#6790)Takashi Kokubun
YJIT: Skip padding jumps to side exits Co-authored-by: Maxime Chevalier-Boisvert <maxime.chevalierboisvert@shopify.com> Co-authored-by: Alan Wu <alansi.xingwu@shopify.com> Co-authored-by: Maxime Chevalier-Boisvert <maxime.chevalierboisvert@shopify.com> Co-authored-by: Alan Wu <alansi.xingwu@shopify.com> Notes: Merged-By: maximecb <maximecb@ruby-lang.org>
2022-11-1832 bit comparison on shape idAaron Patterson
This commit changes the shape id comparisons to use a 32 bit comparison rather than 64 bit. That means we don't need to load the shape id to a register on x86 machines. Given the following program: ```ruby class Foo def initialize @foo = 1 @bar = 1 end def read [@foo, @bar] end end foo = Foo.new foo.read foo.read foo.read foo.read foo.read puts RubyVM::YJIT.disasm(Foo.instance_method(:read)) ``` The machine code we generated _before_ this change is like this: ``` == BLOCK 1/4, ISEQ RANGE [0,3), 65 bytes ====================== # getinstancevariable 0x559a18623023: mov rax, qword ptr [r13 + 0x18] # guard object is heap 0x559a18623027: test al, 7 0x559a1862302a: jne 0x559a1862502d 0x559a18623030: cmp rax, 4 0x559a18623034: jbe 0x559a1862502d # guard shape, embedded, and T_OBJECT 0x559a1862303a: mov rcx, qword ptr [rax] 0x559a1862303d: movabs r11, 0xffff00000000201f 0x559a18623047: and rcx, r11 0x559a1862304a: movabs r11, 0xb000000002001 0x559a18623054: cmp rcx, r11 0x559a18623057: jne 0x559a18625046 0x559a1862305d: mov rax, qword ptr [rax + 0x18] 0x559a18623061: mov qword ptr [rbx], rax == BLOCK 2/4, ISEQ RANGE [3,6), 0 bytes ======================= == BLOCK 3/4, ISEQ RANGE [3,6), 47 bytes ====================== # gen_direct_jmp: fallthrough # getinstancevariable # regenerate_branch # getinstancevariable # regenerate_branch 0x559a18623064: mov rax, qword ptr [r13 + 0x18] # guard shape, embedded, and T_OBJECT 0x559a18623068: mov rcx, qword ptr [rax] 0x559a1862306b: movabs r11, 0xffff00000000201f 0x559a18623075: and rcx, r11 0x559a18623078: movabs r11, 0xb000000002001 0x559a18623082: cmp rcx, r11 0x559a18623085: jne 0x559a18625099 0x559a1862308b: mov rax, qword ptr [rax + 0x20] 0x559a1862308f: mov qword ptr [rbx + 8], rax ``` After this change, it's like this: ``` == BLOCK 1/4, ISEQ RANGE [0,3), 41 bytes ====================== # getinstancevariable 0x5560c986d023: mov rax, qword ptr [r13 + 0x18] # guard object is heap 0x5560c986d027: test al, 7 0x5560c986d02a: jne 0x5560c986f02d 0x5560c986d030: cmp rax, 4 0x5560c986d034: jbe 0x5560c986f02d # guard shape 0x5560c986d03a: cmp word ptr [rax + 6], 0x19 0x5560c986d03f: jne 0x5560c986f046 0x5560c986d045: mov rax, qword ptr [rax + 0x10] 0x5560c986d049: mov qword ptr [rbx], rax == BLOCK 2/4, ISEQ RANGE [3,6), 0 bytes ======================= == BLOCK 3/4, ISEQ RANGE [3,6), 23 bytes ====================== # gen_direct_jmp: fallthrough # getinstancevariable # regenerate_branch # getinstancevariable # regenerate_branch 0x5560c986d04c: mov rax, qword ptr [r13 + 0x18] # guard shape 0x5560c986d050: cmp word ptr [rax + 6], 0x19 0x5560c986d055: jne 0x5560c986f099 0x5560c986d05b: mov rax, qword ptr [rax + 0x18] 0x5560c986d05f: mov qword ptr [rbx + 8], rax ``` The first ivar read is a bit more complex, but the second ivar read is much simpler. I think eventually we could teach the context about the shape, then emit only one shape guard. Notes: Merged: https://github.com/ruby/ruby/pull/6737
2022-11-17Fix bug involving .send and overwritten methods. (#6752)Jimmy Miller
@casperisfine reporting a bug in this gist https://gist.github.com/casperisfine/d59e297fba38eb3905a3d7152b9e9350 After investigating I found it was caused by a combination of send and a c_func that we have overwritten in the JIT. For send calls, we need to do some stack manipulation before making the call. Because of the way exits works, we need to do that stack manipulation at the last possible moment. In this case, we weren't doing that stack manipulation at all. Unfortunately, with how the code is structured there isn't a great place to do that stack manipulation for our overridden C funcs. Each overridden C func can return a boolean stating that it shouldn't be used. We would need to do the stack manipulation after all of those checks are done. We could pass a lambda(?) or separate out the logic for "can I run this override" from "now generate the code for it". Since we are coming up on a release, I went with the path of least resistence and just decided to not use these overrides if we are in a send call. We definitely should revist this in the future. Notes: Merged-By: maximecb <maximecb@ruby-lang.org>
2022-11-16YJIT: Pack BlockId and CodePtr (#6748)Takashi Kokubun
Notes: Merged-By: k0kubun <takashikkbn@gmail.com>
2022-11-16YJIT: Shrink the vectors of Block after mutation (#6739)Takashi Kokubun
Notes: Merged-By: k0kubun <takashikkbn@gmail.com>
2022-11-15YJIT: Count getivar side exits by receiver flag changes (#6735)Takashi Kokubun
Notes: Merged-By: k0kubun <takashikkbn@gmail.com>
2022-11-15YJIT: Invalidate redefined methods only through cme (#6734)Takashi Kokubun
Co-authored-by: Alan Wu <alansi.xingwu@shopify.com> Co-authored-by: Alan Wu <alansi.xingwu@shopify.com> Notes: Merged-By: k0kubun <takashikkbn@gmail.com>
2022-11-13YJIT: Remove unused src_ctx from Block (#6714)Takashi Kokubun
Notes: Merged-By: maximecb <maximecb@ruby-lang.org>
2022-11-11YJIT: Fix staying in invalidated code after proc callsAlan Wu
Previously, there is no instruction boundary patch point after the call to a non-leaf C function we generate for OPTIMIZED_METHOD_TYPE_CALL. This meant that if code GC is triggered while inside the C function, we would keep running invalidated code when we return from the C function. This had the effect of running stale branch stubs, jumping to bad code, etc. Use jit_prepare_routine_call() to make sure we exit from the invalidated region as soon as possible after the C call in case of invalidation. Notes: Merged: https://github.com/ruby/ruby/pull/6711
2022-11-10Enable --yjit-stats for release builds (#6694)Jimmy Miller
* Enable --yjit-stats for release builds In order for people in the real world to report information about how their application runs with YJIT, we want to expose stats without requiring rebuilding ruby. We can do this without overhead, with the exception of count ratio in yjit, since this relies on the interpreter also counting instructions. This change exposes those stats, while not showing ratio in yjit if we are not in a stats build. * Update yjit.rb Co-authored-by: Takashi Kokubun <takashikkbn@gmail.com> Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com> Notes: Merged-By: maximecb <maximecb@ruby-lang.org>
2022-11-10Remove numiv from RObjectJemma Issroff
Since object shapes store the capacity of an object, we no longer need the numiv field on RObjects. This gives us one extra slot which we can use to give embedded objects one more instance variable (for a total of 3 ivs). This commit removes the concept of numiv from RObject. Notes: Merged: https://github.com/ruby/ruby/pull/6699
2022-11-08Implement optimize call (#6691)Jimmy Miller
This dispatches to a c func for doing the dynamic lookup. I experimented with chain on the proc but wasn't able to detect which call sites would be monomorphic vs polymorphic. There is definitely room for optimization here, but it does reduce exits. Notes: Merged-By: maximecb <maximecb@ruby-lang.org>
2022-11-03YJIT: Make Code GC metrics available for non-stats builds (#6665)Takashi Kokubun
Notes: Merged-By: maximecb <maximecb@ruby-lang.org>
2022-11-03YJIT: Fix a wrong type reference (#6661)Takashi Kokubun
* YJIT: Fix a wrong type reference * YJIT: Just remove CapturedSelfOpnd for now Notes: Merged-By: maximecb <maximecb@ruby-lang.org>
2022-11-02YJIT: Support invokeblock (#6640)Takashi Kokubun
* YJIT: Support invokeblock * Update yjit/src/backend/arm64/mod.rs * Update yjit/src/codegen.rs Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com> Notes: Merged-By: maximecb <maximecb@ruby-lang.org>
2022-11-01YJIT: Delete redundant ways to make ContextAlan Wu
Context::new() is the same as Context::default() and Context::new_with_stack_size() was only used in tests. Notes: Merged: https://github.com/ruby/ruby/pull/6656
2022-10-27YJIT: Use guard_known_class() for opt_aref on Arrays (#6643)Alan Wu
This code used to roll its own heap object check before we made a better version in guard_known_class(). The improved version uses one fewer comparison, so let's use that. Notes: Merged-By: maximecb <maximecb@ruby-lang.org>
2022-10-26YJIT: Support nil and blockparamproxy as blockarg in send (#6492)Matthew Draper
Co-authored-by: John Hawthorn <john@hawthorn.email> Co-authored-by: John Hawthorn <john@hawthorn.email> Notes: Merged-By: maximecb <maximecb@ruby-lang.org>
2022-10-25YJIT: GC and recompile all code pages (#6406)Takashi Kokubun
when it fails to allocate a new page. Co-authored-by: Alan Wu <alansi.xingwu@shopify.com> Notes: Merged-By: k0kubun <takashikkbn@gmail.com>
2022-10-21YJIT: Read rb_num_t as usize earlyAlan Wu
This patch makes sure that we're not accidentally reading rb_num_t instruction arguments as VALUE and accidentally baking them into code and marking them. Some of these are simply moving the cast earlier, but some of these avoid potential problems for flag and ID arguments. Follow-up for 39f7eddec4c55711d56f05b085992a83bf23159e. Notes: Merged: https://github.com/ruby/ruby/pull/6606
2022-10-20YJIT: Fix gen_expandarray treating argument as VALUEAlan Wu
The expandarray instruction interpreters its arguments as rb_num_t. YJIT was treating the num argument as a VALUE previously and when it has a certain bit pattern, it can look like a GC pointer. The argument is not a pointer, so YJIT crashed when trying to mark those pointers. This bug existed previously, but our test suite didn't expose it until f55212bce939f736559709a8cd16c409772389c8. TestArgf#test_to_io has a line like: a1, a2, a3, a4, a5, a6, a7, a8 = array Which maps to an expandarray with an argument of 8. Qnil happened to be defined as 8, which masked the issue. Fix it by not using the argument as a VALUE. Notes: Merged: https://github.com/ruby/ruby/pull/6603
2022-10-20YJIT: remove some stale comments [ci skip]Alan Wu
2022-10-20YJIT: incorporate ruby_special_constsNobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/6600
2022-10-20Move "special consts" so `Qundef` and `Qnil` differ just 1 bitNobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/6599
2022-10-18Code clean around unused code for some architectures or features (#6581)Jimmy Miller
Notes: Merged-By: maximecb <maximecb@ruby-lang.org>
2022-10-17YJIT: Allow --yjit-dump-disasm to dump into a file (#6552)Takashi Kokubun
* YJIT: Allow --yjit-dump-disasm to dump into a file * YJIT: Move IO implementation to disasm.rs * YJIT: More consistent naming Notes: Merged-By: k0kubun <takashikkbn@gmail.com>
2022-10-17YJIT: Interleave inline and outlined code blocks (#6460)Takashi Kokubun
Co-authored-by: Alan Wu <alansi.xingwu@shopify.com> Co-authored-by: Maxime Chevalier-Boisvert <maxime.chevalierboisvert@shopify.com> Notes: Merged-By: k0kubun <takashikkbn@gmail.com>
2022-10-17YJIT: call free_block to cleanup block when out of memoryAlan Wu
The commented out instance of free_block() is left over from the port. The addition in gen_single_block() was a place we missed. The new block is allocated in the same function and could have invariants associated with it even though there is no space to hold all the code. Notes: Merged: https://github.com/ruby/ruby/pull/6551
2022-10-14More clippy fixes (#6547)Jimmy Miller
Notes: Merged-By: maximecb <maximecb@ruby-lang.org>
2022-10-13fixes more clippy warnings (#6543)Jimmy Miller
* fixes more clippy warnings * Fix x86 c_callable to have doc_strings Notes: Merged-By: maximecb <maximecb@ruby-lang.org>
2022-10-11Implement optimize send in yjit (#6488)Jimmy Miller
* Implement optimize send in yjit This successfully makes all our benchmarks exit way less for optimize send reasons. It makes some benchmarks faster, but not by as much as I'd like. I think this implementation works, but there are definitely more optimial arrangements. For example, what if we compiled send to a jump table? That seems like perhaps the most optimal we could do, but not obvious (to me) how to implement give our current setup. Co-authored-by: Alan Wu <XrXr@users.noreply.github.com> * Attempt at fixing the issues raised by @XrXr * fix allowlist * returns 0 instead of nil when not found * remove comment about encoding exception * Fix up c changes * Update assert Co-authored-by: Alan Wu <XrXr@users.noreply.github.com> * get rid of unneeded code and fix the flags * Apply suggestions from code review Co-authored-by: Alan Wu <XrXr@users.noreply.github.com> * rename and fix typo Co-authored-by: Alan Wu <XrXr@users.noreply.github.com> Notes: Merged-By: maximecb <maximecb@ruby-lang.org>
2022-10-11Make inline cache reads / writes atomic with object shapesJemma Issroff
Prior to this commit, we were reading and writing ivar index and shape ID in inline caches in two separate instructions when getting and setting ivars. This meant there was a race condition with ractors and these caches where one ractor could change a value in the cache while another was still reading from it. This commit instead reads and writes shape ID and ivar index to inline caches atomically so there is no longer a race condition. Co-Authored-By: Aaron Patterson <tenderlove@ruby-lang.org> Co-Authored-By: John Hawthorn <john@hawthorn.email>
2022-10-11Revert "Revert "This commit implements the Object Shapes technique in CRuby.""Jemma Issroff
This reverts commit 9a6803c90b817f70389cae10d60b50ad752da48f.
2022-10-04YJIT: add support for calling bmethods (#6489)Alan Wu
* YJIT: fix a parameter name * YJIT: add support for calling bmethods This commit adds support for the VM_METHOD_TYPE_BMETHOD method type in YJIT. You can get these type of methods from facilities like Kernel#define_singleton_method and Module#define_method. Even though the body of these methods are blocks, the parameter setup for them is exactly the same as VM_METHOD_TYPE_ISEQ, so we can reuse the same logic in gen_send_iseq(). You can see this from how vm_call_bmethod() eventually calls setup_parameters_complex() with arg_setup_method. Bmethods do need their frame environment to be setup differently. We handle this by allowing callers of gen_send_iseq() to control the iseq, the frame flag, and the prev_ep. The `prev_ep` goes into the same location as the block handler would go into in an iseq method frame. Co-authored-by: John Hawthorn <john@hawthorn.email> Co-authored-by: John Hawthorn <john@hawthorn.email> Notes: Merged-By: maximecb <maximecb@ruby-lang.org>
2022-09-30Fix YJIT args for rb_vm_set_ivar_idxJohn Hawthorn
This was broken accidentally with the revert of shapes (it conflicted with some unrelated cleanup). Notes: Merged: https://github.com/ruby/ruby/pull/6479
2022-09-30Fix YJIT build after shapes-revertJohn Hawthorn
An variable had been renamed in between the merge and revert, so the build was broken. This restores it.
2022-09-30Revert "This commit implements the Object Shapes technique in CRuby."Aaron Patterson
This reverts commit 68bc9e2e97d12f80df0d113e284864e225f771c2.
generated by cgit v1.2.3 (git 2.25.1) at 2026-01-17 17:52:14 +0000