path: root/misc
AgeCommit message (Collapse)Author
2022-07-14Get the insns_address_table from the vm_exec_core module table...Matt Valentine-House
Notes: Merged:
2022-07-06fix lldb scripts on older lldb pythonAaron Patterson
2022-06-21Add T_STRUCT to lldb inspect helperJemma Issroff
Notes: Merged:
2022-06-17[ci skip][lldb] Fix array length representation with USING_RVARGCMatt Valentine-House
This commit makes `rp` report the correct array length in lldb. When USING_RVARGC is set we use 7 bits of the flags to store the array len rather than the usual 2, so they need to be part of the mask when calculating the length in lldb. When calculating whether rvargc is enabled I've used the same approach that's used by `GC.using_rvargc?` which is to detect whether there is more than one size pool in the current objspace. Notes: Merged:
2022-06-15[ci skip] [lldb] Ensure rbbt has loaded the globalsMatt Valentine-House
rb_backtrace relies on the existend of RUBY_T_MASK. This is set up by the global loading code in lldb_init() rb_backtrace does not call lldb_init previously, and therefore would only work if called after another lldb function that _did_ load the globals. Notes: Merged:
2022-06-15[ci skip] Print the rb_classext_t for a class, using an offsetMatt Valentine-House
Now that classes are using VWA, the RCLASS_PTR uses an offset to get the rb_classext_t object. Doing this all the time in lldb is boring. So script lldb to do it for us Notes: Merged:
2022-06-15Add imemo types to global namespace in lldb helpersJemma Issroff
Notes: Merged:
2022-05-27Add more information to lldb dump_page helperJemma Issroff
Notes: Merged:
2022-05-04Update lldb helper for iseq disassembly to use correct var nameJemma Issroff
Notes: Merged:
2022-04-27Rust YJITAlan Wu
In December 2021, we opened an [issue] to solicit feedback regarding the porting of the YJIT codebase from C99 to Rust. There were some reservations, but this project was given the go ahead by Ruby core developers and Matz. Since then, we have successfully completed the port of YJIT to Rust. The new Rust version of YJIT has reached parity with the C version, in that it passes all the CRuby tests, is able to run all of the YJIT benchmarks, and performs similarly to the C version (because it works the same way and largely generates the same machine code). We've even incorporated some design improvements, such as a more fine-grained constant invalidation mechanism which we expect will make a big difference in Ruby on Rails applications. Because we want to be careful, YJIT is guarded behind a configure option: ```shell ./configure --enable-yjit # Build YJIT in release mode ./configure --enable-yjit=dev # Build YJIT in dev/debug mode ``` By default, YJIT does not get compiled and cargo/rustc is not required. If YJIT is built in dev mode, then `cargo` is used to fetch development dependencies, but when building in release, `cargo` is not required, only `rustc`. At the moment YJIT requires Rust 1.60.0 or newer. The YJIT command-line options remain mostly unchanged, and more details about the build process are documented in `doc/yjit/`. The CI tests have been updated and do not take any more resources than before. The development history of the Rust port is available at the following commit for interested parties: Our hope is that Rust YJIT will be compiled and included as a part of system packages and compiled binaries of the Ruby 3.2 release. We do not anticipate any major problems as Rust is well supported on every platform which YJIT supports, but to make sure that this process works smoothly, we would like to reach out to those who take care of building systems packages before the 3.2 release is shipped and resolve any issues that may come up. [issue]: Co-authored-by: Maxime Chevalier-Boisvert <> Co-authored-by: Noah Gibbs <> Co-authored-by: Kevin Newton <> Notes: Merged:
2022-04-04Make heap page sizes 64KiB by defaultPeter Zhu
Commit dde164e968e382d50b07ad4559468885cbff33ef decoupled incremental marking from page sizes. This commit changes Ruby heap page sizes to 64KiB. Doing so will have several benefits: 1. We can use compaction on systems with 64KiB system page sizes (e.g. PowerPC). 2. Larger page sizes will allow Variable Width Allocation to increase slot sizes and embed larger objects. 3. Since commit 002fa2859962f22de8afdbeece04966ea57b7da9, macOS has 64 KiB pages. Making page sizes 64 KiB will bring these systems to parity. I have attached some bechmark results below. Discourse: On Discourse, we saw much better p99 performance (e.g. for "categories" it went from 214ms on master to 134ms on branch, for "home" it went from 265ms to 251ms). We don’t see much change in p60, p75, and p90 performance. We also see a slight decrease in memory usage by 1.04x. Branch RSS: 354.9MB Master RSS: 368.2MB railsbench: On rails bench, we don’t see a big change in RPS or p99 performance. We don’t see a big difference in memory usage. Branch RPS: 826.27 Master RPS: 824.85 Branch p99: 1.67 Master p99: 1.72 Branch RSS: 88.72MB Master RSS: 88.48MB liquid: We don’t see a significant change in liquid performance. Branch parse & render: 28.653 I/s Master parse & render: 28.563 i/s Notes: Merged:
2022-04-01Fix up global name references in misc/lldb_disasm.pyAaron Patterson
Some of the symbols had changed names and the script was no longer finding them. Notes: Merged:
2022-02-02Treat TS_ICVARC cache as separate from TS_IVC cacheJemma Issroff
Notes: Merged:
2022-01-26[lldb] Handle MacOS 64Kb heap pages in the lldb helpersMatt Valentine-House
Notes: Merged: support RVARGC on T_CLASS [ci skip]Nobuyoshi Nakada
2022-01-06Update for VWA stringsPeter Zhu
Notes: Merged:
2021-12-03YJIT: Bounds check every byte in the assemblerAlan Wu
Previously, YJIT assumed that basic blocks never consume more than 1 KiB of memory. This assumption does not hold for long Ruby methods such as the one in the following: ```ruby eval(<<RUBY) def set_local_a_lot #{'_=0;'*0x40000} end RUBY set_local_a_lot ``` For low `--yjit-exec-mem-size` values, one basic block could exhaust the entire buffer. Introduce a new field `codeblock_t::dropped_bytes` that the assembler sets whenever it runs out of space. Check this field in gen_single_block() to respond to out of memory situations and other error conditions. This design avoids making the control flow graph of existing code generation functions more complex. Use POSIX shell in misc/ since bash is expanding `0%/*/*` differently. Co-authored-by: Aaron Patterson <> Notes: Merged:
2021-12-01Mark JIT code as writeable / executable depending on the situationAaron Patterson
Some platforms don't want memory to be marked as writeable and executable at the same time. When we write to the code block, we calculate the OS page that the buffer position maps to. Then we call `mprotect` to allow writes on that particular page. As an optimization, we cache the "last written" aligned page which allows us to amortize the cost of the `mprotect` call. In other words, sequential writes to the same page will only call `mprotect` on the page once. When we're done writing, we call `mprotect` on the entire JIT buffer. This means we don't need to keep track of which pages were marked as writeable, we let the OS take care of that. Co-authored-by: John Hawthorn <> Notes: Merged:
2021-11-05YJIT: use shorter encoding for mov(r64,imm) when unambiguous (#5081)Alan Wu
* YJIT: use shorter encoding for mov(r64,imm) when unambiguous Previously, for small constants such as `mov(RAX, imm_opnd(Qundef))`, we emit an instruction with an 8-byte immediate. This form commonly gets the `movabs` mnemonic. In 64-bit mode, 32-bit operands get zero extended to 64-bit to fill the register, so when the immediate is small enough, we can save 4 bytes by using the `mov` variant that takes a 32-bit immediate and does a zero extension. Not implement with this change, there is an imm32 variant of `mov` that does sign extension we could use. When the constant is negative, we fallback to the `movabs` form. In railsbench, this change yields roughly a 12% code size reduction for the outlined block. Co-authored-by: Jemma Issroff <> * [ci skip] comment edit. Please squash. Co-authored-by: Jemma Issroff <> Notes: Merged-By: maximecb <>
2021-11-04YJIT code pages refactoring for code GC (#5073)Maxime Chevalier-Boisvert
* New code page allocation logic * Fix leaked globals * Fix leaked symbols, yjit asm tests * Make COUNTED_EXIT take a jit argument, so we can eliminate global ocb * Remove extra whitespace * Change block start_pos/end_pos to be pointers instead of uint32_t * Change branch end_pos and start_pos to end_addr, start_addr Notes: Merged-By: maximecb <>
2021-11-02Fix typosNobuyoshi Nakada
2021-10-28Follow up the RString change [ci skip]Nobuyoshi Nakada
Since 46b66eb9e8e6de2d5750591e532310e8f8599d90, already `ary` has been enclosed in `embed`.
2021-10-27Fix yjit_asm_tests.c as C99 compliant (#5033)Nobuyoshi Nakada
* rb_bug should be variadic * Prefer ANSI-style prototypes over old K&R-style definitions * Add missing argument types Notes: Merged-By: maximecb <>
2021-10-25[Feature #18239] Implement VWA for stringsPeter Zhu
This commit adds support for embedded strings with variable capacity and uses Variable Width Allocation to allocate strings. Notes: Merged:
2021-10-21Move the test fileNobuyoshi Nakada
2021-10-21Fix for out-of-place buildNobuyoshi Nakada
2021-10-20Fix non RUBY_DEBUG build warningsAlan Wu
On non RUBY_DEBUG builds, assert() compiles to nothing and the compiler warns about uninitialized variables in those code paths. Replace those asserts with rb_bug() to fix the warnings and do the assert in all builds. Since yjit_asm_tests.c compiles outside of Ruby, it needed a distinct version of rb_bug(). Also put YJIT_STATS check for function delcaration that is only defined in YJIT_STATS builds.
2021-10-20Move into miscAlan Wu
Since conventionally scripts don't live at the top level of the repo.
2021-10-20Add an lldb script to print YJIT commentsAaron Patterson
This script is an lldb helper that just loops through all the comments stored and prints out the comment along with the address corresponding to the comment. For example, I'm crashing in JIT code at address 0x0000000110000168. Using the `lc` helper I can see that it's probably crashing inside the exit back to the interpreter ``` (lldb) bt 5 * thread #1, queue = '', stop reason = EXC_BAD_ACCESS (code=1, address=0x22220021) frame #0: 0x0000000110000168 * frame #1: 0x00000001002b5ff5 miniruby`invoke_block_from_c_bh [inlined] invoke_block(ec=0x0000000100e05350, iseq=0x0000000100c1ff10, self=0x0000000100c76cc0, captured=<unavailable>, cref=0x0000000000000000, type=<unavailable>, opt_pc=<unavailable>) at vm.c:1268:12 frame #2: 0x00000001002b5f7d miniruby`invoke_block_from_c_bh [inlined] invoke_iseq_block_from_c(ec=<unavailable>, captured=<unavailable>, self=0x0000000100c76cc0, argc=2, argv=<unavailable>, kw_splat=0, passed_block_handler=0x0000000000000000, cref=0x0000000000000000, is_lambda=<unavailable>, me=0x0000000000000000) at vm.c:1340 frame #3: 0x00000001002b5e14 miniruby`invoke_block_from_c_bh(ec=<unavailable>, block_handler=<unavailable>, argc=<unavailable>, argv=<unavailable>, kw_splat=0, passed_block_handler=0x0000000000000000, cref=0x0000000000000000, is_lambda=<unavailable>, force_blockarg=0) at vm.c:1358 frame #4: 0x000000010029860b miniruby`rb_yield_values(n=<unavailable>) at vm_eval.c:0 (lldb) lc 0x11000006d "putobject_INT2FIX_1_" 0x110000083 "leave" 0x110000087 "check for interrupts" 0x110000087 "RUBY_VM_CHECK_INTS(ec)" 0x110000098 "check for finish frame" 0x1100000ed "getlocal_WC_0" 0x110000107 "getlocal_WC_1" 0x11000012a "opt_send_without_block" 0x110000139 "opt_send_without_block" 0x11000013c "exit to interpreter" ```
2021-10-20Free block->incoming in invalidate_block_version()Maxime Chevalier-Boisvert
2021-10-20Added synthetic torture test with 30K tiny methodsMaxime Chevalier-Boisvert
2021-10-20Directly link libcapstone for easier developmentAaron Patterson
This lets us use libcapstone directly from miniruby so we don't need a Ruby Gem to to dev work. Example usage: ```ruby def foo(x) if x < 1 "wow" else "neat" end end iseq = RubyVM::InstructionSequence.of(method(:foo)) puts UJIT.disasm(iseq) 100.times { foo 1 } puts UJIT.disasm(iseq) ``` Then in the terminal ``` $ ./miniruby test.rb == disasm: #<ISeq:foo@test.rb:1 (1,0)-(7,3)> (catch: FALSE) local table (size: 1, argc: 1 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1]) [ 1] x@0<Arg> 0000 getlocal_WC_0 x@0 ( 2)[LiCa] 0002 putobject_INT2FIX_1_ 0003 opt_lt <calldata!mid:<, argc:1, ARGS_SIMPLE> 0005 branchunless 10 0007 putstring "wow" ( 3)[Li] 0009 leave ( 7)[Re] 0010 putstring "neat" ( 5)[Li] 0012 leave ( 7)[Re] == ISEQ RANGE: 10 -> 10 ======================================================== 0x0: movabs rax, 0x7fe816e2d1a0 0xa: mov qword ptr [rdi], rax 0xd: mov r8, rax 0x10: mov r9, rax 0x13: mov r11, r12 0x16: jmp qword ptr [rax] == ISEQ RANGE: 0 -> 7 ========================================================== 0x0: mov rax, qword ptr [rdi + 0x20] 0x4: mov rax, qword ptr [rax - 0x18] 0x8: mov qword ptr [rdx], rax 0xb: mov qword ptr [rdx + 8], 3 0x13: movabs rax, 0x7fe817808200 0x1d: test byte ptr [rax + 0x3e6], 1 0x24: jne 0x3ffff7b 0x2a: test byte ptr [rdx], 1 0x2d: je 0x3ffff7b 0x33: test byte ptr [rdx + 8], 1 0x37: je 0x3ffff7b 0x3d: mov rax, qword ptr [rdx] 0x40: cmp rax, qword ptr [rdx + 8] 0x44: movabs rax, 0 0x4e: movabs rcx, 0x14 0x58: cmovl rax, rcx 0x5c: mov qword ptr [rdx], rax 0x5f: test qword ptr [rdx], -9 0x66: jne 0x3ffffd5 ``` Make sure to `brew install pkg-config capstone`
2021-10-20Added comments. Fixed compiler warning.Maxime Chevalier-Boisvert
2021-10-20add a helper script for ujit disasmAaron Patterson
2021-09-29lldb: Get rid of error at unpreserved encodings [ci skip]Nobuyoshi Nakada
2021-09-28lldb: Show encoding of String [ci skip]Nobuyoshi Nakada
2021-08-25[Feature #18045] Implement size classes for GCPeter Zhu
This commits implements size classes in the GC for the Variable Width Allocation feature. Unless `USE_RVARGC` compile flag is set, only a single size class is created, maintaining current behaviour. See the redmine ticket for more details. Co-authored-by: Aaron Patterson <> Notes: Merged:
2021-08-25[Feature #18045] Remove T_PAYLOADPeter Zhu
This commit removes T_PAYLOAD since the new VWA implementation no longer requires T_PAYLOAD types. Co-authored-by: Aaron Patterson <> Notes: Merged:
2021-08-23Revert "[Feature #18045] Implement size classes for GC"Peter Zhu
This reverts commits 48ff7a9f3e47bffb3e4d067a12ba9b936261caa0 and b2e2cf2dedd104acad8610721db5e4d341f135ef because it is causing crashes in SPARC solaris and i386 debian. Notes: Merged:
2021-08-23[Feature #18045] Implement size classes for GCPeter Zhu
This commits implements size classes in the GC for the Variable Width Allocation feature. Unless `USE_RVARGC` compile flag is set, only a single size class is created, maintaining current behaviour. See the redmine ticket for more details. Co-authored-by: Aaron Patterson <> Notes: Merged:
2021-08-23[Feature #18045] Remove T_PAYLOADPeter Zhu
This commit removes T_PAYLOAD since the new VWA implementation no longer requires T_PAYLOAD types. Co-authored-by: Aaron Patterson <> Notes: Merged: push non-flonum float to history [ci skip]Nobuyoshi Nakada fix non-flonum float inspection [ci skip]Nobuyoshi Nakada
2021-05-27Promote net-smtp to the bundled gemsHiroshi SHIBATA
Notes: Merged:
2021-05-27Promote net-pop to the bundled gemsHiroshi SHIBATA
Notes: Merged:
2021-05-27Promote net-imap to the bundled gemsHiroshi SHIBATA
Notes: Merged:
2021-05-27Promote net-ftp to the bundled gemsHiroshi SHIBATA
Notes: Merged:
2021-05-27Promote matrix to the bundled gemsHiroshi SHIBATA
Notes: Merged:
2021-05-25Update the latest list of default gems for misc/expand_tabs.rbHiroshi SHIBATA
2021-05-24add rb_id2str to lldb debugging scriptsAaron Patterson