summaryrefslogtreecommitdiff
path: root/misc
AgeCommit message (Collapse)Author
2021-12-03YJIT: Bounds check every byte in the assemblerAlan Wu
Previously, YJIT assumed that basic blocks never consume more than 1 KiB of memory. This assumption does not hold for long Ruby methods such as the one in the following: ```ruby eval(<<RUBY) def set_local_a_lot #{'_=0;'*0x40000} end RUBY set_local_a_lot ``` For low `--yjit-exec-mem-size` values, one basic block could exhaust the entire buffer. Introduce a new field `codeblock_t::dropped_bytes` that the assembler sets whenever it runs out of space. Check this field in gen_single_block() to respond to out of memory situations and other error conditions. This design avoids making the control flow graph of existing code generation functions more complex. Use POSIX shell in misc/test_yjit_asm.sh since bash is expanding `0%/*/*` differently. Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org> Notes: Merged: https://github.com/ruby/ruby/pull/5209
2021-12-01Mark JIT code as writeable / executable depending on the situationAaron Patterson
Some platforms don't want memory to be marked as writeable and executable at the same time. When we write to the code block, we calculate the OS page that the buffer position maps to. Then we call `mprotect` to allow writes on that particular page. As an optimization, we cache the "last written" aligned page which allows us to amortize the cost of the `mprotect` call. In other words, sequential writes to the same page will only call `mprotect` on the page once. When we're done writing, we call `mprotect` on the entire JIT buffer. This means we don't need to keep track of which pages were marked as writeable, we let the OS take care of that. Co-authored-by: John Hawthorn <john@hawthorn.email> Notes: Merged: https://github.com/ruby/ruby/pull/5032
2021-11-05YJIT: use shorter encoding for mov(r64,imm) when unambiguous (#5081)Alan Wu
* YJIT: use shorter encoding for mov(r64,imm) when unambiguous Previously, for small constants such as `mov(RAX, imm_opnd(Qundef))`, we emit an instruction with an 8-byte immediate. This form commonly gets the `movabs` mnemonic. In 64-bit mode, 32-bit operands get zero extended to 64-bit to fill the register, so when the immediate is small enough, we can save 4 bytes by using the `mov` variant that takes a 32-bit immediate and does a zero extension. Not implement with this change, there is an imm32 variant of `mov` that does sign extension we could use. When the constant is negative, we fallback to the `movabs` form. In railsbench, this change yields roughly a 12% code size reduction for the outlined block. Co-authored-by: Jemma Issroff <jemmaissroff@gmail.com> * [ci skip] comment edit. Please squash. Co-authored-by: Jemma Issroff <jemmaissroff@gmail.com> Notes: Merged-By: maximecb <maximecb@ruby-lang.org>
2021-11-04YJIT code pages refactoring for code GC (#5073)Maxime Chevalier-Boisvert
* New code page allocation logic * Fix leaked globals * Fix leaked symbols, yjit asm tests * Make COUNTED_EXIT take a jit argument, so we can eliminate global ocb * Remove extra whitespace * Change block start_pos/end_pos to be pointers instead of uint32_t * Change branch end_pos and start_pos to end_addr, start_addr Notes: Merged-By: maximecb <maximecb@ruby-lang.org>
2021-11-02Fix typosNobuyoshi Nakada
2021-10-28Follow up the RString change [ci skip]Nobuyoshi Nakada
Since 46b66eb9e8e6de2d5750591e532310e8f8599d90, already `ary` has been enclosed in `embed`.
2021-10-27Fix yjit_asm_tests.c as C99 compliant (#5033)Nobuyoshi Nakada
* rb_bug should be variadic * Prefer ANSI-style prototypes over old K&R-style definitions * Add missing argument types Notes: Merged-By: maximecb <maximecb@ruby-lang.org>
2021-10-25[Feature #18239] Implement VWA for stringsPeter Zhu
This commit adds support for embedded strings with variable capacity and uses Variable Width Allocation to allocate strings. Notes: Merged: https://github.com/ruby/ruby/pull/4933
2021-10-21Move the test fileNobuyoshi Nakada
2021-10-21Fix for out-of-place buildNobuyoshi Nakada
2021-10-20Fix non RUBY_DEBUG build warningsAlan Wu
On non RUBY_DEBUG builds, assert() compiles to nothing and the compiler warns about uninitialized variables in those code paths. Replace those asserts with rb_bug() to fix the warnings and do the assert in all builds. Since yjit_asm_tests.c compiles outside of Ruby, it needed a distinct version of rb_bug(). Also put YJIT_STATS check for function delcaration that is only defined in YJIT_STATS builds.
2021-10-20Move test_yjit_asm.sh into miscAlan Wu
Since conventionally scripts don't live at the top level of the repo.
2021-10-20Add an lldb script to print YJIT commentsAaron Patterson
This script is an lldb helper that just loops through all the comments stored and prints out the comment along with the address corresponding to the comment. For example, I'm crashing in JIT code at address 0x0000000110000168. Using the `lc` helper I can see that it's probably crashing inside the exit back to the interpreter ``` (lldb) bt 5 * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x22220021) frame #0: 0x0000000110000168 * frame #1: 0x00000001002b5ff5 miniruby`invoke_block_from_c_bh [inlined] invoke_block(ec=0x0000000100e05350, iseq=0x0000000100c1ff10, self=0x0000000100c76cc0, captured=<unavailable>, cref=0x0000000000000000, type=<unavailable>, opt_pc=<unavailable>) at vm.c:1268:12 frame #2: 0x00000001002b5f7d miniruby`invoke_block_from_c_bh [inlined] invoke_iseq_block_from_c(ec=<unavailable>, captured=<unavailable>, self=0x0000000100c76cc0, argc=2, argv=<unavailable>, kw_splat=0, passed_block_handler=0x0000000000000000, cref=0x0000000000000000, is_lambda=<unavailable>, me=0x0000000000000000) at vm.c:1340 frame #3: 0x00000001002b5e14 miniruby`invoke_block_from_c_bh(ec=<unavailable>, block_handler=<unavailable>, argc=<unavailable>, argv=<unavailable>, kw_splat=0, passed_block_handler=0x0000000000000000, cref=0x0000000000000000, is_lambda=<unavailable>, force_blockarg=0) at vm.c:1358 frame #4: 0x000000010029860b miniruby`rb_yield_values(n=<unavailable>) at vm_eval.c:0 (lldb) lc 0x11000006d "putobject_INT2FIX_1_" 0x110000083 "leave" 0x110000087 "check for interrupts" 0x110000087 "RUBY_VM_CHECK_INTS(ec)" 0x110000098 "check for finish frame" 0x1100000ed "getlocal_WC_0" 0x110000107 "getlocal_WC_1" 0x11000012a "opt_send_without_block" 0x110000139 "opt_send_without_block" 0x11000013c "exit to interpreter" ```
2021-10-20Free block->incoming in invalidate_block_version()Maxime Chevalier-Boisvert
2021-10-20Added synthetic torture test with 30K tiny methodsMaxime Chevalier-Boisvert
2021-10-20Directly link libcapstone for easier developmentAaron Patterson
This lets us use libcapstone directly from miniruby so we don't need a Ruby Gem to to dev work. Example usage: ```ruby def foo(x) if x < 1 "wow" else "neat" end end iseq = RubyVM::InstructionSequence.of(method(:foo)) puts UJIT.disasm(iseq) 100.times { foo 1 } puts UJIT.disasm(iseq) ``` Then in the terminal ``` $ ./miniruby test.rb == disasm: #<ISeq:foo@test.rb:1 (1,0)-(7,3)> (catch: FALSE) local table (size: 1, argc: 1 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1]) [ 1] x@0<Arg> 0000 getlocal_WC_0 x@0 ( 2)[LiCa] 0002 putobject_INT2FIX_1_ 0003 opt_lt <calldata!mid:<, argc:1, ARGS_SIMPLE> 0005 branchunless 10 0007 putstring "wow" ( 3)[Li] 0009 leave ( 7)[Re] 0010 putstring "neat" ( 5)[Li] 0012 leave ( 7)[Re] == ISEQ RANGE: 10 -> 10 ======================================================== 0x0: movabs rax, 0x7fe816e2d1a0 0xa: mov qword ptr [rdi], rax 0xd: mov r8, rax 0x10: mov r9, rax 0x13: mov r11, r12 0x16: jmp qword ptr [rax] == ISEQ RANGE: 0 -> 7 ========================================================== 0x0: mov rax, qword ptr [rdi + 0x20] 0x4: mov rax, qword ptr [rax - 0x18] 0x8: mov qword ptr [rdx], rax 0xb: mov qword ptr [rdx + 8], 3 0x13: movabs rax, 0x7fe817808200 0x1d: test byte ptr [rax + 0x3e6], 1 0x24: jne 0x3ffff7b 0x2a: test byte ptr [rdx], 1 0x2d: je 0x3ffff7b 0x33: test byte ptr [rdx + 8], 1 0x37: je 0x3ffff7b 0x3d: mov rax, qword ptr [rdx] 0x40: cmp rax, qword ptr [rdx + 8] 0x44: movabs rax, 0 0x4e: movabs rcx, 0x14 0x58: cmovl rax, rcx 0x5c: mov qword ptr [rdx], rax 0x5f: test qword ptr [rdx], -9 0x66: jne 0x3ffffd5 ``` Make sure to `brew install pkg-config capstone`
2021-10-20Added comments. Fixed compiler warning.Maxime Chevalier-Boisvert
2021-10-20add a helper script for ujit disasmAaron Patterson
2021-09-29lldb: Get rid of error at unpreserved encodings [ci skip]Nobuyoshi Nakada
2021-09-28lldb: Show encoding of String [ci skip]Nobuyoshi Nakada
2021-08-25[Feature #18045] Implement size classes for GCPeter Zhu
This commits implements size classes in the GC for the Variable Width Allocation feature. Unless `USE_RVARGC` compile flag is set, only a single size class is created, maintaining current behaviour. See the redmine ticket for more details. Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org> Notes: Merged: https://github.com/ruby/ruby/pull/4773
2021-08-25[Feature #18045] Remove T_PAYLOADPeter Zhu
This commit removes T_PAYLOAD since the new VWA implementation no longer requires T_PAYLOAD types. Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org> Notes: Merged: https://github.com/ruby/ruby/pull/4773
2021-08-23Revert "[Feature #18045] Implement size classes for GC"Peter Zhu
This reverts commits 48ff7a9f3e47bffb3e4d067a12ba9b936261caa0 and b2e2cf2dedd104acad8610721db5e4d341f135ef because it is causing crashes in SPARC solaris and i386 debian. Notes: Merged: https://github.com/ruby/ruby/pull/4764
2021-08-23[Feature #18045] Implement size classes for GCPeter Zhu
This commits implements size classes in the GC for the Variable Width Allocation feature. Unless `USE_RVARGC` compile flag is set, only a single size class is created, maintaining current behaviour. See the redmine ticket for more details. Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org> Notes: Merged: https://github.com/ruby/ruby/pull/4680
2021-08-23[Feature #18045] Remove T_PAYLOADPeter Zhu
This commit removes T_PAYLOAD since the new VWA implementation no longer requires T_PAYLOAD types. Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org> Notes: Merged: https://github.com/ruby/ruby/pull/4680
2021-06-04lldb_cruby.py: push non-flonum float to history [ci skip]Nobuyoshi Nakada
2021-06-04lldb_cruby.py: fix non-flonum float inspection [ci skip]Nobuyoshi Nakada
2021-05-27Promote net-smtp to the bundled gemsHiroshi SHIBATA
Notes: Merged: https://github.com/ruby/ruby/pull/4530
2021-05-27Promote net-pop to the bundled gemsHiroshi SHIBATA
Notes: Merged: https://github.com/ruby/ruby/pull/4530
2021-05-27Promote net-imap to the bundled gemsHiroshi SHIBATA
Notes: Merged: https://github.com/ruby/ruby/pull/4530
2021-05-27Promote net-ftp to the bundled gemsHiroshi SHIBATA
Notes: Merged: https://github.com/ruby/ruby/pull/4530
2021-05-27Promote matrix to the bundled gemsHiroshi SHIBATA
Notes: Merged: https://github.com/ruby/ruby/pull/4530
2021-05-25Update the latest list of default gems for misc/expand_tabs.rbHiroshi SHIBATA
2021-05-24add rb_id2str to lldb debugging scriptsAaron Patterson
2021-05-06lldb: convert heap_page_obj_limit from a float to intPeter Zhu
Notes: Merged: https://github.com/ruby/ruby/pull/4467
2021-05-06lldb: teach rp about T_PAYLOADMatt Valentine-House
Notes: Merged: https://github.com/ruby/ruby/pull/4391
2021-04-29lldb: Warn when attempting to dump invalid pagesMatt Valentine-House
Notes: Merged: https://github.com/ruby/ruby/pull/4431
2021-04-28[ruby/optparse] Add EditorConfig fileAlexander Popov
More info here: https://editorconfig.org/ For example, `ruby/ruby` has it: https://github.com/ruby/ruby/blob/05ebaee/.editorconfig Also fix some offenses. https://github.com/ruby/optparse/commit/29402e7e0e
2021-04-27lldb: highlight the slot when using dump_page_rvalueMatt Valentine-House
Notes: Merged: https://github.com/ruby/ruby/pull/4420
2021-04-27lldb: dump_page_rvalue - dump a heap page containing an RVALUEMatt Valentine-House
rather than having to do this in a two step process: 1. heap_page obj 2. dump_page $2 (or whatever lldb variable heap_page set) we can now just dump_page_rvalue obj Notes: Merged: https://github.com/ruby/ruby/pull/4420
2021-04-27lldb: Add Freelist Index to dump_page outputMatt Valentine-House
Notes: Merged: https://github.com/ruby/ruby/pull/4420
2021-03-29[ruby/optparse] Completion scripts themselves are not executableNobuyoshi Nakada
https://github.com/ruby/optparse/commit/65d8aff935
2021-03-29[ruby/optparse] Use ZDOTDIR env var to locate .zshrcMartin Rey
https://github.com/ruby/optparse/commit/c4977674bf
2021-03-16LLDB: Introduce dump_page helperMatt Valentine-House
This dumps out object type information for every object on a page in the form: bits [LM R ] T_CLASS [389]: Addr: 0x1007ebcf0 (flags: 0x100000062) Notes: Merged: https://github.com/ruby/ruby/pull/4277
2021-03-16LLDB: Extract a dump_bits function from rpMatt Valentine-House
that dumps the heap page bitmaps for a slot Notes: Merged: https://github.com/ruby/ruby/pull/4277
2020-12-10Promote webrick to bundled gemsHiroshi SHIBATA
Notes: Merged: https://github.com/ruby/ruby/pull/3729
2020-10-14Mostly recover a Ruby stack trace from a core fileAaron Patterson
Update the lldb script so it can mostly recover a Ruby stack trace from a core file. It's still missing line numbers and dealing with CFUNCs, but you use it like this: ``` (lldb) rbbt ec rb_control_frame_t TYPE 0x7f6fd6555fa0 EVAL ./bootstraptest/runner.rb error!! 0x7f6fd6555f68 METHOD ./bootstraptest/runner.rb main 0x7f6fd6555f30 METHOD ./bootstraptest/runner.rb in_temporary_working_directory 0x7f6fd6555ef8 METHOD /home/aaron/git/ruby/lib/tmpdir.rb mktmpdir 0x7f6fd6555ec0 BLOCK ./bootstraptest/runner.rb block in in_temporary_working_directory 0x7f6fd6555e88 CFUNC 0x7f6fd6555e50 BLOCK ./bootstraptest/runner.rb block (2 levels) in in_temporary_working_directory 0x7f6fd6555e18 BLOCK ./bootstraptest/runner.rb block in main 0x7f6fd6555de0 METHOD ./bootstraptest/runner.rb exec_test 0x7f6fd6555da8 CFUNC 0x7f6fd6555d70 BLOCK ./bootstraptest/runner.rb block in exec_test 0x7f6fd6555d38 CFUNC 0x7f6fd6555d00 TOP /home/aaron/git/ruby/bootstraptest/test_insns.rb error!! 0x7f6fd6555cc8 CFUNC 0x7f6fd6555c90 BLOCK /home/aaron/git/ruby/bootstraptest/test_insns.rb block in <top (required)> 0x7f6fd6555c58 METHOD ./bootstraptest/runner.rb assert_equal 0x7f6fd6555c20 METHOD ./bootstraptest/runner.rb assert_check 0x7f6fd6555be8 METHOD ./bootstraptest/runner.rb show_progress 0x7f6fd6555bb0 METHOD ./bootstraptest/runner.rb with_stderr 0x7f6fd6555b78 BLOCK ./bootstraptest/runner.rb block in show_progress 0x7f6fd6555b40 BLOCK ./bootstraptest/runner.rb block in assert_check 0x7f6fd6555b08 METHOD ./bootstraptest/runner.rb get_result_string 0x7f6fd6555ad0 METHOD ./bootstraptest/runner.rb make_srcfile 0x7f6fd6555a98 CFUNC 0x7f6fd6555a60 BLOCK ./bootstraptest/runner.rb block in make_srcfile ``` Getting the main execution context is difficult (it is stored in a thread local) so for now you must supply an ec and this will make a backtrace
2020-10-08Fix lldb disassembler so it works with core filesAaron Patterson
This fixes the lldb disassembler script so that it doesn't need a live process when disassembling rb_iseq_t. I also added the PC to the output so you can tell what the VM is executing when it crashed. For example: ``` (lldb) rbdisasm ec->cfp->iseq PC IDX insn_name(operands) 0x56039f0a1720 0000 nop 0x56039f0a1728 0001 getlocal_WC_1( 5 ) 0x56039f0a1738 0003 branchunless( 7 ) 0x56039f0a1748 0005 getlocal_WC_0( 3 ) 0x56039f0a1758 0007 putstring( (VALUE)0x56039f0c7eb8 ) 0x56039f0a1768 0009 opt_send_without_block( (struct rb_call_data *)0x56039f09f140 ) 0x56039f0a1778 0011 pop 0x56039f0a1780 0012 getglobal( ID: 0x7fd7 ) 0x56039f0a1790 0014 branchunless( 7 ) 0x56039f0a17a0 0016 getlocal_WC_0( 3 ) 0x56039f0a17b0 0018 putstring( (VALUE)0x56039f0c7e90 ) 0x56039f0a17c0 0020 opt_send_without_block( (struct rb_call_data *)0x56039f09f150 ) 0x56039f0a17d0 0022 pop 0x56039f0a17d8 0023 getlocal_WC_0( 3 ) 0x56039f0a17e8 0025 putobject( (VALUE)0x56039f0c7e68 ) 0x56039f0a17f8 0027 getlocal_WC_1( 6 ) 0x56039f0a1808 0029 dup 0x56039f0a1810 0030 checktype( 5 ) 0x56039f0a1820 0032 branchif( 4 ) 0x56039f0a1830 0034 dup 0x56039f0a1838 0035 opt_send_without_block( (struct rb_call_data *)0x56039f09f160 ) 0x56039f0a1848 0037 tostring 0x56039f0a1850 0038 putobject( (VALUE)0x56039f0c7e40 ) 0x56039f0a1860 0040 concatstrings( 3 ) 0x56039f0a1870 0042 opt_send_without_block( (struct rb_call_data *)0x56039f09f170 ) 0x56039f0a1880 0044 nop 0x56039f0a1888 0045 leave (lldb) p ec->cfp->pc (const VALUE *) $146 = 0x000056039f0a1848 ``` Here we can see the VM is currently executing `opt_send_without_block` (because the PC is one ahead of the current instruction)
2020-09-28bit table information when printing an objectAaron Patterson
2020-09-22Rudimentary support for disassembling rb_iseq_tAaron Patterson
I need to disassemble instruction sequences while debugging, so I wrote this. Usage is like this: ``` (lldb) p iseq (rb_iseq_t *) $147 = 0x0000000101068400 (lldb) rbdisasm iseq 0000 putspecialobject( 3 ) 0002 putnil 0003 defineclass( ID: 0x560b, (rb_iseq_t *)0x1010681d0, 2 ) 0007 pop 0008 putspecialobject( 3 ) 0010 putnil 0011 defineclass( ID: 0x56eb, (rb_iseq_t *)0x101063b58, 2 ) 0015 leave ``` Also thanks a ton to @kivikakk helping me figure out how to navigate LLDB's Python 😆 Notes: Merged: https://github.com/ruby/ruby/pull/3554
generated by cgit v1.2.3 (git 2.25.1) at 2024-05-06 09:48:46 +0000