| Age | Commit message (Collapse) | Author |
|---|
|
In December 2021, we opened an [issue] to solicit feedback regarding the
porting of the YJIT codebase from C99 to Rust. There were some
reservations, but this project was given the go ahead by Ruby core
developers and Matz. Since then, we have successfully completed the port
of YJIT to Rust.
The new Rust version of YJIT has reached parity with the C version, in
that it passes all the CRuby tests, is able to run all of the YJIT
benchmarks, and performs similarly to the C version (because it works
the same way and largely generates the same machine code). We've even
incorporated some design improvements, such as a more fine-grained
constant invalidation mechanism which we expect will make a big
difference in Ruby on Rails applications.
Because we want to be careful, YJIT is guarded behind a configure
option:
```shell
./configure --enable-yjit # Build YJIT in release mode
./configure --enable-yjit=dev # Build YJIT in dev/debug mode
```
By default, YJIT does not get compiled and cargo/rustc is not required.
If YJIT is built in dev mode, then `cargo` is used to fetch development
dependencies, but when building in release, `cargo` is not required,
only `rustc`. At the moment YJIT requires Rust 1.60.0 or newer.
The YJIT command-line options remain mostly unchanged, and more details
about the build process are documented in `doc/yjit/yjit.md`.
The CI tests have been updated and do not take any more resources than
before.
The development history of the Rust port is available at the following
commit for interested parties:
https://github.com/Shopify/ruby/commit/1fd9573d8b4b65219f1c2407f30a0a60e537f8be
Our hope is that Rust YJIT will be compiled and included as a part of
system packages and compiled binaries of the Ruby 3.2 release. We do not
anticipate any major problems as Rust is well supported on every
platform which YJIT supports, but to make sure that this process works
smoothly, we would like to reach out to those who take care of building
systems packages before the 3.2 release is shipped and resolve any
issues that may come up.
[issue]: https://bugs.ruby-lang.org/issues/18481
Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
Co-authored-by: Noah Gibbs <the.codefolio.guy@gmail.com>
Co-authored-by: Kevin Newton <kddnewton@gmail.com>
Notes:
Merged: https://github.com/ruby/ruby/pull/5826
|
|
- prerequisite of supporting YJIT with VC++.
- note that now can specfily `--yjit` on mswin64, but not enabled
YJIT'ed code because of YJIT requires `OPT_DIRECT_THREADED_CODE`
or `OPT_CALL_THREADED_CODE` in `rb_yjit_compile_iseq`.
|
|
|
|
Previously, YJIT incorrectly discarded the upper 32 bits of the object
pointer when writing out VALUEs to setup default keyword arguments.
In addition to incorrectly truncating, the output pointers were not
properly tracked for handling GC compaction moving the referenced
objects.
YJIT previously attempted to encode a mov instruction with a memory
destination and a 64 bit immediate when there is no such encoding
possible in the ISA. Add an assert to mitigate not being able to
catch this at build time.
Notes:
Merged: https://github.com/ruby/ruby/pull/5274
Merged-By: XrXr
|
|
PUSH DS triggers the #UD processor exception in 64-bit mode, which the
OS translates to a SIGILL. Unlike INT3, this triggers the usual crash
reporter, which makes failures easier to notice. When there is a
debugger attached, the PUSH DS pauses execution just like INT3.
Notes:
Merged: https://github.com/ruby/ruby/pull/5217
|
|
Previously, YJIT assumed that basic blocks never consume more than
1 KiB of memory. This assumption does not hold for long Ruby methods
such as the one in the following:
```ruby
eval(<<RUBY)
def set_local_a_lot
#{'_=0;'*0x40000}
end
RUBY
set_local_a_lot
```
For low `--yjit-exec-mem-size` values, one basic block could exhaust the
entire buffer.
Introduce a new field `codeblock_t::dropped_bytes` that the assembler
sets whenever it runs out of space. Check this field in
gen_single_block() to respond to out of memory situations and other
error conditions. This design avoids making the control flow graph of
existing code generation functions more complex.
Use POSIX shell in misc/test_yjit_asm.sh since bash is expanding
`0%/*/*` differently.
Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
Notes:
Merged: https://github.com/ruby/ruby/pull/5209
|
|
Some platforms don't want memory to be marked as writeable and
executable at the same time. When we write to the code block, we
calculate the OS page that the buffer position maps to. Then we call
`mprotect` to allow writes on that particular page. As an optimization,
we cache the "last written" aligned page which allows us to amortize the
cost of the `mprotect` call. In other words, sequential writes to the
same page will only call `mprotect` on the page once.
When we're done writing, we call `mprotect` on the entire JIT buffer.
This means we don't need to keep track of which pages were marked as
writeable, we let the OS take care of that.
Co-authored-by: John Hawthorn <john@hawthorn.email>
Notes:
Merged: https://github.com/ruby/ruby/pull/5032
|
|
* YJIT: use shorter encoding for mov(r64,imm) when unambiguous
Previously, for small constants such as `mov(RAX, imm_opnd(Qundef))`,
we emit an instruction with an 8-byte immediate. This form commonly
gets the `movabs` mnemonic.
In 64-bit mode, 32-bit operands get zero extended to 64-bit to fill the
register, so when the immediate is small enough, we can save 4 bytes by
using the `mov` variant that takes a 32-bit immediate and does a zero
extension.
Not implement with this change, there is an imm32 variant of `mov` that
does sign extension we could use. When the constant is negative, we
fallback to the `movabs` form.
In railsbench, this change yields roughly a 12% code size reduction for
the outlined block.
Co-authored-by: Jemma Issroff <jemmaissroff@gmail.com>
* [ci skip] comment edit. Please squash.
Co-authored-by: Jemma Issroff <jemmaissroff@gmail.com>
Notes:
Merged-By: maximecb <maximecb@ruby-lang.org>
|
|
* New code page allocation logic
* Fix leaked globals
* Fix leaked symbols, yjit asm tests
* Make COUNTED_EXIT take a jit argument, so we can eliminate global ocb
* Remove extra whitespace
* Change block start_pos/end_pos to be pointers instead of uint32_t
* Change branch end_pos and start_pos to end_addr, start_addr
Notes:
Merged-By: maximecb <maximecb@ruby-lang.org>
|
|
Fixes:
./src/yjit_asm.c:196:8: warning: 'mem_block' may be used uninitialized [-Wmaybe-uninitialized]
|
|
On non RUBY_DEBUG builds, assert() compiles to nothing and the compiler
warns about uninitialized variables in those code paths. Replace
those asserts with rb_bug() to fix the warnings and do the assert in
all builds. Since yjit_asm_tests.c compiles outside of Ruby, it needed
a distinct version of rb_bug().
Also put YJIT_STATS check for function delcaration that is only defined
in YJIT_STATS builds.
|
|
For upstreaming, we want functions we export either prefixed with "rb_"
or made static. Historically we haven't been following this rule, so we
were "leaking" a lot of symbols as `make leak-globals` would tell us.
This change unifies everything YJIT into a single compilation unit,
yjit.o, and makes everything unprefixed static to pass `make leak-globals`.
This manual "unified build" setup is similar to that of vm.o.
Having everything in one compilation unit allows static functions to
be visible across YJIT files and removes the need for declarations in
headers in some cases. Unnecessary declarations were removed.
Other changes of note:
- switched to MJIT_SYMBOL_EXPORT_BEGIN which indicates stuff as being
off limits for native extensions
- the first include of each YJIT file is change to be "internal.h"
- undefined MAP_STACK before explicitly redefining it since it
collide's with a definition in system headers. Consider renaming?
|
|
|
|
Case labels get half an indent and the opening brace is on the same line
as "switch".
|
|
|
|
|
|
It might help figure out why it is failing.
|
|
|
|
|
|
|
|
|
|
* Use INT32_MIN, INT32_MAX, etc. constants in yjit_asm.c
* Print warning on stderr when code past rel32 jump range
* Fix preprocessor snafu
* Move rel32 warning into --yjit-stats
* Try to allocate within rel32 offset on Linux machines
* Update yjit_asm.c
Co-authored-by: Alan Wu <XrXr@users.noreply.github.com>
* On Linux, use sysconf to get the page size
Co-authored-by: Alan Wu <XrXr@users.noreply.github.com>
|
|
|
|
|
|
Mostly unused and uninitialized warnings here and there
|
|
|
|
Renaming uJIT to YJIT. AKA s/ujit/yjit/g.
|
