<feed xmlns='http://www.w3.org/2005/Atom'>
<title>ruby.git/yjit/src/yjit.rs, branch v3_2_11</title>
<subtitle>The Ruby Programming Language</subtitle>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/'/>
<entry>
<title>merge revision(s) 3b83b265f11965582d4b9b439eff8a501792ab68: [Backport #19404]</title>
<updated>2023-02-06T05:00:47+00:00</updated>
<author>
<name>NARUSE, Yui</name>
<email>naruse@airemix.jp</email>
</author>
<published>2023-02-06T05:00:47+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=3a88589399f7f1059be245f766809c49790ad939'/>
<id>3a88589399f7f1059be245f766809c49790ad939</id>
<content type='text'>
	YJIT: Crash with rb_bug() when panicking

	Helps with getting good bug reports in the wild. Intended to be
	backported to the 3.2.x series.
	---
	 yjit/bindgen/src/main.rs       |  3 +++
	 yjit/src/cruby_bindings.inc.rs |  1 +
	 yjit/src/yjit.rs               | 29 +++++++++++++++++++++++++++--
	 3 files changed, 31 insertions(+), 2 deletions(-)
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
	YJIT: Crash with rb_bug() when panicking

	Helps with getting good bug reports in the wild. Intended to be
	backported to the 3.2.x series.
	---
	 yjit/bindgen/src/main.rs       |  3 +++
	 yjit/src/cruby_bindings.inc.rs |  1 +
	 yjit/src/yjit.rs               | 29 +++++++++++++++++++++++++++--
	 3 files changed, 31 insertions(+), 2 deletions(-)
</pre>
</div>
</content>
</entry>
<entry>
<title>YJIT: Add RubyVM::YJIT.code_gc (#6644)</title>
<updated>2022-10-31T18:29:45+00:00</updated>
<author>
<name>Takashi Kokubun</name>
<email>takashikkbn@gmail.com</email>
</author>
<published>2022-10-31T18:29:45+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=2b39640b0bbf7459b305d8a98bb01f197975b8d9'/>
<id>2b39640b0bbf7459b305d8a98bb01f197975b8d9</id>
<content type='text'>
* YJIT: Add RubyVM::YJIT.code_gc

* Rename compiled_page_count to live_page_count</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* YJIT: Add RubyVM::YJIT.code_gc

* Rename compiled_page_count to live_page_count</pre>
</div>
</content>
</entry>
<entry>
<title>Fix code invalidation while OOM and OOM simulation (https://github.com/Shopify/ruby/pull/395)</title>
<updated>2022-08-29T15:47:10+00:00</updated>
<author>
<name>Alan Wu</name>
<email>XrXr@users.noreply.github.com</email>
</author>
<published>2022-08-10T21:22:55+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=4d811d7a2b92d110e3e70cb77e5f499acfa7112a'/>
<id>4d811d7a2b92d110e3e70cb77e5f499acfa7112a</id>
<content type='text'>
`YJIT.simulate_oom!` used to leave one byte of space in the code block,
so our test didn't expose a problem with asserting that the write
position is in bounds in `CodeBlock::set_pos`. We do the following when
patching code:
  1. save current write position
  2. seek to middle of the code block and patch
  3. restore old write position
The bounds check fails on (3) when the code block is already filled up.

Leaving one byte of space also meant that when we write that byte, we
need to fill the entire code region with trapping instruction in
`VirtualMem`, which made the OOM tests unnecessarily slow.

Remove the incorrect bounds check and stop leaving space in the code
block when simulating OOM.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
`YJIT.simulate_oom!` used to leave one byte of space in the code block,
so our test didn't expose a problem with asserting that the write
position is in bounds in `CodeBlock::set_pos`. We do the following when
patching code:
  1. save current write position
  2. seek to middle of the code block and patch
  3. restore old write position
The bounds check fails on (3) when the code block is already filled up.

Leaving one byte of space also meant that when we write that byte, we
need to fill the entire code region with trapping instruction in
`VirtualMem`, which made the OOM tests unnecessarily slow.

Remove the incorrect bounds check and stop leaving space in the code
block when simulating OOM.
</pre>
</div>
</content>
</entry>
<entry>
<title>Rename mjit_exec to jit_exec (#6262)</title>
<updated>2022-08-20T06:57:17+00:00</updated>
<author>
<name>Takashi Kokubun</name>
<email>takashikkbn@gmail.com</email>
</author>
<published>2022-08-20T06:57:17+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=485019c2bd02794b484500c78919b0d1230e4a84'/>
<id>485019c2bd02794b484500c78919b0d1230e4a84</id>
<content type='text'>
* Rename mjit_exec to jit_exec

* Rename mjit_exec_slowpath to mjit_check_iseq

* Remove mjit_exec references from comments</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* Rename mjit_exec to jit_exec

* Rename mjit_exec_slowpath to mjit_check_iseq

* Remove mjit_exec references from comments</pre>
</div>
</content>
</entry>
<entry>
<title>Add ability to trace exit locations in yjit (#5970)</title>
<updated>2022-06-09T16:59:39+00:00</updated>
<author>
<name>Eileen M. Uchitelle</name>
<email>eileencodes@users.noreply.github.com</email>
</author>
<published>2022-06-09T16:59:39+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=473ee328c5be01ac6bb29659afcbe3361664bf68'/>
<id>473ee328c5be01ac6bb29659afcbe3361664bf68</id>
<content type='text'>
When running with `--yjit-stats` turned on, yjit can inform the user
what the most common exits are. While this is useful information it
doesn't tell you the source location of the code that exited or what the
code that exited looks like. This change intends to fix that.

To use the feature, run yjit with the `--yjit-trace-exits` option,
which will record the backtrace for every exit that occurs. This functionality
requires the stats feature to be turned on. Calling `--yjit-trace-exits`
will automatically set the `--yjit-stats` option.

Users must call `RubyVM::YJIT.dump_exit_locations(filename)` which will
Marshal dump the contents of `RubyVM::YJIT.exit_locations` into a file
based on the passed filename.

*Example usage:*

Given the following script, we write to a file called
`concat_array.dump` the results of `RubyVM::YJIT.exit_locations`.

```ruby
def concat_array
  ["t", "r", *x = "u", "e"].join
end

1000.times do
  concat_array
end

RubyVM::YJIT.dump_exit_locations("concat_array.dump")
```

When we run the file with this branch and the appropriate flags the
stacktrace will be recorded. Note Stackprof needs to be installed or you
need to point to the library directly.

```
./ruby --yjit --yjit-call-threshold=1 --yjit-trace-exits -I/Users/eileencodes/open_source/stackprof/lib test.rb
```

We can then read the dump file with Stackprof:

```
./ruby -I/Users/eileencodes/open_source/stackprof/lib/ /Users/eileencodes/open_source/stackprof/bin/stackprof --text concat_array.dump
```

Results will look similar to the following:

```
==================================
  Mode: ()
  Samples: 1817 (0.00% miss rate)
  GC: 0 (0.00%)
==================================
     TOTAL    (pct)     SAMPLES    (pct)     FRAME
      1001  (55.1%)        1001  (55.1%)     concatarray
       335  (18.4%)         335  (18.4%)     invokeblock
       178   (9.8%)         178   (9.8%)     send
       140   (7.7%)         140   (7.7%)     opt_getinlinecache
       ...etc...
```

Simply inspecting the `concatarray` method will give `SOURCE
UNAVAILABLE` because the source is insns.def.

```
./ruby -I/Users/eileencodes/open_source/stackprof/lib/ /Users/eileencodes/open_source/stackprof/bin/stackprof --text concat_array.dump --method concatarray
```

Result:

```
concatarray (nonexistent.def:1)
  samples:  1001 self (55.1%)  /   1001 total (55.1%)
  callers:
    1000  (   99.9%)  Object#concat_array
       1  (    0.1%)  Gem.suffixes
  callees (0 total):
  code:
        SOURCE UNAVAILABLE
```

However if we go deeper to the callee we can see the exact
source of the `concatarray` exit.

```
./ruby -I/Users/eileencodes/open_source/stackprof/lib/ /Users/eileencodes/open_source/stackprof/bin/stackprof --text concat_array.dump --method Object#concat_array
```

```
Object#concat_array (/Users/eileencodes/open_source/rust_ruby/test.rb:1)
  samples:     0 self (0.0%)  /   1000 total (55.0%)
  callers:
    1000  (  100.0%)  block in &lt;main&gt;
  callees (1000 total):
    1000  (  100.0%)  concatarray
  code:
                                  |     1  | def concat_array
 1000   (55.0%)                   |     2  |   ["t", "r", *x = "u", "e"].join
                                  |     3  | end
```

The `--walk` option is recommended for this feature as it make it
easier to traverse the tree of exits.

*Goals of this feature:*

This feature is meant to give more information when working on YJIT.
The idea is that if we know what code is exiting we can decide what
areas to prioritize when fixing exits. In some cases this means adding
prioritizing avoiding certain exits in yjit. In more complex cases it
might mean changing the Ruby code to be more performant when run with
yjit. Ultimately the more information we have about what code is exiting
AND why, the better we can make yjit.

*Known limitations:*

* Due to tracing exits, running this on large codebases like Rails
can be quite slow.
* On complex methods it can still be difficult to pinpoint the exact cause of
an exit.
* Stackprof is a requirement to to view the backtrace information from
the dump file.

Co-authored-by: Aaron Patterson &lt;tenderlove@ruby-lang.org&gt;

Co-authored-by: Aaron Patterson &lt;tenderlove@ruby-lang.org&gt;</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
When running with `--yjit-stats` turned on, yjit can inform the user
what the most common exits are. While this is useful information it
doesn't tell you the source location of the code that exited or what the
code that exited looks like. This change intends to fix that.

To use the feature, run yjit with the `--yjit-trace-exits` option,
which will record the backtrace for every exit that occurs. This functionality
requires the stats feature to be turned on. Calling `--yjit-trace-exits`
will automatically set the `--yjit-stats` option.

Users must call `RubyVM::YJIT.dump_exit_locations(filename)` which will
Marshal dump the contents of `RubyVM::YJIT.exit_locations` into a file
based on the passed filename.

*Example usage:*

Given the following script, we write to a file called
`concat_array.dump` the results of `RubyVM::YJIT.exit_locations`.

```ruby
def concat_array
  ["t", "r", *x = "u", "e"].join
end

1000.times do
  concat_array
end

RubyVM::YJIT.dump_exit_locations("concat_array.dump")
```

When we run the file with this branch and the appropriate flags the
stacktrace will be recorded. Note Stackprof needs to be installed or you
need to point to the library directly.

```
./ruby --yjit --yjit-call-threshold=1 --yjit-trace-exits -I/Users/eileencodes/open_source/stackprof/lib test.rb
```

We can then read the dump file with Stackprof:

```
./ruby -I/Users/eileencodes/open_source/stackprof/lib/ /Users/eileencodes/open_source/stackprof/bin/stackprof --text concat_array.dump
```

Results will look similar to the following:

```
==================================
  Mode: ()
  Samples: 1817 (0.00% miss rate)
  GC: 0 (0.00%)
==================================
     TOTAL    (pct)     SAMPLES    (pct)     FRAME
      1001  (55.1%)        1001  (55.1%)     concatarray
       335  (18.4%)         335  (18.4%)     invokeblock
       178   (9.8%)         178   (9.8%)     send
       140   (7.7%)         140   (7.7%)     opt_getinlinecache
       ...etc...
```

Simply inspecting the `concatarray` method will give `SOURCE
UNAVAILABLE` because the source is insns.def.

```
./ruby -I/Users/eileencodes/open_source/stackprof/lib/ /Users/eileencodes/open_source/stackprof/bin/stackprof --text concat_array.dump --method concatarray
```

Result:

```
concatarray (nonexistent.def:1)
  samples:  1001 self (55.1%)  /   1001 total (55.1%)
  callers:
    1000  (   99.9%)  Object#concat_array
       1  (    0.1%)  Gem.suffixes
  callees (0 total):
  code:
        SOURCE UNAVAILABLE
```

However if we go deeper to the callee we can see the exact
source of the `concatarray` exit.

```
./ruby -I/Users/eileencodes/open_source/stackprof/lib/ /Users/eileencodes/open_source/stackprof/bin/stackprof --text concat_array.dump --method Object#concat_array
```

```
Object#concat_array (/Users/eileencodes/open_source/rust_ruby/test.rb:1)
  samples:     0 self (0.0%)  /   1000 total (55.0%)
  callers:
    1000  (  100.0%)  block in &lt;main&gt;
  callees (1000 total):
    1000  (  100.0%)  concatarray
  code:
                                  |     1  | def concat_array
 1000   (55.0%)                   |     2  |   ["t", "r", *x = "u", "e"].join
                                  |     3  | end
```

The `--walk` option is recommended for this feature as it make it
easier to traverse the tree of exits.

*Goals of this feature:*

This feature is meant to give more information when working on YJIT.
The idea is that if we know what code is exiting we can decide what
areas to prioritize when fixing exits. In some cases this means adding
prioritizing avoiding certain exits in yjit. In more complex cases it
might mean changing the Ruby code to be more performant when run with
yjit. Ultimately the more information we have about what code is exiting
AND why, the better we can make yjit.

*Known limitations:*

* Due to tracing exits, running this on large codebases like Rails
can be quite slow.
* On complex methods it can still be difficult to pinpoint the exact cause of
an exit.
* Stackprof is a requirement to to view the backtrace information from
the dump file.

Co-authored-by: Aaron Patterson &lt;tenderlove@ruby-lang.org&gt;

Co-authored-by: Aaron Patterson &lt;tenderlove@ruby-lang.org&gt;</pre>
</div>
</content>
</entry>
<entry>
<title>YJIT: Enable default rustc lints (warnings) (#5864)</title>
<updated>2022-04-29T22:20:23+00:00</updated>
<author>
<name>Alan Wu</name>
<email>XrXr@users.noreply.github.com</email>
</author>
<published>2022-04-29T22:20:23+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=5c843a1a6e24aeabb3497065a362caf7b3e2d3b1'/>
<id>5c843a1a6e24aeabb3497065a362caf7b3e2d3b1</id>
<content type='text'>
`rustc` performs in depth dead code analysis and issues warning
even for things like unused struct fields and unconstructed enum
variants. This was annoying for us during the port but hopefully
they are less of an issue now.

This patch enables all the unused warnings we disabled and address
all the warnings we previously ignored. Generally, the approach I've
taken is to use `cfg!` instead of using the `cfg` attribute and
to delete code where it makes sense. I've put `#[allow(unused)]`
on things we intentionally keep around for printf style debugging
and on items that are too annoying to keep warning-free in all
build configs.</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
`rustc` performs in depth dead code analysis and issues warning
even for things like unused struct fields and unconstructed enum
variants. This was annoying for us during the port but hopefully
they are less of an issue now.

This patch enables all the unused warnings we disabled and address
all the warnings we previously ignored. Generally, the approach I've
taken is to use `cfg!` instead of using the `cfg` attribute and
to delete code where it makes sense. I've put `#[allow(unused)]`
on things we intentionally keep around for printf style debugging
and on items that are too annoying to keep warning-free in all
build configs.</pre>
</div>
</content>
</entry>
<entry>
<title>Rust YJIT</title>
<updated>2022-04-27T15:00:22+00:00</updated>
<author>
<name>Alan Wu</name>
<email>alanwu@ruby-lang.org</email>
</author>
<published>2022-04-19T18:40:21+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=f90549cd38518231a6a74432fe1168c943a7cc18'/>
<id>f90549cd38518231a6a74432fe1168c943a7cc18</id>
<content type='text'>
In December 2021, we opened an [issue] to solicit feedback regarding the
porting of the YJIT codebase from C99 to Rust. There were some
reservations, but this project was given the go ahead by Ruby core
developers and Matz. Since then, we have successfully completed the port
of YJIT to Rust.

The new Rust version of YJIT has reached parity with the C version, in
that it passes all the CRuby tests, is able to run all of the YJIT
benchmarks, and performs similarly to the C version (because it works
the same way and largely generates the same machine code). We've even
incorporated some design improvements, such as a more fine-grained
constant invalidation mechanism which we expect will make a big
difference in Ruby on Rails applications.

Because we want to be careful, YJIT is guarded behind a configure
option:

```shell
./configure --enable-yjit # Build YJIT in release mode
./configure --enable-yjit=dev # Build YJIT in dev/debug mode
```

By default, YJIT does not get compiled and cargo/rustc is not required.
If YJIT is built in dev mode, then `cargo` is used to fetch development
dependencies, but when building in release, `cargo` is not required,
only `rustc`. At the moment YJIT requires Rust 1.60.0 or newer.

The YJIT command-line options remain mostly unchanged, and more details
about the build process are documented in `doc/yjit/yjit.md`.

The CI tests have been updated and do not take any more resources than
before.

The development history of the Rust port is available at the following
commit for interested parties:
https://github.com/Shopify/ruby/commit/1fd9573d8b4b65219f1c2407f30a0a60e537f8be

Our hope is that Rust YJIT will be compiled and included as a part of
system packages and compiled binaries of the Ruby 3.2 release. We do not
anticipate any major problems as Rust is well supported on every
platform which YJIT supports, but to make sure that this process works
smoothly, we would like to reach out to those who take care of building
systems packages before the 3.2 release is shipped and resolve any
issues that may come up.

[issue]: https://bugs.ruby-lang.org/issues/18481

Co-authored-by: Maxime Chevalier-Boisvert &lt;maximechevalierb@gmail.com&gt;
Co-authored-by: Noah Gibbs &lt;the.codefolio.guy@gmail.com&gt;
Co-authored-by: Kevin Newton &lt;kddnewton@gmail.com&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
In December 2021, we opened an [issue] to solicit feedback regarding the
porting of the YJIT codebase from C99 to Rust. There were some
reservations, but this project was given the go ahead by Ruby core
developers and Matz. Since then, we have successfully completed the port
of YJIT to Rust.

The new Rust version of YJIT has reached parity with the C version, in
that it passes all the CRuby tests, is able to run all of the YJIT
benchmarks, and performs similarly to the C version (because it works
the same way and largely generates the same machine code). We've even
incorporated some design improvements, such as a more fine-grained
constant invalidation mechanism which we expect will make a big
difference in Ruby on Rails applications.

Because we want to be careful, YJIT is guarded behind a configure
option:

```shell
./configure --enable-yjit # Build YJIT in release mode
./configure --enable-yjit=dev # Build YJIT in dev/debug mode
```

By default, YJIT does not get compiled and cargo/rustc is not required.
If YJIT is built in dev mode, then `cargo` is used to fetch development
dependencies, but when building in release, `cargo` is not required,
only `rustc`. At the moment YJIT requires Rust 1.60.0 or newer.

The YJIT command-line options remain mostly unchanged, and more details
about the build process are documented in `doc/yjit/yjit.md`.

The CI tests have been updated and do not take any more resources than
before.

The development history of the Rust port is available at the following
commit for interested parties:
https://github.com/Shopify/ruby/commit/1fd9573d8b4b65219f1c2407f30a0a60e537f8be

Our hope is that Rust YJIT will be compiled and included as a part of
system packages and compiled binaries of the Ruby 3.2 release. We do not
anticipate any major problems as Rust is well supported on every
platform which YJIT supports, but to make sure that this process works
smoothly, we would like to reach out to those who take care of building
systems packages before the 3.2 release is shipped and resolve any
issues that may come up.

[issue]: https://bugs.ruby-lang.org/issues/18481

Co-authored-by: Maxime Chevalier-Boisvert &lt;maximechevalierb@gmail.com&gt;
Co-authored-by: Noah Gibbs &lt;the.codefolio.guy@gmail.com&gt;
Co-authored-by: Kevin Newton &lt;kddnewton@gmail.com&gt;
</pre>
</div>
</content>
</entry>
</feed>
