<feed xmlns='http://www.w3.org/2005/Atom'>
<title>ruby.git/yjit/src/asm/x86_64/tests.rs, branch v3_2_11</title>
<subtitle>The Ruby Programming Language</subtitle>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/'/>
<entry>
<title>Fix YJIT backend to account for unsigned int immediates (#6789)</title>
<updated>2022-11-23T15:48:17+00:00</updated>
<author>
<name>Jemma Issroff</name>
<email>jemmaissroff@gmail.com</email>
</author>
<published>2022-11-23T15:48:17+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=e82b15b6603ddc6754f4cfa7a189c0acb0ccce71'/>
<id>e82b15b6603ddc6754f4cfa7a189c0acb0ccce71</id>
<content type='text'>
YJIT: x86_64: Fix cmp with number where sign bit is set

Before this commit, we were unconditionally treating unsigned ints as
signed ints when counting the number of bits required for representing
the immediate in machine code. When the size of the immediate matches
the size of the other operand, no sign extension happens, so this was
incorrect. `asm.cmp(opnd64, 0x8000_0000)` panicked even though it's
encodable as `CMP r/m32, imm32`. Large shape ids were impacted by this
issue.

Co-Authored-By: Aaron Patterson &lt;tenderlove@ruby-lang.org&gt;
Co-Authored-By: Alan Wu &lt;alanwu@ruby-lang.org&gt;

Co-authored-by: Aaron Patterson &lt;tenderlove@ruby-lang.org&gt;
Co-authored-by: Alan Wu &lt;alanwu@ruby-lang.org&gt;</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
YJIT: x86_64: Fix cmp with number where sign bit is set

Before this commit, we were unconditionally treating unsigned ints as
signed ints when counting the number of bits required for representing
the immediate in machine code. When the size of the immediate matches
the size of the other operand, no sign extension happens, so this was
incorrect. `asm.cmp(opnd64, 0x8000_0000)` panicked even though it's
encodable as `CMP r/m32, imm32`. Large shape ids were impacted by this
issue.

Co-Authored-By: Aaron Patterson &lt;tenderlove@ruby-lang.org&gt;
Co-Authored-By: Alan Wu &lt;alanwu@ruby-lang.org&gt;

Co-authored-by: Aaron Patterson &lt;tenderlove@ruby-lang.org&gt;
Co-authored-by: Alan Wu &lt;alanwu@ruby-lang.org&gt;</pre>
</div>
</content>
</entry>
<entry>
<title>YJIT: Always encode Opnd::Value in 64 bits on x86_64 for GC offsets (#6733)</title>
<updated>2022-11-15T23:23:20+00:00</updated>
<author>
<name>Takashi Kokubun</name>
<email>takashikkbn@gmail.com</email>
</author>
<published>2022-11-15T23:23:20+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=41b0f641ef0671d8cde397e56b1eb3c6b8e0f0db'/>
<id>41b0f641ef0671d8cde397e56b1eb3c6b8e0f0db</id>
<content type='text'>
* YJIT: Always encode Opnd::Value in 64 bits on x86_64

for GC offsets

Co-authored-by: Alan Wu &lt;alansi.xingwu@shopify.com&gt;

* Introduce heap_object_p

* Leave original mov intact

* Remove unneeded branches

* Add a test for movabs

Co-authored-by: Alan Wu &lt;alansi.xingwu@shopify.com&gt;</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* YJIT: Always encode Opnd::Value in 64 bits on x86_64

for GC offsets

Co-authored-by: Alan Wu &lt;alansi.xingwu@shopify.com&gt;

* Introduce heap_object_p

* Leave original mov intact

* Remove unneeded branches

* Add a test for movabs

Co-authored-by: Alan Wu &lt;alansi.xingwu@shopify.com&gt;</pre>
</div>
</content>
</entry>
<entry>
<title>YJIT: fold the "asm_comments" feature into "disasm" (#6591)</title>
<updated>2022-10-19T18:03:07+00:00</updated>
<author>
<name>Alan Wu</name>
<email>XrXr@users.noreply.github.com</email>
</author>
<published>2022-10-19T18:03:07+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=5ca23caa2057fc4760fbefab6087371b11c4bc6c'/>
<id>5ca23caa2057fc4760fbefab6087371b11c4bc6c</id>
<content type='text'>
Previously, enabling only "disasm" didn't actually build. Since these
two features are closely related and we don't really use one without the
other, let's simplify and merge the two features together.</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Previously, enabling only "disasm" didn't actually build. Since these
two features are closely related and we don't really use one without the
other, let's simplify and merge the two features together.</pre>
</div>
</content>
</entry>
<entry>
<title>* Arm64 Beginnings (https://github.com/Shopify/ruby/pull/291)</title>
<updated>2022-08-29T15:46:54+00:00</updated>
<author>
<name>Maxime Chevalier-Boisvert</name>
<email>maxime.chevalierboisvert@shopify.com</email>
</author>
<published>2022-06-15T17:10:13+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=a1b8c947380716a5ffca2b1888a6310e8132b00c'/>
<id>a1b8c947380716a5ffca2b1888a6310e8132b00c</id>
<content type='text'>
* Initial setup for aarch64

* ADDS and SUBS

* ADD and SUB for immediates

* Revert moved code

* Documentation

* Rename Arm64* to A64*

* Comments on shift types

* Share sig_imm_size and unsig_imm_size
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* Initial setup for aarch64

* ADDS and SUBS

* ADD and SUB for immediates

* Revert moved code

* Documentation

* Rename Arm64* to A64*

* Comments on shift types

* Share sig_imm_size and unsig_imm_size
</pre>
</div>
</content>
</entry>
<entry>
<title>YJIT: On-demand executable memory allocation; faster boot (#5944)</title>
<updated>2022-06-14T14:23:13+00:00</updated>
<author>
<name>Alan Wu</name>
<email>XrXr@users.noreply.github.com</email>
</author>
<published>2022-06-14T14:23:13+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=9f09397bfe6762bf19ef47b2f60988e49b80560d'/>
<id>9f09397bfe6762bf19ef47b2f60988e49b80560d</id>
<content type='text'>
This commit makes YJIT allocate memory for generated code gradually as
needed. Previously, YJIT allocates all the memory it needs on boot in
one go, leading to higher than necessary resident set size (RSS) and
time spent on boot initializing the memory with a large memset().

Users should no longer need to search for a magic number to pass to
`--yjit-exec-mem` since physical memory consumption should now more
accurately reflect the requirement of the workload.

YJIT now reserves a range of addresses on boot. This region start out
with no access permission at all so buggy attempts to jump to the region
crashes like before this change. To get this hardening at finer
granularity than the page size, we fill each page with trapping
instructions when we first allocate physical memory for the page.

Most of the time applications don't need 256 MiB of executable code, so
allocating on-demand ends up doing less total work than before. Case in
point, a simple `ruby --yjit-call-threshold=1 -eitself` takes about
half as long after this change. In terms of memory consumption, here is
a table to give a rough summary of the impact:

    | Peak RSS in MiB | -eitself example | railsbench once |
    | :-------------: | ---------------: | --------------: |
    |     before      |              265 |             377 |
    |      after      |               11 |             143 |
    |     no YJIT     |               10 |             101 |

A new module is introduced to handle allocation bookkeeping.
`CodePtr` is moved into the module since it has a close relationship
with the new `VirtualMemory` struct. This new interface has a slightly
smaller surface than before in that marking a region as writable is no
longer a public operation.</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This commit makes YJIT allocate memory for generated code gradually as
needed. Previously, YJIT allocates all the memory it needs on boot in
one go, leading to higher than necessary resident set size (RSS) and
time spent on boot initializing the memory with a large memset().

Users should no longer need to search for a magic number to pass to
`--yjit-exec-mem` since physical memory consumption should now more
accurately reflect the requirement of the workload.

YJIT now reserves a range of addresses on boot. This region start out
with no access permission at all so buggy attempts to jump to the region
crashes like before this change. To get this hardening at finer
granularity than the page size, we fill each page with trapping
instructions when we first allocate physical memory for the page.

Most of the time applications don't need 256 MiB of executable code, so
allocating on-demand ends up doing less total work than before. Case in
point, a simple `ruby --yjit-call-threshold=1 -eitself` takes about
half as long after this change. In terms of memory consumption, here is
a table to give a rough summary of the impact:

    | Peak RSS in MiB | -eitself example | railsbench once |
    | :-------------: | ---------------: | --------------: |
    |     before      |              265 |             377 |
    |      after      |               11 |             143 |
    |     no YJIT     |               10 |             101 |

A new module is introduced to handle allocation bookkeeping.
`CodePtr` is moved into the module since it has a close relationship
with the new `VirtualMemory` struct. This new interface has a slightly
smaller surface than before in that marking a region as writable is no
longer a public operation.</pre>
</div>
</content>
</entry>
<entry>
<title>YJIT: Remove redundant `extern crate` (#5869)</title>
<updated>2022-05-02T14:05:01+00:00</updated>
<author>
<name>Koichi ITO</name>
<email>koic.ito@gmail.com</email>
</author>
<published>2022-05-02T14:05:01+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=8587bacc252e95e533d319cc58b58ec11e5561ff'/>
<id>8587bacc252e95e533d319cc58b58ec11e5561ff</id>
<content type='text'>
Follow up https://github.com/ruby/ruby/commit/0514d81

Rust YJIT requires Rust 1.60.0 or later. So, `extern crate` looks unnecessary
because it can use the following Rust 2018 edition feature:
https://doc.rust-lang.org/stable/edition-guide/rust-2018/path-changes.html#no-more-extern-crate

It passes the following tests.

```console
% cd yjit
% cargo test --features asm_comments,disasm
(snip)

test result: ok. 56 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
```</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Follow up https://github.com/ruby/ruby/commit/0514d81

Rust YJIT requires Rust 1.60.0 or later. So, `extern crate` looks unnecessary
because it can use the following Rust 2018 edition feature:
https://doc.rust-lang.org/stable/edition-guide/rust-2018/path-changes.html#no-more-extern-crate

It passes the following tests.

```console
% cd yjit
% cargo test --features asm_comments,disasm
(snip)

test result: ok. 56 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
```</pre>
</div>
</content>
</entry>
<entry>
<title>YJIT: Enable default rustc lints (warnings) (#5864)</title>
<updated>2022-04-29T22:20:23+00:00</updated>
<author>
<name>Alan Wu</name>
<email>XrXr@users.noreply.github.com</email>
</author>
<published>2022-04-29T22:20:23+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=5c843a1a6e24aeabb3497065a362caf7b3e2d3b1'/>
<id>5c843a1a6e24aeabb3497065a362caf7b3e2d3b1</id>
<content type='text'>
`rustc` performs in depth dead code analysis and issues warning
even for things like unused struct fields and unconstructed enum
variants. This was annoying for us during the port but hopefully
they are less of an issue now.

This patch enables all the unused warnings we disabled and address
all the warnings we previously ignored. Generally, the approach I've
taken is to use `cfg!` instead of using the `cfg` attribute and
to delete code where it makes sense. I've put `#[allow(unused)]`
on things we intentionally keep around for printf style debugging
and on items that are too annoying to keep warning-free in all
build configs.</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
`rustc` performs in depth dead code analysis and issues warning
even for things like unused struct fields and unconstructed enum
variants. This was annoying for us during the port but hopefully
they are less of an issue now.

This patch enables all the unused warnings we disabled and address
all the warnings we previously ignored. Generally, the approach I've
taken is to use `cfg!` instead of using the `cfg` attribute and
to delete code where it makes sense. I've put `#[allow(unused)]`
on things we intentionally keep around for printf style debugging
and on items that are too annoying to keep warning-free in all
build configs.</pre>
</div>
</content>
</entry>
<entry>
<title>YJIT: Adopt Clippy suggestions we like</title>
<updated>2022-04-29T19:03:45+00:00</updated>
<author>
<name>Alan Wu</name>
<email>XrXr@users.noreply.github.com</email>
</author>
<published>2022-04-27T18:08:52+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=fead7107abc494ef051fd26357c21a546b49c7d9'/>
<id>fead7107abc494ef051fd26357c21a546b49c7d9</id>
<content type='text'>
This adopts most suggestions that rust-clippy is confident enough to
auto apply. The manual changes mostly fix manual if-lets and take
opportunities to use the `Default` trait on standard collections.

Co-authored-by: Kevin Newton &lt;kddnewton@gmail.com&gt;
Co-authored-by: Maxime Chevalier-Boisvert &lt;maxime.chevalierboisvert@shopify.com&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This adopts most suggestions that rust-clippy is confident enough to
auto apply. The manual changes mostly fix manual if-lets and take
opportunities to use the `Default` trait on standard collections.

Co-authored-by: Kevin Newton &lt;kddnewton@gmail.com&gt;
Co-authored-by: Maxime Chevalier-Boisvert &lt;maxime.chevalierboisvert@shopify.com&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>Rust YJIT</title>
<updated>2022-04-27T15:00:22+00:00</updated>
<author>
<name>Alan Wu</name>
<email>alanwu@ruby-lang.org</email>
</author>
<published>2022-04-19T18:40:21+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=f90549cd38518231a6a74432fe1168c943a7cc18'/>
<id>f90549cd38518231a6a74432fe1168c943a7cc18</id>
<content type='text'>
In December 2021, we opened an [issue] to solicit feedback regarding the
porting of the YJIT codebase from C99 to Rust. There were some
reservations, but this project was given the go ahead by Ruby core
developers and Matz. Since then, we have successfully completed the port
of YJIT to Rust.

The new Rust version of YJIT has reached parity with the C version, in
that it passes all the CRuby tests, is able to run all of the YJIT
benchmarks, and performs similarly to the C version (because it works
the same way and largely generates the same machine code). We've even
incorporated some design improvements, such as a more fine-grained
constant invalidation mechanism which we expect will make a big
difference in Ruby on Rails applications.

Because we want to be careful, YJIT is guarded behind a configure
option:

```shell
./configure --enable-yjit # Build YJIT in release mode
./configure --enable-yjit=dev # Build YJIT in dev/debug mode
```

By default, YJIT does not get compiled and cargo/rustc is not required.
If YJIT is built in dev mode, then `cargo` is used to fetch development
dependencies, but when building in release, `cargo` is not required,
only `rustc`. At the moment YJIT requires Rust 1.60.0 or newer.

The YJIT command-line options remain mostly unchanged, and more details
about the build process are documented in `doc/yjit/yjit.md`.

The CI tests have been updated and do not take any more resources than
before.

The development history of the Rust port is available at the following
commit for interested parties:
https://github.com/Shopify/ruby/commit/1fd9573d8b4b65219f1c2407f30a0a60e537f8be

Our hope is that Rust YJIT will be compiled and included as a part of
system packages and compiled binaries of the Ruby 3.2 release. We do not
anticipate any major problems as Rust is well supported on every
platform which YJIT supports, but to make sure that this process works
smoothly, we would like to reach out to those who take care of building
systems packages before the 3.2 release is shipped and resolve any
issues that may come up.

[issue]: https://bugs.ruby-lang.org/issues/18481

Co-authored-by: Maxime Chevalier-Boisvert &lt;maximechevalierb@gmail.com&gt;
Co-authored-by: Noah Gibbs &lt;the.codefolio.guy@gmail.com&gt;
Co-authored-by: Kevin Newton &lt;kddnewton@gmail.com&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
In December 2021, we opened an [issue] to solicit feedback regarding the
porting of the YJIT codebase from C99 to Rust. There were some
reservations, but this project was given the go ahead by Ruby core
developers and Matz. Since then, we have successfully completed the port
of YJIT to Rust.

The new Rust version of YJIT has reached parity with the C version, in
that it passes all the CRuby tests, is able to run all of the YJIT
benchmarks, and performs similarly to the C version (because it works
the same way and largely generates the same machine code). We've even
incorporated some design improvements, such as a more fine-grained
constant invalidation mechanism which we expect will make a big
difference in Ruby on Rails applications.

Because we want to be careful, YJIT is guarded behind a configure
option:

```shell
./configure --enable-yjit # Build YJIT in release mode
./configure --enable-yjit=dev # Build YJIT in dev/debug mode
```

By default, YJIT does not get compiled and cargo/rustc is not required.
If YJIT is built in dev mode, then `cargo` is used to fetch development
dependencies, but when building in release, `cargo` is not required,
only `rustc`. At the moment YJIT requires Rust 1.60.0 or newer.

The YJIT command-line options remain mostly unchanged, and more details
about the build process are documented in `doc/yjit/yjit.md`.

The CI tests have been updated and do not take any more resources than
before.

The development history of the Rust port is available at the following
commit for interested parties:
https://github.com/Shopify/ruby/commit/1fd9573d8b4b65219f1c2407f30a0a60e537f8be

Our hope is that Rust YJIT will be compiled and included as a part of
system packages and compiled binaries of the Ruby 3.2 release. We do not
anticipate any major problems as Rust is well supported on every
platform which YJIT supports, but to make sure that this process works
smoothly, we would like to reach out to those who take care of building
systems packages before the 3.2 release is shipped and resolve any
issues that may come up.

[issue]: https://bugs.ruby-lang.org/issues/18481

Co-authored-by: Maxime Chevalier-Boisvert &lt;maximechevalierb@gmail.com&gt;
Co-authored-by: Noah Gibbs &lt;the.codefolio.guy@gmail.com&gt;
Co-authored-by: Kevin Newton &lt;kddnewton@gmail.com&gt;
</pre>
</div>
</content>
</entry>
</feed>
