<feed xmlns='http://www.w3.org/2005/Atom'>
<title>ruby.git/shape.h, branch v3_3_11</title>
<subtitle>The Ruby Programming Language</subtitle>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/'/>
<entry>
<title>[3.3 backport] Do not emit shape transition warnings when YJIT is compiling (#10911)</title>
<updated>2024-06-04T20:21:58+00:00</updated>
<author>
<name>Jean byroot Boussier</name>
<email>jean.boussier+github@shopify.com</email>
</author>
<published>2024-06-04T20:21:58+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=4f00d98b327e3aa23564aa765488d15bc60c9e79'/>
<id>4f00d98b327e3aa23564aa765488d15bc60c9e79</id>
<content type='text'>
Do not emit shape transition warnings when YJIT is compiling

[Bug #20522]

If `Warning.warn` is redefined in Ruby, emitting a warning would invoke
Ruby code, which can't safely be done when YJIT is compiling.

Co-authored-by: Jean Boussier &lt;jean.boussier@gmail.com&gt;
Co-authored-by: Takashi Kokubun &lt;takashikkbn@gmail.com&gt;</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Do not emit shape transition warnings when YJIT is compiling

[Bug #20522]

If `Warning.warn` is redefined in Ruby, emitting a warning would invoke
Ruby code, which can't safely be done when YJIT is compiling.

Co-authored-by: Jean Boussier &lt;jean.boussier@gmail.com&gt;
Co-authored-by: Takashi Kokubun &lt;takashikkbn@gmail.com&gt;</pre>
</div>
</content>
</entry>
<entry>
<title>Use 32-bit integers for redblack_id_t</title>
<updated>2023-12-04T18:57:12+00:00</updated>
<author>
<name>Peter Zhu</name>
<email>peter@peterzhu.ca</email>
</author>
<published>2023-12-04T14:26:26+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=71babe5536bdb2238509752d8706194ee57ff485'/>
<id>71babe5536bdb2238509752d8706194ee57ff485</id>
<content type='text'>
On 32-bit systems, the shape cache size is 1048576 (value of
REDBLACK_CACHE_SIZE), but a 16-bit unsigned integer can only go up to
65536. This means that the redblack_id_t can overflow and lead to a
corrupted red-black tree.

The following script crashes on 32-bit systems:

    o = Object.new
    1_000_000.times do |i|
      o.instance_variable_set(:"@i#{i}", i)
    end
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
On 32-bit systems, the shape cache size is 1048576 (value of
REDBLACK_CACHE_SIZE), but a 16-bit unsigned integer can only go up to
65536. This means that the redblack_id_t can overflow and lead to a
corrupted red-black tree.

The following script crashes on 32-bit systems:

    o = Object.new
    1_000_000.times do |i|
      o.instance_variable_set(:"@i#{i}", i)
    end
</pre>
</div>
</content>
</entry>
<entry>
<title>Don't try compacting ivars on Classes that are "too complex"</title>
<updated>2023-11-21T00:09:48+00:00</updated>
<author>
<name>Aaron Patterson</name>
<email>tenderlove@ruby-lang.org</email>
</author>
<published>2023-10-25T23:52:37+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=6fce8c79807e69cfe475b5291e892567c869fbcc'/>
<id>6fce8c79807e69cfe475b5291e892567c869fbcc</id>
<content type='text'>
Too complex classes use a hash table to store ivs, and should always pin
their IVs.  We shouldn't touch those classes in compaction.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Too complex classes use a hash table to store ivs, and should always pin
their IVs.  We shouldn't touch those classes in compaction.
</pre>
</div>
</content>
</entry>
<entry>
<title>Refactor rb_obj_evacuate_ivs_to_hash_table</title>
<updated>2023-11-17T08:19:21+00:00</updated>
<author>
<name>Jean Boussier</name>
<email>byroot@ruby-lang.org</email>
</author>
<published>2023-11-16T16:50:21+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=94c9f166632a901e563463933efd42e618432d70'/>
<id>94c9f166632a901e563463933efd42e618432d70</id>
<content type='text'>
That function is a bit too low level to called from multiple
places. It's always used in tandem with `rb_shape_set_too_complex`
and both have to know how the object is laid out to update the
`iv_ptr`.

So instead we can provide two higher level function:

  - `rb_obj_copy_ivs_to_hash_table` to prepare a `st_table` from an
    arbitrary oject.
  - `rb_obj_convert_to_too_complex` to assign the new `st_table`
    to the old object, and safely free the old `iv_ptr`.

Unfortunately both can't be combined into one, because `rb_obj_copy_ivar`
need `rb_obj_copy_ivs_to_hash_table` to copy from one object
to another.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
That function is a bit too low level to called from multiple
places. It's always used in tandem with `rb_shape_set_too_complex`
and both have to know how the object is laid out to update the
`iv_ptr`.

So instead we can provide two higher level function:

  - `rb_obj_copy_ivs_to_hash_table` to prepare a `st_table` from an
    arbitrary oject.
  - `rb_obj_convert_to_too_complex` to assign the new `st_table`
    to the old object, and safely free the old `iv_ptr`.

Unfortunately both can't be combined into one, because `rb_obj_copy_ivar`
need `rb_obj_copy_ivs_to_hash_table` to copy from one object
to another.
</pre>
</div>
</content>
</entry>
<entry>
<title>Revert "Revert "Remove SHAPE_CAPACITY_CHANGE shapes""</title>
<updated>2023-11-13T23:26:36+00:00</updated>
<author>
<name>Peter Zhu</name>
<email>peter@peterzhu.ca</email>
</author>
<published>2023-11-10T21:17:39+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=68869e9bd921e4021a9ee8eee5a36377dd2b2a90'/>
<id>68869e9bd921e4021a9ee8eee5a36377dd2b2a90</id>
<content type='text'>
This reverts commit 5f3fb4f4e397735783743fe52a7899b614bece20.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This reverts commit 5f3fb4f4e397735783743fe52a7899b614bece20.
</pre>
</div>
</content>
</entry>
<entry>
<title>Revert "Remove SHAPE_CAPACITY_CHANGE shapes"</title>
<updated>2023-11-10T16:27:49+00:00</updated>
<author>
<name>Peter Zhu</name>
<email>peter@peterzhu.ca</email>
</author>
<published>2023-11-10T16:27:49+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=5f3fb4f4e397735783743fe52a7899b614bece20'/>
<id>5f3fb4f4e397735783743fe52a7899b614bece20</id>
<content type='text'>
This reverts commit f6910a61122931e4193bcc0fad18d839c319b720.

We're seeing crashes in the test suite of Shopify's core monolith after
this change.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This reverts commit f6910a61122931e4193bcc0fad18d839c319b720.

We're seeing crashes in the test suite of Shopify's core monolith after
this change.
</pre>
</div>
</content>
</entry>
<entry>
<title>Remove SHAPE_CAPACITY_CHANGE shapes</title>
<updated>2023-11-09T14:25:02+00:00</updated>
<author>
<name>Peter Zhu</name>
<email>peter@peterzhu.ca</email>
</author>
<published>2023-11-08T15:19:06+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=f6910a61122931e4193bcc0fad18d839c319b720'/>
<id>f6910a61122931e4193bcc0fad18d839c319b720</id>
<content type='text'>
We don't need to create a shape to transition capacity as we can
transition the capacity when the capacity of the SHAPE_IVAR changes.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
We don't need to create a shape to transition capacity as we can
transition the capacity when the capacity of the SHAPE_IVAR changes.
</pre>
</div>
</content>
</entry>
<entry>
<title>Refactor rb_shape_transition_shape_capa out</title>
<updated>2023-11-08T10:02:55+00:00</updated>
<author>
<name>Jean Boussier</name>
<email>byroot@ruby-lang.org</email>
</author>
<published>2023-11-07T17:09:55+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=d898e8d6f89fba34a9ee5c0e139f38ac807059e6'/>
<id>d898e8d6f89fba34a9ee5c0e139f38ac807059e6</id>
<content type='text'>
Right now the `rb_shape_get_next` shape caller need to
first check if there is capacity left, and if not call
`rb_shape_transition_shape_capa` before it can call `rb_shape_get_next`.

And on each of these it needs to checks if we got a TOO_COMPLEX
back.

All this logic is duplicated in the interpreter, YJIT and RJIT.

Instead we can have `rb_shape_get_next` do the capacity transition
when needed. The caller can compare the old and new shapes capacity
to know if resizing is needed. It also can check for TOO_COMPLEX
only once.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Right now the `rb_shape_get_next` shape caller need to
first check if there is capacity left, and if not call
`rb_shape_transition_shape_capa` before it can call `rb_shape_get_next`.

And on each of these it needs to checks if we got a TOO_COMPLEX
back.

All this logic is duplicated in the interpreter, YJIT and RJIT.

Instead we can have `rb_shape_get_next` do the capacity transition
when needed. The caller can compare the old and new shapes capacity
to know if resizing is needed. It also can check for TOO_COMPLEX
only once.
</pre>
</div>
</content>
</entry>
<entry>
<title>vm_getivar: assume the cached shape_id like have a common ancestor</title>
<updated>2023-11-03T11:47:43+00:00</updated>
<author>
<name>Jean Boussier</name>
<email>byroot@ruby-lang.org</email>
</author>
<published>2023-10-26T09:08:05+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=b92b9e1e9e078d6bc1066a9335efb9e31f3bd4d9'/>
<id>b92b9e1e9e078d6bc1066a9335efb9e31f3bd4d9</id>
<content type='text'>
When an inline cache misses, it is very likely that the stale shape_id
and the current instance shape_id have a close common ancestor.

For example if the instance variable is sometimes frozen sometimes
not, one of the two shape will be the direct parent of the other.

Another pattern that commonly cause IC misses is "memoization",
in such case the object will have a "base common shape" and then
a number of close descendants.

In addition, when we find a common ancestor, we store it in the
inline cache instead of the current shape. This help prevent the
cache from flip-flopping, ensuring the next lookup will be marginally
faster and more generally avoid writing in memory too much.

However, now that shapes have an ancestors index, we only check
for a few ancestors before falling back to use the index.

So overall this change speeds up what is assumed to be the more common
case, but makes what is assumed to be the less common case a bit slower.

```
compare-ruby: ruby 3.3.0dev (2023-10-26T05:30:17Z master 701ca070b4) [arm64-darwin22]
built-ruby: ruby 3.3.0dev (2023-10-26T09:25:09Z shapes_double_sear.. a723a85235) [arm64-darwin22]
warming up......

|                                     |compare-ruby|built-ruby|
|:------------------------------------|-----------:|---------:|
|vm_ivar_stable_shape                 |     11.672M|   11.679M|
|                                     |           -|     1.00x|
|vm_ivar_memoize_unstable_shape       |      7.551M|   10.506M|
|                                     |           -|     1.39x|
|vm_ivar_memoize_unstable_shape_miss  |     11.591M|   11.624M|
|                                     |           -|     1.00x|
|vm_ivar_unstable_undef               |      9.037M|    7.981M|
|                                     |       1.13x|         -|
|vm_ivar_divergent_shape              |      8.034M|    6.657M|
|                                     |       1.21x|         -|
|vm_ivar_divergent_shape_imbalanced   |     10.471M|    9.231M|
|                                     |       1.13x|         -|
```

Co-Authored-By: John Hawthorn &lt;john@hawthorn.email&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
When an inline cache misses, it is very likely that the stale shape_id
and the current instance shape_id have a close common ancestor.

For example if the instance variable is sometimes frozen sometimes
not, one of the two shape will be the direct parent of the other.

Another pattern that commonly cause IC misses is "memoization",
in such case the object will have a "base common shape" and then
a number of close descendants.

In addition, when we find a common ancestor, we store it in the
inline cache instead of the current shape. This help prevent the
cache from flip-flopping, ensuring the next lookup will be marginally
faster and more generally avoid writing in memory too much.

However, now that shapes have an ancestors index, we only check
for a few ancestors before falling back to use the index.

So overall this change speeds up what is assumed to be the more common
case, but makes what is assumed to be the less common case a bit slower.

```
compare-ruby: ruby 3.3.0dev (2023-10-26T05:30:17Z master 701ca070b4) [arm64-darwin22]
built-ruby: ruby 3.3.0dev (2023-10-26T09:25:09Z shapes_double_sear.. a723a85235) [arm64-darwin22]
warming up......

|                                     |compare-ruby|built-ruby|
|:------------------------------------|-----------:|---------:|
|vm_ivar_stable_shape                 |     11.672M|   11.679M|
|                                     |           -|     1.00x|
|vm_ivar_memoize_unstable_shape       |      7.551M|   10.506M|
|                                     |           -|     1.39x|
|vm_ivar_memoize_unstable_shape_miss  |     11.591M|   11.624M|
|                                     |           -|     1.00x|
|vm_ivar_unstable_undef               |      9.037M|    7.981M|
|                                     |       1.13x|         -|
|vm_ivar_divergent_shape              |      8.034M|    6.657M|
|                                     |       1.21x|         -|
|vm_ivar_divergent_shape_imbalanced   |     10.471M|    9.231M|
|                                     |       1.13x|         -|
```

Co-Authored-By: John Hawthorn &lt;john@hawthorn.email&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>Make every initial size pool shape a root shape</title>
<updated>2023-11-02T17:42:11+00:00</updated>
<author>
<name>Peter Zhu</name>
<email>peter@peterzhu.ca</email>
</author>
<published>2023-11-02T13:49:23+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=38ba040d8bda65321daad8d5bfa8956b5072e826'/>
<id>38ba040d8bda65321daad8d5bfa8956b5072e826</id>
<content type='text'>
This commit makes every initial size pool shape a root shape and assigns
it a capacity of 0.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This commit makes every initial size pool shape a root shape and assigns
it a capacity of 0.
</pre>
</div>
</content>
</entry>
</feed>
