Some TYPEDDATA objects allocate struct fields using the GC right after they get created, and in that case the VM can try to perform a GC and join a barrier if another ractor started one. If we're dumping the heap in another ractor, this acquires a barrier and it will call the `rb_obj_memsize` function on this object. We can't assume these struct fields are non-null. This also goes for C extensions, which may cause problems with heap dumping from a ractor if their memsize functions aren't coded correctly to check for NULL fields. Because dumping the heap from a ractor is likely a rare scenario and it has only recently been introduced, we'll have to see how this works in practice and if it causes bugs.
This saves one pointer in `struct set_table`, which would allow `Set` objects to still fit in 80B TypedData slots even if RTypedData goes from 32B to 40B large. The existing set benchmark seem to show this doesn't have a very significant impact. Smaller sets are a bit faster, larger sets a bit slower. It seem consistent over multiple runs, but it's unclear how much of that is just error margin. ``` compare-ruby: ruby 3.5.0dev (2025-08-12T02:14:57Z master 428937a536) +YJIT +PRISM [arm64-darwin24] built-ruby: ruby 3.5.0dev (2025-08-12T07:22:26Z set-entries-bounds da30024fdc) +YJIT +PRISM [arm64-darwin24] warming up........ | |compare-ruby|built-ruby| |:------------------------|-----------:|---------:| |new_0 | 15.459M| 15.823M| | | -| 1.02x| |new_10 | 3.484M| 3.574M| | | -| 1.03x| |new_100 | 546.992k| 564.679k| | | -| 1.03x| |new_1000 | 49.391k| 48.169k| | | 1.03x| -| |aref_0 | 18.643M| 19.350M| | | -| 1.04x| |aref_10 | 5.941M| 6.006M| | | -| 1.01x| |aref_100 | 822.197k| 814.219k| | | 1.01x| -| |aref_1000 | 83.230k| 79.411k| | | 1.05x| -| ```
These functions conflict with the planned C-API functions. Since they deal with the underlying set_table pointers and not Set instances, this seems like a more accurate name as well.
[Bug #21438] Previously GC could trigger a table rebuild of the generic fields st_table in the middle of calling the st_update callback. This could cause entries to be reallocated or rearranged and the update to be for the wrong entry. This commit adds an assertion to make that case easier to detect, and replaces the st_update with a separate st_lookup and st_insert. Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org> Co-authored-by: Jean Boussier <byroot@ruby-lang.org>
Fixes [Bug #21201] This change addresses a performance regression where defining methods inside `refine` blocks caused severe slowdowns. The issue was due to `rb_clear_all_refinement_method_cache()` triggering a full object space scan via `rb_objspace_each_objects` to find and invalidate affected callcaches, which is very inefficient. To fix this, I introduce `vm->cc_refinement_table` to track callcaches related to refinements. This allows us to invalidate only the necessary callcaches without scanning the entire heap, resulting in significant performance improvement.
Ivars will longer be the only thing stored inline via shapes, so keeping the `iv_index` and `ivptr` names would be confusing. Instance variables won't be the only thing stored inline via shapes, so keeping the `ivptr` name would be confusing. `field` encompass anything that can be stored in a VALUE array. Similarly, `gen_ivtbl` becomes `gen_fields_tbl`.
Now that we have a `set_table` implementation, we can use it to track const caches and save some memory. We could even save some more memory if `numtable` didn't store a copy of the `hash` and instead recomputed it every time, but this is a quick win.
Set has been an autoloaded standard library since Ruby 3.2. The standard library Set is less efficient than it could be, as it uses Hash for storage, which stores unnecessary values for each key. Implementation details: * Core Set uses a modified version of `st_table`, named `set_table`. than `s/st_/set_/`, the main difference is that the stored records do not have values, making them 1/3 smaller. `st_table_entry` stores `hash`, `key`, and `record` (value), while `set_table_entry` only stores `hash` and `key`. This results in large sets using ~33% less memory compared to stdlib Set. For small sets, core Set uses 12% more memory (160 byte object slot and 64 malloc bytes, while stdlib set uses 40 for Set and 160 for Hash). More memory is used because the set_table is embedded and 72 bytes in the object slot are currently wasted. Hopefully we can make this more efficient and have it stored in an 80 byte object slot in the future. * All methods are implemented as cfuncs, except the pretty_print methods, which were moved to `lib/pp.rb` (which is where the pretty_print methods for other core classes are defined). As is typical for core classes, internal calls call C functions and not Ruby methods. For example, to check if something is a Set, `rb_obj_is_kind_of` is used, instead of calling `is_a?(Set)` on the related object. * Almost all methods use the same algorithm that the pure-Ruby implementation used. The exception is when calling `Set#divide` with a block with 2-arity. The pure-Ruby method used tsort to implement this. I developed an algorithm that only allocates a single intermediate hash and does not need tsort. * The `flatten_merge` protected method is no longer necessary, so it is not implemented (it could be). * Similar to Hash/Array, subclasses of Set are no longer reflected in `inspect` output. * RDoc from stdlib Set was moved to core Set, with minor updates. This includes a comprehensive benchmark suite for all public Set methods. As you would expect, the native version is faster in the vast majority of cases, and multiple times faster in many cases. There are a few cases where it is significantly slower: * Set.new with no arguments (~1.6x) * Set#compare_by_identity for small sets (~1.3x) * Set#clone for small sets (~1.5x) * Set#dup for small sets (~1.7x) These are slower as Set does not currently use the AR table optimization that Hash does, so a new set_table is initialized for each call. I'm not sure it's worth the complexity to have an AR table-like optimization for small sets (for hashes it makes sense, as small hashes are used everywhere in Ruby). The rbs and repl_type_completor bundled gems will need updates to support core Set. The pull request marks them as allowed failures. This passes all set tests with no changes. The following specs needed modification: * Modifying frozen set error message (changed for the better) * `Set#divide` when passed a 2-arity block no longer yields the same object as both the first and second argument (this seems like an issue with the previous implementation). * Set-like objects that override `is_a?` such that `is_a?(Set)` return `true` are no longer treated as Set instances. * `Set.allocate.hash` is no longer the same as `nil.hash` * `Set#join` no longer calls `Set#to_a` (it calls the underlying C function). * `Set#flatten_merge` protected method is not implemented. Previously, `set.rb` added a `SortedSet` autoload, which loads `set/sorted_set.rb`. This replaces the `Set` autoload in `prelude.rb` with a `SortedSet` autoload, but I recommend removing it and `set/sorted_set.rb`. This moves `test/set/test_set.rb` to `test/ruby/test_set.rb`, reflecting that switch to a core class. This does not move the spec files, as I'm not sure how they should be handled. Internally, this uses the st_* types and functions as much as possible, and only adds set_* types and functions as needed. The underlying set_table implementation is stored in st.c, but there is no public C-API for it, nor is there one planned, in order to keep the ability to change the internals going forward. For internal uses of st_table with Qtrue values, those can probably be replaced with set_table. To do that, include internal/set_table.h. To handle symbol visibility (rb_ prefix), internal/set_table.h uses the same macro approach that include/ruby/st.h uses. The Set class (rb_cSet) and all methods are defined in set.c. There isn't currently a C-API for the Set class, though C-API functions can be added as needed going forward. Implements [Feature #21216] Co-authored-by: Jean Boussier <jean.boussier@gmail.com> Co-authored-by: Oliver Nutter <mrnoname1000@riseup.net>
[Bug #21170] st_table reserves -1 as a special hash value to indicate that an entry has been deleted. So that that's a valid value to be returned from the hash function, do_hash replaces -1 with 0 so that it is not mistaken for the sentinel. Previously, when upgrading an AR table to an ST table, rb_st_add_direct_with_hash was used which did not perform the same conversion, this could lead to a hash in a broken state where one if its entries which was supposed to exist being marked as a tombstone. The hash could then become further corrupted when the ST table required resizing as the falsely tombstoned entry would be skipped but it would be counted in num entries, leading to an uninitialized entry at index 15. In most cases this will be really rare, unless using a very poorly implemented custom hash function. This also adds two debug assertions, one that st_add_direct_with_hash does not receive the reserved hash value, and a second in rebuild_table_with, which ensures that after we rebuild/compact a table it contains the expected number of elements. Co-authored-by: Alan Wu <alanwu@ruby-lang.org>