| Age | Commit message (Collapse) | Author |
|---|
|
RSTRUCT_LEN / RSTRUCT_GET / RSTRUCT_SET all existing in two
versions, one public that does type and frozens checks
and one private that doesn't.
The problem is that this is error prone because the public version
is always accessible, but the private one require to include
`internal/struct.h`. So you may have some code that rely on the
public version, and later on the private header is included and
changes the behavior.
This already led to introducing a bug in YJIT & ZJIT:
https://github.com/ruby/ruby/pull/15835
|
|
|
|
|
|
|
|
|
|
|
|
|
|
[Feature #21084]
# Summary
The current way of marking weak references uses `rb_gc_mark_weak(VALUE *ptr)`.
This presents challenges because Ruby's GC is incremental, meaning that if the
`ptr` changes (e.g. realloc'd or free'd), then we could have an invalid memory
access. This also overwrites `*ptr = Qundef` if `*ptr` is dead, which prevents
any cleanup to be run (e.g. freeing memory or deleting entries from hash
tables). This ticket proposes `rb_gc_declare_weak_references` which declares
that an object has weak references and calls a cleanup function after marking,
allowing the object to clean up any memory for dead objects.
# Introduction
In [[Feature #19783]](https://bugs.ruby-lang.org/issues/19783), I introduced an
API allowing objects to mark weak references, the function signature looks like
this:
```c
void rb_gc_mark_weak(VALUE *ptr);
```
`rb_gc_mark_weak` is called during the marking phase of the GC to specify that
the memory at `ptr` holds a pointer to a Ruby object that is weakly referenced.
`rb_gc_mark_weak` appends this pointer to a list that is processed after the
marking phase of the GC. If the object at `*ptr` is no longer alive, then it
overwrites the object reference with a special value (`*ptr = Qundef`).
However, this API resulted in two challenges:
1. Ruby's default GC is incremental, which means that the GC is not ran in one
phase, but rather split into chunks of work that interleaves with Ruby
execution. The `ptr` passed into `rb_gc_mark_weak` could be on the malloc
heap, and that memory could be realloc'd or even free'd. We had to use
workarounds such as `rb_gc_remove_weak` to ensure that there were no illegal
memory accesses. This made `rb_gc_mark_weak` difficult to use, impacted
runtime performance, and increased memory usage.
2. When an object dies, `rb_gc_mark_weak` only overwites the reference with
`Qundef`. This means that if we want to do any cleanup (e.g. free a piece of
memory or delete a hash table entry), we could not do that and had to defer
this process elsewhere (e.g. during marking or runtime).
In this ticket, I'm proposing a new API for weak references. Instead of an
object marking its weak references during the marking phase, the object declares
that it has weak references using the `rb_gc_declare_weak_references` function.
This declaration occurs during runtime (e.g. after the object has been created)
rather than during GC.
After an object declares that it has weak references, it will have its callback
function called after marking as long as that object is alive. This callback
function can then call a special function `rb_gc_handle_weak_references_alive_p`
to determine whether its references are alive. This will allow the callback
function to do whatever it wants on the object, allowing it to perform any
cleanup work it needs.
This significantly simplifies the code for `ObjectSpace::WeakMap` and
`ObjectSpace::WeakKeyMap` because it no longer needs to have the workarounds for
the limitations of `rb_gc_mark_weak`.
# Performance
The performance results below demonstrate that `ObjectSpace::WeakMap#[]=` is now
about 60% faster because the implementation has been simplified and the number
of allocations has been reduced. We can see that there is not a significant
impact on the performance of `ObjectSpace::WeakMap#[]`.
Base:
```
ObjectSpace::WeakMap#[]=
4.620M (± 6.4%) i/s (216.44 ns/i) - 23.342M in 5.072149s
ObjectSpace::WeakMap#[]
30.967M (± 1.9%) i/s (32.29 ns/i) - 154.998M in 5.007157s
```
Branch:
```
ObjectSpace::WeakMap#[]=
7.336M (± 2.8%) i/s (136.31 ns/i) - 36.755M in 5.013983s
ObjectSpace::WeakMap#[]
30.902M (± 5.4%) i/s (32.36 ns/i) - 155.901M in 5.064060s
```
Code:
```
require "bundler/inline"
gemfile do
source "https://rubygems.org"
gem "benchmark-ips"
end
wmap = ObjectSpace::WeakMap.new
key = Object.new
val = Object.new
wmap[key] = val
Benchmark.ips do |x|
x.report("ObjectSpace::WeakMap#[]=") do |times|
i = 0
while i < times
wmap[Object.new] = Object.new
i += 1
end
end
x.report("ObjectSpace::WeakMap#[]") do |times|
i = 0
while i < times
wmap[key]
wmap[val] # does not exist
i += 1
end
end
end
```
# Alternative designs
Currently, `rb_gc_declare_weak_references` is designed to be an internal-only
API. This allows us to assume the object types that call
`rb_gc_declare_weak_references`. In the future, if we want to open up this API
to third parties, we may want to change this function to something like:
```c
void rb_gc_add_cleaner(VALUE obj, void (*callback)(VALUE obj));
```
This will allow the third party to implement a custom `callback` that gets
called after the marking phase of GC to clean up any dead references. I chose
not to implement this design because it is less efficient as we would need to
store a mapping from `obj` to `callback`, which requires extra memory.
|
|
Since it now live in the EC.
|
|
Mutexes spend a significant amount of time in `rb_fiber_serial`
because it can't be inlined (except with LTO).
The fiber struct is opaque the so function can't be defined as inlineable.
Ideally the while fiber struct would not be opaque to the rest of
Ruby core, but it's tricky to do.
Instead we can store the fiber serial in the execution context
itself, and make its access cheaper:
```
$ hyperfine './miniruby-baseline --yjit /tmp/mut.rb' './miniruby-inline-serial --yjit /tmp/mut.rb'
Benchmark 1: ./miniruby-baseline --yjit /tmp/mut.rb
Time (mean ± σ): 4.011 s ± 0.084 s [User: 3.977 s, System: 0.011 s]
Range (min … max): 3.950 s … 4.245 s 10 runs
Benchmark 2: ./miniruby-inline-serial --yjit /tmp/mut.rb
Time (mean ± σ): 3.495 s ± 0.150 s [User: 3.448 s, System: 0.009 s]
Range (min … max): 3.340 s … 3.869 s 10 runs
Summary
./miniruby-inline-serial --yjit /tmp/mut.rb ran
1.15 ± 0.05 times faster than ./miniruby-baseline --yjit /tmp/mut.rb
```
```ruby
i = 10_000_000
mut = Mutex.new
while i > 0
i -= 1
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
end
```
|
|
The changes are to `io.c` and `thread.c`.
I changed the API of 2 exported thread functions from `internal/thread.h` that
didn't look like they had any use in C extensions:
* rb_thread_wait_for_single_fd
* rb_thread_io_wait
I didn't change the following exported internal function because it's
used in C extensions:
* rb_thread_fd_select
I added a comment to note that this function, although internal, is used
in C extensions.
|
|
Saves one more call to GET_EC()
|
|
That call is surprisingly expensive, so trying doing it once
in `#synchronize` and then passing the EC to lock and unlock
saves quite a few cycles.
Before:
```
ruby 4.0.0dev (2025-12-10T09:30:18Z master c5608ab4d7) +YJIT +PRISM [arm64-darwin25]
Warming up --------------------------------------
Mutex 1.888M i/100ms
Monitor 1.633M i/100ms
Calculating -------------------------------------
Mutex 22.610M (± 0.2%) i/s (44.23 ns/i) - 113.258M in 5.009097s
Monitor 19.148M (± 0.3%) i/s (52.22 ns/i) - 96.366M in 5.032755s
```
After:
```
ruby 4.0.0dev (2025-12-10T10:40:07Z speedup-mutex 1c901cd4f8) +YJIT +PRISM [arm64-darwin25]
Warming up --------------------------------------
Mutex 2.095M i/100ms
Monitor 1.578M i/100ms
Calculating -------------------------------------
Mutex 24.456M (± 0.4%) i/s (40.89 ns/i) - 123.584M in 5.053418s
Monitor 19.176M (± 0.1%) i/s (52.15 ns/i) - 96.243M in 5.018977s
```
Bench:
```
require 'bundler/inline'
gemfile do
gem "benchmark-ips"
end
mutex = Mutex.new
require "monitor"
monitor = Monitor.new
Benchmark.ips do |x|
x.report("Mutex") { mutex.synchronize { } }
x.report("Monitor") { monitor.synchronize { } }
end
```
|
|
This rewrites the class allocator search to be faster. Instead of using
RCLASS_SUPER, which is now even slower due to Box, we can scan the
superclasses list to find a class where the allocator is defined.
This also disallows allocating from an ICLASS. Previously I believe that
was only done for FrozenCore, and that was changed in
e596cf6e93dbf121e197cccfec8a69902e00eda3.
|
|
These macros have been defined here and there, so collect them.
|
|
|
|
If we don't have uint128, then rb_int128_to_numeric emits a strict
aliasing warning:
numeric.c:3641:39: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
3641 | return rb_uint128_to_numeric(*(rb_uint128_t*)&n);
| ^~~~~~~~~~~~~~~~~
|
|
It is exported only for the extension library to test, but the method
is no longer used since 29e31e72fb5a14194a78ec974c4ba56c33ad8d45.
|
|
It is used only in hash.c, when `ENV['TZ']` is set.
|
|
The #ifdef is currently not taken because include/ruby/backward.h is
not included at this point. The attribute is unnecessary in an internal
header, so remove it.
|
|
(#15399)
|
|
Previously we only allocated bignums from the 40 byte sizepool, and
embedded bignum used a fixed size.
|
|
|
|
[Bug #21710]
- struct.c: `struct_alloc`
It is possible for a `NEWOBJ` tracepoint call back to write fields
into a newly allocated object before `struct_alloc` had the time
to set the `RSTRUCT_GEN_FIELDS` flags and such.
Hence we can't blindly initialize the `fields_obj` reference to `0`
we first need to check no fields were added yet.
- object.c: `rb_class_allocate_instance`
Similarly, if a `NEWOBJ` tracepoint tries to set fields on the object,
the `shape_id` must already be set, as it's required on T_OBJECT to
know where to write fields.
`NEWOBJ_OF` had to be refactored to accept a `shape_id`.
|
|
|
|
This will help JITs (and maybe later the interpreter) optimize
Integer#>>.
|
|
Previously this held a pointer to the Fiber itself, which requires
marking it (which was only implemented recently, prior to that it was
buggy). Using a monotonically increasing integer instead allows us to
avoid having a free function and keeps everything simpler.
My main motivations in making this change are that the root fiber lazily
allocates self, which makes the writebarrier implementation challenging
to do correctly, and wanting to avoid sending Mutexes to the remembered
set when locked by a short-lived Fiber.
|
|
|
|
Visual C++ 2005 (8.0):
- _MSC_VER: 1400
- MSVCRT_VERSION: 80
|
|
|
|
This reverts commit 2998c8d6b99ec49925ebea42198b29c3e27b34a7.
We need to find a better way to fix this bug. Even with this refcount
change, errors were still being seen in CI. For now we need to remove
this failing test.
|
|
|
|
|
|
|
|
|
|
|
|
We were seeing errors like:
```
* thread #8, stop reason = EXC_BAD_ACCESS (code=1, address=0x803)
* frame #0: 0x00000001001fe944 ruby`rb_st_lookup(tab=0x00000000000007fb, key=1, value=0x00000001305b7490) at st.c:1066:22
frame #1: 0x000000010002d658 ruby`remove_class_from_subclasses [inlined] class_get_subclasses_for_ns(tbl=0x00000000000007fb, ns_id=1) at class.c:604:9
frame #2: 0x000000010002d650 ruby`remove_class_from_subclasses(tbl=0x00000000000007fb, ns_id=1, klass=4754039232) at class.c:620:34
frame #3: 0x000000010002c8a8 ruby`rb_class_classext_free_subclasses(ext=0x000000011b5ce1d8, klass=4754039232, replacing=<unavailable>) at class.c:700:9
frame #4: 0x000000010002c760 ruby`rb_class_classext_free(klass=4754039232, ext=0x000000011b5ce1d8, is_prime=true) at class.c:105:5
frame #5: 0x00000001000e770c ruby`classext_free(ext=<unavailable>, is_prime=<unavailable>, namespace=<unavailable>, arg=<unavailable>) at gc.c:1231:5 [artificial]
frame #6: 0x000000010002d178 ruby`rb_class_classext_foreach(klass=<unavailable>, func=(ruby`classext_free at gc.c:1228), arg=0x00000001305b75c0) at class.c:518:5
frame #7: 0x00000001000e745c ruby`rb_gc_obj_free(objspace=0x000000012500c400, obj=4754039232) at gc.c:1282:9
frame #8: 0x00000001000e70d4 ruby`gc_sweep_plane(objspace=0x000000012500c400, heap=<unavailable>, p=4754039232, bitset=4095, ctx=0x00000001305b76e8) at default.c:3482:21
frame #9: 0x00000001000e6e9c ruby`gc_sweep_page(objspace=0x000000012500c400, heap=0x000000012500c540, ctx=0x00000001305b76e8) at default.c:3567:13
frame #10: 0x00000001000e51d0 ruby`gc_sweep_step(objspace=0x000000012500c400, heap=0x000000012500c540) at default.c:3848:9
frame #11: 0x00000001000e1880 ruby`gc_continue [inlined] gc_sweep_continue(objspace=0x000000012500c400, sweep_heap=0x000000012500c540) at default.c:3931:13
frame #12: 0x00000001000e1754 ruby`gc_continue(objspace=0x000000012500c400, heap=0x000000012500c540) at default.c:2037:9
frame #13: 0x00000001000e10bc ruby`newobj_cache_miss [inlined] heap_prepare(objspace=0x000000012500c400, heap=0x000000012500c540) at default.c:2056:5
frame #14: 0x00000001000e1074 ruby`newobj_cache_miss [inlined] heap_next_free_page(objspace=0x000000012500c400, heap=0x000000012500c540) at default.c:2280:9
frame #15: 0x00000001000e106c ruby`newobj_cache_miss(objspace=0x000000012500c400, cache=0x0000600001b00300, heap_idx=2, vm_locked=false) at default.c:2387:38
frame #16: 0x00000001000e0d28 ruby`newobj_alloc(objspace=<unavailable>, cache=<unavailable>, heap_idx=<unavailable>, vm_locked=<unavailable>) at default.c:2411:15 [artificial]
frame #17: 0x00000001000d7214 ruby`newobj_of [inlined] rb_gc_impl_new_obj(objspace_ptr=<unavailable>, cache_ptr=<unavailable>, klass=<unavailable>, flags=<unavailable>, wb_protected=<unavailable>, alloc_size=<unavailable>) at default.c:2490:15
frame #18: 0x00000001000d719c ruby`newobj_of(cr=<unavailable>, klass=4313971728, flags=258, wb_protected=<unavailable>, size=<unavailable>) at gc.c:995:17
frame #19: 0x00000001000d73ec ruby`rb_wb_protected_newobj_of(ec=<unavailable>, klass=<unavailable>, flags=<unavailable>, size=<unavailable>) at gc.c:1044:12 [artificial]
frame #20: 0x0000000100032d34 ruby`class_alloc0(type=<unavailable>, klass=4313971728, namespaceable=<unavailable>) at class.c:803:5
```
|
|
Calling the usual rb_iclass_classext_free() causes SEGV because
duplicating a newer classext of iclass had set the reference from superclass
to the newer classext, but calling rb_iclass_classext_free() deletes it.
|
|
to adopt strict shareable rule.
* (basically) shareable objects only refer shareable objects
* (exception) shareable objects can refere unshareable objects
but should not leak reference to unshareable objects to Ruby world
|
|
* `RB_OBJ_SET_SHAREABLE(obj)` makes obj shareable.
All of reachable objects from `obj` should be shareable.
* `RB_OBJ_SET_FROZEN_SHAREABLE(obj)` same as above
but freeze `obj` before making it shareable.
Also `rb_gc_verify_shareable(obj)` is introduced to check
the `obj` does not violate shareable rule (an shareable object
only refers shareable objects) strictly.
The rule has some exceptions (some shareable objects can refer to
unshareable objects, such as a Ractor object (which is a shareable
object) can refer to the Ractor local objects.
To handle such case, `check_shareable` flag is also introduced.
`STRICT_VERIFY_SHAREABLE` macro is also introduced to verify
the strict shareable rule at `SET_SHAREABLE`.
|
|
The st_insert in RCLASS_SET_NAMESPACE_CLASSEXT may overwrite an existing
rb_classext_t, causing it to leak memory. This commit changes it to use
st_update to free the existing one before overwriting it.
|
|
|
|
Inspired by 42ba82424d908c290a4a34ced8853f0a403b734b, I looked for other
occurrences of "the the".
|
|
To avoid crashes during displaying crash reports.
|
|
|
|
|
|
|
|
to fix inconsistent and wrong current namespace detections.
This includes:
* Moving load_path and related things from rb_vm_t to rb_namespace_t to simplify
accessing those values via namespace (instead of accessing either vm or ns)
* Initializing root_namespace earlier and consolidate builtin_namespace into root_namespace
* Adding VM_FRAME_FLAG_NS_REQUIRE for checkpoints to detect a namespace to load/require files
* Removing implicit refinements in the root namespace which was used to determine
the namespace to be loaded (replaced by VM_FRAME_FLAG_NS_REQUIRE)
* Removing namespaces from rb_proc_t because its namespace can be identified by lexical context
* Starting to use ep[VM_ENV_DATA_INDEX_SPECVAL] to store the current namespace when
the frame type is MAGIC_TOP or MAGIC_CLASS (block handlers don't exist in this case)
|
|
Nothing needs rb_imemo_new exported anymore.
|
|
These used to be used by the parser
|
