https://github.com/ruby/mmtk/commit/c4cca6c1c3
Adding a fast path for bump pointer allocator can improve allocation
performance.
For the following microbenchmark with MMTK_HEAP_MIN=100MiB:
10_000_000.times { String.new }
Before:
810.7 ms ± 8.3 ms [User: 790.9 ms, System: 40.3 ms]
After:
777.9 ms ± 10.4 ms [User: 759.0 ms, System: 37.9 ms]
https://github.com/ruby/mmtk/commit/0ff5c9f579
https://github.com/ruby/mmtk/commit/2185189df4
This will allow the Ruby backtrace, memory mapping, etc. to be outputted when a Ruby mutator thread panics. https://github.com/ruby/mmtk/commit/d10fd325dd
https://github.com/ruby/mmtk/commit/ed9036c295
https://github.com/ruby/mmtk/commit/4e789e118b
This reverts commit 228d13f6ed914d1e7f6bd2416e3f5be8283be865. This commit makes default.c and mmtk.c depend on shape.h, which prevents them from building independently.
Attempt to fix the following SEGV: ``` ruby(gc_mark) ../src/gc/default/default.c:4429 ruby(gc_mark_children+0x45) [0x560b380bf8b5] ../src/gc/default/default.c:4625 ruby(gc_mark_stacked_objects) ../src/gc/default/default.c:4647 ruby(gc_mark_stacked_objects_all) ../src/gc/default/default.c:4685 ruby(gc_marks_rest) ../src/gc/default/default.c:5707 ruby(gc_marks+0x4e7) [0x560b380c41c1] ../src/gc/default/default.c:5821 ruby(gc_start) ../src/gc/default/default.c:6502 ruby(heap_prepare+0xa4) [0x560b380c4efc] ../src/gc/default/default.c:2074 ruby(heap_next_free_page) ../src/gc/default/default.c:2289 ruby(newobj_cache_miss) ../src/gc/default/default.c:2396 ruby(RB_SPECIAL_CONST_P+0x0) [0x560b380c5df4] ../src/gc/default/default.c:2420 ruby(RB_BUILTIN_TYPE) ../src/include/ruby/internal/value_type.h:184 ruby(newobj_init) ../src/gc/default/default.c:2136 ruby(rb_gc_impl_new_obj) ../src/gc/default/default.c:2500 ruby(newobj_of) ../src/gc.c:996 ruby(rb_imemo_new+0x37) [0x560b380d8bed] ../src/imemo.c:46 ruby(imemo_fields_new) ../src/imemo.c:105 ruby(rb_imemo_fields_new) ../src/imemo.c:120 ``` I have no reproduction, but my understanding based on the backtrace and error is that GC is triggered inside `newobj_init` causing the new object to be marked while in a incomplete state. I believe the fix is to pass the `shape_id` down to `newobj_init` so it can be set before the GC has a chance to trigger.
In rb_gc_impl_before_fork, it locks the VM and barriers all the Ractors
before calling mmtk_before_fork. However, since rb_mmtk_block_for_gc is
a barrier point, one or more Ractors could be paused there. However,
mmtk_before_fork is not compatible with that because it assumes that the
MMTk workers are idle, but the workers are not idle because they are
busy working on a GC.
This commit essentially implements a trylock. It will optimistically
lock but will release the lock if it detects that any other Ractors are
waiting in rb_mmtk_block_for_gc.
For example, the following script demonstrates the issue:
puts "Hello #{Process.pid}"
100.times do |i|
puts "i = #{i}"
Ractor.new(i) do |j|
puts "Ractor #{j} hello"
1000.times do |i|
s = "#{j}-#{i}"
end
Ractor.receive
puts "Ractor #{j} goodbye"
end
pid = fork { }
puts "Child pid is #{pid}"
_, status = Process.waitpid2 pid
puts status.success?
end
puts "Goodbye"
We can see the MMTk worker thread is waiting to start the GC:
#4 0x00007ffff66538b1 in rb_mmtk_stop_the_world () at gc/mmtk/mmtk.c:101
#5 0x00007ffff6d04caf in mmtk_ruby::collection::{impl#0}::stop_all_mutators<mmtk::scheduler::gc_work::{impl#14}::do_work::{closure_env#0}<mmtk::plan::immix::gc_work::ImmixGCWorkContext<mmtk_ruby::Ruby, 0>>> (_tls=..., mutator_visitor=...) at src/collection.rs:23
However, the mutator thread is stuck in mmtk_before_fork trying to stop
that worker thread:
#4 0x00007ffff6c0b621 in std::sys::thread::unix::Thread::join () at library/std/src/sys/thread/unix.rs:134
#5 0x00007ffff6658b6e in std::thread::JoinInner<()>::join<()> (self=...)
#6 0x00007ffff6658d4c in std::thread::JoinHandle<()>::join<()> (self=...)
#7 0x00007ffff665795e in mmtk_ruby::binding::RubyBinding::join_all_gc_threads (self=0x7ffff72462d0 <mmtk_ruby::BINDING+8>) at src/binding.rs:115
#8 0x00007ffff66561a8 in mmtk_ruby::api::mmtk_before_fork () at src/api.rs:309
#9 0x00007ffff66556ff in rb_gc_impl_before_fork (objspace_ptr=0x555555d17980) at gc/mmtk/mmtk.c:1054
#10 0x00005555556bbc3e in rb_gc_before_fork () at gc.c:5429
https://github.com/ruby/mmtk/commit/1a629504a7
We need the VM barrier in rb_gc_impl_before_fork to stop the other Ractors
because otherwise they could be allocating objects in the fast path which
could be calling mmtk_add_obj_free_candidate. Since mmtk_add_obj_free_candidate
acquires a lock on obj_free_candidates in weak_proc.rs, this lock may not
be released in the child process after the Ractor dies.
For example, the following script demonstrates the issue:
puts "Hello #{Process.pid}"
100.times do |i|
puts "i = #{i}"
Ractor.new(i) do |j|
puts "Ractor #{j} hello"
1000.times do |i|
s = "#{j}-#{i}"
end
Ractor.receive
puts "Ractor #{j} goodbye"
end
pid = fork { }
puts "Child pid is #{pid}"
_, status = Process.waitpid2 pid
puts status.success?
end
puts "Goodbye"
In the child process, we can see that it is stuck trying to acquire the
lock on obj_free_candidates:
#5 0x00007192bfb53f10 in mmtk_ruby::weak_proc::WeakProcessor::get_all_obj_free_candidates (self=0x7192c0657498 <mmtk_ruby::BINDING+72>) at src/weak_proc.rs:52
#6 0x00007192bfa634c3 in mmtk_ruby::api::mmtk_get_all_obj_free_candidates () at src/api.rs:295
#7 0x00007192bfa61d50 in rb_gc_impl_shutdown_call_finalizer (objspace_ptr=0x578c17abfc50) at gc/mmtk/mmtk.c:1032
#8 0x0000578c1601e48e in rb_ec_finalize (ec=0x578c17ac06d0) at eval.c:166
#9 rb_ec_cleanup (ec=<optimized out>, ex=<optimized out>) at eval.c:257
#10 0x0000578c1601ebf6 in ruby_cleanup (ex=<optimized out>) at eval.c:180
#11 ruby_stop (ex=<optimized out>) at eval.c:292
#12 0x0000578c16127124 in rb_f_fork (obj=<optimized out>) at process.c:4291
#13 rb_f_fork (obj=<optimized out>) at process.c:4281
https://github.com/ruby/mmtk/commit/eb4b229858