Fix flakiness in TestGc#test_thrashing_for_young_objects

I caught a reproduction of this test failing under rr, and was able to
replay it to isolate the failure. The call to
`before_stat_heap = GC.stat_heap` is itself allocating a hash, which in
unlucky circumstances can result in a new page being allocated and thus
`before_stats[:heap_allocated_pages]` no longer equals
`after_stats[:heap_allocated_pages]`.

The solution is to use the form of GC.stat/stat_heap which takes a hash
as an argument, and thus needs to perform no Ruby allocations itself.

1 files changed, 14 insertions, 3 deletions

diff --git a/test/ruby/test_gc.rb b/test/ruby/test_gc.rb
index 7c49f0f5bb..f0ab0b0c1a 100644
--- a/test/ruby/test_gc.rb
+++ b/test/ruby/test_gc.rb
@@ -636,8 +636,19 @@ class TestGc < Test::Unit::TestCase
       # Warmup to make sure heap stabilizes
       1_000_000.times { Object.new }
 
-      before_stats = GC.stat
+      # We need to pre-allocate all the hashes for GC.stat calls, because
+      # otherwise the call to GC.stat/GC.stat_heap itself could cause a new
+      # page to be allocated and the before/after assertions will fail
+      before_stats = {}
+      after_stats = {}
+      # stat_heap needs a hash of hashes for each heap; easiest way to get the
+      # right shape for that is just to call stat_heap with no argument
       before_stat_heap = GC.stat_heap
+      after_stat_heap = GC.stat_heap
+
+      # Now collect the actual stats
+      GC.stat before_stats
+      GC.stat_heap nil, before_stat_heap
 
       1_000_000.times { Object.new }
 
@@ -645,8 +656,8 @@ class TestGc < Test::Unit::TestCase
       # running a minor GC here will guarantee that GC will be complete
       GC.start(full_mark: false)
 
-      after_stats = GC.stat
-      after_stat_heap = GC.stat_heap
+      GC.stat after_stats
+      GC.stat_heap nil, after_stat_heap
 
       # Debugging output to for failures in trunk-repeat50@phosphorus-docker
       debug_msg = "before_stats: #{before_stats}\nbefore_stat_heap: #{before_stat_heap}\nafter_stats: #{after_stats}\nafter_stat_heap: #{after_stat_heap}"