Fix buffer overrun with auto-compact for shapes

The following script crashes:

```ruby
GC.auto_compact = true
GC.stress = true

class Foo
  def initialize
    @a = @b = @c = 0
  end

  def add_ivars
    @d = @e = @f = 0
  end
end

ary = 1_000.times.map { Foo.new }
ary.each { |f| f.add_ivars }
```

This is because in rb_grow_iv_list, it first calls
rb_ensure_iv_list_size to allocate the buffer (and also unsets the
embed bit) then rb_shape_transition_shape_capa to get the new shape.
However, auto-compact can trigger in rb_shape_transition_shape_capa
which would re-embed the object since it doesn't have the new shape yet.
This causes a crash as the object is now embedded but has a non-embed
shape which would cause the object to have a buffer overrun.

1 files changed, 4 insertions, 1 deletions

diff --git a/variable.c b/variable.c
index 0c283738aa..0fd4a80887 100644
--- a/variable.c
+++ b/variable.c
@@ -1409,10 +1409,13 @@ rb_grow_iv_list(VALUE obj)
     uint32_t len = initial_shape->capacity;
     RUBY_ASSERT(len > 0);
     uint32_t newsize = (uint32_t)(len * 2);
-    rb_ensure_iv_list_size(obj, len, newsize);
 
     rb_shape_t * res = rb_shape_transition_shape_capa(initial_shape, newsize);
+
+    rb_ensure_iv_list_size(obj, len, newsize);
+
     rb_shape_set_shape(obj, res);
+
     return res;
 }