<feed xmlns='http://www.w3.org/2005/Atom'>
<title>ruby.git/test/ruby/test_weakmap.rb, branch v3_3_11</title>
<subtitle>The Ruby Programming Language</subtitle>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/'/>
<entry>
<title>[Backport 3.3] [Bug #20688] Fix use-after-free for WeakMap and WeakKeyMap (#11439)</title>
<updated>2024-08-22T22:54:55+00:00</updated>
<author>
<name>Peter Zhu</name>
<email>peter@peterzhu.ca</email>
</author>
<published>2024-08-22T22:54:55+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=8657de70aa9d16066f9761710a5a74211046dc67'/>
<id>8657de70aa9d16066f9761710a5a74211046dc67</id>
<content type='text'>
* Add struct weakmap_entry for WeakMap entries

* Refactor wmap_foreach to pass weakmap_entry

* Use wmap_foreach for wmap_mark

* Refactor wmap_compact to use wmap_foreach

* Remove wmap_free_entry

* Fix WeakMap use-after-free

[Bug #20688]

We cannot free the weakmap_entry before the ST_DELETE because it could
hash the key which would read the weakmap_entry and would cause a
use-after-free. Instead, we store the entry and free it on the next
iteration.

For example, the following script triggers a use-after-free in Valgrind:

    weakmap = ObjectSpace::WeakMap.new
    10_000.times { weakmap[Object.new] = Object.new }

    ==25795== Invalid read of size 8
    ==25795==    at 0x462297: wmap_cmp (weakmap.c:165)
    ==25795==    by 0x3A2B1C: find_table_bin_ind (st.c:930)
    ==25795==    by 0x3A5EAA: st_general_foreach (st.c:1599)
    ==25795==    by 0x3A5EAA: rb_st_foreach (st.c:1640)
    ==25795==    by 0x25C991: gc_mark_children (default.c:4870)
    ==25795==    by 0x25C991: gc_marks_wb_unprotected_objects_plane (default.c:5565)
    ==25795==    by 0x25C991: rgengc_rememberset_mark_plane (default.c:5557)
    ==25795==    by 0x25C991: rgengc_rememberset_mark (default.c:6233)
    ==25795==    by 0x25C991: gc_marks_start (default.c:6057)
    ==25795==    by 0x25C991: gc_marks (default.c:6077)
    ==25795==    by 0x25C991: gc_start (default.c:6723)
    ==25795==    by 0x260F96: heap_prepare (default.c:2282)
    ==25795==    by 0x260F96: heap_next_free_page (default.c:2489)
    ==25795==    by 0x260F96: newobj_cache_miss (default.c:2598)
    ==25795==    by 0x26197F: newobj_alloc (default.c:2622)
    ==25795==    by 0x26197F: rb_gc_impl_new_obj (default.c:2701)
    ==25795==    by 0x26197F: newobj_of (gc.c:890)
    ==25795==    by 0x26197F: rb_wb_protected_newobj_of (gc.c:917)
    ==25795==    by 0x2DEA88: rb_class_allocate_instance (object.c:131)
    ==25795==    by 0x2E3B18: class_call_alloc_func (object.c:2141)
    ==25795==    by 0x2E3B18: rb_class_alloc (object.c:2113)
    ==25795==    by 0x2E3B18: rb_class_new_instance_pass_kw (object.c:2172)
    ==25795==    by 0x429DDC: vm_call_cfunc_with_frame_ (vm_insnhelper.c:3786)
    ==25795==    by 0x44B08D: vm_sendish (vm_insnhelper.c:5953)
    ==25795==    by 0x44B08D: vm_exec_core (insns.def:898)
    ==25795==    by 0x43A7A4: rb_vm_exec (vm.c:2564)
    ==25795==    by 0x234914: rb_ec_exec_node (eval.c:281)
    ==25795==  Address 0x21603710 is 0 bytes inside a block of size 16 free'd
    ==25795==    at 0x4849B2C: free (vg_replace_malloc.c:989)
    ==25795==    by 0x249651: rb_gc_impl_free (default.c:8527)
    ==25795==    by 0x249651: rb_gc_impl_free (default.c:8508)
    ==25795==    by 0x249651: ruby_sized_xfree.constprop.0 (gc.c:4178)
    ==25795==    by 0x4626EC: ruby_sized_xfree_inlined (gc.h:277)
    ==25795==    by 0x4626EC: wmap_free_entry (weakmap.c:45)
    ==25795==    by 0x4626EC: wmap_mark_weak_table_i (weakmap.c:61)
    ==25795==    by 0x3A5CEF: apply_functor (st.c:1633)
    ==25795==    by 0x3A5CEF: st_general_foreach (st.c:1543)
    ==25795==    by 0x3A5CEF: rb_st_foreach (st.c:1640)
    ==25795==    by 0x25C991: gc_mark_children (default.c:4870)
    ==25795==    by 0x25C991: gc_marks_wb_unprotected_objects_plane (default.c:5565)
    ==25795==    by 0x25C991: rgengc_rememberset_mark_plane (default.c:5557)
    ==25795==    by 0x25C991: rgengc_rememberset_mark (default.c:6233)
    ==25795==    by 0x25C991: gc_marks_start (default.c:6057)
    ==25795==    by 0x25C991: gc_marks (default.c:6077)
    ==25795==    by 0x25C991: gc_start (default.c:6723)
    ==25795==    by 0x260F96: heap_prepare (default.c:2282)
    ==25795==    by 0x260F96: heap_next_free_page (default.c:2489)
    ==25795==    by 0x260F96: newobj_cache_miss (default.c:2598)
    ==25795==    by 0x26197F: newobj_alloc (default.c:2622)
    ==25795==    by 0x26197F: rb_gc_impl_new_obj (default.c:2701)
    ==25795==    by 0x26197F: newobj_of (gc.c:890)
    ==25795==    by 0x26197F: rb_wb_protected_newobj_of (gc.c:917)
    ==25795==    by 0x2DEA88: rb_class_allocate_instance (object.c:131)
    ==25795==    by 0x2E3B18: class_call_alloc_func (object.c:2141)
    ==25795==    by 0x2E3B18: rb_class_alloc (object.c:2113)
    ==25795==    by 0x2E3B18: rb_class_new_instance_pass_kw (object.c:2172)
    ==25795==    by 0x429DDC: vm_call_cfunc_with_frame_ (vm_insnhelper.c:3786)
    ==25795==    by 0x44B08D: vm_sendish (vm_insnhelper.c:5953)
    ==25795==    by 0x44B08D: vm_exec_core (insns.def:898)
    ==25795==    by 0x43A7A4: rb_vm_exec (vm.c:2564)
    ==25795==  Block was alloc'd at
    ==25795==    at 0x484680F: malloc (vg_replace_malloc.c:446)
    ==25795==    by 0x25CE9E: rb_gc_impl_malloc (default.c:8542)
    ==25795==    by 0x462A39: wmap_aset_replace (weakmap.c:423)
    ==25795==    by 0x3A5542: rb_st_update (st.c:1487)
    ==25795==    by 0x462B8E: wmap_aset (weakmap.c:452)
    ==25795==    by 0x429DDC: vm_call_cfunc_with_frame_ (vm_insnhelper.c:3786)
    ==25795==    by 0x44B08D: vm_sendish (vm_insnhelper.c:5953)
    ==25795==    by 0x44B08D: vm_exec_core (insns.def:898)
    ==25795==    by 0x43A7A4: rb_vm_exec (vm.c:2564)
    ==25795==    by 0x234914: rb_ec_exec_node (eval.c:281)
    ==25795==    by 0x2369B8: ruby_run_node (eval.c:319)
    ==25795==    by 0x15D675: rb_main (main.c:43)
    ==25795==    by 0x15D675: main (main.c:62)

* Fix use-after-free for WeakKeyMap

[Bug #20688]

We cannot free the key before the ST_DELETE because it could hash the
key which would read the key and would cause a use-after-free. Instead,
we store the key and free it on the next iteration.</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* Add struct weakmap_entry for WeakMap entries

* Refactor wmap_foreach to pass weakmap_entry

* Use wmap_foreach for wmap_mark

* Refactor wmap_compact to use wmap_foreach

* Remove wmap_free_entry

* Fix WeakMap use-after-free

[Bug #20688]

We cannot free the weakmap_entry before the ST_DELETE because it could
hash the key which would read the weakmap_entry and would cause a
use-after-free. Instead, we store the entry and free it on the next
iteration.

For example, the following script triggers a use-after-free in Valgrind:

    weakmap = ObjectSpace::WeakMap.new
    10_000.times { weakmap[Object.new] = Object.new }

    ==25795== Invalid read of size 8
    ==25795==    at 0x462297: wmap_cmp (weakmap.c:165)
    ==25795==    by 0x3A2B1C: find_table_bin_ind (st.c:930)
    ==25795==    by 0x3A5EAA: st_general_foreach (st.c:1599)
    ==25795==    by 0x3A5EAA: rb_st_foreach (st.c:1640)
    ==25795==    by 0x25C991: gc_mark_children (default.c:4870)
    ==25795==    by 0x25C991: gc_marks_wb_unprotected_objects_plane (default.c:5565)
    ==25795==    by 0x25C991: rgengc_rememberset_mark_plane (default.c:5557)
    ==25795==    by 0x25C991: rgengc_rememberset_mark (default.c:6233)
    ==25795==    by 0x25C991: gc_marks_start (default.c:6057)
    ==25795==    by 0x25C991: gc_marks (default.c:6077)
    ==25795==    by 0x25C991: gc_start (default.c:6723)
    ==25795==    by 0x260F96: heap_prepare (default.c:2282)
    ==25795==    by 0x260F96: heap_next_free_page (default.c:2489)
    ==25795==    by 0x260F96: newobj_cache_miss (default.c:2598)
    ==25795==    by 0x26197F: newobj_alloc (default.c:2622)
    ==25795==    by 0x26197F: rb_gc_impl_new_obj (default.c:2701)
    ==25795==    by 0x26197F: newobj_of (gc.c:890)
    ==25795==    by 0x26197F: rb_wb_protected_newobj_of (gc.c:917)
    ==25795==    by 0x2DEA88: rb_class_allocate_instance (object.c:131)
    ==25795==    by 0x2E3B18: class_call_alloc_func (object.c:2141)
    ==25795==    by 0x2E3B18: rb_class_alloc (object.c:2113)
    ==25795==    by 0x2E3B18: rb_class_new_instance_pass_kw (object.c:2172)
    ==25795==    by 0x429DDC: vm_call_cfunc_with_frame_ (vm_insnhelper.c:3786)
    ==25795==    by 0x44B08D: vm_sendish (vm_insnhelper.c:5953)
    ==25795==    by 0x44B08D: vm_exec_core (insns.def:898)
    ==25795==    by 0x43A7A4: rb_vm_exec (vm.c:2564)
    ==25795==    by 0x234914: rb_ec_exec_node (eval.c:281)
    ==25795==  Address 0x21603710 is 0 bytes inside a block of size 16 free'd
    ==25795==    at 0x4849B2C: free (vg_replace_malloc.c:989)
    ==25795==    by 0x249651: rb_gc_impl_free (default.c:8527)
    ==25795==    by 0x249651: rb_gc_impl_free (default.c:8508)
    ==25795==    by 0x249651: ruby_sized_xfree.constprop.0 (gc.c:4178)
    ==25795==    by 0x4626EC: ruby_sized_xfree_inlined (gc.h:277)
    ==25795==    by 0x4626EC: wmap_free_entry (weakmap.c:45)
    ==25795==    by 0x4626EC: wmap_mark_weak_table_i (weakmap.c:61)
    ==25795==    by 0x3A5CEF: apply_functor (st.c:1633)
    ==25795==    by 0x3A5CEF: st_general_foreach (st.c:1543)
    ==25795==    by 0x3A5CEF: rb_st_foreach (st.c:1640)
    ==25795==    by 0x25C991: gc_mark_children (default.c:4870)
    ==25795==    by 0x25C991: gc_marks_wb_unprotected_objects_plane (default.c:5565)
    ==25795==    by 0x25C991: rgengc_rememberset_mark_plane (default.c:5557)
    ==25795==    by 0x25C991: rgengc_rememberset_mark (default.c:6233)
    ==25795==    by 0x25C991: gc_marks_start (default.c:6057)
    ==25795==    by 0x25C991: gc_marks (default.c:6077)
    ==25795==    by 0x25C991: gc_start (default.c:6723)
    ==25795==    by 0x260F96: heap_prepare (default.c:2282)
    ==25795==    by 0x260F96: heap_next_free_page (default.c:2489)
    ==25795==    by 0x260F96: newobj_cache_miss (default.c:2598)
    ==25795==    by 0x26197F: newobj_alloc (default.c:2622)
    ==25795==    by 0x26197F: rb_gc_impl_new_obj (default.c:2701)
    ==25795==    by 0x26197F: newobj_of (gc.c:890)
    ==25795==    by 0x26197F: rb_wb_protected_newobj_of (gc.c:917)
    ==25795==    by 0x2DEA88: rb_class_allocate_instance (object.c:131)
    ==25795==    by 0x2E3B18: class_call_alloc_func (object.c:2141)
    ==25795==    by 0x2E3B18: rb_class_alloc (object.c:2113)
    ==25795==    by 0x2E3B18: rb_class_new_instance_pass_kw (object.c:2172)
    ==25795==    by 0x429DDC: vm_call_cfunc_with_frame_ (vm_insnhelper.c:3786)
    ==25795==    by 0x44B08D: vm_sendish (vm_insnhelper.c:5953)
    ==25795==    by 0x44B08D: vm_exec_core (insns.def:898)
    ==25795==    by 0x43A7A4: rb_vm_exec (vm.c:2564)
    ==25795==  Block was alloc'd at
    ==25795==    at 0x484680F: malloc (vg_replace_malloc.c:446)
    ==25795==    by 0x25CE9E: rb_gc_impl_malloc (default.c:8542)
    ==25795==    by 0x462A39: wmap_aset_replace (weakmap.c:423)
    ==25795==    by 0x3A5542: rb_st_update (st.c:1487)
    ==25795==    by 0x462B8E: wmap_aset (weakmap.c:452)
    ==25795==    by 0x429DDC: vm_call_cfunc_with_frame_ (vm_insnhelper.c:3786)
    ==25795==    by 0x44B08D: vm_sendish (vm_insnhelper.c:5953)
    ==25795==    by 0x44B08D: vm_exec_core (insns.def:898)
    ==25795==    by 0x43A7A4: rb_vm_exec (vm.c:2564)
    ==25795==    by 0x234914: rb_ec_exec_node (eval.c:281)
    ==25795==    by 0x2369B8: ruby_run_node (eval.c:319)
    ==25795==    by 0x15D675: rb_main (main.c:43)
    ==25795==    by 0x15D675: main (main.c:62)

* Fix use-after-free for WeakKeyMap

[Bug #20688]

We cannot free the key before the ST_DELETE because it could hash the
key which would read the key and would cause a use-after-free. Instead,
we store the key and free it on the next iteration.</pre>
</div>
</content>
</entry>
<entry>
<title>Skip under_gc_compact_stress on s390x (#10073)</title>
<updated>2024-05-29T18:13:02+00:00</updated>
<author>
<name>Takashi Kokubun</name>
<email>takashikkbn@gmail.com</email>
</author>
<published>2024-02-22T22:34:19+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=2f4fe76eff0a8c6ab7a1d2fb845453acfc3cb206'/>
<id>2f4fe76eff0a8c6ab7a1d2fb845453acfc3cb206</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Make WeakMap safe for compaction during allocation</title>
<updated>2023-12-12T14:01:21+00:00</updated>
<author>
<name>Peter Zhu</name>
<email>peter@peterzhu.ca</email>
</author>
<published>2023-12-08T20:17:51+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=33cf8f640bab35c463186ef0856689c22559fbeb'/>
<id>33cf8f640bab35c463186ef0856689c22559fbeb</id>
<content type='text'>
During allocation, the table may not have been allocated yet which would
crash in the st_foreach.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
During allocation, the table may not have been allocated yet which would
crash in the st_foreach.
</pre>
</div>
</content>
</entry>
<entry>
<title>Fix crash in WeakMap during compaction</title>
<updated>2023-09-06T18:20:23+00:00</updated>
<author>
<name>Peter Zhu</name>
<email>peter@peterzhu.ca</email>
</author>
<published>2023-09-06T18:20:23+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=12102d101af258d7a3e9695b736a189cd3658df1'/>
<id>12102d101af258d7a3e9695b736a189cd3658df1</id>
<content type='text'>
WeakMap can crash during compaction because the st_insert could allocate
memory.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
WeakMap can crash during compaction because the st_insert could allocate
memory.
</pre>
</div>
</content>
</entry>
<entry>
<title>Implement WeakMap using weak references</title>
<updated>2023-08-25T13:01:21+00:00</updated>
<author>
<name>Peter Zhu</name>
<email>peter@peterzhu.ca</email>
</author>
<published>2023-07-24T19:26:10+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=ee9cc8e32e04597a4a86b391df2c19aed9fde3e5'/>
<id>ee9cc8e32e04597a4a86b391df2c19aed9fde3e5</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Remove --disable-gems for assert_separately</title>
<updated>2023-08-03T00:11:08+00:00</updated>
<author>
<name>Peter Zhu</name>
<email>peter@peterzhu.ca</email>
</author>
<published>2023-08-02T21:02:37+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=4b6c584023f41827c891f33a16cb5db221b7cd19'/>
<id>4b6c584023f41827c891f33a16cb5db221b7cd19</id>
<content type='text'>
assert_separately adds --disable=gems so we don't need to add
--disable-gems when calling assert_separately.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
assert_separately adds --disable=gems so we don't need to add
--disable-gems when calling assert_separately.
</pre>
</div>
</content>
</entry>
<entry>
<title>Implement ObjectSpace::WeakMap#delete and ObjectSpace::WeakKeyMap#delete</title>
<updated>2023-04-15T14:29:46+00:00</updated>
<author>
<name>Jean Boussier</name>
<email>byroot@ruby-lang.org</email>
</author>
<published>2023-03-30T14:23:14+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=52449b5b75068872ce568ed00c4c7fb8e6c28072'/>
<id>52449b5b75068872ce568ed00c4c7fb8e6c28072</id>
<content type='text'>
[Feature #19561]

It's useful to be able to remove references from weak maps.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
[Feature #19561]

It's useful to be able to remove references from weak maps.
</pre>
</div>
</content>
</entry>
<entry>
<title>Add guard to compaction test in WeakMap</title>
<updated>2023-04-06T17:35:25+00:00</updated>
<author>
<name>Peter Zhu</name>
<email>peter@peterzhu.ca</email>
</author>
<published>2023-04-06T17:35:25+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=bffadcd6d46ccfccade79ce0efb60ced8eac4483'/>
<id>bffadcd6d46ccfccade79ce0efb60ced8eac4483</id>
<content type='text'>
Some platforms don't support compaction, so we should skip this test.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Some platforms don't support compaction, so we should skip this test.
</pre>
</div>
</content>
</entry>
<entry>
<title>ObjectSpace::WeakMap: clean inverse reference when an entry is re-assigned</title>
<updated>2023-03-17T17:50:08+00:00</updated>
<author>
<name>Jean Boussier</name>
<email>byroot@ruby-lang.org</email>
</author>
<published>2023-03-16T08:52:22+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=3592b24cdc07ed89eecb39161f21fe721a89a5de'/>
<id>3592b24cdc07ed89eecb39161f21fe721a89a5de</id>
<content type='text'>
[Bug #19531]

```ruby
wmap[1] = "A"
wmap[1] = "B"
```

In the example above, we need to remove the `"A" =&gt; 1` inverse reference
so that when `"A"` is GCed the `1` key isn't deleted.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
[Bug #19531]

```ruby
wmap[1] = "A"
wmap[1] = "B"
```

In the example above, we need to remove the `"A" =&gt; 1` inverse reference
so that when `"A"` is GCed the `1` key isn't deleted.
</pre>
</div>
</content>
</entry>
<entry>
<title>Fix crash during compaction</title>
<updated>2023-03-15T03:18:11+00:00</updated>
<author>
<name>Peter Zhu</name>
<email>peter@peterzhu.ca</email>
</author>
<published>2023-03-15T03:18:11+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=3dc8cde70078ccb38f5f4b0818ad5eecded01bd5'/>
<id>3dc8cde70078ccb38f5f4b0818ad5eecded01bd5</id>
<content type='text'>
[Bug #19529]

The fix for [Bug #19529] in commit 548086b contained a bug that crashes
on the following script:

```
wm = ObjectSpace::WeakMap.new
obj = Object.new
100.times do
  wm[Object.new] = obj
  GC.start
end
GC.compact
```
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
[Bug #19529]

The fix for [Bug #19529] in commit 548086b contained a bug that crashes
on the following script:

```
wm = ObjectSpace::WeakMap.new
obj = Object.new
100.times do
  wm[Object.new] = obj
  GC.start
end
GC.compact
```
</pre>
</div>
</content>
</entry>
</feed>
