ruby.git/test, branch v3_3_5

merge revision(s) a3562c2a0abf1c2bdd1d50377b4f929580782594: [Backport #20701]

2024-09-02T10:19:13+00:00

	Remove incorrect setting of KW_SPLAT_MUT flag

	Fixes [Bug #20701]

	Co-authored-by: Pablo Herrero

merge revision(s) 992596fb7af18a7f472589a607d0eb3fbb03b49a: [Backport #20344]

2024-09-02T10:10:19+00:00

	Fix next inside block argument stack underflow

	[Bug #20344]
	Fix compile_next adding removable adjust label

merge revision(s) 97449338d6cb42d9dd7c9ca61550616e7e6b6ef6: [Backport #20649]

2024-09-02T10:07:09+00:00

	[Bug #20649] Allow `nil` as 2nd argument of `assign_error`

	Fallback to the last token element in that case, for the backward
	compatibilities.

merge revision(s) 1870505f478cc75993b296b7144a45137ace6937: [Backport #20651]

2024-09-02T09:56:18+00:00

	Fix wrong unreachable chunk remove when jump destination label is unremovable

[Backport 3.3] [Bug #20691] Fix use-after-free in WeakKeyMap#clear (#11443)

2024-08-23T22:48:08+00:00

Fix use-after-free in WeakKeyMap#clear

[Bug #20691]

If the WeakKeyMap has been marked but sweeping hasn't started yet and we
cann WeakKeyMap#clear, then there could be a use-after-free because we do
not call rb_gc_remove_weak to remove the key from the GC.

For example, the following code triggers use-after-free errors in Valgrind:

    map = ObjectSpace::WeakKeyMap.new

    1_000.times do
      1_000.times do
        map[Object.new] = nil
      end

      map.clear
    end

Output from Valgrind:

    ==61230== Invalid read of size 8
    ==61230==    at 0x25CAF8: gc_update_weak_references (default.c:5593)
    ==61230==    by 0x25CAF8: gc_marks_finish (default.c:5641)
    ==61230==    by 0x26031C: gc_marks_continue (default.c:5987)
    ==61230==    by 0x26031C: gc_continue (default.c:2255)
    ==61230==    by 0x2605FC: newobj_cache_miss (default.c:2589)
    ==61230==    by 0x26111F: newobj_alloc (default.c:2622)
    ==61230==    by 0x26111F: rb_gc_impl_new_obj (default.c:2701)
    ==61230==    by 0x26111F: newobj_of (gc.c:890)
    ==61230==    by 0x26111F: rb_wb_protected_newobj_of (gc.c:917)
    ==61230==    by 0x2DE218: rb_class_allocate_instance (object.c:131)
    ==61230==    by 0x2E32A8: class_call_alloc_func (object.c:2141)
    ==61230==    by 0x2E32A8: rb_class_alloc (object.c:2113)
    ==61230==    by 0x2E32A8: rb_class_new_instance_pass_kw (object.c:2172)
    ==61230==    by 0x4296BC: vm_call_cfunc_with_frame_ (vm_insnhelper.c:3788)
    ==61230==    by 0x44A9CD: vm_sendish (vm_insnhelper.c:5955)
    ==61230==    by 0x44A9CD: vm_exec_core (insns.def:898)
    ==61230==    by 0x43A0E4: rb_vm_exec (vm.c:2564)
    ==61230==    by 0x2341B4: rb_ec_exec_node (eval.c:281)
    ==61230==    by 0x236258: ruby_run_node (eval.c:319)
    ==61230==    by 0x15D665: rb_main (main.c:43)
    ==61230==    by 0x15D665: main (main.c:62)
    ==61230==  Address 0x2159cb00 is 0 bytes inside a block of size 8 free'd
    ==61230==    at 0x4849B2C: free (vg_replace_malloc.c:989)
    ==61230==    by 0x248EF1: rb_gc_impl_free (default.c:8512)
    ==61230==    by 0x248EF1: rb_gc_impl_free (default.c:8493)
    ==61230==    by 0x248EF1: ruby_sized_xfree.constprop.0 (gc.c:4178)
    ==61230==    by 0x4627EC: wkmap_free_table_i (weakmap.c:652)
    ==61230==    by 0x3A54AF: apply_functor (st.c:1633)
    ==61230==    by 0x3A54AF: st_general_foreach (st.c:1543)
    ==61230==    by 0x3A54AF: rb_st_foreach (st.c:1640)
    ==61230==    by 0x46203C: wkmap_clear (weakmap.c:973)
    ==61230==    by 0x4296BC: vm_call_cfunc_with_frame_ (vm_insnhelper.c:3788)
    ==61230==    by 0x44A9CD: vm_sendish (vm_insnhelper.c:5955)
    ==61230==    by 0x44A9CD: vm_exec_core (insns.def:898)
    ==61230==    by 0x43A0E4: rb_vm_exec (vm.c:2564)
    ==61230==    by 0x2341B4: rb_ec_exec_node (eval.c:281)
    ==61230==    by 0x236258: ruby_run_node (eval.c:319)
    ==61230==    by 0x15D665: rb_main (main.c:43)
    ==61230==    by 0x15D665: main (main.c:62)
    ==61230==  Block was alloc'd at
    ==61230==    at 0x484680F: malloc (vg_replace_malloc.c:446)
    ==61230==    by 0x25C68E: rb_gc_impl_malloc (default.c:8527)
    ==61230==    by 0x4622E9: wkmap_aset_replace (weakmap.c:817)
    ==61230==    by 0x3A4D02: rb_st_update (st.c:1487)
    ==61230==    by 0x4623E4: wkmap_aset (weakmap.c:854)
    ==61230==    by 0x4296BC: vm_call_cfunc_with_frame_ (vm_insnhelper.c:3788)
    ==61230==    by 0x44A9CD: vm_sendish (vm_insnhelper.c:5955)
    ==61230==    by 0x44A9CD: vm_exec_core (insns.def:898)
    ==61230==    by 0x43A0E4: rb_vm_exec (vm.c:2564)
    ==61230==    by 0x2341B4: rb_ec_exec_node (eval.c:281)
    ==61230==    by 0x236258: ruby_run_node (eval.c:319)
    ==61230==    by 0x15D665: rb_main (main.c:43)
    ==61230==    by 0x15D665: main (main.c:62)
    ==61230==
    ==61230== Invalid write of size 8
    ==61230==    at 0x25CB3B: gc_update_weak_references (default.c:5598)
    ==61230==    by 0x25CB3B: gc_marks_finish (default.c:5641)
    ==61230==    by 0x26031C: gc_marks_continue (default.c:5987)
    ==61230==    by 0x26031C: gc_continue (default.c:2255)
    ==61230==    by 0x2605FC: newobj_cache_miss (default.c:2589)
    ==61230==    by 0x26111F: newobj_alloc (default.c:2622)
    ==61230==    by 0x26111F: rb_gc_impl_new_obj (default.c:2701)
    ==61230==    by 0x26111F: newobj_of (gc.c:890)
    ==61230==    by 0x26111F: rb_wb_protected_newobj_of (gc.c:917)
    ==61230==    by 0x2DE218: rb_class_allocate_instance (object.c:131)
    ==61230==    by 0x2E32A8: class_call_alloc_func (object.c:2141)
    ==61230==    by 0x2E32A8: rb_class_alloc (object.c:2113)
    ==61230==    by 0x2E32A8: rb_class_new_instance_pass_kw (object.c:2172)
    ==61230==    by 0x4296BC: vm_call_cfunc_with_frame_ (vm_insnhelper.c:3788)
    ==61230==    by 0x44A9CD: vm_sendish (vm_insnhelper.c:5955)
    ==61230==    by 0x44A9CD: vm_exec_core (insns.def:898)
    ==61230==    by 0x43A0E4: rb_vm_exec (vm.c:2564)
    ==61230==    by 0x2341B4: rb_ec_exec_node (eval.c:281)
    ==61230==    by 0x236258: ruby_run_node (eval.c:319)
    ==61230==    by 0x15D665: rb_main (main.c:43)
    ==61230==    by 0x15D665: main (main.c:62)
    ==61230==  Address 0x2159cb00 is 0 bytes inside a block of size 8 free'd
    ==61230==    at 0x4849B2C: free (vg_replace_malloc.c:989)
    ==61230==    by 0x248EF1: rb_gc_impl_free (default.c:8512)
    ==61230==    by 0x248EF1: rb_gc_impl_free (default.c:8493)
    ==61230==    by 0x248EF1: ruby_sized_xfree.constprop.0 (gc.c:4178)
    ==61230==    by 0x4627EC: wkmap_free_table_i (weakmap.c:652)
    ==61230==    by 0x3A54AF: apply_functor (st.c:1633)
    ==61230==    by 0x3A54AF: st_general_foreach (st.c:1543)
    ==61230==    by 0x3A54AF: rb_st_foreach (st.c:1640)
    ==61230==    by 0x46203C: wkmap_clear (weakmap.c:973)
    ==61230==    by 0x4296BC: vm_call_cfunc_with_frame_ (vm_insnhelper.c:3788)
    ==61230==    by 0x44A9CD: vm_sendish (vm_insnhelper.c:5955)
    ==61230==    by 0x44A9CD: vm_exec_core (insns.def:898)
    ==61230==    by 0x43A0E4: rb_vm_exec (vm.c:2564)
    ==61230==    by 0x2341B4: rb_ec_exec_node (eval.c:281)
    ==61230==    by 0x236258: ruby_run_node (eval.c:319)
    ==61230==    by 0x15D665: rb_main (main.c:43)
    ==61230==    by 0x15D665: main (main.c:62)
    ==61230==  Block was alloc'd at
    ==61230==    at 0x484680F: malloc (vg_replace_malloc.c:446)
    ==61230==    by 0x25C68E: rb_gc_impl_malloc (default.c:8527)
    ==61230==    by 0x4622E9: wkmap_aset_replace (weakmap.c:817)
    ==61230==    by 0x3A4D02: rb_st_update (st.c:1487)
    ==61230==    by 0x4623E4: wkmap_aset (weakmap.c:854)
    ==61230==    by 0x4296BC: vm_call_cfunc_with_frame_ (vm_insnhelper.c:3788)
    ==61230==    by 0x44A9CD: vm_sendish (vm_insnhelper.c:5955)
    ==61230==    by 0x44A9CD: vm_exec_core (insns.def:898)
    ==61230==    by 0x43A0E4: rb_vm_exec (vm.c:2564)
    ==61230==    by 0x2341B4: rb_ec_exec_node (eval.c:281)
    ==61230==    by 0x236258: ruby_run_node (eval.c:319)
    ==61230==    by 0x15D665: rb_main (main.c:43)
    ==61230==    by 0x15D665: main (main.c:62)

Co-authored-by: Jean Boussier

[Backport 3.3] [Bug #20688] Fix use-after-free for WeakMap and WeakKeyMap (#11439)

2024-08-22T22:54:55+00:00

* Add struct weakmap_entry for WeakMap entries

* Refactor wmap_foreach to pass weakmap_entry

* Use wmap_foreach for wmap_mark

* Refactor wmap_compact to use wmap_foreach

* Remove wmap_free_entry

* Fix WeakMap use-after-free

[Bug #20688]

We cannot free the weakmap_entry before the ST_DELETE because it could
hash the key which would read the weakmap_entry and would cause a
use-after-free. Instead, we store the entry and free it on the next
iteration.

For example, the following script triggers a use-after-free in Valgrind:

    weakmap = ObjectSpace::WeakMap.new
    10_000.times { weakmap[Object.new] = Object.new }

    ==25795== Invalid read of size 8
    ==25795==    at 0x462297: wmap_cmp (weakmap.c:165)
    ==25795==    by 0x3A2B1C: find_table_bin_ind (st.c:930)
    ==25795==    by 0x3A5EAA: st_general_foreach (st.c:1599)
    ==25795==    by 0x3A5EAA: rb_st_foreach (st.c:1640)
    ==25795==    by 0x25C991: gc_mark_children (default.c:4870)
    ==25795==    by 0x25C991: gc_marks_wb_unprotected_objects_plane (default.c:5565)
    ==25795==    by 0x25C991: rgengc_rememberset_mark_plane (default.c:5557)
    ==25795==    by 0x25C991: rgengc_rememberset_mark (default.c:6233)
    ==25795==    by 0x25C991: gc_marks_start (default.c:6057)
    ==25795==    by 0x25C991: gc_marks (default.c:6077)
    ==25795==    by 0x25C991: gc_start (default.c:6723)
    ==25795==    by 0x260F96: heap_prepare (default.c:2282)
    ==25795==    by 0x260F96: heap_next_free_page (default.c:2489)
    ==25795==    by 0x260F96: newobj_cache_miss (default.c:2598)
    ==25795==    by 0x26197F: newobj_alloc (default.c:2622)
    ==25795==    by 0x26197F: rb_gc_impl_new_obj (default.c:2701)
    ==25795==    by 0x26197F: newobj_of (gc.c:890)
    ==25795==    by 0x26197F: rb_wb_protected_newobj_of (gc.c:917)
    ==25795==    by 0x2DEA88: rb_class_allocate_instance (object.c:131)
    ==25795==    by 0x2E3B18: class_call_alloc_func (object.c:2141)
    ==25795==    by 0x2E3B18: rb_class_alloc (object.c:2113)
    ==25795==    by 0x2E3B18: rb_class_new_instance_pass_kw (object.c:2172)
    ==25795==    by 0x429DDC: vm_call_cfunc_with_frame_ (vm_insnhelper.c:3786)
    ==25795==    by 0x44B08D: vm_sendish (vm_insnhelper.c:5953)
    ==25795==    by 0x44B08D: vm_exec_core (insns.def:898)
    ==25795==    by 0x43A7A4: rb_vm_exec (vm.c:2564)
    ==25795==    by 0x234914: rb_ec_exec_node (eval.c:281)
    ==25795==  Address 0x21603710 is 0 bytes inside a block of size 16 free'd
    ==25795==    at 0x4849B2C: free (vg_replace_malloc.c:989)
    ==25795==    by 0x249651: rb_gc_impl_free (default.c:8527)
    ==25795==    by 0x249651: rb_gc_impl_free (default.c:8508)
    ==25795==    by 0x249651: ruby_sized_xfree.constprop.0 (gc.c:4178)
    ==25795==    by 0x4626EC: ruby_sized_xfree_inlined (gc.h:277)
    ==25795==    by 0x4626EC: wmap_free_entry (weakmap.c:45)
    ==25795==    by 0x4626EC: wmap_mark_weak_table_i (weakmap.c:61)
    ==25795==    by 0x3A5CEF: apply_functor (st.c:1633)
    ==25795==    by 0x3A5CEF: st_general_foreach (st.c:1543)
    ==25795==    by 0x3A5CEF: rb_st_foreach (st.c:1640)
    ==25795==    by 0x25C991: gc_mark_children (default.c:4870)
    ==25795==    by 0x25C991: gc_marks_wb_unprotected_objects_plane (default.c:5565)
    ==25795==    by 0x25C991: rgengc_rememberset_mark_plane (default.c:5557)
    ==25795==    by 0x25C991: rgengc_rememberset_mark (default.c:6233)
    ==25795==    by 0x25C991: gc_marks_start (default.c:6057)
    ==25795==    by 0x25C991: gc_marks (default.c:6077)
    ==25795==    by 0x25C991: gc_start (default.c:6723)
    ==25795==    by 0x260F96: heap_prepare (default.c:2282)
    ==25795==    by 0x260F96: heap_next_free_page (default.c:2489)
    ==25795==    by 0x260F96: newobj_cache_miss (default.c:2598)
    ==25795==    by 0x26197F: newobj_alloc (default.c:2622)
    ==25795==    by 0x26197F: rb_gc_impl_new_obj (default.c:2701)
    ==25795==    by 0x26197F: newobj_of (gc.c:890)
    ==25795==    by 0x26197F: rb_wb_protected_newobj_of (gc.c:917)
    ==25795==    by 0x2DEA88: rb_class_allocate_instance (object.c:131)
    ==25795==    by 0x2E3B18: class_call_alloc_func (object.c:2141)
    ==25795==    by 0x2E3B18: rb_class_alloc (object.c:2113)
    ==25795==    by 0x2E3B18: rb_class_new_instance_pass_kw (object.c:2172)
    ==25795==    by 0x429DDC: vm_call_cfunc_with_frame_ (vm_insnhelper.c:3786)
    ==25795==    by 0x44B08D: vm_sendish (vm_insnhelper.c:5953)
    ==25795==    by 0x44B08D: vm_exec_core (insns.def:898)
    ==25795==    by 0x43A7A4: rb_vm_exec (vm.c:2564)
    ==25795==  Block was alloc'd at
    ==25795==    at 0x484680F: malloc (vg_replace_malloc.c:446)
    ==25795==    by 0x25CE9E: rb_gc_impl_malloc (default.c:8542)
    ==25795==    by 0x462A39: wmap_aset_replace (weakmap.c:423)
    ==25795==    by 0x3A5542: rb_st_update (st.c:1487)
    ==25795==    by 0x462B8E: wmap_aset (weakmap.c:452)
    ==25795==    by 0x429DDC: vm_call_cfunc_with_frame_ (vm_insnhelper.c:3786)
    ==25795==    by 0x44B08D: vm_sendish (vm_insnhelper.c:5953)
    ==25795==    by 0x44B08D: vm_exec_core (insns.def:898)
    ==25795==    by 0x43A7A4: rb_vm_exec (vm.c:2564)
    ==25795==    by 0x234914: rb_ec_exec_node (eval.c:281)
    ==25795==    by 0x2369B8: ruby_run_node (eval.c:319)
    ==25795==    by 0x15D675: rb_main (main.c:43)
    ==25795==    by 0x15D675: main (main.c:62)

* Fix use-after-free for WeakKeyMap

[Bug #20688]

We cannot free the key before the ST_DELETE because it could hash the
key which would read the key and would cause a use-after-free. Instead,
we store the key and free it on the next iteration.

parse.y: const_decl_path don't replace destination node by a literal (#11314)

2024-08-06T17:34:39+00:00

[Bug #20668]

The `dest` node is assumed to be a `CDECL`, so overwriting it with
a `LIT` cause a crash on the next iteration.

Co-authored-by: Jean Boussier

Update RubyGems 3.5.16 and Bundler 2.5.16 for Ruby 3.3 (#11252)

2024-07-30T16:05:54+00:00

* Merge RubyGems-3.5.12 and Bundler-2.5.12

* Merge RubyGems-3.5.13 and Bundler-2.5.13

* Merge RubyGems-3.5.14 and Bundler-2.5.14

* Merge RubyGems-3.5.15 and Bundler-2.5.15

* Merge RubyGems-3.5.16 and Bundler-2.5.16

[Bug #20654] Fix floor and ceil when ndigits is large (#11277)

2024-07-30T16:05:09+00:00

* Fix floor when ndigits is large

[Bug #20654]

This commit fixes Integer#floor and Float#floor when the number is
negative and ndigits is large such that 10**ndigits is a bignum.

Previously, it would return 0 in such cases. However, this would cause
unexpected behaviour such as:

    puts -1.floor(-5) # => -100000
    puts -1.floor(-10) # => -10000000000
    puts -1.floor(-20) # => 0

This commit changes the last result so that it will return
-100000000000000000000.

* Fix ceil when ndigits is large

[Bug #20654]

This commit fixes Integer#ceil and Float#ceil when the number is
negative and ndigits is large such that 10**ndigits is a bignum.

Previously, it would return 0 in such cases. However, this would cause
unexpected behaviour such as:

    puts 1.ceil(-5) # => 100000
    puts 1.ceil(-10) # => 10000000000
    puts 1.ceil(-20) # => 0

This commit changes the last result so that it will return
100000000000000000000.

[Bug #20653] Fix memory leak in String#start_with? when regexp times out (#11255)

2024-07-30T00:45:26+00:00

Fix memory leak in String#start_with? when regexp times out

[Bug #20653]

This commit refactors how Onigmo handles timeout. Instead of raising a
timeout error, onig_search will return a ONIGERR_TIMEOUT which the
caller can free memory, and then raise a timeout error.

This fixes a memory leak in String#start_with when the regexp times out.
For example:

    regex = Regexp.new("^#{"(a*)" * 10_000}x$", timeout: 0.000001)
    str = "a" * 1000000 + "x"

    10.times do
      100.times do
        str.start_with?(regex)
      rescue
      end

      puts `ps -o rss= -p #{$$}`
    end

Before:

    33216
    51936
    71152
    81728
    97152
    103248
    120384
    133392
    133520
    133616

After:

    14912
    15376
    15824
    15824
    16128
    16128
    16144
    16144
    16160
    16160