summaryrefslogtreecommitdiff
path: root/test
AgeCommit message (Collapse)Author
2024-06-18Fix flaky TestWeakMap#test_inspect_garbagePeter Zhu
If a GC is ran before the assert_match, then the WeakMap would be empty and would not have any objects, so the regular expression match would fail. This changes the regular expression to work even if the WeakMap is empty.
2024-06-18fix allocation assertionsAaron Patterson
2024-06-18Optimized forwarding callers and calleesAaron Patterson
This patch optimizes forwarding callers and callees. It only optimizes methods that only take `...` as their parameter, and then pass `...` to other calls. Calls it optimizes look like this: ```ruby def bar(a) = a def foo(...) = bar(...) # optimized foo(123) ``` ```ruby def bar(a) = a def foo(...) = bar(1, 2, ...) # optimized foo(123) ``` ```ruby def bar(*a) = a def foo(...) list = [1, 2] bar(*list, ...) # optimized end foo(123) ``` All variants of the above but using `super` are also optimized, including a bare super like this: ```ruby def foo(...) super end ``` This patch eliminates intermediate allocations made when calling methods that accept `...`. We can observe allocation elimination like this: ```ruby def m x = GC.stat(:total_allocated_objects) yield GC.stat(:total_allocated_objects) - x end def bar(a) = a def foo(...) = bar(...) def test m { foo(123) } end test p test # allocates 1 object on master, but 0 objects with this patch ``` ```ruby def bar(a, b:) = a + b def foo(...) = bar(...) def test m { foo(1, b: 2) } end test p test # allocates 2 objects on master, but 0 objects with this patch ``` How does it work? ----------------- This patch works by using a dynamic stack size when passing forwarded parameters to callees. The caller's info object (known as the "CI") contains the stack size of the parameters, so we pass the CI object itself as a parameter to the callee. When forwarding parameters, the forwarding ISeq uses the caller's CI to determine how much stack to copy, then copies the caller's stack before calling the callee. The CI at the forwarded call site is adjusted using information from the caller's CI. I think this description is kind of confusing, so let's walk through an example with code. ```ruby def delegatee(a, b) = a + b def delegator(...) delegatee(...) # CI2 (FORWARDING) end def caller delegator(1, 2) # CI1 (argc: 2) end ``` Before we call the delegator method, the stack looks like this: ``` Executing Line | Code | Stack ---------------+---------------------------------------+-------- 1| def delegatee(a, b) = a + b | self 2| | 1 3| def delegator(...) | 2 4| # | 5| delegatee(...) # CI2 (FORWARDING) | 6| end | 7| | 8| def caller | -> 9| delegator(1, 2) # CI1 (argc: 2) | 10| end | ``` The ISeq for `delegator` is tagged as "forwardable", so when `caller` calls in to `delegator`, it writes `CI1` on to the stack as a local variable for the `delegator` method. The `delegator` method has a special local called `...` that holds the caller's CI object. Here is the ISeq disasm fo `delegator`: ``` == disasm: #<ISeq:delegator@-e:1 (1,0)-(1,39)> local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1]) [ 1] "..."@0 0000 putself ( 1)[LiCa] 0001 getlocal_WC_0 "..."@0 0003 send <calldata!mid:delegatee, argc:0, FCALL|FORWARDING>, nil 0006 leave [Re] ``` The local called `...` will contain the caller's CI: CI1. Here is the stack when we enter `delegator`: ``` Executing Line | Code | Stack ---------------+---------------------------------------+-------- 1| def delegatee(a, b) = a + b | self 2| | 1 3| def delegator(...) | 2 -> 4| # | CI1 (argc: 2) 5| delegatee(...) # CI2 (FORWARDING) | cref_or_me 6| end | specval 7| | type 8| def caller | 9| delegator(1, 2) # CI1 (argc: 2) | 10| end | ``` The CI at `delegatee` on line 5 is tagged as "FORWARDING", so it knows to memcopy the caller's stack before calling `delegatee`. In this case, it will memcopy self, 1, and 2 to the stack before calling `delegatee`. It knows how much memory to copy from the caller because `CI1` contains stack size information (argc: 2). Before executing the `send` instruction, we push `...` on the stack. The `send` instruction pops `...`, and because it is tagged with `FORWARDING`, it knows to memcopy (using the information in the CI it just popped): ``` == disasm: #<ISeq:delegator@-e:1 (1,0)-(1,39)> local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1]) [ 1] "..."@0 0000 putself ( 1)[LiCa] 0001 getlocal_WC_0 "..."@0 0003 send <calldata!mid:delegatee, argc:0, FCALL|FORWARDING>, nil 0006 leave [Re] ``` Instruction 001 puts the caller's CI on the stack. `send` is tagged with FORWARDING, so it reads the CI and _copies_ the callers stack to this stack: ``` Executing Line | Code | Stack ---------------+---------------------------------------+-------- 1| def delegatee(a, b) = a + b | self 2| | 1 3| def delegator(...) | 2 4| # | CI1 (argc: 2) -> 5| delegatee(...) # CI2 (FORWARDING) | cref_or_me 6| end | specval 7| | type 8| def caller | self 9| delegator(1, 2) # CI1 (argc: 2) | 1 10| end | 2 ``` The "FORWARDING" call site combines information from CI1 with CI2 in order to support passing other values in addition to the `...` value, as well as perfectly forward splat args, kwargs, etc. Since we're able to copy the stack from `caller` in to `delegator`'s stack, we can avoid allocating objects. I want to do this to eliminate object allocations for delegate methods. My long term goal is to implement `Class#new` in Ruby and it uses `...`. I was able to implement `Class#new` in Ruby [here](https://github.com/ruby/ruby/pull/9289). If we adopt the technique in this patch, then we can optimize allocating objects that take keyword parameters for `initialize`. For example, this code will allocate 2 objects: one for `SomeObject`, and one for the kwargs: ```ruby SomeObject.new(foo: 1) ``` If we combine this technique, plus implement `Class#new` in Ruby, then we can reduce allocations for this common operation. Co-Authored-By: John Hawthorn <john@hawthorn.email> Co-Authored-By: Alan Wu <XrXr@users.noreply.github.com>
2024-06-18[ruby/irb] Improve how command calls' return value is handledStan Lo
(https://github.com/ruby/irb/pull/972) In #934, we changed command calls to return nil only. This PR improves the behaviour even further by: - Not echoing the `nil` returned by command calls - Not overriding previous return value stored in `_` with the `nil` from commands https://github.com/ruby/irb/commit/c844176842
2024-06-18[ruby/reline] Fix vi_yank or vi_delete_meta copies nil bugtomoya ishida
(https://github.com/ruby/reline/pull/726) https://github.com/ruby/reline/commit/46b30b07c9
2024-06-18[rubygems/rubygems] Delete extra spaces left after rubocop autofixAlexey Schepin
https://github.com/rubygems/rubygems/commit/a552732bed
2024-06-18[rubygems/rubygems] Disable `install_extension_in_lib` when cross-compilingYuta Saito
https://github.com/rubygems/rubygems/commit/643e154f32
2024-06-18[rubygems/rubygems] Add `--target-rbconfig` option to `gem install` and `gem ↵Yuta Saito
update` commands This patch adds `--target-rbconfig` option to specify the rbconfig.rb file for the deployment target platform. This is useful when cross-compiling gems. At the moment, this option is only available for `extconf.rb`-based extensions. https://github.com/rubygems/rubygems/commit/cf2843f7a2
2024-06-14[ruby/prism] Ensure ranges are non-associativeKevin Newton
https://github.com/ruby/prism/commit/f59295938b
2024-06-14[Bug #20579] ripper: Dispatch spaces at END-OF-INPUT without newlineNobuyoshi Nakada
2024-06-14[rubygems/rubygems] Never remove executables that may belong to a default gemDavid Rodríguez
https://github.com/rubygems/rubygems/commit/ed585f2fca
2024-06-14[rubygems/rubygems] Reuse `write_file` helperDavid Rodríguez
https://github.com/rubygems/rubygems/commit/339f099870
2024-06-14[Bug #20578] ripper: Fix dispatching part at invalid escapesNobuyoshi Nakada
2024-06-13[ruby/prism] Handle implicit array precedenceKevin Newton
When an implicit array is used in a write, is causes the whole expression to become a statement. For example: ```ruby a = *b a = 1, 2, 3 ``` Even though these expressions are exactly equivalent to their explicit array counterparts: ```ruby a = [*b] a = [1, 2, 3] ``` As such, these expressions cannot be joined with other expressions by operators or modifiers except if, unless, while, until, or rescue. https://github.com/ruby/prism/commit/7cd2407272
2024-06-13Add rb_str_resize coderange testtompng
2024-06-13[rubygems/rubygems] Fix default gem priority when sorting specsDavid Rodríguez
https://github.com/rubygems/rubygems/commit/8dbe1dbdc7 Co-authored-by: MSP-Greg <Greg.mpls@gmail.com>
2024-06-13clear `kw_flag` if given hash is nilKoichi Sasada
https://bugs.ruby-lang.org/issues/20570 is caused I missed to clear the `kw_flag` even if `keyword_hash` is nil.
2024-06-12Add regression test for Bug #20573Aaron Patterson
Just a regression test to ensure behavior remains the same
2024-06-12[ruby/irb] Invalid encoding symbol now raises SyntaxError also intomoya ishida
3.3 (https://github.com/ruby/irb/pull/969) https://github.com/ruby/irb/commit/bad7492ab0
2024-06-12[Bug #20572] Abandon if replacing destination is the sameNobuyoshi Nakada
2024-06-12[ruby/irb] Cleanup irbrc generator cache always at teardowntomoya ishida
(https://github.com/ruby/irb/pull/968) https://github.com/ruby/irb/commit/905184ff9c
2024-06-11[ruby/openssl] Pass through nil as digest when signing certificatesgartens
(https://github.com/ruby/openssl/pull/761) In order to sign certificates with Ed25519 keys, NULL must be passed as md to X509_sign. This NULL is then passed (via ASN1_item_sign_ex) as type to EVP_DigestSignInit. The documentation[1] of EVP_DigestSignInit states that type must be NULL for various key types, including Ed25519. [1]: https://www.openssl.org/docs/manmaster/man3/EVP_DigestSignInit.html https://github.com/ruby/openssl/commit/b0fc100091
2024-06-11[ruby/prism] Match CRuby error message for unknown regexp optionsKevin Newton
https://github.com/ruby/prism/commit/73669b59f6
2024-06-11compile.c: use putspecialobject for RubyVM::FrozenCoreJean Boussier
[Bug #20569] `putobject RubyVM::FrozenCore`, is not serializable, we have to use `putspecialobject VM_SPECIAL_OBJECT_VMCORE`.
2024-06-10[ruby/prism] Provide ability to lock encoding while parsingKevin Newton
https://github.com/ruby/prism/commit/f7faedfb3f
2024-06-10Don't skip test_inspect_under_gc_compact_stressPeter Zhu
Commit 1471a16 seems to have fixed this flaky test, so we don't need to skip it for YJIT or RJIT anymore.
2024-06-10[ruby/prism] Allow block exits in loop predicatesKevin Newton
https://github.com/ruby/prism/commit/f09db18e46
2024-06-09[ruby/irb] Suppress Ruby warnings in certain backtrace filteringStan Lo
tests (https://github.com/ruby/irb/pull/966) Since they're sensitive to the warnings, and the warnings are not relevant to the tests, we can suppress them to keep the tests simple. https://github.com/ruby/irb/commit/ad642795da
2024-06-08[ruby/openssl] Add X509::Certificate#tbs_bytesSamuel Giddins
Ref https://github.com/ruby/openssl/issues/519 This makes verifying embedded certificate transparency signatures significantly easier, as otherwise the alternative was manipulating the ASN1 sequence, as in https://github.com/segiddins/sigstore-cosign-verify/pull/2/commits/656d992fa816613fd9936f53ce30972c2f2f4957 https://github.com/ruby/openssl/commit/99128bea5d
2024-06-07[ruby/prism] Handle chomped bytesize with lines without newlinesKevin Newton
https://github.com/ruby/prism/commit/1528d3c019
2024-06-07[ruby/prism] Document that nested heredocs are not properly parsed for parserKevin Newton
https://github.com/ruby/prism/commit/d218e65561
2024-06-07[ruby/prism] Ensure inner heredoc nodes have the correct locationKevin Newton
https://github.com/ruby/prism/commit/100340bc6b
2024-06-07[ruby/prism] Use correct newlines for heredoc inner linesKevin Newton
https://github.com/ruby/prism/commit/4a9a7a62af Co-authored-by: Jason Kim <jasonkim@github.com> Co-authored-by: Adam Hess <HParker@github.com>
2024-06-07Remove prism compiler warningKevin Newton
2024-06-07Don't use SEGV signal when timeout in test_gc_compactPeter Zhu
Using a SEGV signal for timeout makes it difficult to tell if it's a real SEGV or if it timed out, so we should just use the default signals.
2024-06-07TestRequire#test_loading_fifo_threading_success: Extend the timeout limitYusuke Endoh
2024-06-07TestRegexp#test_match_cache_positive_look_behind: Extend the timeout limitYusuke Endoh
2024-06-07TestRegexp#test_timeout_shorter_than_global: Extend the timeout limitYusuke Endoh
2024-06-07TestRequire#test_loading_fifo_fd_leak: Extend the timeout limitYusuke Endoh
2024-06-07TestGc#test_thrashing_for_young_objects: extend the timeout limitYusuke Endoh
2024-06-07TestRegexp#test_s_timeout: accept timeout errors more tolerantlyYusuke Endoh
This test seems flaky on macOS GitHub Actions
2024-06-07[ruby/error_highlight] Support for the prism compilerKevin Newton
https://github.com/ruby/error_highlight/commit/69fbacfd49
2024-06-07Extend timeout of TestGCCompect#test_moving_objects_between_size_poolsYusuke Endoh
It is too flaky on macOS GitHub Actions
2024-06-07ripper: Fix excess `compile_error` at simple backref op_asgnNobuyoshi Nakada
Fix up 89cfc1520717257073012ec07105c551e4b8af7c.
2024-06-06Fix Module#define_method to change visibility when passed existing method bodyJeremy Evans
Fixes [Bug #19749]
2024-06-06Remove circular parameter syntax errorKevin Newton
https://bugs.ruby-lang.org/issues/20478
2024-06-07Make subclasses to define methodsNobuyoshi Nakada
2024-06-06[Bug #20521] ripper: Clean up strtermNobuyoshi Nakada
2024-06-06[ruby/reline] Remove instance variable @first_chartomoya ishida
(https://github.com/ruby/reline/pull/717) When Reline reads EOF, Reline.readline should return nil if and only if input is empty https://github.com/ruby/reline/commit/cc74b3686a
2024-06-06[ruby/logger] Add support for symbols in #shift_ageBaron Bloomer
Resolves issue: https://github.com/ruby/logger/issues/46 https://github.com/ruby/logger/commit/83502c2107