| Age | Commit message (Collapse) | Author |
|---|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/11276
|
|
This is slowing down benchmarks on x86, so lets revert it for now.
Notes:
Merged: https://github.com/ruby/ruby/pull/11275
|
|
Use an enum for the method arg instead of needing to add an id
that doesn't map to an actual method name.
$ ruby --dump=insns -e 'b = "x"; [v].pack("E*", buffer: b)'
before:
```
== disasm: #<ISeq:<main>@-e:1 (1,0)-(1,34)>
local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
[ 1] b@0
0000 putchilledstring "x" ( 1)[Li]
0002 setlocal_WC_0 b@0
0004 putself
0005 opt_send_without_block <calldata!mid:v, argc:0, FCALL|VCALL|ARGS_SIMPLE>
0007 newarray 1
0009 putchilledstring "E*"
0011 getlocal_WC_0 b@0
0013 opt_send_without_block <calldata!mid:pack, argc:2, kw:[#<Symbol:0x000000000023110c>], KWARG>
0015 leave
```
after:
```
== disasm: #<ISeq:<main>@-e:1 (1,0)-(1,34)>
local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
[ 1] b@0
0000 putchilledstring "x" ( 1)[Li]
0002 setlocal_WC_0 b@0
0004 putself
0005 opt_send_without_block <calldata!mid:v, argc:0, FCALL|VCALL|ARGS_SIMPLE>
0007 putchilledstring "E*"
0009 getlocal b@0, 0
0012 opt_newarray_send 3, 5
0015 leave
```
Notes:
Merged: https://github.com/ruby/ruby/pull/11249
|
|
[Bug #20653]
This commit refactors how Onigmo handles timeout. Instead of raising a
timeout error, onig_search will return a ONIGERR_TIMEOUT which the
caller can free memory, and then raise a timeout error.
This fixes a memory leak in String#start_with when the regexp times out.
For example:
regex = Regexp.new("^#{"(a*)" * 10_000}x$", timeout: 0.000001)
str = "a" * 1000000 + "x"
10.times do
100.times do
str.start_with?(regex)
rescue
end
puts `ps -o rss= -p #{$$}`
end
Before:
33216
51936
71152
81728
97152
103248
120384
133392
133520
133616
After:
14912
15376
15824
15824
16128
16128
16144
16144
16160
16160
Notes:
Merged: https://github.com/ruby/ruby/pull/11247
|
|
Hash.new no longer takes keyword arguments except for capacity, so we
should just use a method that takes keyword arguments instead.
Notes:
Merged: https://github.com/ruby/ruby/pull/11246
|
|
[Bug #20650]
The capture group allocates memory that is leaked when it times out.
For example:
re = Regexp.new("^#{"(a*)" * 10_000}x$", timeout: 0.000001)
str = "a" * 1000000 + "x"
10.times do
100.times do
re =~ str
rescue Regexp::TimeoutError
end
puts `ps -o rss= -p #{$$}`
end
Before:
34688
56416
78288
100368
120784
140704
161904
183568
204320
224800
After:
16288
16288
16880
16896
16912
16928
16944
17184
17184
17200
Notes:
Merged: https://github.com/ruby/ruby/pull/11238
|
|
GitHub Actions macos-14 machine is stupid.
https://app.launchableinc.com/organizations/ruby/workspaces/ruby/data/test-paths/file%3Dtest%2Fruby%2Ftest_file.rb%23%23%23class%3DTestFile%23%23%23testcase%3Dtest_stat?organizationId=ruby&workspaceId=ruby&testPathId=file%3Dtest%2Fruby%2Ftest_file.rb%23%23%23class%3DTestFile%23%23%23testcase%3Dtest_stat&testSessionStatus=flake
Notes:
Merged: https://github.com/ruby/ruby/pull/11241
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/11234
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/11227
|
|
This commit introduce `RubyVM::AbstractSyntaxTree::Node#locations` method
and `RubyVM::AbstractSyntaxTree::Location` class.
Ruby AST node will hold multiple locations information.
`RubyVM::AbstractSyntaxTree::Node#locations` provides a way to access
these locations information.
`RubyVM::AbstractSyntaxTree::Location` is a class which holds these location information:
* `#first_lineno`
* `#first_column`
* `#last_lineno`
* `#last_column`
Notes:
Merged: https://github.com/ruby/ruby/pull/11226
|
|
Unparenthesize the argument and make `command_call` when calling with
`do`-block.
Notes:
Merged: https://github.com/ruby/ruby/pull/11216
|
|
call
Notes:
Merged: https://github.com/ruby/ruby/pull/11215
|
|
Enumerator#zip yields a single array, not multiple arguments,
so Enumerator::Lazy#zip should do the same.
Fixes [#20623]
Notes:
Merged: https://github.com/ruby/ruby/pull/11212
|
|
When calling a method that does not accept a positional splat
parameter with a splatted array with a ruby2_keywords flagged hash,
there is no need to duplicate the splatted array. Previously,
Ruby would duplicate the splatted array and potentially modify
it before flattening it to the VM stack
Use a similar approach as the f(*ary, **hash) optimization,
flattening the splatted array to the VM stack without modifying
it, and make any modifications needed to the VM stack.
Notes:
Merged: https://github.com/ruby/ruby/pull/11161
|
|
When calling a method that accepts keywords but not a keyword
splat with a splatted array with a ruby2_keywords flagged hash,
there is no need to duplicate the ruby2_keywords flagged hash,
since it will be accessed to get the keyword values, but it will
not be modified.
Notes:
Merged: https://github.com/ruby/ruby/pull/11161
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/11161
|
|
This avoids an array allocation when calling a method that does
not accept a positional splat or keywords with both a positional
splat and keywords. Previously, Ruby would dup the positional
splat to append the keyword splat to it. Then it would flatten
the dupped positional splat array to the VM stack.
This flattens the given positional splat to the VM stack, then
adds the keyword splat hash after the last positional splat
element on the VM stack, avoiding the need to modify
the positional splat array.
Notes:
Merged: https://github.com/ruby/ruby/pull/11161
|
|
a: lvar), and other calls
The `f(arg, *arg, **arg, **arg)` case was previously not optimized.
The optimizer didn't optimize this case because of the multiple
keyword splats, and the compiler didn't optimize it because the
`f(*arg, **arg, **arg)` optimization added in
0ee3960685e283d8e75149a8777eb0109d41509a didn't apply.
I found it difficult to apply this optimization without changing
the `setup_args_core` API, since by the time you get to the ARGSCAT
case, you don't know whether you were called recursively or directly,
so I'm not sure if it was possible to know at that point whether the
array allocation could be avoided.
This changes the dup_rest argument in `setup_args_core` from an int
to a pointer to int. This allows us to track whether we have allocated
a caller side array for multiple splats or splat+post across
recursive calls. Check the pointed value (*dup_rest) to determine the
`splatarray` argument. If dup_rest is 1, then use `splatarray true`
(caller-side array allocation), then set *dup_rest back to 0, ensuring
only a single `splatarray true` per method call.
Before calling `setup_args_core`, check whether the array allocation
can be avoided safely using `splatarray false`. Optimizable cases are:
```
// f(*arg)
SPLAT
// f(1, *arg)
ARGSCAT
LIST
// f(*arg, **arg)
ARGSPUSH
SPLAT
HASH nd_brace=0
// f(1, *arg, **arg)
ARGSPUSH
ARGSCAT
LIST
HASH nd_brace=0
```
If so, dup_rest is set to 0 instead of 1 to avoid the allocation.
After calling `setup_args_core`, check the flag. If the flag
includes `VM_CALL_ARGS_SPLAT`, and the pointed value has changed,
indicating `splatarray true` was used, then also set
`VM_CALL_ARGS_SPLAT_MUT` in the flag.
My initial attempt at this broke the `f(*ary, &ary.pop)` test,
because we were not duplicating the ary in the splat even though
it was modified later (evaluation order issue). The initial attempt
would also break `f(*ary, **ary.pop)` or `f(*ary, kw: ary.pop)` cases
for the same reason. I added test cases for those evaluation
order issues.
Add setup_args_dup_rest_p static function that checks that a given
node is safe. Call that on the block pass node to determine if
the block pass node is safe. Also call it on each of the hash
key/value nodes to test that they are safe. If any are not safe,
then set dup_rest = 1 so that `splatarray true` will be used to
avoid the evaluation order issue.
This new approach has the affect of optimizing most cases of
literal keywords after positional splats. Previously, only
static keyword hashes after positional splats avoided array
allocation for the splat. Now, most dynamic keyword hashes
after positional splats also avoid array allocation.
Add allocation tests for dynamic keyword keyword hashes after
positional splats.
setup_args_dup_rest_p is currently fairly conservative. It
could definitely be expanded to handle additional node types
to reduce allocations in additional cases.
Notes:
Merged: https://github.com/ruby/ruby/pull/11161
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/11201
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/11192
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/11184
|
|
|
|
These methods return the byte-based offset of the beginning or end of the specified match.
[Feature #20576]
|
|
|
|
This feature provides a new method `GC.config` that configures internal
GC configuration variables provided by an individual GC implementation.
Implemented in this PR is the option `full_mark`: a boolean value that
will determine whether the Ruby GC is allowed to run a major collection
while the process is running.
It has the following semantics
This feature configures Ruby's GC to only run minor GC's. It's designed
to give users relying on Out of Band GC complete control over when a
major GC is run. Configuring `full_mark: false` does two main things:
* Never runs a Major GC. When the heap runs out of space during a minor
and when a major would traditionally be run, instead we allocate more
heap pages, and mark objspace as needing a major GC.
* Don't increment object ages. We don't promote objects during GC, this
will cause every object to be scanned on every minor. This is an
intentional trade-off between minor GC's doing more work every time,
and potentially promoting objects that will then never be GC'd.
The intention behind not aging objects is that users of this feature
should use a preforking web server, or some other method of pre-warming
the oldgen (like Nakayoshi fork)before disabling Majors. That way most
objects that are going to be old will have already been promoted.
This will interleave major and minor GC collections in exactly the same
what that the Ruby GC runs in versions previously to this. This is the
default behaviour.
* This new method has the following extra semantics:
- `GC.config` with no arguments returns a hash of the keys of the
currently configured GC
- `GC.config` with a key pair (eg. `GC.config(full_mark: true)` sets
the matching config key to the corresponding value and returns the
entire known config hash, including the new values. If the key does
not exist, `nil` is returned
* When a minor GC is run, Ruby sets an internal status flag to determine
whether the next GC will be a major or a minor. When `full_mark:
false` this flag is ignored and every GC will be a minor.
This status flag can be accessed at
`GC.latest_gc_info(:needs_major_by)`. Any value other than `nil` means
that the next collection would have been a major.
Thus it's possible to use this feature to check at a predetermined
time, whether a major GC is necessary and run one if it is. eg. After
a request has finished processing.
```ruby
if GC.latest_gc_info(:needs_major_by)
GC.start(full_mark: true)
end
```
[Feature #20443]
|
|
16 beta
|
|
Include € euro sign from CCSID 864
|
|
This is a follow up to 182822683f86c8f8d63b05765addf5a04d112aa2.
Co-Authored-By: Aaron Patterson <tenderlove@ruby-lang.org>
|
|
ruby2_keywords method
Treat this similar to keyword splatting nil, using goto ignore.
However, keep previous behavior if the method accepts a keyword
splat, to avoid double hash allocation.
This also can avoid an array allocation when calling a method
that doesn't have any splat parameters but supports literal
keyword parameters, because ignore_keyword_hash_p was not
ignoring the keyword hash in that case.
This change doesn't remove the empty ruby2_keywords hash from
the array, which caused an assertion failure if the method
being called accepted keywords in some cases. Modify the
assertion to handle this case. An alternative approach would
add a flag to the args struct so the args_argc calculation could
handle this case and report the correct argc, but such an approach
would likely be slower.
|
|
For calls such as:
m(*ary, a: 2, **h)
m(*ary, **h, **h, **h)
Where m does not take a positional argument splat, there was previously
an array allocation (splatarray true) to dup ary, even though it was not
necessary to do so. This is because the elimination of the array allocation
(splatarray false) was performed in the optimizer, and the optimizer didn't
handle this case, because the instructions for the keywords can be of
arbitrary length.
Move part of the optimization from the optimizer to the compiler,
detecting parse trees of the form:
ARGS_PUSH:
head: SPLAT
tail: HASH (without brace)
And using splatarray false instead of splatarray true for them.
Unfortunately, moving part of the optimization to the compiler broke
the hash allocation elimination optimization for calls of the
form:
m(*ary, a: 2)
That's because the compiler had already set splatarray false,
and the optimizer code was looking for splatarray true.
Split the array allocation elimination and hash allocation
elimination in the optimizer so that the hash allocation
elimination will still apply if the compiler performs the
splatarray false optimization.
|
|
We only collect GC.stat_heap(nil, stat_heap_all)
once, outside of the loop, but assert_equal could
allocate objects which can cause a GC to run and
cause stat_heap_all to be out-of-sync.
|
|
As the document states, it should return `self`, not `nil`.
Fix up of f4b313f7338f5fbe37f73aae29f70aeb474f7f5b.
|
|
Improves activerecord by about 1% on the interpreter:
```
before: ruby 3.4.0dev (2024-07-03T18:40:10Z master f88841b8f3) [arm64-darwin23]
after: ruby 3.4.0dev (2024-07-03T18:41:14Z ruby-map 6c0df4eb32) [arm64-darwin23]
------------ ----------- ---------- ---------- ---------- ------------- ------------
bench before (ms) stddev (%) after (ms) stddev (%) after 1st itr before/after
activerecord 235.2 0.8 233.6 0.7 1.01 1.01
------------ ----------- ---------- ---------- ---------- ------------- ------------
Legend:
- after 1st itr: ratio of before/after time for the first benchmarking iteration.
- before/after: ratio of before/after time. Higher is better for after. Above 1 represents a speedup.
```
Improves YJIT by about 4%:
```
before: ruby 3.4.0dev (2024-07-03T18:40:10Z master f88841b8f3) +YJIT [arm64-darwin23]
after: ruby 3.4.0dev (2024-07-03T18:41:14Z ruby-map 6c0df4eb32) +YJIT [arm64-darwin23]
------------ ----------- ---------- ---------- ---------- ------------- ------------
bench before (ms) stddev (%) after (ms) stddev (%) after 1st itr before/after
activerecord 142.1 1.2 137.0 0.6 1.00 1.04
------------ ----------- ---------- ---------- ---------- ------------- ------------
Legend:
- after 1st itr: ratio of before/after time for the first benchmarking iteration.
- before/after: ratio of before/after time. Higher is better for after. Above 1 represents a speedup.
```
|
|
Following [Feature #20589] it can happen that we change the
capacity of a frozen array, so these assertions no longer make
sense.
Normally we don't hit them because `Array#freeze` shrinks the
array, but if somehow the Array was frozen using `Object#freeze`
then we may shrink it after it was frozen.
|
|
This commit splits gc.c into two files:
- gc.c now only contains code not specific to Ruby GC. This includes
code to mark objects (which the GC implementation may choose not to
use) and wrappers for internal APIs that the implementation may need
to use (e.g. locking the VM).
- gc_impl.c now contains the implementation of Ruby's GC. This includes
marking, sweeping, compaction, and statistics. Most importantly,
gc_impl.c only uses public APIs in Ruby and a limited set of functions
exposed in gc.c. This allows us to build gc_impl.c independently of
Ruby and plug Ruby's GC into itself.
|
|
It doesn't look like there was a test added for this bug, so I'm adding
it.
Code is from here:
https://web.archive.org/web/20160908192307/http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-dev/24445
|
|
[Bug #20598]
Just like [Bug #20595], Encoding#name_list and Encoding#aliases can have
their strings corrupted when Encoding.default_internal is set to nil.
Co-authored-by: Matthew Valentine-House <matt@eightbitraptor.com>
|
|
[Bug #20595]
enc_set_default_encoding will free the C string if the encoding is nil,
but the C string can be used by the encoding name string. This will cause
the encoding name string to be corrupted.
Consider the following code:
Encoding.default_internal = Encoding::ASCII_8BIT
names = Encoding.default_internal.names
p names
Encoding.default_internal = nil
p names
It outputs:
["ASCII-8BIT", "BINARY", "internal"]
["ASCII-8BIT", "BINARY", "\x00\x00\x00\x00\x00\x00\x00\x00"]
Co-authored-by: Matthew Valentine-House <matt@eightbitraptor.com>
|
|
|
|
|
|
This was an optimization for versions prior to 1.9 that traverse the
AST at runtime.
|
|
This supports the nodes in both in the parse.y and prism compilers.
Fixes [Bug #20043]
Co-authored-by: Kevin Newton <kddnewton@gmail.com>
|
|
`--disable-frozen-string-literal`
[Feature #20205]
This was an undesired side effect. Now that this value is a triplet, we can't
assume it's disabled by default.
|
|
GitHub Actions macos-arm-oss is often too slow and does not timestamp as
expected.
|
|
|
|
... to respect RUBY_TEST_TIMEOUT_SCALE. This test somehow fails
frequently on macos-arm-oss with --repeat-count=2
https://app.launchableinc.com/organizations/ruby/workspaces/ruby/data/test-paths/file%3Dtest%2Fruby%2Ftest_file.rb%23%23%23class%3DTestFile%23%23%23testcase%3Dtest_stat?organizationId=ruby&workspaceId=ruby&testPathId=file%3Dtest%2Fruby%2Ftest_file.rb%23%23%23class%3DTestFile%23%23%23testcase%3Dtest_stat&testSessionStatus=flake
|
|
These tests are flaky and are duplicative of other tests that are
run in CI when parser=prism.
|
|
If a GC is ran before the assert_match, then the WeakMap would be empty
and would not have any objects, so the regular expression match would
fail. This changes the regular expression to work even if the WeakMap
is empty.
|
|
|
|
This patch optimizes forwarding callers and callees. It only optimizes methods that only take `...` as their parameter, and then pass `...` to other calls.
Calls it optimizes look like this:
```ruby
def bar(a) = a
def foo(...) = bar(...) # optimized
foo(123)
```
```ruby
def bar(a) = a
def foo(...) = bar(1, 2, ...) # optimized
foo(123)
```
```ruby
def bar(*a) = a
def foo(...)
list = [1, 2]
bar(*list, ...) # optimized
end
foo(123)
```
All variants of the above but using `super` are also optimized, including a bare super like this:
```ruby
def foo(...)
super
end
```
This patch eliminates intermediate allocations made when calling methods that accept `...`.
We can observe allocation elimination like this:
```ruby
def m
x = GC.stat(:total_allocated_objects)
yield
GC.stat(:total_allocated_objects) - x
end
def bar(a) = a
def foo(...) = bar(...)
def test
m { foo(123) }
end
test
p test # allocates 1 object on master, but 0 objects with this patch
```
```ruby
def bar(a, b:) = a + b
def foo(...) = bar(...)
def test
m { foo(1, b: 2) }
end
test
p test # allocates 2 objects on master, but 0 objects with this patch
```
How does it work?
-----------------
This patch works by using a dynamic stack size when passing forwarded parameters to callees.
The caller's info object (known as the "CI") contains the stack size of the
parameters, so we pass the CI object itself as a parameter to the callee.
When forwarding parameters, the forwarding ISeq uses the caller's CI to determine how much stack to copy, then copies the caller's stack before calling the callee.
The CI at the forwarded call site is adjusted using information from the caller's CI.
I think this description is kind of confusing, so let's walk through an example with code.
```ruby
def delegatee(a, b) = a + b
def delegator(...)
delegatee(...) # CI2 (FORWARDING)
end
def caller
delegator(1, 2) # CI1 (argc: 2)
end
```
Before we call the delegator method, the stack looks like this:
```
Executing Line | Code | Stack
---------------+---------------------------------------+--------
1| def delegatee(a, b) = a + b | self
2| | 1
3| def delegator(...) | 2
4| # |
5| delegatee(...) # CI2 (FORWARDING) |
6| end |
7| |
8| def caller |
-> 9| delegator(1, 2) # CI1 (argc: 2) |
10| end |
```
The ISeq for `delegator` is tagged as "forwardable", so when `caller` calls in
to `delegator`, it writes `CI1` on to the stack as a local variable for the
`delegator` method. The `delegator` method has a special local called `...`
that holds the caller's CI object.
Here is the ISeq disasm fo `delegator`:
```
== disasm: #<ISeq:delegator@-e:1 (1,0)-(1,39)>
local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
[ 1] "..."@0
0000 putself ( 1)[LiCa]
0001 getlocal_WC_0 "..."@0
0003 send <calldata!mid:delegatee, argc:0, FCALL|FORWARDING>, nil
0006 leave [Re]
```
The local called `...` will contain the caller's CI: CI1.
Here is the stack when we enter `delegator`:
```
Executing Line | Code | Stack
---------------+---------------------------------------+--------
1| def delegatee(a, b) = a + b | self
2| | 1
3| def delegator(...) | 2
-> 4| # | CI1 (argc: 2)
5| delegatee(...) # CI2 (FORWARDING) | cref_or_me
6| end | specval
7| | type
8| def caller |
9| delegator(1, 2) # CI1 (argc: 2) |
10| end |
```
The CI at `delegatee` on line 5 is tagged as "FORWARDING", so it knows to
memcopy the caller's stack before calling `delegatee`. In this case, it will
memcopy self, 1, and 2 to the stack before calling `delegatee`. It knows how much
memory to copy from the caller because `CI1` contains stack size information
(argc: 2).
Before executing the `send` instruction, we push `...` on the stack. The
`send` instruction pops `...`, and because it is tagged with `FORWARDING`, it
knows to memcopy (using the information in the CI it just popped):
```
== disasm: #<ISeq:delegator@-e:1 (1,0)-(1,39)>
local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
[ 1] "..."@0
0000 putself ( 1)[LiCa]
0001 getlocal_WC_0 "..."@0
0003 send <calldata!mid:delegatee, argc:0, FCALL|FORWARDING>, nil
0006 leave [Re]
```
Instruction 001 puts the caller's CI on the stack. `send` is tagged with
FORWARDING, so it reads the CI and _copies_ the callers stack to this stack:
```
Executing Line | Code | Stack
---------------+---------------------------------------+--------
1| def delegatee(a, b) = a + b | self
2| | 1
3| def delegator(...) | 2
4| # | CI1 (argc: 2)
-> 5| delegatee(...) # CI2 (FORWARDING) | cref_or_me
6| end | specval
7| | type
8| def caller | self
9| delegator(1, 2) # CI1 (argc: 2) | 1
10| end | 2
```
The "FORWARDING" call site combines information from CI1 with CI2 in order
to support passing other values in addition to the `...` value, as well as
perfectly forward splat args, kwargs, etc.
Since we're able to copy the stack from `caller` in to `delegator`'s stack, we
can avoid allocating objects.
I want to do this to eliminate object allocations for delegate methods.
My long term goal is to implement `Class#new` in Ruby and it uses `...`.
I was able to implement `Class#new` in Ruby
[here](https://github.com/ruby/ruby/pull/9289).
If we adopt the technique in this patch, then we can optimize allocating
objects that take keyword parameters for `initialize`.
For example, this code will allocate 2 objects: one for `SomeObject`, and one
for the kwargs:
```ruby
SomeObject.new(foo: 1)
```
If we combine this technique, plus implement `Class#new` in Ruby, then we can
reduce allocations for this common operation.
Co-Authored-By: John Hawthorn <john@hawthorn.email>
Co-Authored-By: Alan Wu <XrXr@users.noreply.github.com>
|
