| Age | Commit message (Collapse) | Author |
|---|
|
When loading a non-ASCII compatible file, an error is raised which
causes memory leak.
For example:
require "tempfile"
Tempfile.create do |f|
f.write("# -*- coding: UTF-16BE -*-")
f.flush
10.times do
20_000.times do
begin
load(f.path)
rescue
end
end
puts `ps -o rss= -p #{$$}`
end
end
Before:
33904
49072
64528
79216
94576
109504
124768
139536
154928
170256
After:
19568
21296
21664
21728
22192
22256
22416
22272
22272
22272
Notes:
Merged: https://github.com/ruby/ruby/pull/11220
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/11227
|
|
`$5`, `brace_block` is no longer assigned in this action.
|
|
call
Notes:
Merged: https://github.com/ruby/ruby/pull/11215
|
|
For example:
```
undef a, b
```
Before:
```
@ NODE_UNDEF (id: 1, line: 1, location: (1,6)-(1,10))*
```
After:
```
@ NODE_UNDEF (id: 1, line: 1, location: (1,0)-(1,10))*
```
Notes:
Merged: https://github.com/ruby/ruby/pull/11214
|
|
Change UNDEF Node to hold their items to keep the original grammar
structure.
For example:
```
undef a, b
```
Before:
```
@ NODE_BLOCK (id: 4, line: 1, location: (1,6)-(1,10))*
+- nd_head (1):
| @ NODE_UNDEF (id: 1, line: 1, location: (1,6)-(1,7))
| +- nd_undef:
| @ NODE_SYM (id: 0, line: 1, location: (1,6)-(1,7))
| +- string: :a
+- nd_head (2):
@ NODE_UNDEF (id: 3, line: 1, location: (1,9)-(1,10))
+- nd_undef:
@ NODE_SYM (id: 2, line: 1, location: (1,9)-(1,10))
+- string: :b
```
After:
```
@ NODE_UNDEF (id: 1, line: 1, location: (1,6)-(1,10))*
+- nd_undefs:
+- length: 2
+- element (0):
| @ NODE_SYM (id: 0, line: 1, location: (1,6)-(1,7))
| +- string: :a
+- element (1):
@ NODE_SYM (id: 2, line: 1, location: (1,9)-(1,10))
+- string: :b
```
Notes:
Merged: https://github.com/ruby/ruby/pull/11213
|
|
For example:
10.times do
100_000.times do
RubyVM::AbstractSyntaxTree.parse("x = 1 + 2 +", keep_tokens: true)
rescue SyntaxError
end
puts `ps -o rss= -p #{$$}`
end
Before:
28944
44816
60720
76496
92336
108160
123968
139808
155648
171408
After:
11984
12704
12816
12832
13072
13088
13088
13136
13136
13152
Notes:
Merged: https://github.com/ruby/ruby/pull/11193
|
|
We face `[BUG] unexpected rb_parser_ary_data_type (0) for script lines`
on master branch recently.
This commit changes `enum rb_parser_ary_data_type` to start with `1`
and `0` to be invalid then it makes clear `rb_parser_ary_data_type (0)`
is not intentional.
|
|
This was an optimization for versions prior to 1.9 that traverse the
AST at runtime.
|
|
|
|
This patch optimizes forwarding callers and callees. It only optimizes methods that only take `...` as their parameter, and then pass `...` to other calls.
Calls it optimizes look like this:
```ruby
def bar(a) = a
def foo(...) = bar(...) # optimized
foo(123)
```
```ruby
def bar(a) = a
def foo(...) = bar(1, 2, ...) # optimized
foo(123)
```
```ruby
def bar(*a) = a
def foo(...)
list = [1, 2]
bar(*list, ...) # optimized
end
foo(123)
```
All variants of the above but using `super` are also optimized, including a bare super like this:
```ruby
def foo(...)
super
end
```
This patch eliminates intermediate allocations made when calling methods that accept `...`.
We can observe allocation elimination like this:
```ruby
def m
x = GC.stat(:total_allocated_objects)
yield
GC.stat(:total_allocated_objects) - x
end
def bar(a) = a
def foo(...) = bar(...)
def test
m { foo(123) }
end
test
p test # allocates 1 object on master, but 0 objects with this patch
```
```ruby
def bar(a, b:) = a + b
def foo(...) = bar(...)
def test
m { foo(1, b: 2) }
end
test
p test # allocates 2 objects on master, but 0 objects with this patch
```
How does it work?
-----------------
This patch works by using a dynamic stack size when passing forwarded parameters to callees.
The caller's info object (known as the "CI") contains the stack size of the
parameters, so we pass the CI object itself as a parameter to the callee.
When forwarding parameters, the forwarding ISeq uses the caller's CI to determine how much stack to copy, then copies the caller's stack before calling the callee.
The CI at the forwarded call site is adjusted using information from the caller's CI.
I think this description is kind of confusing, so let's walk through an example with code.
```ruby
def delegatee(a, b) = a + b
def delegator(...)
delegatee(...) # CI2 (FORWARDING)
end
def caller
delegator(1, 2) # CI1 (argc: 2)
end
```
Before we call the delegator method, the stack looks like this:
```
Executing Line | Code | Stack
---------------+---------------------------------------+--------
1| def delegatee(a, b) = a + b | self
2| | 1
3| def delegator(...) | 2
4| # |
5| delegatee(...) # CI2 (FORWARDING) |
6| end |
7| |
8| def caller |
-> 9| delegator(1, 2) # CI1 (argc: 2) |
10| end |
```
The ISeq for `delegator` is tagged as "forwardable", so when `caller` calls in
to `delegator`, it writes `CI1` on to the stack as a local variable for the
`delegator` method. The `delegator` method has a special local called `...`
that holds the caller's CI object.
Here is the ISeq disasm fo `delegator`:
```
== disasm: #<ISeq:delegator@-e:1 (1,0)-(1,39)>
local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
[ 1] "..."@0
0000 putself ( 1)[LiCa]
0001 getlocal_WC_0 "..."@0
0003 send <calldata!mid:delegatee, argc:0, FCALL|FORWARDING>, nil
0006 leave [Re]
```
The local called `...` will contain the caller's CI: CI1.
Here is the stack when we enter `delegator`:
```
Executing Line | Code | Stack
---------------+---------------------------------------+--------
1| def delegatee(a, b) = a + b | self
2| | 1
3| def delegator(...) | 2
-> 4| # | CI1 (argc: 2)
5| delegatee(...) # CI2 (FORWARDING) | cref_or_me
6| end | specval
7| | type
8| def caller |
9| delegator(1, 2) # CI1 (argc: 2) |
10| end |
```
The CI at `delegatee` on line 5 is tagged as "FORWARDING", so it knows to
memcopy the caller's stack before calling `delegatee`. In this case, it will
memcopy self, 1, and 2 to the stack before calling `delegatee`. It knows how much
memory to copy from the caller because `CI1` contains stack size information
(argc: 2).
Before executing the `send` instruction, we push `...` on the stack. The
`send` instruction pops `...`, and because it is tagged with `FORWARDING`, it
knows to memcopy (using the information in the CI it just popped):
```
== disasm: #<ISeq:delegator@-e:1 (1,0)-(1,39)>
local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
[ 1] "..."@0
0000 putself ( 1)[LiCa]
0001 getlocal_WC_0 "..."@0
0003 send <calldata!mid:delegatee, argc:0, FCALL|FORWARDING>, nil
0006 leave [Re]
```
Instruction 001 puts the caller's CI on the stack. `send` is tagged with
FORWARDING, so it reads the CI and _copies_ the callers stack to this stack:
```
Executing Line | Code | Stack
---------------+---------------------------------------+--------
1| def delegatee(a, b) = a + b | self
2| | 1
3| def delegator(...) | 2
4| # | CI1 (argc: 2)
-> 5| delegatee(...) # CI2 (FORWARDING) | cref_or_me
6| end | specval
7| | type
8| def caller | self
9| delegator(1, 2) # CI1 (argc: 2) | 1
10| end | 2
```
The "FORWARDING" call site combines information from CI1 with CI2 in order
to support passing other values in addition to the `...` value, as well as
perfectly forward splat args, kwargs, etc.
Since we're able to copy the stack from `caller` in to `delegator`'s stack, we
can avoid allocating objects.
I want to do this to eliminate object allocations for delegate methods.
My long term goal is to implement `Class#new` in Ruby and it uses `...`.
I was able to implement `Class#new` in Ruby
[here](https://github.com/ruby/ruby/pull/9289).
If we adopt the technique in this patch, then we can optimize allocating
objects that take keyword parameters for `initialize`.
For example, this code will allocate 2 objects: one for `SomeObject`, and one
for the kwargs:
```ruby
SomeObject.new(foo: 1)
```
If we combine this technique, plus implement `Class#new` in Ruby, then we can
reduce allocations for this common operation.
Co-Authored-By: John Hawthorn <john@hawthorn.email>
Co-Authored-By: Alan Wu <XrXr@users.noreply.github.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Fix up 89cfc1520717257073012ec07105c551e4b8af7c.
|
|
https://bugs.ruby-lang.org/issues/20478
|
|
|
|
|
|
At least LLDB needs an actual variable not only casts to access the
type in debugger sessions.
|
|
|
|
|
|
|
|
Blocks and keywords are allowed in regular index.
Also update NEWS to make this more clear.
Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
|
|
Recently, `TestRubyLiteral#test_float` fails randomly.
```
1) Error:
TestRubyLiteral#test_float:
ArgumentError: SyntaxError#path changed: "(eval at /home/chkbuild/chkbuild/tmp/build/20240527T050036Z/ruby/test/ruby/test_literal.rb:642)"->"(eval at /home/chkbuild/chkbuild/tmp/build/20240527T050036Z/ruby/test/ruby/test_literal.rb:642)"
```
https://rubyci.s3.amazonaws.com/s390x/ruby-master/log/20240527T050036Z.fail.html.gz
According to Launchable, the first failure was on Apr 30.
This is just when 528c4501f46fbe1e06028d673a777ef124d29829 was
committed. I don't know if the change is really the cause, but I want to
revert it once to see if the random failure disappears.
|
|
|
|
Since 140512d2225e6fd046ba1bdbcd1a27450f55c233, `else` without
`rescue` has been a syntax error.
|
|
I think this fixes the following random test failure that could not be
fixed for a long time:
```
1) Failure:
TestSymbol#test_inspect_under_gc_compact_stress [/home/chkbuild/chkbuild/tmp/build/20240522T003003Z/ruby/test/ruby/test_symbol.rb:126]:
<":testing"> expected but was
<":\"\\x00\\x00\\x00\\x00\\x00\\x00\\x00\"">.
```
The value passed to this function is the return value of `rb_id2str`, so
it is never collected. However, if auto_compact is enabled, the string
may move and `RSTRING_PTR(str)` became invalid.
This change prevents the string from being moved by RB_GC_GUARD.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Prepare `lhs` as `$:$` before `assignable` and update it there.
Remove `ripper_assignable` which is no longer used.
|
|
Prepare `path` as `$:$` before `const_decl` and update it there.
Remove `ripper_const_decl` which is no longer used.
|
|
Now it is used only for wheter `opt_paren_args` is `none`. Introduce
a new special node to distinguish an empty parentheses from it .
|
|
|
|
|
|
Ripper DSL uses these values for callbacks, but does not need indexes.
|
|
|
|
|
|
|
|
* Add types to `tLAMBDA` and `tSTRING_DBEG` to store corresponding
information when returning these tokens.
* Add `enum lex_state_e state` to `%union` for `tSTRING_DBEG`.
|
|
In the past, these codes were used by both parser and ripper.
On ripper, the type of LHS is `<val>` then type cast was needed.
However currently these are only used by parser then no need to
cast.
|
