Age | Commit message (Collapse) | Author |
|
Notes:
Merged: https://github.com/ruby/ruby/pull/6700
|
|
This was introduced by b609bdeb5307e280137b4b2838af0fe4e4b46f1c
to suppress warnings. However these warngins were deleted by
beae6cbf0fd8b6619e5212552de98022d4c4d4d4. Therefore these codes
are not needed anymore.
Notes:
Merged: https://github.com/ruby/ruby/pull/6826
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/6818
|
|
[Feature #19134]
Notes:
Merged: https://github.com/ruby/ruby/pull/6818
|
|
If the rescue clause has only exc_var and not exc_list, use the
exc_var position instead of the rescue body position.
This issue appears to have been introduced in
688169fd83b24564b653c03977c168cea50ccd35 when "opt_list" was split
into "exc_list exc_var".
Fixes [Bug #18974]
Notes:
Merged: https://github.com/ruby/ruby/pull/6283
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/6776
|
|
Implementation for Language Server Protocol (LSP) sometimes needs token information.
For example both `m(1)` and `m(1, )` has same AST structure other than node locations
then it's impossible to check the existence of `,` from AST. However in later case,
it might be better to suggest variables list for the second argument.
Token information is important for such case.
This commit adds these methods.
* Add `keep_tokens` option for `RubyVM::AbstractSyntaxTree.parse`, `.parse_file` and `.of`
* Add `RubyVM::AbstractSyntaxTree::Node#tokens` which returns tokens for the node including tokens for descendants nodes.
* Add `RubyVM::AbstractSyntaxTree::Node#all_tokens` which returns all tokens for the input script regardless the receiver node.
[Feature #19070]
Impacts on memory usage and performance are below:
Memory usage:
```
$ cat test.rb
root = RubyVM::AbstractSyntaxTree.parse_file(File.expand_path('../test/ruby/test_keyword.rb', __FILE__), keep_tokens: true)
$ /usr/bin/time -f %Mkb /usr/local/bin/ruby -v
ruby 3.2.0dev (2022-11-19T09:41:54Z 19070-keep_tokens d3af1b8057) [x86_64-linux]
11408kb
# keep_tokens :false
$ /usr/bin/time -f %Mkb /usr/local/bin/ruby test.rb
17508kb
# keep_tokens :true
$ /usr/bin/time -f %Mkb /usr/local/bin/ruby test.rb
30960kb
```
Performance:
```
$ cat ../ast_keep_tokens.yml
prelude: |
src = <<~SRC
module M
class C
def m1(a, b)
1 + a + b
end
end
end
SRC
benchmark:
without_keep_tokens: |
RubyVM::AbstractSyntaxTree.parse(src, keep_tokens: false)
with_keep_tokens: |
RubyVM::AbstractSyntaxTree.parse(src, keep_tokens: true)
$ make benchmark COMPARE_RUBY="./ruby" ARGS=../ast_keep_tokens.yml
/home/kaneko.y/.rbenv/shims/ruby --disable=gems -rrubygems -I../benchmark/lib ../benchmark/benchmark-driver/exe/benchmark-driver \
--executables="compare-ruby::./ruby -I.ext/common --disable-gem" \
--executables="built-ruby::./miniruby -I../lib -I. -I.ext/common ../tool/runruby.rb --extout=.ext -- --disable-gems --disable-gem" \
--output=markdown --output-compare -v ../ast_keep_tokens.yml
compare-ruby: ruby 3.2.0dev (2022-11-19T09:41:54Z 19070-keep_tokens d3af1b8057) [x86_64-linux]
built-ruby: ruby 3.2.0dev (2022-11-19T09:41:54Z 19070-keep_tokens d3af1b8057) [x86_64-linux]
warming up..
| |compare-ruby|built-ruby|
|:--------------------|-----------:|---------:|
|without_keep_tokens | 21.659k| 21.303k|
| | 1.02x| -|
|with_keep_tokens | 6.220k| 5.691k|
| | 1.09x| -|
```
Notes:
Merged: https://github.com/ruby/ruby/pull/6770
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/6760
|
|
Fixes [Bug #19132]
Notes:
Merged: https://github.com/ruby/ruby/pull/6743
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/6721
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/6698
|
|
Before:
```
Reducing stack by rule 639 (line 5062):
$1 = token "integer literal" (1.0-1.1: 1)
-> $$ = nterm simple_numeric (1.0-1.1: )
```
After:
```
Reducing stack by rule 641 (line 5078):
$1 = token "integer literal" (1.0-1.1: 1)
-> $$ = nterm simple_numeric (1.0-1.1: NODE_LIT)
```
`"<*>"` is supported by Bison 2.3b (2008-05-27) or later.
https://git.savannah.gnu.org/cgit/bison.git/commit/?id=12e3584054c16ab255672c07af0ffc7bb220e8bc
Therefore developers need to install Bison 2.3b+ to build ruby from
source codes if their Bison is older.
Minimum version requirement for Bison is changed to 3.0.
See: https://bugs.ruby-lang.org/issues/19068 [Feature #19068]
Notes:
Merged: https://github.com/ruby/ruby/pull/6579
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/6517
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/6515
|
|
Assign internal_id to semantic value so that dump parsetree option
can render the tree for these codes without SEGV.
* `def m(&); end`
* `def m(*); end`
* `def m(**); end`
Notes:
Merged: https://github.com/ruby/ruby/pull/6514
|
|
So that "IF" node is kept in the case below
```
def m
if
end
```
[Feature #19013]
Notes:
Merged: https://github.com/ruby/ruby/pull/6512
|
|
By this change, syntax error is recovered smaller units.
In the case below, "DEFN :bar" is same level with "CLASS :Foo"
now.
```
module Z
class Foo
foo.
end
def bar
end
end
```
[Feature #19013]
Notes:
Merged: https://github.com/ruby/ruby/pull/6512
|
|
"end" after "." or "::" is treated as local variable or method,
see `EXPR_DOT_bit` for detail.
However this "changes" where `bar` method is defined. In the example
below it is not module Z but class Foo.
```
module Z
class Foo
foo.
end
def bar
end
end
```
[Feature #19013]
Notes:
Merged: https://github.com/ruby/ruby/pull/6512
|
|
but "end" tokens are needed for correct language.
[Feature #19013]
Notes:
Merged: https://github.com/ruby/ruby/pull/6512
|
|
If this option is enabled, SyntaxError is not raised and Node is
returned even if passed script is broken.
[Feature #19013]
Notes:
Merged: https://github.com/ruby/ruby/pull/6512
|
|
The following warning appears without this fix:
```
parse.y:78:1: warning: unknown warning group '-Wpsabi', ignored
[-Wunknown-warning-option]
RBIMPL_WARNING_IGNORED(-Wpsabi)
^
./include/ruby/internal/warning_push.h:103:39: note: expanded from macro
'RBIMPL_WARNING_IGNORED'
^
./include/ruby/internal/warning_push.h:99:39: note: expanded from macro
'RBIMPL_WARNING_PRAGMA2'
^
./include/ruby/internal/warning_push.h:98:39: note: expanded from macro
'RBIMPL_WARNING_PRAGMA1'
^
./include/ruby/internal/warning_push.h:97:39: note: expanded from macro
'RBIMPL_WARNING_PRAGMA0'
^
<scratch space>:49:27: note: expanded from here
clang diagnostic ignored "-Wpsabi"
^
1 warning generated.
```
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/6373
Merged-By: nobu <nobu@ruby-lang.org>
|
|
matching
[Bug #18990]
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/6292
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/6263
Merged-By: nobu <nobu@ruby-lang.org>
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/5977
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/6073
|
|
`Ripper::Lexer#parse` re-parses the source code with syntax errors
when `raise_errors: false`.
Co-Authored-By: tompng <tomoyapenguin@gmail.com>
Notes:
Merged: https://github.com/ruby/ruby/pull/6233
|
|
* Fix some UBSAN false positives.
* ruby tool/update-deps --fix
Notes:
Merged-By: jhawthorn <john@hawthorn.email>
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/6087
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/6065
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/6064
Merged-By: nobu <nobu@ruby-lang.org>
|
|
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/5867
|
|
Do not override the input string encoding at the time of preparation,
the source encoding is not determined from the input yet.
Notes:
Merged: https://github.com/ruby/ruby/pull/6015
|
|
`no_blockarg` is called for non-null `node` only.
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/6016
|
|
Invalid escapes are handled at multiple levels. The first level
is in parse.y, so skip invalid unicode escape checks for regexps
in parse.y.
Make rb_reg_preprocess and unescape_nonascii accept the regexp
options. In unescape_nonascii, if the regexp is an extended
regexp, when "#" is encountered, ignore all characters until the
end of line or end of regexp.
Unfortunately, in extended regexps, you can use "#" as a non-comment
character inside a character class, so also parse "[" and "]"
specially for extended regexps, and only skip comments if "#" is
not inside a character class. Handle nested character classes as well.
This issue doesn't just affect extended regexps, it also affects
"(#?" comments inside all regexps. So for those comments, scan
until trailing ")" and ignore content inside.
I'm not sure if there are other corner cases not handled. A
better fix would be to redesign the regexp parser so that it
unescaped during parsing instead of before parsing, so you already
know the current parsing state.
Fixes [Bug #18294]
Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
Notes:
Merged: https://github.com/ruby/ruby/pull/5721
Merged-By: jeremyevans <code@jeremyevans.net>
|
|
|
|
|
|
Record block ID before vtable_pop, so the incorrect one doesn't
override it.
Fixes [Bug #18673]
Notes:
Merged: https://github.com/ruby/ruby/pull/5761
|
|
|
|
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/5545
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/5540
|
|
|
|
Get rid of depending on using $/ internally in String#chomp!, and
chomp the separator at once.
Notes:
Merged: https://github.com/ruby/ruby/pull/5403
|
|
Get rid of depending on using $/ internally in ARGF.gets.
Notes:
Merged: https://github.com/ruby/ruby/pull/5403
|
|
This allows for the following syntax:
```ruby
def foo(*)
bar(*)
end
def baz(**)
quux(**)
end
```
This is a natural addition after the introduction of anonymous
block forwarding. Anonymous rest and keyword rest arguments were
already supported in method parameters, this just allows them to
be used as arguments to other methods. The same advantages of
anonymous block forwarding apply to rest and keyword rest argument
forwarding.
This has some minor changes to #parameters output. Now, instead
of `[:rest], [:keyrest]`, you get `[:rest, :*], [:keyrest, :**]`.
These were already used for `...` forwarding, so I think it makes
it more consistent to include them in other cases. If we want to
use `[:rest], [:keyrest]` in both cases, that is also possible.
I don't think the previous behavior of `[:rest], [:keyrest]` in
the non-... case and `[:rest, :*], [:keyrest, :**]` in the ...
case makes sense, but if we did want that behavior, we'll have to
make more substantial changes, such as using a different ID in the
... forwarding case.
Implements [Feature #18351]
Notes:
Merged: https://github.com/ruby/ruby/pull/5148
|
|
Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
Co-authored-by: Marivaldo Cavalheiro <marivaldo@gmail.com>
Notes:
Merged: https://github.com/ruby/ruby/pull/5089
Merged-By: nobu <nobu@ruby-lang.org>
|