summaryrefslogtreecommitdiff
path: root/parse.y
AgeCommit message (Collapse)Author
2022-12-05Use loc to fix a failure of test_ast.rbShugo Maeda
``` 1) Failure: TestAst#test_ranges:test/fiber/scheduler.rb [/home/runner/work/ruby/ruby/src/test/ruby/test_ast.rb:122]: <[]> expected but was <[{:type=>:max_validation_error, :max=> #<RubyVM::AbstractSyntaxTree::Node::CodePosition:0x00007f80d630b598 @column=20, @lineno=203>, :end_pos=> #<RubyVM::AbstractSyntaxTree::Node::CodePosition:0x00007f80d630b778 @column=19, @lineno=203>, :node=> (BLOCK_PASS@203:15-203:19 (ARGSPUSH@203:15-203:20 (SPLAT@203:16-203:19 (LVAR@203:16-203:19 :*)) (HASH@203:16-203:19 (LIST@203:16-203:19 nil (LVAR@203:16-203:19 :**) nil))) (LVAR@203:16-203:19 :&))}]>. ```
2022-12-05Should use argsloc for the last argument for arg_append()Shugo Maeda
2022-12-02Introduce encoding check macroS-H-GAMELINKS
Notes: Merged: https://github.com/ruby/ruby/pull/6700
2022-11-29Remove ruby2_keywords related to args forwardingyui-knk
This was introduced by b609bdeb5307e280137b4b2838af0fe4e4b46f1c to suppress warnings. However these warngins were deleted by beae6cbf0fd8b6619e5212552de98022d4c4d4d4. Therefore these codes are not needed anymore. Notes: Merged: https://github.com/ruby/ruby/pull/6826
2022-11-29Use idFWD_* instead of ANON_*_IDShugo Maeda
Notes: Merged: https://github.com/ruby/ruby/pull/6818
2022-11-29Allow ** in def foo(...)Shugo Maeda
[Feature #19134] Notes: Merged: https://github.com/ruby/ruby/pull/6818
2022-11-24Fix the position of rescue clause without exc_listJeremy Evans
If the rescue clause has only exc_var and not exc_list, use the exc_var position instead of the rescue body position. This issue appears to have been introduced in 688169fd83b24564b653c03977c168cea50ccd35 when "opt_list" was split into "exc_list exc_var". Fixes [Bug #18974] Notes: Merged: https://github.com/ruby/ruby/pull/6283
2022-11-21Refactor to use has_delayed_token macroyui-knk
Notes: Merged: https://github.com/ruby/ruby/pull/6776
2022-11-21Enhance keep_tokens option for RubyVM::AbstractSyntaxTree parsing methodsyui-knk
Implementation for Language Server Protocol (LSP) sometimes needs token information. For example both `m(1)` and `m(1, )` has same AST structure other than node locations then it's impossible to check the existence of `,` from AST. However in later case, it might be better to suggest variables list for the second argument. Token information is important for such case. This commit adds these methods. * Add `keep_tokens` option for `RubyVM::AbstractSyntaxTree.parse`, `.parse_file` and `.of` * Add `RubyVM::AbstractSyntaxTree::Node#tokens` which returns tokens for the node including tokens for descendants nodes. * Add `RubyVM::AbstractSyntaxTree::Node#all_tokens` which returns all tokens for the input script regardless the receiver node. [Feature #19070] Impacts on memory usage and performance are below: Memory usage: ``` $ cat test.rb root = RubyVM::AbstractSyntaxTree.parse_file(File.expand_path('../test/ruby/test_keyword.rb', __FILE__), keep_tokens: true) $ /usr/bin/time -f %Mkb /usr/local/bin/ruby -v ruby 3.2.0dev (2022-11-19T09:41:54Z 19070-keep_tokens d3af1b8057) [x86_64-linux] 11408kb # keep_tokens :false $ /usr/bin/time -f %Mkb /usr/local/bin/ruby test.rb 17508kb # keep_tokens :true $ /usr/bin/time -f %Mkb /usr/local/bin/ruby test.rb 30960kb ``` Performance: ``` $ cat ../ast_keep_tokens.yml prelude: | src = <<~SRC module M class C def m1(a, b) 1 + a + b end end end SRC benchmark: without_keep_tokens: | RubyVM::AbstractSyntaxTree.parse(src, keep_tokens: false) with_keep_tokens: | RubyVM::AbstractSyntaxTree.parse(src, keep_tokens: true) $ make benchmark COMPARE_RUBY="./ruby" ARGS=../ast_keep_tokens.yml /home/kaneko.y/.rbenv/shims/ruby --disable=gems -rrubygems -I../benchmark/lib ../benchmark/benchmark-driver/exe/benchmark-driver \ --executables="compare-ruby::./ruby -I.ext/common --disable-gem" \ --executables="built-ruby::./miniruby -I../lib -I. -I.ext/common ../tool/runruby.rb --extout=.ext -- --disable-gems --disable-gem" \ --output=markdown --output-compare -v ../ast_keep_tokens.yml compare-ruby: ruby 3.2.0dev (2022-11-19T09:41:54Z 19070-keep_tokens d3af1b8057) [x86_64-linux] built-ruby: ruby 3.2.0dev (2022-11-19T09:41:54Z 19070-keep_tokens d3af1b8057) [x86_64-linux] warming up.. | |compare-ruby|built-ruby| |:--------------------|-----------:|---------:| |without_keep_tokens | 21.659k| 21.303k| | | 1.02x| -| |with_keep_tokens | 6.220k| 5.691k| | | 1.09x| -| ``` Notes: Merged: https://github.com/ruby/ruby/pull/6770
2022-11-18Make anonymous rest arg (*) and block arg (&) accessible from ARGS nodeyui-knk
Notes: Merged: https://github.com/ruby/ruby/pull/6760
2022-11-18Allow anonymous keyword rest parameter with other keyword parametersShugo Maeda
Fixes [Bug #19132] Notes: Merged: https://github.com/ruby/ruby/pull/6743
2022-11-16Using UNDEF_P macroS-H-GAMELINKS
Notes: Merged: https://github.com/ruby/ruby/pull/6721
2022-11-09Now bison 3.0 or later is requiredNobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/6698
2022-11-08Set default %printer for NODE ntermsyui-knk
Before: ``` Reducing stack by rule 639 (line 5062): $1 = token "integer literal" (1.0-1.1: 1) -> $$ = nterm simple_numeric (1.0-1.1: ) ``` After: ``` Reducing stack by rule 641 (line 5078): $1 = token "integer literal" (1.0-1.1: 1) -> $$ = nterm simple_numeric (1.0-1.1: NODE_LIT) ``` `"<*>"` is supported by Bison 2.3b (2008-05-27) or later. https://git.savannah.gnu.org/cgit/bison.git/commit/?id=12e3584054c16ab255672c07af0ffc7bb220e8bc Therefore developers need to install Bison 2.3b+ to build ruby from source codes if their Bison is older. Minimum version requirement for Bison is changed to 3.0. See: https://bugs.ruby-lang.org/issues/19068 [Feature #19068] Notes: Merged: https://github.com/ruby/ruby/pull/6579
2022-10-09Do not set `$!` to `SyntaxError` when error tolerantNobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/6517
2022-10-08Fix typosyui-knk
Notes: Merged: https://github.com/ruby/ruby/pull/6515
2022-10-08Fix SEGV of dump parsetreeyui-knk
Assign internal_id to semantic value so that dump parsetree option can render the tree for these codes without SEGV. * `def m(&); end` * `def m(*); end` * `def m(**); end` Notes: Merged: https://github.com/ruby/ruby/pull/6514
2022-10-08"expr_value" can be erroryui-knk
So that "IF" node is kept in the case below ``` def m if end ``` [Feature #19013] Notes: Merged: https://github.com/ruby/ruby/pull/6512
2022-10-08Move `error` from top_stmts and top_stmt to stmtyui-knk
By this change, syntax error is recovered smaller units. In the case below, "DEFN :bar" is same level with "CLASS :Foo" now. ``` module Z class Foo foo. end def bar end end ``` [Feature #19013] Notes: Merged: https://github.com/ruby/ruby/pull/6512
2022-10-08Treat "end" as reserved word with consideration of indentyui-knk
"end" after "." or "::" is treated as local variable or method, see `EXPR_DOT_bit` for detail. However this "changes" where `bar` method is defined. In the example below it is not module Z but class Foo. ``` module Z class Foo foo. end def bar end end ``` [Feature #19013] Notes: Merged: https://github.com/ruby/ruby/pull/6512
2022-10-08Generates "end" tokens if parser hits end of inputyui-knk
but "end" tokens are needed for correct language. [Feature #19013] Notes: Merged: https://github.com/ruby/ruby/pull/6512
2022-10-08Add error_tolerant option to RubyVM::ASTyui-knk
If this option is enabled, SyntaxError is not raised and Node is returned even if passed script is broken. [Feature #19013] Notes: Merged: https://github.com/ruby/ruby/pull/6512
2022-09-26Suppress a warning on clangShugo Maeda
The following warning appears without this fix: ``` parse.y:78:1: warning: unknown warning group '-Wpsabi', ignored [-Wunknown-warning-option] RBIMPL_WARNING_IGNORED(-Wpsabi) ^ ./include/ruby/internal/warning_push.h:103:39: note: expanded from macro 'RBIMPL_WARNING_IGNORED' ^ ./include/ruby/internal/warning_push.h:99:39: note: expanded from macro 'RBIMPL_WARNING_PRAGMA2' ^ ./include/ruby/internal/warning_push.h:98:39: note: expanded from macro 'RBIMPL_WARNING_PRAGMA1' ^ ./include/ruby/internal/warning_push.h:97:39: note: expanded from macro 'RBIMPL_WARNING_PRAGMA0' ^ <scratch space>:49:27: note: expanded from here clang diagnostic ignored "-Wpsabi" ^ 1 warning generated. ```
2022-09-14Reuse `opt_arg_append` functionS.H
Notes: Merged: https://github.com/ruby/ruby/pull/6373 Merged-By: nobu <nobu@ruby-lang.org>
2022-09-09Fix unexpected "duplicated key name" error in paren-less one line pattern ↵Kazuki Tsujimoto
matching [Bug #18990]
2022-08-28[Bug #18963] Separate string contents by here document terminatorNobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/6292
2022-08-22Reuse `nonlocal_var` patternsS.H
Notes: Merged: https://github.com/ruby/ruby/pull/6263 Merged-By: nobu <nobu@ruby-lang.org>
2022-08-19Reuse opt_nl ruleS-H-GAMELINKS
Notes: Merged: https://github.com/ruby/ruby/pull/5977
2022-08-19Repalce to NIL_P macroS-H-GAMELINKS
Notes: Merged: https://github.com/ruby/ruby/pull/6073
2022-08-12[Bug #18962] Do not read again once reached EOFNobuyoshi Nakada
`Ripper::Lexer#parse` re-parses the source code with syntax errors when `raise_errors: false`. Co-Authored-By: tompng <tomoyapenguin@gmail.com> Notes: Merged: https://github.com/ruby/ruby/pull/6233
2022-07-12Fix some UBSAN false positives (#6115)Kevin Backhouse
* Fix some UBSAN false positives. * ruby tool/update-deps --fix Notes: Merged-By: jhawthorn <john@hawthorn.email>
2022-07-06[Bug #18890] newline should be insignificant after pattern labelNobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/6087
2022-06-30[Bug #18877] Let `lex_ctxt` not to eat escaped whitespaceNobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/6065
2022-06-29[Bug #18884] `class` cannot be just followed by modifiersNobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/6064 Merged-By: nobu <nobu@ruby-lang.org>
2022-06-23Suppress notes for old gccNobuyoshi Nakada
2022-06-17Using is_ascii_string to check encodingS-H-GAMELINKS
Notes: Merged: https://github.com/ruby/ruby/pull/5867
2022-06-17Respect the encoding of the source [Bug #18827]Nobuyoshi Nakada
Do not override the input string encoding at the time of preparation, the source encoding is not determined from the input yet. Notes: Merged: https://github.com/ruby/ruby/pull/6015
2022-06-15Remove unnecessary conditionNobuyoshi Nakada
`no_blockarg` is called for non-null `node` only.
2022-06-14ripper: Check if anonymous parameters defined [Bug #18828]Nobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/6016
2022-06-06Ignore invalid escapes in regexp commentsJeremy Evans
Invalid escapes are handled at multiple levels. The first level is in parse.y, so skip invalid unicode escape checks for regexps in parse.y. Make rb_reg_preprocess and unescape_nonascii accept the regexp options. In unescape_nonascii, if the regexp is an extended regexp, when "#" is encountered, ignore all characters until the end of line or end of regexp. Unfortunately, in extended regexps, you can use "#" as a non-comment character inside a character class, so also parse "[" and "]" specially for extended regexps, and only skip comments if "#" is not inside a character class. Handle nested character classes as well. This issue doesn't just affect extended regexps, it also affects "(#?" comments inside all regexps. So for those comments, scan until trailing ")" and ignore content inside. I'm not sure if there are other corner cases not handled. A better fix would be to redesign the regexp parser so that it unescaped during parsing instead of before parsing, so you already know the current parsing state. Fixes [Bug #18294] Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org> Notes: Merged: https://github.com/ruby/ruby/pull/5721 Merged-By: jeremyevans <code@jeremyevans.net>
2022-04-21Private local variables should shadow outer variables [Bug #18629]Nobuyoshi Nakada
2022-04-18Fix the wrong index of the previous component [Bug #18739]Nobuyoshi Nakada
2022-04-05Fix using anonymous block in method accepting explicit keywordsJeremy Evans
Record block ID before vtable_pop, so the incorrect one doesn't override it. Fixes [Bug #18673] Notes: Merged: https://github.com/ruby/ruby/pull/5761
2022-02-19Find pattern is no longer experimental [Feature #18585]Kazuki Tsujimoto
2022-02-19Fix location of NODE_LIT in p_kwKazuki Tsujimoto
2022-02-12Reuse operation rule for operation2S-H-GAMELINKS
Notes: Merged: https://github.com/ruby/ruby/pull/5545
2022-02-12Reuse p_kwnorest rule for f_no_kwargS-H-GAMELINKS
Notes: Merged: https://github.com/ruby/ruby/pull/5540
2022-01-11Reduce p_args rules with p_restVladimir Dementyev
2022-01-07Use chomp: option when chomp modeNobuyoshi Nakada
Get rid of depending on using $/ internally in String#chomp!, and chomp the separator at once. Notes: Merged: https://github.com/ruby/ruby/pull/5403
2022-01-07Explicitly pass $/ when loop modeNobuyoshi Nakada
Get rid of depending on using $/ internally in ARGF.gets. Notes: Merged: https://github.com/ruby/ruby/pull/5403
generated by cgit v1.2.3 (git 2.25.1) at 2024-05-25 16:46:19 +0000