summaryrefslogtreecommitdiff
path: root/test/ruby/test_ast.rb
AgeCommit message (Collapse)Author
2023-01-19merge revision(s) 979dd02e2f0a70f3f30d47c6115780d8fac6147e: [Backport #19262]NARUSE, Yui
Check if the argument is Thread::Backtrace::Location object [Bug #19262] --- ast.c | 5 +++++ test/ruby/test_ast.rb | 6 ++++++ 2 files changed, 11 insertions(+)
2022-12-15Disallow mixed usage of ... and */**Shugo Maeda
[Feature #19134] Notes: Merged: https://github.com/ruby/ruby/pull/6934
2022-11-29Add a test case for argument forwardingyui-knk
Notes: Merged: https://github.com/ruby/ruby/pull/6828
2022-11-21Enhance keep_tokens option for RubyVM::AbstractSyntaxTree parsing methodsyui-knk
Implementation for Language Server Protocol (LSP) sometimes needs token information. For example both `m(1)` and `m(1, )` has same AST structure other than node locations then it's impossible to check the existence of `,` from AST. However in later case, it might be better to suggest variables list for the second argument. Token information is important for such case. This commit adds these methods. * Add `keep_tokens` option for `RubyVM::AbstractSyntaxTree.parse`, `.parse_file` and `.of` * Add `RubyVM::AbstractSyntaxTree::Node#tokens` which returns tokens for the node including tokens for descendants nodes. * Add `RubyVM::AbstractSyntaxTree::Node#all_tokens` which returns all tokens for the input script regardless the receiver node. [Feature #19070] Impacts on memory usage and performance are below: Memory usage: ``` $ cat test.rb root = RubyVM::AbstractSyntaxTree.parse_file(File.expand_path('../test/ruby/test_keyword.rb', __FILE__), keep_tokens: true) $ /usr/bin/time -f %Mkb /usr/local/bin/ruby -v ruby 3.2.0dev (2022-11-19T09:41:54Z 19070-keep_tokens d3af1b8057) [x86_64-linux] 11408kb # keep_tokens :false $ /usr/bin/time -f %Mkb /usr/local/bin/ruby test.rb 17508kb # keep_tokens :true $ /usr/bin/time -f %Mkb /usr/local/bin/ruby test.rb 30960kb ``` Performance: ``` $ cat ../ast_keep_tokens.yml prelude: | src = <<~SRC module M class C def m1(a, b) 1 + a + b end end end SRC benchmark: without_keep_tokens: | RubyVM::AbstractSyntaxTree.parse(src, keep_tokens: false) with_keep_tokens: | RubyVM::AbstractSyntaxTree.parse(src, keep_tokens: true) $ make benchmark COMPARE_RUBY="./ruby" ARGS=../ast_keep_tokens.yml /home/kaneko.y/.rbenv/shims/ruby --disable=gems -rrubygems -I../benchmark/lib ../benchmark/benchmark-driver/exe/benchmark-driver \ --executables="compare-ruby::./ruby -I.ext/common --disable-gem" \ --executables="built-ruby::./miniruby -I../lib -I. -I.ext/common ../tool/runruby.rb --extout=.ext -- --disable-gems --disable-gem" \ --output=markdown --output-compare -v ../ast_keep_tokens.yml compare-ruby: ruby 3.2.0dev (2022-11-19T09:41:54Z 19070-keep_tokens d3af1b8057) [x86_64-linux] built-ruby: ruby 3.2.0dev (2022-11-19T09:41:54Z 19070-keep_tokens d3af1b8057) [x86_64-linux] warming up.. | |compare-ruby|built-ruby| |:--------------------|-----------:|---------:| |without_keep_tokens | 21.659k| 21.303k| | | 1.02x| -| |with_keep_tokens | 6.220k| 5.691k| | | 1.09x| -| ``` Notes: Merged: https://github.com/ruby/ruby/pull/6770
2022-11-18Make anonymous rest arg (*) and block arg (&) accessible from ARGS nodeyui-knk
Notes: Merged: https://github.com/ruby/ruby/pull/6760
2022-11-18Allow anonymous keyword rest parameter with other keyword parametersShugo Maeda
Fixes [Bug #19132] Notes: Merged: https://github.com/ruby/ruby/pull/6743
2022-10-31Add `node_id_for_backtrace_location` functioneileencodes
We want to use error highlight with eval'd code, specifically ERB templates. We're able to recover the generated code for eval'd templates and can get a parse tree for the ERB generated code, but we don't have a way to get the node id from the backtrace location. So we can't pass the right node into error highlight. This patch gives us an API to get the node id from the backtrace location so we can find the node in the AST. Error Highlight PR: https://github.com/ruby/error_highlight/pull/26 Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org> Notes: Merged: https://github.com/ruby/ruby/pull/6593
2022-10-12Suppress warnings in test/ruby/test_ast.rbYusuke Endoh
The tests for error tolerance printed some warnings. This change suppresses them.
2022-10-09Do not set `$!` to `SyntaxError` when error tolerantNobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/6517
2022-10-09Extract `assert_error_tolerant`Nobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/6519
2022-10-09Set AST prettyprint width to 80 explicitlyNobuyoshi Nakada
`PrettyPrint` results depend on the `COLUMNS` environment variable by default.
2022-10-08"expr_value" can be erroryui-knk
So that "IF" node is kept in the case below ``` def m if end ``` [Feature #19013] Notes: Merged: https://github.com/ruby/ruby/pull/6512
2022-10-08Move `error` from top_stmts and top_stmt to stmtyui-knk
By this change, syntax error is recovered smaller units. In the case below, "DEFN :bar" is same level with "CLASS :Foo" now. ``` module Z class Foo foo. end def bar end end ``` [Feature #19013] Notes: Merged: https://github.com/ruby/ruby/pull/6512
2022-10-08Treat "end" as reserved word with consideration of indentyui-knk
"end" after "." or "::" is treated as local variable or method, see `EXPR_DOT_bit` for detail. However this "changes" where `bar` method is defined. In the example below it is not module Z but class Foo. ``` module Z class Foo foo. end def bar end end ``` [Feature #19013] Notes: Merged: https://github.com/ruby/ruby/pull/6512
2022-10-08Generates "end" tokens if parser hits end of inputyui-knk
but "end" tokens are needed for correct language. [Feature #19013] Notes: Merged: https://github.com/ruby/ruby/pull/6512
2022-10-08Add error_tolerant option to RubyVM::ASTyui-knk
If this option is enabled, SyntaxError is not raised and Node is returned even if passed script is broken. [Feature #19013] Notes: Merged: https://github.com/ruby/ruby/pull/6512
2022-06-20Prevent a warning "possibly useless use of a literal in void context"Yusuke Endoh
2022-06-17Respect the encoding of the source [Bug #18827]Nobuyoshi Nakada
Do not override the input string encoding at the time of preparation, the source encoding is not determined from the input yet. Notes: Merged: https://github.com/ruby/ruby/pull/6015
2022-01-31pend some tests while using to_binaryKoichi Sasada
ISeqs from ISeq#to_binary/load do not support `keep_script_lines` now so some tests should be pending tests with `RUBY_ISEQ_DUMP_DEBUG=to_binary` Notes: Merged: https://github.com/ruby/ruby/pull/5508
2021-12-26Make RubyVM::AST.of work with code written in `-e` command-line optionYusuke Endoh
[Bug #18434] Notes: Merged: https://github.com/ruby/ruby/pull/5347
2021-12-20test/ruby/test_ast.rb: Avoid a "method redefined" warningYusuke Endoh
2021-12-19Make AST.of possible even under eval when keep_script_lines is enabledYusuke Endoh
Now the following code works without an exception. ``` RubyVM.keep_script_lines = true eval(<<END) def foo end END p RubyVM::AbstractSyntaxTree.of(method(:foo)) ```
2021-12-19Make RubyVM::AbstractSyntaxTree.of raise for backtrace location in evalYusuke Endoh
This check is needed to fix a bug of error_highlight when NameError occurred in eval'ed code. https://github.com/ruby/error_highlight/pull/16 The same check for proc/method has been already introduced since 64ac984129a7a4645efe5ac57c168ef880b479b2.
2021-09-18ast.c: AST.of against C method should return nil (as Ruby 2.6--3.0)Yusuke Endoh
2021-09-18ast.c: AST.of checks if a given method object is defined in CYusuke Endoh
[Bug #18178]
2021-08-20ast.c: Rename "save_script_lines" to "keep_script_lines"Yusuke Endoh
... as per ko1's preference. He is preparing to extend this feature to ISeq for his new debugger. He prefers "keep" to "save" for this wording. This API is internal and not included in any released version, so I change it in advance.
2021-07-29Make RubyVM::AbstractSyntaxTree.of raise for method/proc created in evalJeremy Evans
This changes Thread::Location::Backtrace#absolute_path to return nil for methods/procs defined in eval. If the realpath of an iseq is nil, that indicates it was defined in eval, in which case you cannot use RubyVM::AbstractSyntaxTree.of. Fixes [Bug #16983] Co-authored-by: Koichi Sasada <ko1@atdot.net> Notes: Merged: https://github.com/ruby/ruby/pull/4519
2021-06-18ast.rb: RubyVM::AST.parse and .of accepts `save_script_lines: true`Yusuke Endoh
This option makes the parser keep the original source as an array of the original code lines. This feature exploits the mechanism of `SCRIPT_LINES__` but records only the specified code that is passed to RubyVM::AST.of or .parse, instead of recording all parsed program texts. Notes: Merged: https://github.com/ruby/ruby/pull/4581
2021-01-01Added AST assertions for method definition arguments [Bug #17495]Nobuyoshi Nakada
2021-01-01Added AST tests for endless method definitionsNobuyoshi Nakada
2020-07-08Added `NODE_SPECIAL_EXCESSIVE_COMMA` info to `ARGS` of ↵manga_osyo
`RubyVM::AbstractSyntaxTree`. Notes: Merged: https://github.com/ruby/ruby/pull/3298
2020-07-06Add operator info to `OP_ASGN2` of `RubyVM::AbstractSyntaxTree`.manga_osyo
Notes: Merged: https://github.com/ruby/ruby/pull/3294
2020-01-17Added test for f38b3e8c707ebdcad05aa9485cf1760640b74fbbNobuyoshi Nakada
2019-11-12Revert "Method reference operator"Nobuyoshi Nakada
This reverts commit 67c574736912003c377218153f9d3b9c0c96a17b. [Feature #16275]
2019-09-07Rename NODE_ARRAY to NODE_LIST to reflect its actual use casesYusuke Endoh
and NODE_ZARRAY to NODE_ZLIST. NODE_ARRAY is used not only by an Array literal, but also the contents of Hash literals, method call arguments, dynamic string literals, etc. In addition, the structure of NODE_ARRAY is a linked list, not an array. This is very confusing, so I believe `NODE_LIST` is a better name.
2019-08-30Make RubyVM::AbstractSyntaxTree handle **nil syntaxJeremy Evans
Use false instead of nil for the keyword and keyword rest values in that case. Notes: Merged: https://github.com/ruby/ruby/pull/2395
2019-06-30Suppress "literal in condition" warningsNobuyoshi Nakada
2019-06-30Suppress void context warnings in verbose modeNobuyoshi Nakada
2019-06-30Suppress unused variable warningsNobuyoshi Nakada
2019-05-18Add test for UNTILNobuyoshi Nakada
2019-05-18Distinguish pre-condition and post-condition loopsNobuyoshi Nakada
2019-01-14ast.c: fix missing head part in dynamic literalnobu
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66819 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-01-06ast.c: argument must be a stringnobu
[ruby-core:90904] [Bug #15511] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66736 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-31Method reference operatornobu
Introduce the new operator for method reference, `.:`. [Feature #12125] [Feature #13581] [EXPERIMENTAL] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66667 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-03Refine RubyVM::AbstractSyntaxTree::Node#typenobu
* ast.c (rb_ast_node_type): simplified to return a Symbol without "NODE_" prefix. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66142 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-25parser: improve error messagesnobu
[Fix GH-2011] From: Akim Demaille <akim.demaille@gmail.com> git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65969 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-09Get rid of setting SCRIPT_LINES__ by AST.parsenobu
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65647 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-09Rename `AST` module to `AbstractSyntaxTree`yui-knk
Follow the same naming convention of `InstructionSequence` class. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65641 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-05Implement `RubyVM::AST.of` [Feature #14836]yui-knk
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65542 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-09-07test/ruby/test_ast.rb: assert error messages toonobu
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64654 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
generated by cgit v1.2.3 (git 2.25.1) at 2026-01-18 19:52:52 +0000