summaryrefslogtreecommitdiff
path: root/test/ruby/test_ast.rb
AgeCommit message (Collapse)Author
2025-07-14merge revision(s) 9f91f3617bab2ee220d298ddb874ef73b10dac23: [Backport #21383]Takashi Kokubun
Fix memory leak with invalid yield in prism [Bug #21383] The following script leaks memory: 10.times do 20_000.times do eval("class C; yield; end") rescue SyntaxError end puts `ps -o rss= -p #{$$}` end Before: 16464 25536 29424 35904 39552 44576 46736 51600 56096 59824 After: 13488 16160 18240 20528 19760 21808 21680 22272 22064 22336
2024-10-03Update ruby test for colon-style hash inspecttompng
Notes: Merged: https://github.com/ruby/ruby/pull/10924
2024-09-30Implement SPLAT NODE keyword locationsydah
Notes: Merged: https://github.com/ruby/ruby/pull/11673
2024-09-28Implement OP_ASGN2 NODE locationsydah
Notes: Merged: https://github.com/ruby/ruby/pull/11702
2024-09-27Implement OP_ASGN1 NODE locationsydah
Notes: Merged: https://github.com/ruby/ruby/pull/11701
2024-09-25Implement BLOCK_PASS NODE keyword locationsydah
Notes: Merged: https://github.com/ruby/ruby/pull/11663
2024-09-25Implement RETURN NODE keyword locationsydah
Notes: Merged: https://github.com/ruby/ruby/pull/11589
2024-09-23Implement CASE3 NODE keyword locationsydah
Notes: Merged: https://github.com/ruby/ruby/pull/11661
2024-09-23Implement CASE2 NODE keyword locationsydah
Notes: Merged: https://github.com/ruby/ruby/pull/11661
2024-09-23Implement CASE NODE keyword locationsydah
Notes: Merged: https://github.com/ruby/ruby/pull/11661
2024-09-17Prevent warnings for RubyVM::AbstractSyntaxTree.parse in test_ast.rbYusuke Endoh
2024-09-12[PRISM] Omit some TestAST tests when Prism is enabledKevin Newton
Notes: Merged: https://github.com/ruby/ruby/pull/11497
2024-09-11Implement BREAK, NEXT and REDO NODE locationsydah
Notes: Merged: https://github.com/ruby/ruby/pull/11584
2024-09-11Implement WHILE and UNTIL NODE locationsydah
Notes: Merged: https://github.com/ruby/ruby/pull/11579
2024-09-09Implement WHEN NODE locationsydah
Notes: Merged: https://github.com/ruby/ruby/pull/11553
2024-09-05Implement AND/OR NODE operator locationsydah
Notes: Merged: https://github.com/ruby/ruby/pull/11543
2024-09-04Implement VALIAS NODE keyword locationsydah
Notes: Merged: https://github.com/ruby/ruby/pull/11531
2024-09-03Implement ALIAS NODE keyword locationsydah
Notes: Merged: https://github.com/ruby/ruby/pull/11530
2024-09-03Implement UNDEF NODE keyword locationsydah
Notes: Merged: https://github.com/ruby/ruby/pull/11523
2024-08-25[Bug #20680] `ensure` block is always void contextNobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/11451
2024-07-23Implement UNLESS NODE keyword locationsyui-knk
Notes: Merged: https://github.com/ruby/ruby/pull/11227
2024-07-23[Feature #20624] Enhance `RubyVM::AbstractSyntaxTree::Node#locations`yui-knk
This commit introduce `RubyVM::AbstractSyntaxTree::Node#locations` method and `RubyVM::AbstractSyntaxTree::Location` class. Ruby AST node will hold multiple locations information. `RubyVM::AbstractSyntaxTree::Node#locations` provides a way to access these locations information. `RubyVM::AbstractSyntaxTree::Location` is a class which holds these location information: * `#first_lineno` * `#first_column` * `#last_lineno` * `#last_column` Notes: Merged: https://github.com/ruby/ruby/pull/11226
2024-07-18[PRISM] Refactor parser support into its own moduleKevin Newton
Notes: Merged: https://github.com/ruby/ruby/pull/11201
2024-07-16[PRISM] Omit two more ast tests that will not work without RubyVM::AstKevin Newton
2024-06-25[Bug #20457] Do not remove final `return` nodeNobuyoshi Nakada
This was an optimization for versions prior to 1.9 that traverse the AST at runtime.
2024-05-02[PRISM] Enable test_ast.rbKevin Newton
2024-05-01Test for memory leakNobuyoshi Nakada
2024-04-11[Bug #20417] Block local variables do not need to warn about unusedNobuyoshi Nakada
2024-04-04Separate SCRIPT_LINES__ from ast.cHASUMI Hitoshi
This patch suggests relocating the code dealing with `SCRIPT_LINES__` from ast.c to ruby_parser.c. ## Background - I guess `AbstractSyntaxTree.of` method used to use `SCRIPT_LINES__` internally for some reason before - However, now it appears `SCRIPT_LINES__` is no longer used meaningfully by the method - As evidence of this, (and as my patch shows,) removing the function call of `rb_script_lines_for()` from `ast_s_of()` does not affect the result of `test/ruby/test_ast.rb` Given the above, I think two possibilities can be considered: - (A) `AbstractSyntaxTree.of` has not needed `SCRIPT_LINES__` already (I pick this) - (B) We lack a test case of `AbstractSyntaxTree.of` that needs to use `SCRIPT_LINES__` ## Besides, The current implementation causes strange behavior: ```console ruby -e"SCRIPT_LINES__ = {__FILE__ => []}; puts RubyVM::AbstractSyntaxTree.of(->{ 1 + 2 }, keep_script_lines: true).script_lines" => `-e:1:in '<main>': undefined method 'script_lines' for nil (NoMethodError)` ``` I think this is a bug because `AbstractSyntaxTree.of` is not supposed to return `nil` even in this case. This happens due to the ast.c's dependence on `SCRIPT_LINES__`. And at the end of the `ast_s_of()`, `node_find()` can not find the target child node obviously because it doesn't make sense to look for a corresponding node made from the parameter of `AbstractSyntaxTree.of` in the AST tree made from the value of `{__FILE__ => []}` ## Solution Since I think it's good enough `SCRIPT_LINES__` to be only referred by ruby.c, I chose the possibility "(A)" and wrote this patch which moves `rb_script_lines_for()` from ast.c to ruby_parser.c. So as the result: - `ast_s_of()` function no longer look up `SCRIPT_LINES__` - Even so, this patched code passes the existing tests - The strange behavior above no longer happens (I also added a test for it) Please correct me if I miss something🙏
2024-01-24Do not use ruby2_keywords for ... argument forwardingJeremy Evans
This allows ... argument forwarding to benefit from Allocationless Anonymous Splat Forwarding, allowing the `f` call below to not allocate an array or a hash. ```ruby a = [1] kw = {b: 2} def c(a, b:) end def f(...) c(...) end f(*a, **kw) ``` This temporarily skips prism locals tests until prism is changed to use * and ** for ..., instead of using ruby2_keywords. Ignore failures in rbs bundled gems tests, since they fail due to this change.
2024-01-09Introduce NODE_SYM to manage symbol literalyui-knk
`:sym` was managed by `NODE_LIT` with `Symbol` object. This commit introduces `NODE_SYM` so that 1. Symbol literal is detectable from AST Node 2. Reduce dependency on ruby object
2024-01-07Introduce Numeric Node'sS-H-GAMELINKS
2023-11-05ast.rb: Fix bug for source of multibyte charactersalpaca-tc
first_column and last_column return byte positions, but existing implementations did not consider multibyte.
2023-10-02`yield` cannot be placed outside methods even in blocksNobuyoshi Nakada
2023-09-29Fix `retry` in nested `rescue` blocksNobuyoshi Nakada
Restore `rescue`-context from the outer context. `retry` targets the next outer block except for between `rescue` and `else` or `ensure`, otherwise, if there is no enclosing block, it should be syntax error.
2023-09-28Syntax check of `yield` in the parserNobuyoshi Nakada
2023-09-28Suppress void context warnings in verbose modeNobuyoshi Nakada
2023-09-27Out of place jumps are valid in `defined?`Nobuyoshi Nakada
2023-09-27Syntax check of `retry` in the parserNobuyoshi Nakada
2023-09-27Syntax check of block exits in the parserNobuyoshi Nakada
2023-08-25[Bug #19848] Flush BOMNobuyoshi Nakada
The token just after BOM needs to position at column 0, so that the indentation matches closing line. Notes: Merged: https://github.com/ruby/ruby/pull/8281
2023-03-30[Bug #19547] Add token for unescaped backslashNobuyoshi Nakada
This token is exposed only when `RubyVM::AbstractSyntaxTree` with `keep_tokens` option. Notes: Merged: https://github.com/ruby/ruby/pull/7622
2023-01-06[Bug #19312] Return end-of-input at `__END__`Nobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/7077
2023-01-06Check if the argument is Thread::Backtrace::Location objectyui-knk
[Bug #19262] Notes: Merged: https://github.com/ruby/ruby/pull/7029
2022-12-15Disallow mixed usage of ... and */**Shugo Maeda
[Feature #19134] Notes: Merged: https://github.com/ruby/ruby/pull/6934
2022-11-29Add a test case for argument forwardingyui-knk
Notes: Merged: https://github.com/ruby/ruby/pull/6828
2022-11-21Enhance keep_tokens option for RubyVM::AbstractSyntaxTree parsing methodsyui-knk
Implementation for Language Server Protocol (LSP) sometimes needs token information. For example both `m(1)` and `m(1, )` has same AST structure other than node locations then it's impossible to check the existence of `,` from AST. However in later case, it might be better to suggest variables list for the second argument. Token information is important for such case. This commit adds these methods. * Add `keep_tokens` option for `RubyVM::AbstractSyntaxTree.parse`, `.parse_file` and `.of` * Add `RubyVM::AbstractSyntaxTree::Node#tokens` which returns tokens for the node including tokens for descendants nodes. * Add `RubyVM::AbstractSyntaxTree::Node#all_tokens` which returns all tokens for the input script regardless the receiver node. [Feature #19070] Impacts on memory usage and performance are below: Memory usage: ``` $ cat test.rb root = RubyVM::AbstractSyntaxTree.parse_file(File.expand_path('../test/ruby/test_keyword.rb', __FILE__), keep_tokens: true) $ /usr/bin/time -f %Mkb /usr/local/bin/ruby -v ruby 3.2.0dev (2022-11-19T09:41:54Z 19070-keep_tokens d3af1b8057) [x86_64-linux] 11408kb # keep_tokens :false $ /usr/bin/time -f %Mkb /usr/local/bin/ruby test.rb 17508kb # keep_tokens :true $ /usr/bin/time -f %Mkb /usr/local/bin/ruby test.rb 30960kb ``` Performance: ``` $ cat ../ast_keep_tokens.yml prelude: | src = <<~SRC module M class C def m1(a, b) 1 + a + b end end end SRC benchmark: without_keep_tokens: | RubyVM::AbstractSyntaxTree.parse(src, keep_tokens: false) with_keep_tokens: | RubyVM::AbstractSyntaxTree.parse(src, keep_tokens: true) $ make benchmark COMPARE_RUBY="./ruby" ARGS=../ast_keep_tokens.yml /home/kaneko.y/.rbenv/shims/ruby --disable=gems -rrubygems -I../benchmark/lib ../benchmark/benchmark-driver/exe/benchmark-driver \ --executables="compare-ruby::./ruby -I.ext/common --disable-gem" \ --executables="built-ruby::./miniruby -I../lib -I. -I.ext/common ../tool/runruby.rb --extout=.ext -- --disable-gems --disable-gem" \ --output=markdown --output-compare -v ../ast_keep_tokens.yml compare-ruby: ruby 3.2.0dev (2022-11-19T09:41:54Z 19070-keep_tokens d3af1b8057) [x86_64-linux] built-ruby: ruby 3.2.0dev (2022-11-19T09:41:54Z 19070-keep_tokens d3af1b8057) [x86_64-linux] warming up.. | |compare-ruby|built-ruby| |:--------------------|-----------:|---------:| |without_keep_tokens | 21.659k| 21.303k| | | 1.02x| -| |with_keep_tokens | 6.220k| 5.691k| | | 1.09x| -| ``` Notes: Merged: https://github.com/ruby/ruby/pull/6770
2022-11-18Make anonymous rest arg (*) and block arg (&) accessible from ARGS nodeyui-knk
Notes: Merged: https://github.com/ruby/ruby/pull/6760
2022-11-18Allow anonymous keyword rest parameter with other keyword parametersShugo Maeda
Fixes [Bug #19132] Notes: Merged: https://github.com/ruby/ruby/pull/6743
2022-10-31Add `node_id_for_backtrace_location` functioneileencodes
We want to use error highlight with eval'd code, specifically ERB templates. We're able to recover the generated code for eval'd templates and can get a parse tree for the ERB generated code, but we don't have a way to get the node id from the backtrace location. So we can't pass the right node into error highlight. This patch gives us an API to get the node id from the backtrace location so we can find the node in the AST. Error Highlight PR: https://github.com/ruby/error_highlight/pull/26 Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org> Notes: Merged: https://github.com/ruby/ruby/pull/6593
generated by cgit v1.2.3 (git 2.25.1) at 2026-01-16 13:52:05 +0000