| Age | Commit message (Collapse) | Author |
|---|
|
98637d421dbe8bcf86cc2effae5e26bb96a6a4da changes the name of
the function. However this function is exported as global,
then change the name to origin one for keeping compatibility.
Notes:
Merged: https://github.com/ruby/ruby/pull/7852
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/7844
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/7299
|
|
[Bug #19262]
Notes:
Merged: https://github.com/ruby/ruby/pull/7029
|
|
Implementation for Language Server Protocol (LSP) sometimes needs token information.
For example both `m(1)` and `m(1, )` has same AST structure other than node locations
then it's impossible to check the existence of `,` from AST. However in later case,
it might be better to suggest variables list for the second argument.
Token information is important for such case.
This commit adds these methods.
* Add `keep_tokens` option for `RubyVM::AbstractSyntaxTree.parse`, `.parse_file` and `.of`
* Add `RubyVM::AbstractSyntaxTree::Node#tokens` which returns tokens for the node including tokens for descendants nodes.
* Add `RubyVM::AbstractSyntaxTree::Node#all_tokens` which returns all tokens for the input script regardless the receiver node.
[Feature #19070]
Impacts on memory usage and performance are below:
Memory usage:
```
$ cat test.rb
root = RubyVM::AbstractSyntaxTree.parse_file(File.expand_path('../test/ruby/test_keyword.rb', __FILE__), keep_tokens: true)
$ /usr/bin/time -f %Mkb /usr/local/bin/ruby -v
ruby 3.2.0dev (2022-11-19T09:41:54Z 19070-keep_tokens d3af1b8057) [x86_64-linux]
11408kb
# keep_tokens :false
$ /usr/bin/time -f %Mkb /usr/local/bin/ruby test.rb
17508kb
# keep_tokens :true
$ /usr/bin/time -f %Mkb /usr/local/bin/ruby test.rb
30960kb
```
Performance:
```
$ cat ../ast_keep_tokens.yml
prelude: |
src = <<~SRC
module M
class C
def m1(a, b)
1 + a + b
end
end
end
SRC
benchmark:
without_keep_tokens: |
RubyVM::AbstractSyntaxTree.parse(src, keep_tokens: false)
with_keep_tokens: |
RubyVM::AbstractSyntaxTree.parse(src, keep_tokens: true)
$ make benchmark COMPARE_RUBY="./ruby" ARGS=../ast_keep_tokens.yml
/home/kaneko.y/.rbenv/shims/ruby --disable=gems -rrubygems -I../benchmark/lib ../benchmark/benchmark-driver/exe/benchmark-driver \
--executables="compare-ruby::./ruby -I.ext/common --disable-gem" \
--executables="built-ruby::./miniruby -I../lib -I. -I.ext/common ../tool/runruby.rb --extout=.ext -- --disable-gems --disable-gem" \
--output=markdown --output-compare -v ../ast_keep_tokens.yml
compare-ruby: ruby 3.2.0dev (2022-11-19T09:41:54Z 19070-keep_tokens d3af1b8057) [x86_64-linux]
built-ruby: ruby 3.2.0dev (2022-11-19T09:41:54Z 19070-keep_tokens d3af1b8057) [x86_64-linux]
warming up..
| |compare-ruby|built-ruby|
|:--------------------|-----------:|---------:|
|without_keep_tokens | 21.659k| 21.303k|
| | 1.02x| -|
|with_keep_tokens | 6.220k| 5.691k|
| | 1.09x| -|
```
Notes:
Merged: https://github.com/ruby/ruby/pull/6770
|
|
We want to use error highlight with eval'd code, specifically ERB
templates. We're able to recover the generated code for eval'd templates
and can get a parse tree for the ERB generated code, but we don't have a
way to get the node id from the backtrace location. So we can't pass the
right node into error highlight.
This patch gives us an API to get the node id from the backtrace
location so we can find the node in the AST.
Error Highlight PR: https://github.com/ruby/error_highlight/pull/26
Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
Notes:
Merged: https://github.com/ruby/ruby/pull/6593
|
|
By this change, syntax error is recovered smaller units.
In the case below, "DEFN :bar" is same level with "CLASS :Foo"
now.
```
module Z
class Foo
foo.
end
def bar
end
end
```
[Feature #19013]
Notes:
Merged: https://github.com/ruby/ruby/pull/6512
|
|
If this option is enabled, SyntaxError is not raised and Node is
returned even if passed script is broken.
[Feature #19013]
Notes:
Merged: https://github.com/ruby/ruby/pull/6512
|
|
Use ISEQ_BODY macro to get the rb_iseq_constant_body of the ISeq. Using
this macro will make it easier for us to change the allocation strategy
of rb_iseq_constant_body when using Variable Width Allocation.
Notes:
Merged: https://github.com/ruby/ruby/pull/5698
|
|
[Bug #18434]
Notes:
Merged: https://github.com/ruby/ruby/pull/5347
|
|
Now the following code works without an exception.
```
RubyVM.keep_script_lines = true
eval(<<END)
def foo
end
END
p RubyVM::AbstractSyntaxTree.of(method(:foo))
```
|
|
This check is needed to fix a bug of error_highlight when NameError
occurred in eval'ed code.
https://github.com/ruby/error_highlight/pull/16
The same check for proc/method has been already introduced since
64ac984129a7a4645efe5ac57c168ef880b479b2.
|
|
This `NODE` type was used in pre-YARV implementation, to improve
the performance of assignment to dynamic local variable defined at
the innermost scope. It has no longer any actual difference with
`NODE_DASGN`, except for the node dump.
Notes:
Merged: https://github.com/ruby/ruby/pull/5251
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/5091
Merged-By: nobu <nobu@ruby-lang.org>
|
|
The implementation of a local variable tables was represented as `ID*`,
but it was very hacky: the first element is not an ID but the size of
the table, and, the last element is (sometimes) a link to the next local
table only when the id tables are a linked list.
This change converts the hacky implementation to a normal struct.
Notes:
Merged: https://github.com/ruby/ruby/pull/5136
|
|
ast.c: Use kept script_lines data instead of re-open the source file
Notes:
Merged-By: mame <mame@ruby-lang.org>
|
|
|
|
[Bug #18178]
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/4791
|
|
... as per ko1's preference. He is preparing to extend this feature to
ISeq for his new debugger. He prefers "keep" to "save" for this wording.
This API is internal and not included in any released version, so I
change it in advance.
|
|
This changes Thread::Location::Backtrace#absolute_path to return
nil for methods/procs defined in eval. If the realpath of an iseq
is nil, that indicates it was defined in eval, in which case you
cannot use RubyVM::AbstractSyntaxTree.of.
Fixes [Bug #16983]
Co-authored-by: Koichi Sasada <ko1@atdot.net>
Notes:
Merged: https://github.com/ruby/ruby/pull/4519
|
|
Now ISeq#to_a includes the node_id list for each bytecode instruction.
I want a way to retrieve the AST::Node instance corresponding to an
instruction for a research purpose including TypeProf-based LSP server.
|
|
... which is formally called EXPERIMENTAL_ISEQ_NODE_ID.
See also ff69ef27b06eed1ba750e7d9cab8322f351ed245.
https://bugs.ruby-lang.org/issues/17930
Notes:
Merged: https://github.com/ruby/ruby/pull/4558
|
|
RubyVM::AST.of(Thread::Backtrace::Location) returns a node that
corresponds to the location. Typically, the node is a method call, but
not always.
This change also includes iseq's dump/load support of node_ids for each
instructions.
Notes:
Merged: https://github.com/ruby/ruby/pull/4558
|
|
by merging `rb_ast_body_t#line_count` and `#script_lines`.
Fortunately `line_count == RARRAY_LEN(script_lines)` was always
satisfied. When script_lines is saved, it has an array of lines, and
when not saved, it has a Fixnum that represents the old line_count.
Notes:
Merged: https://github.com/ruby/ruby/pull/4581
|
|
This option makes the parser keep the original source as an array of
the original code lines. This feature exploits the mechanism of
`SCRIPT_LINES__` but records only the specified code that is passed to
RubyVM::AST.of or .parse, instead of recording all parsed program texts.
Notes:
Merged: https://github.com/ruby/ruby/pull/4581
|
|
... then, new_insn_core extracts nd_line(node).
Also, if a macro "EXPERIMENTAL_ISEQ_NODE_ID" is defined, this changeset
keeps nd_node_id(node) for each instruction. This is intended for
TypeProf to identify what AST::Node corresponds to each instruction.
This patch is originally authored by @yui-knk for showing which column a
NoMethodError occurred.
https://github.com/ruby/ruby/compare/master...yui-knk:feature/node_id
Co-Authored-By: Yuichiro Kaneko <yui-knk@ruby-lang.org>
Notes:
Merged: https://github.com/ruby/ruby/pull/4470
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/4270
Merged-By: nobu <nobu@ruby-lang.org>
|
|
[Feature #17104]
|
|
|
|
|
|
`RubyVM::AbstractSyntaxTree`.
Notes:
Merged: https://github.com/ruby/ruby/pull/3298
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/3294
|
|
Was this an autogenerated function? This tendency of avoiding empty
branches are no longer preserved (see for instance NODE_IVAR). Let's
just delete those unnecessary jumps into branches.
Notes:
Merged: https://github.com/ruby/ruby/pull/3247
|
|
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/2792
|
|
Saves comitters' daily life by avoid #include-ing everything from
internal.h to make each file do so instead. This would significantly
speed up incremental builds.
We take the following inclusion order in this changeset:
1. "ruby/config.h", where _GNU_SOURCE is defined (must be the very
first thing among everything).
2. RUBY_EXTCONF_H if any.
3. Standard C headers, sorted alphabetically.
4. Other system headers, maybe guarded by #ifdef
5. Everything else, sorted alphabetically.
Exceptions are those win32-related headers, which tend not be self-
containing (headers have inclusion order dependencies).
Notes:
Merged: https://github.com/ruby/ruby/pull/2711
|
|
This reverts commit 67c574736912003c377218153f9d3b9c0c96a17b.
[Feature #16275]
|
|
|
|
Define RubyVM::AbstractSyntaxTree in ast.rb
with __builtin functions.
Notes:
Merged: https://github.com/ruby/ruby/pull/2655
|
|
and NODE_ZARRAY to NODE_ZLIST.
NODE_ARRAY is used not only by an Array literal, but also the contents
of Hash literals, method call arguments, dynamic string literals, etc.
In addition, the structure of NODE_ARRAY is a linked list, not an array.
This is very confusing, so I believe `NODE_LIST` is a better name.
|
|
|
|
Use false instead of nil for the keyword and keyword rest values
in that case.
Notes:
Merged: https://github.com/ruby/ruby/pull/2395
|
|
MRI (#2113) [ci skip]
* Make it clear as possible that RubyVM is MRI-specific and only exists on MRI
* See [Bug #15743].
* Use "CRuby VM" instead of "Ruby VM" for clarity.
* Use YARV rather than "CRuby VM" for documenting RubyVM::InstructionSequence
* Avoid introducing a new "CRuby VM" term in documentation
|
|
|
|
|
|
|
|
Use NODE_SPECIAL_NO_NAME_REST instead.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67629 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
Add symbol to the result to make pattern match easily.
For example:
(1) NODE_MASGN * NODE_SPECIAL_NO_NAME_REST
```
$ ./miniruby -e 'p RubyVM::AbstractSyntaxTree.parse("a, * = b").children[-1].children'
[#<RubyVM::AbstractSyntaxTree::Node:VCALL@1:7-1:8>, #<RubyVM::AbstractSyntaxTree::Node:ARRAY@1:0-1:1>, :NODE_SPECIAL_NO_NAME_REST]
```
(2) NODE_POSTARG * NODE_SPECIAL_NO_NAME_REST
```
$ ./miniruby -e 'p RubyVM::AbstractSyntaxTree.parse("a, *, _ = b").children[-1].children[-1].children'
[:NODE_SPECIAL_NO_NAME_REST, #<RubyVM::AbstractSyntaxTree::Node:ARRAY@1:6-1:7>]
```
(3) NODE_LASGN * NODE_SPECIAL_REQUIRED_KEYWORD
```
$ ./miniruby -e 'p RubyVM::AbstractSyntaxTree.parse("def a(k:) end").children[-1].children[-1].children[1].children[7].children[0].children'
[:k, :NODE_SPECIAL_REQUIRED_KEYWORD]
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67627 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
[ruby-core:87945] [Feature #14912]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67586 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
