| Age | Commit message (Collapse) | Author |
|---|
|
https://github.com/ruby/prism/commit/47cb73ce69
|
|
https://github.com/ruby/prism/commit/073e8ba307
|
|
https://github.com/ruby/prism/commit/5985ab7687
|
|
https://github.com/ruby/prism/commit/f7faedfb3f
|
|
https://github.com/ruby/prism/commit/1528d3c019
|
|
https://github.com/ruby/prism/commit/100340bc6b
|
|
https://github.com/ruby/prism/commit/4a9a7a62af
Co-authored-by: Jason Kim <jasonkim@github.com>
Co-authored-by: Adam Hess <HParker@github.com>
|
|
https://github.com/ruby/prism/commit/85b4a5f804
|
|
https://github.com/ruby/prism/commit/461aa5e658
|
|
https://github.com/ruby/prism/commit/aecce571d8
|
|
https://github.com/ruby/prism/commit/12e079c97e
|
|
translation""
This reverts commit https://github.com/ruby/prism/commit/d8ae19d0334a.
https://github.com/ruby/prism/commit/df1eda2811
|
|
This reverts commit https://github.com/ruby/prism/commit/823e931ff230.
https://github.com/ruby/prism/commit/d8ae19d033
|
|
https://github.com/ruby/prism/commit/823e931ff2
|
|
https://github.com/ruby/prism/commit/a4e164e22b
|
|
https://github.com/ruby/prism/commit/840185110f
|
|
in parser
https://github.com/ruby/prism/commit/beed43922c
|
|
https://github.com/ruby/prism/commit/f2a327449a
|
|
https://github.com/ruby/prism/commit/785de2c39d
|
|
https://github.com/ruby/prism/commit/6f886be0a4
|
|
https://github.com/ruby/prism/commit/4b06eae0df
|
|
https://github.com/ruby/prism/commit/53bbcfe513
|
|
https://github.com/ruby/prism/commit/79cec4be22
|
|
This eliminates the subnode on RationalNode and replaces it with two
integer fields, which represent the ratio for the rational. It also
reduces those two integers if they both fit into 32 bits.
Importantly, this PR does not implement bignum reduction. That's something
I'd like to consider for the future, but it's simple enough for now to
leave them unreduced, which makes it more useful than it used to be.
https://github.com/ruby/prism/commit/86e06c7068
|
|
https://github.com/ruby/prism/commit/d4eb13e703
|
|
Resolves https://github.com/ruby/prism/pull/2803.
This PR adds error handling for missing `parser` gem in `Prism::Translation`.
The `parser` gem is a required runtime dependency when using `Prism::Translation::Parser`.
But it is not required for other uses of Prism. To avoid unnecessary dependencies,
it is not added as a `runtime_dependency` in the prism.gemspec. Instead, if the dependency is missing,
instructions are given to add it to Gemfile.
## Before
```console
$ bundle exec ruby -e 'require "prism"; require "prism/translation/parser33"'
/Users/koic/.rbenv/versions/3.3.1/lib/ruby/3.3.0/bundled_gems.rb:74:in `require': cannot load such file -- parser (LoadError)
from /Users/koic/.rbenv/versions/3.3.1/lib/ruby/3.3.0/bundled_gems.rb:74:in `block (2 levels) in replace_require'
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:3:in `<top (required)>'
from /Users/koic/.rbenv/versions/3.3.1/lib/ruby/3.3.0/bundled_gems.rb:74:in `require'
from /Users/koic/.rbenv/versions/3.3.1/lib/ruby/3.3.0/bundled_gems.rb:74:in `block (2 levels) in replace_require'
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser33.rb:6:in `<module:Translation>'
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser33.rb:4:in `<module:Prism>'
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser33.rb:3:in `<top (required)>'
from /Users/koic/.rbenv/versions/3.3.1/lib/ruby/3.3.0/bundled_gems.rb:74:in `require'
from /Users/koic/.rbenv/versions/3.3.1/lib/ruby/3.3.0/bundled_gems.rb:74:in `block (2 levels) in replace_require'
from -e:1:in `<main>'
```
## After
```console
$ bundle exec ruby -e 'require "prism"; require "prism/translation/parser33"'
Error: Unable to load parser. Add `gem "parser"` to your Gemfile.
```
https://github.com/ruby/prism/commit/4880aec22d
|
|
https://github.com/ruby/prism/commit/b5e47f5c42
|
|
Follow up https://github.com/ruby/prism/pull/2760.
This PR updates the `Translation::Parser` to use version 3.3.1 when the version 3.3 is specified.
The Parser gem is structured to support the latest patch versions, hence this aligns with Parser-compatible versioning.
As noted in https://github.com/ruby/prism/pull/2760, the behavior remains unchanged with this switch from 3.3.0 to 3.3.1.
https://github.com/ruby/prism/commit/efde09d318
|
|
https://github.com/ruby/prism/commit/14d9b431e2
|
|
This has been requested for a long time, and I'm finally doing it
now. Unfortunately this is a breaking change for all of the APIs.
I've added in a Ruby method for `#child` that is deprecated so that
existing usage doesn't break, but for everyone else this is going
to be a bit of a pain.
https://github.com/ruby/prism/commit/9cbe74464e
|
|
https://github.com/ruby/prism/commit/e4d6984892
|
|
This is similar to https://github.com/davidwessman/syntax_tree-erb/issues/81 but for RuboCop
The parser gem doesn't support these types of checks,
see https://github.com/whitequark/parser?tab=readme-ov-file#syntax-check-of-block-exits
While this is technically a bug in the parser gem, it does increase compatibility
and allows prism to be used when linting erb or haml with a RuboCop extension.
https://github.com/ruby/prism/commit/6c59ae6a00
|
|
https://github.com/ruby/prism/commit/445a0f0d22
|
|
https://github.com/ruby/prism/commit/3604aa15e7
|
|
Fixes https://github.com/ruby/prism/issues/2572
https://github.com/ruby/prism/commit/a446580e75
|
|
https://github.com/ruby/prism/commit/b0fa4b7cd8
|
|
https://github.com/ruby/prism/commit/2f3feb8d51
|
|
https://github.com/ruby/prism/commit/593d637178
|
|
Given this example:
```rb
<<~HEREDOC
#{x}
HEREDOC
```
Both the parser gem and Prism's translation layer would generate the following AST:
```
s(:dstr,
s(:begin,
s(:int, 1)),
s(:str, " a\n"))
```
However, the parser gem inserts a empty string node into this node's location, like:
```
<Parser::Source::Map::Heredoc:0x0000000104ce73b8
@expression=#<Parser::Source::Range (string) 0...10>,
@heredoc_body=#<Parser::Source::Range (string) 11...20>,
@heredoc_end=#<Parser::Source::Range (string) 20...27>,
@node=s(:dstr,
s(:str, ""),
s(:begin,
s(:int, 1)),
s(:str, " a\n"))>
```
This is required to calculate the correct whitespace for the heredoc body.
We need to adjust the translation layer to account for this.
With this fix, we also won't need to ignore the tilde heredoc fixture anymore.
https://github.com/ruby/prism/commit/e7372e3ba5
|
|
https://github.com/ruby/prism/commit/e2147cddd8
|
|
Fixes https://github.com/ruby/prism/pull/2617.
There was an issue with the lexer as follows.
The following are valid regexp options:
```console
$ bundle exec ruby -Ilib -rprism -ve 'p Prism.lex("/x/io").value.map {|token| token[0].type }'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[:REGEXP_BEGIN, :STRING_CONTENT, :REGEXP_END, :EOF]
```
The following are invalid regexp options. Unnecessary the `IDENTIFIER` token is appearing:
```console
$ bundle exec ruby -Ilib -rprism -ve 'p Prism.lex("/x/az").value.map {|token| token[0].type }'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[:REGEXP_BEGIN, :STRING_CONTENT, :REGEXP_END, :IDENTIFIER, :EOF]
```
As a behavior of Ruby, when given `A` to `Z` and `a` to `z`, they act as invalid regexp options. e.g.,
```console
$ ruby -e '/regexp/az'
-e:1: unknown regexp options - az
/regexp/az
-e: compile error (SyntaxError)
```
Thus, it should probably not be construed as `IDENTIFIER` token.
Therefore, `pm_byte_table` has been adapted to accept those invalid regexp option values.
Whether it is a valid regexp option or not is checked by `pm_regular_expression_flags_create`.
For invalid regexp options, `PM_ERR_REGEXP_UNKNOWN_OPTIONS` is added to diagnostics.
https://github.com/ruby/prism/commit/d2a6096fcf
|
|
for RubyParser translation
https://github.com/ruby/prism/commit/a37169621a
|
|
This PR fixes a diagnostic incompatibility when using no anonymous keyword rest parameter:
```ruby
foo(**)
```
Note, although the actual update applies only to the `foo(**)` case, for reference,
`foo(*)` and `foo(&) are also mentioned below.
## Ruby (Expected)
```console
$ ruby -cve 'foo(*)'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
-e:1: no anonymous rest parameter
-e: compile error (SyntaxError)
$ ruby -cve 'foo(**)'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
-e:1: no anonymous keyword rest parameter
-e: compile error (SyntaxError)
$ ruby -cve 'foo(&)'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
-e:1: no anonymous block parameter
-e: compile error (SyntaxError)
```
## Prism (Actual)
Before:
```console
$ bundle exec ruby -Ilib -rprism -wve 'p Prism.parse("foo(*)").errors'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[#<Prism::ParseError @type=:argument_no_forwarding_star @message="unexpected `*` when the parent method is not forwarding"
@location=#<Prism::Location @start_offset=4 @length=1 start_line=1> @level=:fatal>]
$ bundle exec ruby -Ilib -rprism -wve 'p Prism.parse("foo(**)").errors'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[#<Prism::ParseError @type=:expect_expression_after_splat_hash @message="expected an expression after `**` in a hash"
@location=#<Prism::Location @start_offset=4 @length=2 start_line=1> @level=:fatal>]
$ bundle exec ruby -Ilib -rprism -wve 'p Prism.parse("foo(&)").errors'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[#<Prism::ParseError @type=:argument_no_forwarding_amp @message="unexpected `&` when the parent method is not forwarding"
@location=#<Prism::Location @start_offset=4 @length=1 start_line=1> @level=:fatal>]
```
After:
```console
$ bundle exec ruby -Ilib -rprism -wve 'p Prism.parse("foo(*)").errors'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[#<Prism::ParseError @type=:argument_no_forwarding_star @message="unexpected `*` when the parent method is not forwarding"
@location=#<Prism::Location @start_offset=4 @length=1 start_line=1> @level=:fatal>]
$ bundle exec ruby -Ilib -rprism -wve 'p Prism.parse("foo(**)").errors'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[#<Prism::ParseError @type=:argument_no_forwarding_star_star @message="unexpected `**` when the parent method is not forwarding"
@location=#<Prism::Location @start_offset=4 @length=2 start_line=1> @level=:fatal>]
$ bundle exec ruby -Ilib -rprism -wve 'p Prism.parse("foo(&)").errors'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[#<Prism::ParseError @type=:argument_no_forwarding_amp @message="unexpected `&` when the parent method is not forwarding"
@location=#<Prism::Location @start_offset=4 @length=1 start_line=1> @level=:fatal>]
```
https://github.com/ruby/prism/commit/633c9d9fd4
|
|
This PR fixes a diagnostic incompatibility for `Prism::Translation::Parser` when using constant argument:
```ruby
def foo(A)
end
```
## Parser gem (Expected)
Displays `formal argument cannot be a constant (Parser::SyntaxError)`:
```console
$ bundle exec ruby -Ilib -rparser/ruby33 -ve 'p Parser::Ruby33.parse("def foo(A) end")'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
(string):1:9: error: formal argument cannot be a constant
(string):1: def foo(A) end
(string):1: ^
/Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/diagnostic/engine.rb:72:
in `process': formal argument cannot be a constant (Parser::SyntaxError)
from /Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/base.rb:274:in `diagnostic'
from /Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/ruby33.rb:12177:in `_reduce_663'
from /Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/racc-1.7.3/lib/racc/parser.rb:267:in `_racc_do_parse_c'
from /Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/racc-1.7.3/lib/racc/parser.rb:267:in `do_parse'
from /Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/base.rb:190:in `parse'
from /Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/base.rb:33:in `parse'
from -e:1:in `<main>'
```
## `Prism::Translation::Parser` (Actual)
Previously, the error messages displayed by the Parser gem were different.
Before:
```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve 'Prism::Translation::Parser33.parse("def foo(A) end")'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
(string):1:9: error:
(string):1: def foo(A) end
(string):1: ^
/Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/diagnostic/engine.rb:72:
in `process': Parser::SyntaxError (Parser::SyntaxError)
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:218:in `block in unwrap'
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:216:in `each'
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:216:in `unwrap'
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:49:in `parse'
from /Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/base.rb:33:in `parse'
from -e:1:in `<main>'
```
After:
```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve 'Prism::Translation::Parser33.parse("def foo(A) end")'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
(string):1:9: error: formal argument cannot be a constant
(string):1: def foo(A) end
(string):1: ^
/Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/diagnostic/engine.rb:72:
in `process': formal argument cannot be a constant (Parser::SyntaxError)
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:218:in `block in unwrap'
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:216:in `each'
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:216:in `unwrap'
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:49:in `parse'
from /Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/base.rb:33:in `parse'
from -e:1:in `<main>'
```
https://github.com/ruby/prism/commit/4f2af88520
|
|
If there are many searches in the `$LOAD_PATH` in the user environment,
require will perform unnecessary searches that are not needed.
In contrast, `require_relative` is efficient because it uses a relative path.
https://github.com/ruby/prism/commit/438ccc67bd
|
|
This PR fixes token incompatibility for `Prism::Translation::Parser::Lexer` when using backquoted heredoc indetiner:
```ruby
<<-` FOO`
a
b
FOO
```
## Parser gem (Expected)
Returns `tXSTRING_BEG` as the first token:
```console
$ bundle exec ruby -Ilib -rparser/ruby33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = File.read("example.rb"); p Parser::Ruby33.new.tokenize(buf)'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[s(:xstr,
s(:str, "a\n"),
s(:str, "b\n")), [], [[:tXSTRING_BEG, ["<<`", #<Parser::Source::Range example.rb 0...10>]],
[:tSTRING_CONTENT, ["a\n", #<Parser::Source::Range example.rb 11...13>]],
[:tSTRING_CONTENT, ["b\n", #<Parser::Source::Range example.rb 13...15>]],
[:tSTRING_END, [" FOO", #<Parser::Source::Range example.rb 15...23>]], [:tNL, [nil, #<Parser::Source::Range example.rb 10...11>]]]]
```
## `Prism::Translation::Parser` (Actual)
Previously, the tokens returned by the Parser gem were different. The escaped backslash does not match in the `tSTRING_BEG` token and
value of `tSTRING_END` token.
```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = File.read("example.rb"); p Prism::Translation::Parser33.new.tokenize(buf)'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[s(:xstr,
s(:str, "a\n"),
s(:str, "b\n")), [], [[:tSTRING_BEG, ["<<\"", #<Parser::Source::Range example.rb 0...10>]],
[:tSTRING_CONTENT, ["a\n", #<Parser::Source::Range example.rb 11...13>]],
[:tSTRING_CONTENT, ["b\n", #<Parser::Source::Range example.rb 13...15>]],
[:tSTRING_END, ["` FOO`", #<Parser::Source::Range example.rb 15...23>]], [:tNL, [nil, #<Parser::Source::Range example.rb 10...11>]]]]
```
After this correction, the AST and tokens returned by the Parser gem are the same:
```console
$ bunlde exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = File.read("example.rb"); p Prism::Translation::Parser33.new.tokenize(buf)'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[s(:xstr,
s(:str, "a\n"),
s(:str, "b\n")), [], [[:tXSTRING_BEG, ["<<`", #<Parser::Source::Range example.rb 0...10>]],
[:tSTRING_CONTENT, ["a\n", #<Parser::Source::Range example.rb 11...13>]],
[:tSTRING_CONTENT, ["b\n", #<Parser::Source::Range example.rb 13...15>]],
[:tSTRING_END, [" FOO", #<Parser::Source::Range example.rb 15...23>]], [:tNL, [nil, #<Parser::Source::Range example.rb 10...11>]]]]
```
https://github.com/ruby/prism/commit/308f8d85a1
|
|
This PR fixes token incompatibility for `Prism::Translation::Parser::Lexer`
when using escaped backslash in string literal:
```ruby
"\\ foo \\ bar"
```
## Parser gem (Expected)
```console
$ bundle exec ruby -Ilib -rparser/ruby33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = File.read("example.rb"); p Parser::Ruby33.new.tokenize(buf)'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[s(:str, "\\ foo \\ bar"), [], [[:tSTRING, ["\\ foo \\ bar", #<Parser::Source::Range example.rb 0...15>]],
[:tNL, [nil, #<Parser::Source::Range example.rb 15...16>]]]]
```
## `Prism::Translation::Parser` (Actual)
Previously, the tokens returned by the Parser gem were different. The escaped backslash does not match in the `tSTRING` token:
```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = File.read("example.rb"); p Prism::Translation::Parser33.new.tokenize(buf)'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[s(:str, "\\ foo \\ bar"), [], [[:tSTRING, ["\\\\ foo \\\\ bar", #<Parser::Source::Range example.rb 0...15>]],
[:tNL, [nil, #<Parser::Source::Range example.rb 15...16>]]]]
```
After this correction, the AST and tokens returned by the Parser gem are the same:
```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = File.read("example.rb"); p Prism::Translation::Parser33.new.tokenize(buf)'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[s(:str, "\\ foo \\ bar"), [], [[:tSTRING, ["\\ foo \\ bar", #<Parser::Source::Range example.rb 0...15>]],
[:tNL, [nil, #<Parser::Source::Range example.rb 15...16>]]]]
```
The reproduction test is based on the following strings.txt and exists:
https://github.com/ruby/prism/blob/v0.24.0/test/prism/fixtures/strings.txt#L79
But, the restoration has not yet been performed due to remaining other issues in strings.txt.
https://github.com/ruby/prism/commit/2c44e7e307
|
|
This PR fixes a token incompatibility between Parser gem and `Prism::Translation::Parser`
for the heredocs_leading_whitespace.txt test.
https://github.com/ruby/prism/commit/7d45fb1eed
|
|
`Prism::Translation::Parser`
This PR fixes an AST and token incompatibility between Parser gem and `Prism::Translation::Parser`
for dstring literal:
```ruby
"foo
#{bar}"
```
## Parser gem (Expected)
```console
$ bundle exec ruby -Ilib -rparser/ruby33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = File.read("example.rb"); p Parser::Ruby33.new.tokenize(buf)'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[s(:dstr,
s(:str, "foo\n"),
s(:str, " "),
s(:begin,
s(:send, nil, :bar))), [], [[:tSTRING_BEG, ["\"", #<Parser::Source::Range example.rb 0...1>]],
[:tSTRING_CONTENT, ["foo\n", #<Parser::Source::Range example.rb 1...5>]],
[:tSTRING_CONTENT, [" ", #<Parser::Source::Range example.rb 5...7>]],
[:tSTRING_DBEG, ["\#{", #<Parser::Source::Range example.rb 7...9>]],
[:tIDENTIFIER, ["bar", #<Parser::Source::Range example.rb 9...12>]],
[:tSTRING_DEND, ["}", #<Parser::Source::Range example.rb 12...13>]],
[:tSTRING_END, ["\"", #<Parser::Source::Range example.rb 13...14>]], [:tNL, [nil, #<Parser::Source::Range example.rb 14...15>]]]]
```
## `Prism::Translation::Parser` (Actual)
Previously, the AST and tokens returned by the Parser gem were different. In this case, `dstr` node should not be nested:
```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = File.read("example.rb"); p Prism::Translation::Parser33.new.tokenize(buf)'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[s(:dstr,
s(:dstr,
s(:str, "foo\n"),
s(:str, " ")),
s(:begin,
s(:send, nil, :bar))), [], [[:tSTRING_BEG, ["\"", #<Parser::Source::Range example.rb 0...1>]],
[:tSTRING_CONTENT, ["foo\n", #<Parser::Source::Range example.rb 1...5>]],
[:tSTRING_CONTENT, [" ", #<Parser::Source::Range example.rb 5...7>]],
[:tSTRING_DBEG, ["\#{", #<Parser::Source::Range example.rb 7...9>]],
[:tIDENTIFIER, ["bar", #<Parser::Source::Range example.rb 9...12>]],
[:tSTRING_DEND, ["}", #<Parser::Source::Range example.rb 12...13>]],
[:tSTRING_END, ["\"", #<Parser::Source::Range example.rb 13...14>]], [:tNL, [nil, #<Parser::Source::Range example.rb 14...15>]]]]
```
After this correction, the AST and tokens returned by the Parser gem are the same:
```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = File.read("example.rb"); p Prism::Translation::Parser33.new.tokenize(buf)'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[s(:dstr,
s(:str, "foo\n"),
s(:str, " "),
s(:begin,
s(:send, nil, :bar))), [], [[:tSTRING_BEG, ["\"", #<Parser::Source::Range example.rb 0...1>]],
[:tSTRING_CONTENT, ["foo\n", #<Parser::Source::Range example.rb 1...5>]],
[:tSTRING_CONTENT, [" ", #<Parser::Source::Range example.rb 5...7>]],
[:tSTRING_DBEG, ["\#{", #<Parser::Source::Range example.rb 7...9>]],
[:tIDENTIFIER, ["bar", #<Parser::Source::Range example.rb 9...12>]],
[:tSTRING_DEND, ["}", #<Parser::Source::Range example.rb 12...13>]],
[:tSTRING_END, ["\"", #<Parser::Source::Range example.rb 13...14>]], [:tNL, [nil, #<Parser::Source::Range example.rb 14...15>]]]]
```
https://github.com/ruby/prism/commit/c1652a9ee7
|
|
https://github.com/ruby/prism/commit/473cfed6d0
|
