diff options
| author | Koichi ITO <koic.ito@gmail.com> | 2024-03-12 23:02:27 +0900 |
|---|---|---|
| committer | git <svn-admin@ruby-lang.org> | 2024-03-12 14:15:34 +0000 |
| commit | 72a57076e272e1c36304eb5fe1a54091a2136cd7 (patch) | |
| tree | 7a87ec47b99de241019125929aecf63003f332b3 /lib/prism/translation/parser | |
| parent | 1e7ee871cbf10375ca149a32d71a29e5e60eed6c (diff) | |
[ruby/prism] Fix a token incompatibility for `Prism::Translation::Parser::Lexer`
This PR fixes a token incompatibility between Parser gem and `Prism::Translation::Parser` for
beginless range:
## Parser gem (Expected)
Returns `tBDOT2` token:
```console
$ bundle exec ruby -Ilib -rparser/ruby33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "..42"; p Parser::Ruby33.new.tokenize(buf)[2]'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[[:tBDOT2, ["..", #<Parser::Source::Range example.rb 0...2>]], [:tINTEGER, [42, #<Parser::Source::Range example.rb 2...4>]]]
```
## `Prism::Translation::Parser` (Actual)
Previously, the parser returned `tDOT2` token when parsing the following:
```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "..42"; p Prism::Translation::Parser33.new.tokenize(buf)[2]'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[[:tDOT2, ["..", #<Parser::Source::Range example.rb 0...2>]], [:tINTEGER, [42, #<Parser::Source::Range example.rb 2...4>]]]
```
After the update, the parser now returns `tBDOT2` token for the same input:
```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "..42"; p Prism::Translation::Parser33.new.tokenize(buf)[2]'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[[:tBDOT2, ["..", #<Parser::Source::Range example.rb 0...2>]], [:tINTEGER, [42, #<Parser::Source::Range example.rb 2...4>]]]
```
This correction enables the restoration of `endless_range_in_conditional.txt` as a test case.
https://github.com/ruby/prism/commit/f624b99ab0
Diffstat (limited to 'lib/prism/translation/parser')
| -rw-r--r-- | lib/prism/translation/parser/lexer.rb | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/lib/prism/translation/parser/lexer.rb b/lib/prism/translation/parser/lexer.rb index 8bb3084577..590adef323 100644 --- a/lib/prism/translation/parser/lexer.rb +++ b/lib/prism/translation/parser/lexer.rb @@ -167,7 +167,7 @@ module Prism TILDE: :tTILDE, UAMPERSAND: :tAMPER, UCOLON_COLON: :tCOLON3, - UDOT_DOT: :tDOT2, + UDOT_DOT: :tBDOT2, UDOT_DOT_DOT: :tBDOT3, UMINUS: :tUMINUS, UMINUS_NUM: :tUNARY_NUM, |