diff options
| author | Koichi ITO <koic.ito@gmail.com> | 2024-03-13 02:05:57 +0900 |
|---|---|---|
| committer | git <svn-admin@ruby-lang.org> | 2024-03-12 17:33:14 +0000 |
| commit | f8cab4ef8e214dc05efe8a0d4178864bce3b790a (patch) | |
| tree | 27708629dd6957888f51fe91dc02f016afe51614 /lib/prism/translation/parser/lexer.rb | |
| parent | 83790e5fe1e254c315552a6927891ed697da590a (diff) | |
[ruby/prism] Fix a token incompatibility for `Prism::Translation::Parser::Lexer`
In practice, the `BACKTICK` is mapped either as `:tXSTRING_BEG` or `:tBACK_REF2`.
The former is used in xstrings like `` `foo` ``, while the latter is utilized as
a back reference in contexts like `` A::` ``.
This PR will make corrections to differentiate the use of `BACKTICK`.
This mistake was discovered through the investigation of xstring.txt file.
The PR will run tests from xstring.txt file except for `` `f\oo` ``, which will still fail,
hence it will be separated into xstring_with_backslash.txt file.
This separation will facilitate addressing the correction at a different time.
https://github.com/ruby/prism/commit/49ad8df40a
Diffstat (limited to 'lib/prism/translation/parser/lexer.rb')
| -rw-r--r-- | lib/prism/translation/parser/lexer.rb | 6 |
1 files changed, 5 insertions, 1 deletions
diff --git a/lib/prism/translation/parser/lexer.rb b/lib/prism/translation/parser/lexer.rb index 054e56a4ba..92495ab3d2 100644 --- a/lib/prism/translation/parser/lexer.rb +++ b/lib/prism/translation/parser/lexer.rb @@ -24,7 +24,7 @@ module Prism AMPERSAND_DOT: :tANDDOT, AMPERSAND_EQUAL: :tOP_ASGN, BACK_REFERENCE: :tBACK_REF, - BACKTICK: :tBACK_REF2, + BACKTICK: :tXSTRING_BEG, BANG: :tBANG, BANG_EQUAL: :tNEQ, BANG_TILDE: :tNMATCH, @@ -325,6 +325,10 @@ module Prism if !tokens.empty? && tokens.dig(-1, 0) == :kDEF type = :tIDENTIFIER end + when :tXSTRING_BEG + if (next_token = lexed[index][0]) && next_token.type != :STRING_CONTENT + type = :tBACK_REF2 + end end tokens << [type, [value, location]] |