| Age | Commit message (Collapse) | Author |
|---|
|
[Backport #21187]
|
|
|
|
Followup to https://github.com/ruby/prism/pull/3079
https://github.com/ruby/prism/commit/68f434e356
|
|
https://github.com/ruby/prism/commit/f7faedfb3f
|
|
https://github.com/ruby/prism/commit/d4eb13e703
|
|
Resolves https://github.com/ruby/prism/pull/2803.
This PR adds error handling for missing `parser` gem in `Prism::Translation`.
The `parser` gem is a required runtime dependency when using `Prism::Translation::Parser`.
But it is not required for other uses of Prism. To avoid unnecessary dependencies,
it is not added as a `runtime_dependency` in the prism.gemspec. Instead, if the dependency is missing,
instructions are given to add it to Gemfile.
## Before
```console
$ bundle exec ruby -e 'require "prism"; require "prism/translation/parser33"'
/Users/koic/.rbenv/versions/3.3.1/lib/ruby/3.3.0/bundled_gems.rb:74:in `require': cannot load such file -- parser (LoadError)
from /Users/koic/.rbenv/versions/3.3.1/lib/ruby/3.3.0/bundled_gems.rb:74:in `block (2 levels) in replace_require'
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:3:in `<top (required)>'
from /Users/koic/.rbenv/versions/3.3.1/lib/ruby/3.3.0/bundled_gems.rb:74:in `require'
from /Users/koic/.rbenv/versions/3.3.1/lib/ruby/3.3.0/bundled_gems.rb:74:in `block (2 levels) in replace_require'
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser33.rb:6:in `<module:Translation>'
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser33.rb:4:in `<module:Prism>'
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser33.rb:3:in `<top (required)>'
from /Users/koic/.rbenv/versions/3.3.1/lib/ruby/3.3.0/bundled_gems.rb:74:in `require'
from /Users/koic/.rbenv/versions/3.3.1/lib/ruby/3.3.0/bundled_gems.rb:74:in `block (2 levels) in replace_require'
from -e:1:in `<main>'
```
## After
```console
$ bundle exec ruby -e 'require "prism"; require "prism/translation/parser33"'
Error: Unable to load parser. Add `gem "parser"` to your Gemfile.
```
https://github.com/ruby/prism/commit/4880aec22d
|
|
Follow up https://github.com/ruby/prism/pull/2760.
This PR updates the `Translation::Parser` to use version 3.3.1 when the version 3.3 is specified.
The Parser gem is structured to support the latest patch versions, hence this aligns with Parser-compatible versioning.
As noted in https://github.com/ruby/prism/pull/2760, the behavior remains unchanged with this switch from 3.3.0 to 3.3.1.
https://github.com/ruby/prism/commit/efde09d318
|
|
This is similar to https://github.com/davidwessman/syntax_tree-erb/issues/81 but for RuboCop
The parser gem doesn't support these types of checks,
see https://github.com/whitequark/parser?tab=readme-ov-file#syntax-check-of-block-exits
While this is technically a bug in the parser gem, it does increase compatibility
and allows prism to be used when linting erb or haml with a RuboCop extension.
https://github.com/ruby/prism/commit/6c59ae6a00
|
|
https://github.com/ruby/prism/commit/445a0f0d22
|
|
Fixes https://github.com/ruby/prism/pull/2617.
There was an issue with the lexer as follows.
The following are valid regexp options:
```console
$ bundle exec ruby -Ilib -rprism -ve 'p Prism.lex("/x/io").value.map {|token| token[0].type }'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[:REGEXP_BEGIN, :STRING_CONTENT, :REGEXP_END, :EOF]
```
The following are invalid regexp options. Unnecessary the `IDENTIFIER` token is appearing:
```console
$ bundle exec ruby -Ilib -rprism -ve 'p Prism.lex("/x/az").value.map {|token| token[0].type }'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[:REGEXP_BEGIN, :STRING_CONTENT, :REGEXP_END, :IDENTIFIER, :EOF]
```
As a behavior of Ruby, when given `A` to `Z` and `a` to `z`, they act as invalid regexp options. e.g.,
```console
$ ruby -e '/regexp/az'
-e:1: unknown regexp options - az
/regexp/az
-e: compile error (SyntaxError)
```
Thus, it should probably not be construed as `IDENTIFIER` token.
Therefore, `pm_byte_table` has been adapted to accept those invalid regexp option values.
Whether it is a valid regexp option or not is checked by `pm_regular_expression_flags_create`.
For invalid regexp options, `PM_ERR_REGEXP_UNKNOWN_OPTIONS` is added to diagnostics.
https://github.com/ruby/prism/commit/d2a6096fcf
|
|
This PR fixes a diagnostic incompatibility when using no anonymous keyword rest parameter:
```ruby
foo(**)
```
Note, although the actual update applies only to the `foo(**)` case, for reference,
`foo(*)` and `foo(&) are also mentioned below.
## Ruby (Expected)
```console
$ ruby -cve 'foo(*)'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
-e:1: no anonymous rest parameter
-e: compile error (SyntaxError)
$ ruby -cve 'foo(**)'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
-e:1: no anonymous keyword rest parameter
-e: compile error (SyntaxError)
$ ruby -cve 'foo(&)'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
-e:1: no anonymous block parameter
-e: compile error (SyntaxError)
```
## Prism (Actual)
Before:
```console
$ bundle exec ruby -Ilib -rprism -wve 'p Prism.parse("foo(*)").errors'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[#<Prism::ParseError @type=:argument_no_forwarding_star @message="unexpected `*` when the parent method is not forwarding"
@location=#<Prism::Location @start_offset=4 @length=1 start_line=1> @level=:fatal>]
$ bundle exec ruby -Ilib -rprism -wve 'p Prism.parse("foo(**)").errors'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[#<Prism::ParseError @type=:expect_expression_after_splat_hash @message="expected an expression after `**` in a hash"
@location=#<Prism::Location @start_offset=4 @length=2 start_line=1> @level=:fatal>]
$ bundle exec ruby -Ilib -rprism -wve 'p Prism.parse("foo(&)").errors'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[#<Prism::ParseError @type=:argument_no_forwarding_amp @message="unexpected `&` when the parent method is not forwarding"
@location=#<Prism::Location @start_offset=4 @length=1 start_line=1> @level=:fatal>]
```
After:
```console
$ bundle exec ruby -Ilib -rprism -wve 'p Prism.parse("foo(*)").errors'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[#<Prism::ParseError @type=:argument_no_forwarding_star @message="unexpected `*` when the parent method is not forwarding"
@location=#<Prism::Location @start_offset=4 @length=1 start_line=1> @level=:fatal>]
$ bundle exec ruby -Ilib -rprism -wve 'p Prism.parse("foo(**)").errors'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[#<Prism::ParseError @type=:argument_no_forwarding_star_star @message="unexpected `**` when the parent method is not forwarding"
@location=#<Prism::Location @start_offset=4 @length=2 start_line=1> @level=:fatal>]
$ bundle exec ruby -Ilib -rprism -wve 'p Prism.parse("foo(&)").errors'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[#<Prism::ParseError @type=:argument_no_forwarding_amp @message="unexpected `&` when the parent method is not forwarding"
@location=#<Prism::Location @start_offset=4 @length=1 start_line=1> @level=:fatal>]
```
https://github.com/ruby/prism/commit/633c9d9fd4
|
|
This PR fixes a diagnostic incompatibility for `Prism::Translation::Parser` when using constant argument:
```ruby
def foo(A)
end
```
## Parser gem (Expected)
Displays `formal argument cannot be a constant (Parser::SyntaxError)`:
```console
$ bundle exec ruby -Ilib -rparser/ruby33 -ve 'p Parser::Ruby33.parse("def foo(A) end")'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
(string):1:9: error: formal argument cannot be a constant
(string):1: def foo(A) end
(string):1: ^
/Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/diagnostic/engine.rb:72:
in `process': formal argument cannot be a constant (Parser::SyntaxError)
from /Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/base.rb:274:in `diagnostic'
from /Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/ruby33.rb:12177:in `_reduce_663'
from /Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/racc-1.7.3/lib/racc/parser.rb:267:in `_racc_do_parse_c'
from /Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/racc-1.7.3/lib/racc/parser.rb:267:in `do_parse'
from /Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/base.rb:190:in `parse'
from /Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/base.rb:33:in `parse'
from -e:1:in `<main>'
```
## `Prism::Translation::Parser` (Actual)
Previously, the error messages displayed by the Parser gem were different.
Before:
```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve 'Prism::Translation::Parser33.parse("def foo(A) end")'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
(string):1:9: error:
(string):1: def foo(A) end
(string):1: ^
/Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/diagnostic/engine.rb:72:
in `process': Parser::SyntaxError (Parser::SyntaxError)
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:218:in `block in unwrap'
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:216:in `each'
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:216:in `unwrap'
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:49:in `parse'
from /Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/base.rb:33:in `parse'
from -e:1:in `<main>'
```
After:
```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve 'Prism::Translation::Parser33.parse("def foo(A) end")'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
(string):1:9: error: formal argument cannot be a constant
(string):1: def foo(A) end
(string):1: ^
/Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/diagnostic/engine.rb:72:
in `process': formal argument cannot be a constant (Parser::SyntaxError)
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:218:in `block in unwrap'
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:216:in `each'
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:216:in `unwrap'
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:49:in `parse'
from /Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/base.rb:33:in `parse'
from -e:1:in `<main>'
```
https://github.com/ruby/prism/commit/4f2af88520
|
|
In the case of the `**` and `&` ambiguous prefixes, incompatibilities
remained among https://github.com/ruby/prism/issues/2513.
https://github.com/ruby/prism/commit/3b8b231aae
|
|
https://github.com/ruby/prism/commit/1a8a0063dc
|
|
https://github.com/ruby/prism/commit/c6c771d1fa
|
|
Parser::Diagnostic
https://github.com/ruby/prism/commit/6a84a3c9fb
|
|
https://github.com/ruby/prism/commit/c3cc282343
Co-authored-by: Kevin Newton <kddnewton@gmail.com>
|
|
https://github.com/ruby/prism/commit/ea7e400f85
|
|
## Summary
Fixes https://github.com/ruby/prism/pull/2356.
I'm working on integrating Prism into RuboCop.
This PR introduces `Prism::Translation::Parser33` and `Prism::Translation::Parser34`, named
in accordance with the following comments.
https://github.com/rubocop/rubocop/issues/12600#issuecomment-1932707748
Currently, `Prism::Translation::Parser` always operates in Ruby 3.4 mode.
This means it will not parse as Ruby 3.3 even if `TargetRubyVersion: 80_82_73_83_77.33` is specified.
Therefore, the `it` introduced in Ruby 3.4 is parsed incompatibly with Ruby 3.3. In Ruby 3.3,
the expected name for an `lvar` is `:it`, not `:"0it"`.
### Expected AST
The following is an expected AST when parsing Ruby 3.3 code:
```console
$ bundle exec ruby -rprism -rprism/translation/parser33 -ve "p Prism::Translation::Parser33.parse('items.map { it.do_something }')"
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
s(:block,
s(:send,
s(:send, nil, :items), :map),
s(:args),
s(:send,
s(:send, nil, :it), :do_something))
```
### Actual AST
The following is an actual AST when parsing Ruby 3.3 code:
```console
$ ruby -rprism -ve "p Prism::Translation::Parser.parse('items.map { it.do_something }')"
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
s(:block,
s(:send,
s(:send, nil, :items), :map),
s(:args),
s(:send,
s(:lvar, :"0it"), :do_something))
```
`Prism::Translation::Parser33` and `Prism::Translation::Parser34` aim to correspond to Ruby 3.3 and Ruby 3.4, respectively.
And, The hack of specifying `TargetRubyVersion: 80_82_73_83_77.33` is expected to become unnecessary in the future,
but the behavior will be maintained until RuboCop's support is finalized:
https://github.com/rubocop/rubocop/issues/12600#issuecomment-1933657732
## Additional Information
A private method named `convert_for_prism` is prepared to convert the `version` from Parser to the `version` expected by Prism.
For example, a Parser-compatible value is `3.3`, whereas Prism expects `"3.3.0"`.
`Parser#version` is not used in RuboCop, but it's unclear how it is utilized in other libraries that rely on the Parser gem.
Therefore, logic to maintain compatibility between Parser and Prism is implemented.
https://github.com/ruby/prism/commit/62d3991e22
|
|
https://github.com/ruby/prism/commit/8cd92eef79
|
|
translation
https://github.com/ruby/prism/commit/63979de21d
|
|
https://github.com/ruby/prism/commit/4327051c86
|
|
https://github.com/ruby/prism/commit/102b4a16f5
|
|
https://github.com/ruby/prism/commit/8cdec8070c
|
