| Age | Commit message (Collapse) | Author |
|
[Backport #21187]
|
|
|
|
|
|
|
|
The purpose of this commit is to fix Bug #21188. We need to detect when
stdin has run in to an EOF case. Unfortunately we can't _call_ the eof
function on IO because it will block.
Here is a short script to demonstrate the issue:
```ruby
x = STDIN.gets
puts x
puts x.eof?
```
If you run the script, then type some characters (but _NOT_ a newline),
then hit Ctrl-D twice, it will print the input string. Unfortunately,
calling `eof?` will try to read from STDIN again causing us to need a
3rd Ctrl-D to exit the program.
Before introducing the EOF callback to Prism, the input loop looked
kind of like this:
```ruby
loop do
str = STDIN.gets
process(str)
if str.nil?
p :DONE
end
end
```
Which required 3 Ctrl-D to exit. If we naively changed it to something
like this:
```ruby
loop do
str = STDIN.gets
process(str)
if STDIN.eof?
p :DONE
end
end
```
It would still require 3 Ctrl-D because `eof?` would block. In this
patch, we're wrapping the IO object, checking the buffer for a newline
and length, and then using that to simulate a non-blocking eof? method.
This commit wraps STDIN and emulates a non-blocking `eof` function.
[Backport #21188]
|
|
(#13327)
* [ruby/prism] Accept a newline after the defined? keyword
[Bug #21197]
https://github.com/ruby/prism/commit/22be955ce9
* Fix a compilation error
Co-authored-by: Stan Lo <stan001212@gmail.com>
---------
Co-authored-by: Kevin Newton <kddnewton@gmail.com>
Co-authored-by: Stan Lo <stan001212@gmail.com>
|
|
[Backport #21197]"
This reverts commit e630a0f7ae909dafe2e1dbc47baf90cd795b4a5f.
|
|
[ruby/prism] Accept a newline after the defined? keyword
[Bug #21197]
https://github.com/ruby/prism/commit/22be955ce9
|
|
[ruby/prism] No writing to numbered parameters
Fixes [Bug #21117]
https://github.com/ruby/prism/commit/19d4bab5a0
|
|
[ruby/prism] Fix `not` receiver
`not foo` should be `!foo`
`not()` should be `!nil`
Fixes [Bug #21027]
https://github.com/ruby/prism/commit/871ed4b462
|
|
https://github.com/ruby/prism/commit/b283a72c88
Notes:
Merged: https://github.com/ruby/ruby/pull/12358
|
|
https://github.com/ruby/prism/commit/34efacc618
Notes:
Merged: https://github.com/ruby/ruby/pull/12358
|
|
https://github.com/ruby/prism/commit/9686897290
Notes:
Merged: https://github.com/ruby/ruby/pull/12358
|
|
https://github.com/ruby/prism/commit/817a8e39d9
Notes:
Merged: https://github.com/ruby/ruby/pull/12358
|
|
https://github.com/ruby/prism/commit/f80026883d
Notes:
Merged: https://github.com/ruby/ruby/pull/12358
|
|
https://github.com/ruby/prism/commit/230c8b8a48
|
|
default gem
* This is notably necessary on TruffleRuby, which is updating to Ruby 3.3 which introduces Prism as a default gem.
* Using the existing path is not an option as it would end up in truffleruby/lib/build/libprism.so and
"truffleruby/lib/include/#{header}" which are not good places for such files.
https://github.com/ruby/prism/commit/5d16473e69
|
|
https://github.com/ruby/prism/commit/5ea6042408
|
|
Introduce StringQuery to provide methods to access some metadata
about the Ruby lexer.
https://github.com/ruby/prism/commit/d3f55b67b9
|
|
Calculating code unit offsets for a source can be very expensive,
especially when the source is large. This commit introduces a new
class that wraps the source and desired encoding into a cache that
reuses pre-computed offsets. It performs quite a bit better.
There are still some problems with this approach, namely character
boundaries and the fact that the cache is unbounded, but both of
these may be addressed in subsequent commits.
https://github.com/ruby/prism/commit/2e3e1a4d4d
|
|
https://github.com/ruby/prism/commit/343197e4ff
|
|
https://github.com/ruby/prism/commit/25a4cf6794
Co-authored-by: Kevin Newton <kddnewton@users.noreply.github.com>
|
|
Followup to https://github.com/ruby/prism/pull/3079
https://github.com/ruby/prism/commit/68f434e356
|
|
continuation
https://github.com/ruby/prism/commit/84a9251915
|
|
https://github.com/ruby/prism/commit/098f1c4607
|
|
https://github.com/ruby/prism/commit/a4fcd5339a
|
|
`Prism::Translation::Parser::Lexer`
## Summary
This PR fixes `kDO_LAMBDA` token incompatibility between Parser gem and `Prism::Translation::Parser` for lambda `do` block.
### Parser gem (Expected)
Returns `kDO_LAMBDA` token:
```console
$ bundle exec ruby -Ilib -rparser/ruby33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "-> do end"; p Parser::Ruby33.new.tokenize(buf)[2]'
ruby 3.4.0dev (2024-09-01T11:00:13Z master https://github.com/ruby/prism/commit/eb144ef91e) [x86_64-darwin23]
[[:tLAMBDA, ["->", #<Parser::Source::Range example.rb 0...2>]], [:kDO_LAMBDA, ["do", #<Parser::Source::Range example.rb 3...5>]],
[:kEND, ["end", #<Parser::Source::Range example.rb 6...9>]]]
```
### `Prism::Translation::Parser` (Actual)
Previously, the parser returned `kDO` token when parsing the following:
```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "-> do end"; p Prism::Translation::Parser33.new.tokenize(buf)[2]'
ruby 3.4.0dev (2024-09-01T11:00:13Z master https://github.com/ruby/prism/commit/eb144ef91e) [x86_64-darwin23]
[[:tLAMBDA, ["->", #<Parser::Source::Range example.rb 0...2>]], [:kDO, ["do", #<Parser::Source::Range example.rb 3...5>]],
[:kEND, ["end", #<Parser::Source::Range example.rb 6...9>]]]
```
After the update, the parser now returns `kDO_LAMBDA` token for the same input:
```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "-> do end"; p Prism::Translation::Parser33.new.tokenize(buf)[2]'
ruby 3.4.0dev (2024-09-01T11:00:13Z master https://github.com/ruby/prism/commit/eb144ef91e) [x86_64-darwin23]
[[:tLAMBDA, ["->", #<Parser::Source::Range example.rb 0...2>]], [:kDO_LAMBDA, ["do", #<Parser::Source::Range example.rb 3...5>]],
[:kEND, ["end", #<Parser::Source::Range example.rb 6...9>]]]
```
## Additional Information
Unfortunately, this kind of edge case doesn't work as expected; `kDO` is returned instead of `kDO_LAMBDA`.
However, since `kDO` is already being returned in this case, there is no change in behavior.
### Parser gem
Returns `tLAMBDA` token:
```console
$ bundle exec ruby -Ilib -rparser/ruby33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "-> (foo = -> (bar) {}) do end"; p Parser::Ruby33.new.tokenize(buf)[2]'
ruby 3.3.5 (2024-09-03 revision https://github.com/ruby/prism/commit/ef084cc8f4) [x86_64-darwin23]
[[:tLAMBDA, ["->", #<Parser::Source::Range example.rb 0...2>]], [:tLPAREN2, ["(", #<Parser::Source::Range example.rb 3...4>]],
[:tIDENTIFIER, ["foo", #<Parser::Source::Range example.rb 4...7>]], [:tEQL, ["=", #<Parser::Source::Range example.rb 8...9>]],
[:tLAMBDA, ["->", #<Parser::Source::Range example.rb 10...12>]], [:tLPAREN2, ["(", #<Parser::Source::Range example.rb 13...14>]],
[:tIDENTIFIER, ["bar", #<Parser::Source::Range example.rb 14...17>]], [:tRPAREN, [")", #<Parser::Source::Range example.rb 17...18>]],
[:tLAMBEG, ["{", #<Parser::Source::Range example.rb 19...20>]], [:tRCURLY, ["}", #<Parser::Source::Range example.rb 20...21>]],
[:tRPAREN, [")", #<Parser::Source::Range example.rb 21...22>]], [:kDO_LAMBDA, ["do", #<Parser::Source::Range example.rb 23...25>]],
[:kEND, ["end", #<Parser::Source::Range example.rb 26...29>]]]
```
### `Prism::Translation::Parser`
Returns `kDO` token:
```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "-> (foo = -> (bar) {}) do end"; p Prism::Translation::Parser33.new.tokenize(buf)[2]'
ruby 3.3.5 (2024-09-03 revision https://github.com/ruby/prism/commit/ef084cc8f4) [x86_64-darwin23]
[[:tLAMBDA, ["->", #<Parser::Source::Range example.rb 0...2>]], [:tLPAREN2, ["(", #<Parser::Source::Range example.rb 3...4>]],
[:tIDENTIFIER, ["foo", #<Parser::Source::Range example.rb 4...7>]], [:tEQL, ["=", #<Parser::Source::Range example.rb 8...9>]],
[:tLAMBDA, ["->", #<Parser::Source::Range example.rb 10...12>]], [:tLPAREN2, ["(", #<Parser::Source::Range example.rb 13...14>]],
[:tIDENTIFIER, ["bar", #<Parser::Source::Range example.rb 14...17>]], [:tRPAREN, [")", #<Parser::Source::Range example.rb 17...18>]],
[:tLAMBEG, ["{", #<Parser::Source::Range example.rb 19...20>]], [:tRCURLY, ["}", #<Parser::Source::Range example.rb 20...21>]],
[:tRPAREN, [")", #<Parser::Source::Range example.rb 21...22>]], [:kDO, ["do", #<Parser::Source::Range example.rb 23...25>]],
[:kEND, ["end", #<Parser::Source::Range example.rb 26...29>]]]
```
As the intention is not to address such special cases at this point, a comment has been left indicating that this case still returns `kDO`.
In other words, `kDO_LAMBDA` will now be returned except for edge cases after this PR.
https://github.com/ruby/prism/commit/2ee480654c
|
|
https://github.com/ruby/prism/commit/b28877fa4f
|
|
Fixes [Bug #20744]
https://github.com/ruby/prism/commit/f1b8b1b2a2
|
|
https://github.com/ruby/prism/commit/0b527ca93f
|
|
https://github.com/ruby/prism/commit/d68ea29d04
Notes:
Merged: https://github.com/ruby/ruby/pull/11497
|
|
This PR fixes a token incompatibility between Parser gem and `Prism::Translation::Parser` for double splat argument.
## Parser gem (Expected)
Returns `tDSTAR` token:
```console
$ bundle exec ruby -Ilib -rparser/ruby33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "def f(**foo) end"; p Parser::Ruby33.new.tokenize(buf)[2]'
ruby 3.4.0dev (2024-09-01T11:00:13Z master https://github.com/ruby/prism/commit/eb144ef91e) [x86_64-darwin23]
[[:kDEF, ["def", #<Parser::Source::Range example.rb 0...3>]], [:tIDENTIFIER, ["f", #<Parser::Source::Range example.rb 4...5>]],
[:tLPAREN2, ["(", #<Parser::Source::Range example.rb 5...6>]], [:tDSTAR, ["**", #<Parser::Source::Range example.rb 6...8>]],
[:tIDENTIFIER, ["foo", #<Parser::Source::Range example.rb 8...11>]], [:tRPAREN, [")", #<Parser::Source::Range example.rb 11...12>]],
[:kEND, ["end", #<Parser::Source::Range example.rb 13...16>]]]
```
## `Prism::Translation::Parser` (Actual)
Previously, the parser returned `tPOW` token when parsing the following:
```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "def f(**foo) end"; p Prism::Translation::Parser33.new.tokenize(buf)[2]'
ruby 3.4.0dev (2024-09-01T11:00:13Z master https://github.com/ruby/prism/commit/eb144ef91e) [x86_64-darwin23]
[[:kDEF, ["def", #<Parser::Source::Range example.rb 0...3>]], [:tIDENTIFIER, ["f", #<Parser::Source::Range example.rb 4...5>]],
[:tLPAREN2, ["(", #<Parser::Source::Range example.rb 5...6>]], [:tPOW, ["**", #<Parser::Source::Range example.rb 6...8>]],
[:tIDENTIFIER, ["foo", #<Parser::Source::Range example.rb 8...11>]], [:tRPAREN, [")", #<Parser::Source::Range example.rb 11...12>]],
[:kEND, ["end", #<Parser::Source::Range example.rb 13...16>]]]
```
After the update, the parser now returns `tDSTAR` token for the same input:
```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "def f(**foo) end"; p Prism::Translation::Parser33.new.tokenize(buf)[2]'
ruby 3.4.0dev (2024-09-01T11:00:13Z master https://github.com/ruby/prism/commit/eb144ef91e) [x86_64-darwin23]
[[:kDEF, ["def", #<Parser::Source::Range example.rb 0...3>]], [:tIDENTIFIER, ["f", #<Parser::Source::Range example.rb 4...5>]],
[:tLPAREN2, ["(", #<Parser::Source::Range example.rb 5...6>]], [:tDSTAR, ["**", #<Parser::Source::Range example.rb 6...8>]],
[:tIDENTIFIER, ["foo", #<Parser::Source::Range example.rb 8...11>]], [:tRPAREN, [")", #<Parser::Source::Range example.rb 11...12>]],
[:kEND, ["end", #<Parser::Source::Range example.rb 13...16>]]]
```
With this change, the following code could be removed from test/prism/ruby/parser_test.rb:
```diff
- when :tPOW
- actual_token[0] = expected_token[0] if expected_token[0] == :tDSTAR
```
`tPOW` is the token type for the behavior of `a ** b`, and its behavior remains unchanged:
```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "a ** b"; p Prism::Translation::Parser33.new.tokenize(buf)[2]'
ruby 3.4.0dev (2024-09-01T11:00:13Z master https://github.com/ruby/prism/commit/eb144ef91e) [x86_64-darwin23]
[[:tIDENTIFIER, ["a", #<Parser::Source::Range example.rb 0...1>]], [:tPOW, ["**", #<Parser::Source::Range example.rb 2...4>]],
[:tIDENTIFIER, ["b", #<Parser::Source::Range example.rb 5...6>]]]
```
https://github.com/ruby/prism/commit/66bde35a44
|
|
This PR fixes a token incompatibility between Parser gem and `Prism::Translation::Parser` for left parenthesis.
## Parser gem (Expected)
Returns `tLPAREN2` token:
```console
$ bundle exec ruby -Ilib -rparser/ruby33 \
-ve 'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "foo(:bar)"; p Parser::Ruby33.new.tokenize(buf)[2]'
ruby 3.4.0dev (2024-09-01T11:00:13Z master https://github.com/ruby/prism/commit/eb144ef91e) [x86_64-darwin23]
[[:tIDENTIFIER, ["foo", #<Parser::Source::Range example.rb 0...3>]], [:tLPAREN2, ["(", #<Parser::Source::Range example.rb 3...4>]],
[:tSYMBOL, ["bar", #<Parser::Source::Range example.rb 4...8>]], [:tRPAREN, [")", #<Parser::Source::Range example.rb 8...9>]]]
```
## `Prism::Translation::Parser` (Actual)
Previously, the parser returned `tLPAREN` token when parsing the following:
```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "foo(:bar)"; p Prism::Translation::Parser33.new.tokenize(buf)[2]'
ruby 3.4.0dev (2024-09-01T11:00:13Z master https://github.com/ruby/prism/commit/eb144ef91e) [x86_64-darwin23]
[[:tIDENTIFIER, ["foo", #<Parser::Source::Range example.rb 0...3>]], [:tLPAREN, ["(", #<Parser::Source::Range example.rb 3...4>]],
[:tSYMBOL, ["bar", #<Parser::Source::Range example.rb 4...8>]], [:tRPAREN, [")", #<Parser::Source::Range example.rb 8...9>]]]
```
After the update, the parser now returns `tLPAREN2` token for the same input:
```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "foo(:bar)"; p Prism::Translation::Parser33.new.tokenize(buf)[2]'
ruby 3.4.0dev (2024-09-01T11:00:13Z master https://github.com/ruby/prism/commit/eb144ef91e) [x86_64-darwin23]
[[:tIDENTIFIER, ["foo", #<Parser::Source::Range example.rb 0...3>]], [:tLPAREN2, ["(", #<Parser::Source::Range example.rb 3...4>]],
[:tSYMBOL, ["bar", #<Parser::Source::Range example.rb 4...8>]], [:tRPAREN, [")", #<Parser::Source::Range example.rb 8...9>]]]
```
The `PARENTHESIS_LEFT` token in Prism is classified as either `tLPAREN` or `tLPAREN2` in the Parser gem.
The tokens that were previously all classified as `tLPAREN` are now also classified to `tLPAREN2`.
With this change, the following code could be removed from `test/prism/ruby/parser_test.rb`:
```diff
- when :tLPAREN
- actual_token[0] = expected_token[0] if expected_token[0] == :tLPAREN2
```
https://github.com/ruby/prism/commit/04d6f3478d
|
|
https://github.com/ruby/prism/commit/9c68c01bcd
|
|
Follow up https://github.com/ruby/prism/pull/2558.
This PR removes deprecated lib/prism/translation/parser/rubocop.rb file.
The file was a workaround to allow setting `TargetRubyVersion: 80_82_73_83_77.xx` until Prism (`Prism::Translation::Parser`) is integrated into RuboCop.
RuboCop already supports Prism (`Prism::Translation::Parser`) as of https://github.com/rubocop/rubocop/pull/12724.
It has been several months since the file was deprecated in ruby/prism#2558.
And, yesterday, Prism 1.0.0 was released, but perhaps the file should have been removed before then.
Although this might be seen as incompatible with semver 1.0.0, I think there is no longer a reason to keep the file.
https://github.com/ruby/prism/commit/646a10270e
|
|
|
|
https://github.com/ruby/prism/commit/bc21c9f3ee
|
|
Rename some fields that do not quite make sense.
* CaseMatchNode#consequent -> CaseMatchNode#else_clause
* CaseNode#consequent -> CaseNode#else_clause
* IfNode#consequent -> IfNode#subsequent
* RescueNode#consequent -> RescueNode#subsequent
* UnlessNode#consequent -> UnlessNode#else_clause
Notes:
Merged: https://github.com/ruby/ruby/pull/11480
|
|
This PR tweaks inspect representation of `Prism::Location`.
## Before
During debugging, the meaning of `@location=https://github.com/ruby/prism/commit/21474836481` was unclear:
```console
$ ruby -Ilib -rprism -e 'p Prism.lex("puts :hi").value.map(&:first)[1]'
#<Prism::Token:0x000000010cd74e40 @source=#<Prism::ASCIISource:0x000000010cb5f808 @source="puts :hi", @start_line=1, @offsets=[0]>,
@type=:SYMBOL_BEGIN, @value=":", @location=https://github.com/ruby/prism/commit/21474836481>
```
## After
This PR clarifies the contents of the location object, aligning with what I think user expects:
```console
$ ruby -Ilib -rprism -e 'p Prism.lex("puts :hi").value.map(&:first)[1]'
#<Prism::Token:0x000000010e174d50 @source=#<Prism::ASCIISource:0x000000010df5efe8 @source="puts :hi", @start_line=1, @offsets=[0]>,
@type=:SYMBOL_BEGIN, @value=":", @location=#<Prism::Location @start_offset=5 @length=1 start_line=1>>
```
Although it is uncertain whether Prism will accept this change in the inspect representation, it is submitted here as a suggestion.
https://github.com/ruby/prism/commit/e7421ce1c5
|
|
https://github.com/ruby/prism/commit/afc7c9344a
|
|
https://github.com/ruby/prism/commit/47cb73ce69
|
|
https://github.com/ruby/prism/commit/073e8ba307
|
|
https://github.com/ruby/prism/commit/86cf82794a
|
|
https://github.com/ruby/prism/commit/bf16ade7f9
|
|
https://github.com/ruby/prism/commit/08a71f6259
|
|
https://github.com/ruby/prism/commit/9f12a56fd6
|
|
https://github.com/ruby/prism/commit/4cc0eda4ca
|
|
https://github.com/ruby/prism/commit/fb7e1ebb7f
|
|
https://github.com/ruby/prism/commit/5985ab7687
|
|
https://github.com/ruby/prism/commit/f7faedfb3f
|