<feed xmlns='http://www.w3.org/2005/Atom'>
<title>ruby.git/test/prism/fixtures/dstring.txt, branch v4.0.2</title>
<subtitle>The Ruby Programming Language</subtitle>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/'/>
<entry>
<title>[ruby/prism] Add a multiple statements flag to parentheses</title>
<updated>2025-03-18T17:36:53+00:00</updated>
<author>
<name>Kevin Newton</name>
<email>kddnewton@gmail.com</email>
</author>
<published>2025-03-13T17:59:45+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=dc48c1aca347f43ef9cb122342c80d699c5860fa'/>
<id>dc48c1aca347f43ef9cb122342c80d699c5860fa</id>
<content type='text'>
This can get triggered even if the list of statements only contains
a single statement. This is necessary to properly support compiling

```ruby
defined? (;a)
defined? (a;)
```

as "expression". Previously these were parsed as statements lists
with single statements in them.

https://github.com/ruby/prism/commit/b63b5d67a9
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This can get triggered even if the list of statements only contains
a single statement. This is necessary to properly support compiling

```ruby
defined? (;a)
defined? (a;)
```

as "expression". Previously these were parsed as statements lists
with single statements in them.

https://github.com/ruby/prism/commit/b63b5d67a9
</pre>
</div>
</content>
</entry>
<entry>
<title>[ruby/prism] Fix parser translator edge-case when multiline string ends with \n</title>
<updated>2025-03-18T17:36:53+00:00</updated>
<author>
<name>Earlopain</name>
<email>14981592+Earlopain@users.noreply.github.com</email>
</author>
<published>2025-01-20T14:49:06+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=ac728389e219480dd678a01e6f91f96098d3b6d6'/>
<id>ac728389e219480dd678a01e6f91f96098d3b6d6</id>
<content type='text'>
When the line contains no real newline but contains unescaped ones, then there will be one less entry

https://github.com/ruby/prism/commit/4ef093b600
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
When the line contains no real newline but contains unescaped ones, then there will be one less entry

https://github.com/ruby/prism/commit/4ef093b600
</pre>
</div>
</content>
</entry>
<entry>
<title>[ruby/prism] Better handle all kinds of multiline strings in the parser translator</title>
<updated>2025-03-18T17:36:53+00:00</updated>
<author>
<name>Earlopain</name>
<email>14981592+Earlopain@users.noreply.github.com</email>
</author>
<published>2025-01-08T20:17:21+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=0fcb7fc21d1923a61ffe4d127e6e72cc55560972'/>
<id>0fcb7fc21d1923a61ffe4d127e6e72cc55560972</id>
<content type='text'>
This is a followup to #3373, where the implementation
was extracted

https://github.com/ruby/prism/commit/2637007929
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This is a followup to #3373, where the implementation
was extracted

https://github.com/ruby/prism/commit/2637007929
</pre>
</div>
</content>
</entry>
<entry>
<title>[ruby/prism] Fix an incompatibility with the parser translator</title>
<updated>2025-03-18T17:36:53+00:00</updated>
<author>
<name>Earlopain</name>
<email>14981592+Earlopain@users.noreply.github.com</email>
</author>
<published>2024-12-21T21:00:56+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=acf404e20e3d7bc18cbb8abd2b16c62f9c1f996f'/>
<id>acf404e20e3d7bc18cbb8abd2b16c62f9c1f996f</id>
<content type='text'>
The offset cache contains an entry for each byte so it can't be accessed via the string length.

Adds tests for all variants except for this:
```
"fo
o" "ba
’"
```

For some reason, this still has the wrong offset.

https://github.com/ruby/prism/commit/a651126458
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The offset cache contains an entry for each byte so it can't be accessed via the string length.

Adds tests for all variants except for this:
```
"fo
o" "ba
’"
```

For some reason, this still has the wrong offset.

https://github.com/ruby/prism/commit/a651126458
</pre>
</div>
</content>
</entry>
<entry>
<title>[ruby/prism] Better handle all kinds of multiline strings in the parser translator</title>
<updated>2025-01-12T00:09:05+00:00</updated>
<author>
<name>Earlopain</name>
<email>14981592+Earlopain@users.noreply.github.com</email>
</author>
<published>2025-01-08T20:17:21+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=283037f7e3aa54e281c098c43cea830a52b1ec2d'/>
<id>283037f7e3aa54e281c098c43cea830a52b1ec2d</id>
<content type='text'>
This is a followup to #3373, where the implementation
was extracted

https://github.com/ruby/prism/commit/2637007929
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This is a followup to #3373, where the implementation
was extracted

https://github.com/ruby/prism/commit/2637007929
</pre>
</div>
</content>
</entry>
<entry>
<title>[ruby/prism] Fix an incompatibility with the parser translator</title>
<updated>2025-01-12T00:09:05+00:00</updated>
<author>
<name>Earlopain</name>
<email>14981592+Earlopain@users.noreply.github.com</email>
</author>
<published>2024-12-21T21:00:56+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=7cbaa3b9298b4ab5027d75a7317ca43a9e745c16'/>
<id>7cbaa3b9298b4ab5027d75a7317ca43a9e745c16</id>
<content type='text'>
The offset cache contains an entry for each byte so it can't be accessed via the string length.

Adds tests for all variants except for this:
```
"fo
o" "ba
’"
```

For some reason, this still has the wrong offset.

https://github.com/ruby/prism/commit/a651126458
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The offset cache contains an entry for each byte so it can't be accessed via the string length.

Adds tests for all variants except for this:
```
"fo
o" "ba
’"
```

For some reason, this still has the wrong offset.

https://github.com/ruby/prism/commit/a651126458
</pre>
</div>
</content>
</entry>
<entry>
<title>[ruby/prism] Fix an AST and token incompatibility for `Prism::Translation::Parser`</title>
<updated>2024-03-15T12:31:40+00:00</updated>
<author>
<name>Koichi ITO</name>
<email>koic.ito@gmail.com</email>
</author>
<published>2024-03-14T16:22:14+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=c0b8dee95a5412f395486a9bcb4959f93509cecb'/>
<id>c0b8dee95a5412f395486a9bcb4959f93509cecb</id>
<content type='text'>
This PR fixes an AST and token incompatibility between Parser gem and `Prism::Translation::Parser`
for dstring literal:

```ruby
"foo
  #{bar}"
```

## Parser gem (Expected)

```console
$ bundle exec ruby -Ilib -rparser/ruby33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = File.read("example.rb"); p Parser::Ruby33.new.tokenize(buf)'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[s(:dstr,
  s(:str, "foo\n"),
  s(:str, "  "),
  s(:begin,
    s(:send, nil, :bar))), [], [[:tSTRING_BEG, ["\"", #&lt;Parser::Source::Range example.rb 0...1&gt;]],
[:tSTRING_CONTENT, ["foo\n", #&lt;Parser::Source::Range example.rb 1...5&gt;]],
[:tSTRING_CONTENT, ["  ", #&lt;Parser::Source::Range example.rb 5...7&gt;]],
[:tSTRING_DBEG, ["\#{", #&lt;Parser::Source::Range example.rb 7...9&gt;]],
[:tIDENTIFIER, ["bar", #&lt;Parser::Source::Range example.rb 9...12&gt;]],
[:tSTRING_DEND, ["}", #&lt;Parser::Source::Range example.rb 12...13&gt;]],
[:tSTRING_END, ["\"", #&lt;Parser::Source::Range example.rb 13...14&gt;]], [:tNL, [nil, #&lt;Parser::Source::Range example.rb 14...15&gt;]]]]
```

## `Prism::Translation::Parser` (Actual)

Previously, the AST and tokens returned by the Parser gem were different. In this case, `dstr` node should not be nested:

```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = File.read("example.rb"); p Prism::Translation::Parser33.new.tokenize(buf)'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[s(:dstr,
  s(:dstr,
    s(:str, "foo\n"),
    s(:str, "  ")),
  s(:begin,
    s(:send, nil, :bar))), [], [[:tSTRING_BEG, ["\"", #&lt;Parser::Source::Range example.rb 0...1&gt;]],
[:tSTRING_CONTENT, ["foo\n", #&lt;Parser::Source::Range example.rb 1...5&gt;]],
[:tSTRING_CONTENT, ["  ", #&lt;Parser::Source::Range example.rb 5...7&gt;]],
[:tSTRING_DBEG, ["\#{", #&lt;Parser::Source::Range example.rb 7...9&gt;]],
[:tIDENTIFIER, ["bar", #&lt;Parser::Source::Range example.rb 9...12&gt;]],
[:tSTRING_DEND, ["}", #&lt;Parser::Source::Range example.rb 12...13&gt;]],
[:tSTRING_END, ["\"", #&lt;Parser::Source::Range example.rb 13...14&gt;]], [:tNL, [nil, #&lt;Parser::Source::Range example.rb 14...15&gt;]]]]
```

After this correction, the AST and tokens returned by the Parser gem are the same:

```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = File.read("example.rb"); p Prism::Translation::Parser33.new.tokenize(buf)'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[s(:dstr,
  s(:str, "foo\n"),
  s(:str, "  "),
  s(:begin,
    s(:send, nil, :bar))), [], [[:tSTRING_BEG, ["\"", #&lt;Parser::Source::Range example.rb 0...1&gt;]],
[:tSTRING_CONTENT, ["foo\n", #&lt;Parser::Source::Range example.rb 1...5&gt;]],
[:tSTRING_CONTENT, ["  ", #&lt;Parser::Source::Range example.rb 5...7&gt;]],
[:tSTRING_DBEG, ["\#{", #&lt;Parser::Source::Range example.rb 7...9&gt;]],
[:tIDENTIFIER, ["bar", #&lt;Parser::Source::Range example.rb 9...12&gt;]],
[:tSTRING_DEND, ["}", #&lt;Parser::Source::Range example.rb 12...13&gt;]],
[:tSTRING_END, ["\"", #&lt;Parser::Source::Range example.rb 13...14&gt;]], [:tNL, [nil, #&lt;Parser::Source::Range example.rb 14...15&gt;]]]]
```

https://github.com/ruby/prism/commit/c1652a9ee7
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This PR fixes an AST and token incompatibility between Parser gem and `Prism::Translation::Parser`
for dstring literal:

```ruby
"foo
  #{bar}"
```

## Parser gem (Expected)

```console
$ bundle exec ruby -Ilib -rparser/ruby33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = File.read("example.rb"); p Parser::Ruby33.new.tokenize(buf)'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[s(:dstr,
  s(:str, "foo\n"),
  s(:str, "  "),
  s(:begin,
    s(:send, nil, :bar))), [], [[:tSTRING_BEG, ["\"", #&lt;Parser::Source::Range example.rb 0...1&gt;]],
[:tSTRING_CONTENT, ["foo\n", #&lt;Parser::Source::Range example.rb 1...5&gt;]],
[:tSTRING_CONTENT, ["  ", #&lt;Parser::Source::Range example.rb 5...7&gt;]],
[:tSTRING_DBEG, ["\#{", #&lt;Parser::Source::Range example.rb 7...9&gt;]],
[:tIDENTIFIER, ["bar", #&lt;Parser::Source::Range example.rb 9...12&gt;]],
[:tSTRING_DEND, ["}", #&lt;Parser::Source::Range example.rb 12...13&gt;]],
[:tSTRING_END, ["\"", #&lt;Parser::Source::Range example.rb 13...14&gt;]], [:tNL, [nil, #&lt;Parser::Source::Range example.rb 14...15&gt;]]]]
```

## `Prism::Translation::Parser` (Actual)

Previously, the AST and tokens returned by the Parser gem were different. In this case, `dstr` node should not be nested:

```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = File.read("example.rb"); p Prism::Translation::Parser33.new.tokenize(buf)'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[s(:dstr,
  s(:dstr,
    s(:str, "foo\n"),
    s(:str, "  ")),
  s(:begin,
    s(:send, nil, :bar))), [], [[:tSTRING_BEG, ["\"", #&lt;Parser::Source::Range example.rb 0...1&gt;]],
[:tSTRING_CONTENT, ["foo\n", #&lt;Parser::Source::Range example.rb 1...5&gt;]],
[:tSTRING_CONTENT, ["  ", #&lt;Parser::Source::Range example.rb 5...7&gt;]],
[:tSTRING_DBEG, ["\#{", #&lt;Parser::Source::Range example.rb 7...9&gt;]],
[:tIDENTIFIER, ["bar", #&lt;Parser::Source::Range example.rb 9...12&gt;]],
[:tSTRING_DEND, ["}", #&lt;Parser::Source::Range example.rb 12...13&gt;]],
[:tSTRING_END, ["\"", #&lt;Parser::Source::Range example.rb 13...14&gt;]], [:tNL, [nil, #&lt;Parser::Source::Range example.rb 14...15&gt;]]]]
```

After this correction, the AST and tokens returned by the Parser gem are the same:

```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = File.read("example.rb"); p Prism::Translation::Parser33.new.tokenize(buf)'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[s(:dstr,
  s(:str, "foo\n"),
  s(:str, "  "),
  s(:begin,
    s(:send, nil, :bar))), [], [[:tSTRING_BEG, ["\"", #&lt;Parser::Source::Range example.rb 0...1&gt;]],
[:tSTRING_CONTENT, ["foo\n", #&lt;Parser::Source::Range example.rb 1...5&gt;]],
[:tSTRING_CONTENT, ["  ", #&lt;Parser::Source::Range example.rb 5...7&gt;]],
[:tSTRING_DBEG, ["\#{", #&lt;Parser::Source::Range example.rb 7...9&gt;]],
[:tIDENTIFIER, ["bar", #&lt;Parser::Source::Range example.rb 9...12&gt;]],
[:tSTRING_DEND, ["}", #&lt;Parser::Source::Range example.rb 12...13&gt;]],
[:tSTRING_END, ["\"", #&lt;Parser::Source::Range example.rb 13...14&gt;]], [:tNL, [nil, #&lt;Parser::Source::Range example.rb 14...15&gt;]]]]
```

https://github.com/ruby/prism/commit/c1652a9ee7
</pre>
</div>
</content>
</entry>
<entry>
<title>[ruby/prism] Fix an AST and token incompatibility for `Prism::Translation::Parser`</title>
<updated>2024-03-12T10:45:56+00:00</updated>
<author>
<name>Koichi ITO</name>
<email>koic.ito@gmail.com</email>
</author>
<published>2024-03-01T15:43:15+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=2af6bc26c593951fc591a9d77461d092fab728d8'/>
<id>2af6bc26c593951fc591a9d77461d092fab728d8</id>
<content type='text'>
Fixes https://github.com/ruby/prism/pull/2515.

This PR fixes an AST and token incompatibility between Parser gem and `Prism::Translation::Parser`
for string literal with line breaks.

https://github.com/ruby/prism/commit/c58466e5bf
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Fixes https://github.com/ruby/prism/pull/2515.

This PR fixes an AST and token incompatibility between Parser gem and `Prism::Translation::Parser`
for string literal with line breaks.

https://github.com/ruby/prism/commit/c58466e5bf
</pre>
</div>
</content>
</entry>
</feed>
