Introduce PM_TOKEN_KEYWORD_DO_BLOCK to distinguish do-blocks on
command-style calls from regular `do` keywords. Add
parse_command_do_block to attach these blocks to call nodes. Track
in_endless_def_body to prevent do-block consumption inside endless
method definitions, allowing blocks to correctly bubble up to outer
contexts like `private def f = puts "Hello" do end`.

https://github.com/ruby/prism/commit/7d17fd254b

(https://github.com/ruby/prism/pull/3908)

* Add missing nodocs for FFI

When generating docs in ruby/prism, this doesn't seem to matter.

But on docs.ruby-lang.org these take over the actual documentation.

* Nodoc various utility methods

They don't tell much of a story and just pull attention away from actually unique methods

* Nodoc some private functions/modules

* Nodoc translator implementation details

As well as some functionality from ripper that is nodoc in ruby

* Stop documenting visitor implementation details

These result in so many methods that are searchable
and give no actual benefit.
Toplevel docs are all you need to understand the usecase.

https://github.com/ruby/prism/commit/0bd9a7d373

In the C API, we want to use slices instead of locations in the
AST. In this case a "slice" is effectively the same thing as the
location, expect it is represented using a 32-bit offset and a
32-bit length. This will cut down on half of the space of all of
the locations in the AST.

Note that from the Ruby/Java/JavaScript side, this is effectively
an invisible change. This only impacts the C/Rust side.

The same also applies to `break`/`next`.

https://bugs.ruby-lang.org/issues/21540

https://github.com/ruby/prism/commit/3a38b192e3

Generally I have been good about safely accessing the tokens but failed
to properly guard against no tokens in places
where it could theoretically happen through invalid syntax.

I added a test case for one occurance, other changes are theoretical only.

https://github.com/ruby/prism/commit/4a3866af19

https://docs.ruby-lang.org/en/master/syntax/literals_rdoc.html#label-25w+and+-25W-3A+String-Array+Literals
> %W allow escape sequences described in Escape Sequences. However the continuation line <newline> is not usable because it is interpreted as the escaped newline described above.

https://github.com/ruby/prism/commit/f5c7460ad5

Instead, prefer `scan_byte` over `get_byte` since that already returns the byte as an integer, sidestepping conversion issues.

Fixes https://github.com/ruby/prism/issues/3582

https://github.com/ruby/prism/commit/7f3008b2b5

https://github.com/ruby/prism/commit/12af4e144e

`StringNode` and `SymbolNode` don't have the same shape
(`content` vs `value`) and that wasn't handled.

I believe the logic for the common case can be reused.
I simply left the special handling for implicit nodes in pattern matching
and fall through otherwise.

NOTE: patterns.txt is not actually tested at the moment,
because it contains syntax that `parser` mistakenly rejects.
But I checked manually that this doesn't introduce other failures.
https://github.com/whitequark/parser/pull/1060

https://github.com/ruby/prism/commit/55adfaa895