utf-8 is the default for source files but can be overwritten via options

https://github.com/ruby/prism/commit/355f451528

When we hit EOF and still have lex modes left, it means some content was unterminated.
Heredocs specifically have logic that needs to happen when the body finished lexing.
If we don't reset the mode back to how it was before, it will not continue lexing at the correct place.

Followup to https://github.com/ruby/prism/pull/3918.
We can't call into `parser_lex` since it resets token locations.

https://github.com/ruby/prism/commit/27c24fdc0d

This reverts commit https://github.com/ruby/prism/commit/8f35e8ef255b.

https://github.com/ruby/prism/commit/ed48f6dbfb

When we hit EOF and still have lex modes left, it means some content was unterminated.
Heredocs specifically have logic that needs to happen when the body finished lexing.
If we don't reset the mode back to how it was before, it will not continue lexing at the correct place.

https://github.com/ruby/prism/commit/8f35e8ef25

Since `on_sp` is emitted, it doesn't do a whole lot anymore.

This leaves one incompatibility for code like `"x#$%"`

Ripper confuses this for bare interpolation with a global, but `$%` is not a valid global name. Still,
it emits two string tokens in such a case. It doesn't make sense for prism to work around this bug,
so the affected files are added as excludes.

Since the only usage of this method makes sense for testing in prism itself,
the method is removed instead of deprecated.

https://github.com/ruby/prism/commit/31be379f98

`RipperTest` already does this (added in https://github.com/ruby/prism/pull/3849)
Since it doesn't use the token classes, it also lists out all the excludes instead of just claiming some are passing.

https://github.com/ruby/prism/commit/e86a28263c

Mostly not having to list version-specific excludes when testing against ripper/parse.y

Also don't test new syntax additions against the parser gems. The version support
for them may (or may not) be expanded but we shouldn't bother while the ruby version
hasn't even released yet.
(ruby_parser translation is not versioned, so let as is for now)

I also removed excludes that have since been implemented by parse.y

https://github.com/ruby/prism/commit/e5a0221c37

See https://github.com/ruby/ruby/commit/6d81969b475262aba251e99b518181bdf7c5a523

It leaves the old variant around. RuboCop for examples accesses `Prism::Translation::Parser35`
to test against ruby-head. For now I left these simply as an alias

https://github.com/ruby/prism/commit/d0a823f045

This one has been on my mind for a while now.

Currently, there are only tests against the latest syntax version.

This changes the snapshot structure as follows:
* Snapshots at their current location are tested against all syntax versions
* Snapshots inside a version folder like "3.3" are tested against all versions starting from that version
* Snapshots inside a version folder like "3.3-4.2" are tested against all versions in the given range.

This makes sure that as new syntax is added, older versions still work as expected.
I also added a few tests for now valid syntax that should be invalid in older versions (and the other way around as well)

These tests run really fast. So even though it does 3x the work for these, I am still able to run the whole test suite in just 11 seconds.

https://github.com/ruby/prism/commit/5191b1aa68

https://bugs.ruby-lang.org/issues/21168#note-5

The added code samples align with `parse.y`, except for `foo(bar baz do end)` which `parse.y` currently rejects but shouldn't.

https://github.com/ruby/prism/commit/3a4e102d80