ruby.git/prism/encoding.c, branch v4.0.2 The Ruby Programming Language [ruby/prism] Avoid out-of-bounds reads 2025-12-05T20:20:18+00:00 Steven Johnstone steven.james.johnstone@gmail.com 2025-12-05T16:07:33+00:00 834adc358ae967dfa52890a9abba551a070ab75e Fixes https://github.com/ruby/prism/pull/3784. https://github.com/ruby/prism/commit/3fe862534b 
Fixes https://github.com/ruby/prism/pull/3784.

https://github.com/ruby/prism/commit/3fe862534b
[ruby/prism] Update unicode tables to match that of CRuby 2025-11-30T04:24:06+00:00 Kevin Newton kddnewton@gmail.com 2025-11-29T20:58:10+00:00 3bd9583a52e9c3f55adb2097a84c4adb09cf74a3 The unicode version has been updated upstream, which means new codepoints mapped to alpha/alnum/isupper flags. We need to update our tables to match. I'm purposefully not adding a version check here, since that is such a large amount of code. It's possible that we could include different tables depending on a macro (like UNICODE_VERSION) or something to that effect, but it's such a minimal impact on the running of the actual parser that I don't think it's necessary. https://github.com/ruby/prism/commit/78925fe5b6 
The unicode version has been updated upstream, which means new
codepoints mapped to alpha/alnum/isupper flags. We need to update
our tables to match.

I'm purposefully not adding a version check here, since that is
such a large amount of code. It's possible that we could include
different tables depending on a macro (like UNICODE_VERSION) or
something to that effect, but it's such a minimal impact on the
running of the actual parser that I don't think it's necessary.

https://github.com/ruby/prism/commit/78925fe5b6
[ruby/prism] Provide options for reducing size 2024-03-20T21:32:03+00:00 Kevin Newton kddnewton@gmail.com 2024-03-20T14:08:13+00:00 af7bf9e0d8fd5e542781ca7ac9550e3011913924 https://github.com/ruby/prism/commit/592128de4d 
https://github.com/ruby/prism/commit/592128de4d
[ruby/prism] Triple-check prism encodings 2024-02-26T18:29:00+00:00 Kevin Newton kddnewton@gmail.com 2024-02-26T17:45:42+00:00 34bad6d69f7a7fa10a4e5aa48a6895afc9aebf1e https://github.com/ruby/prism/commit/ab7f261354 
https://github.com/ruby/prism/commit/ab7f261354
[ruby/prism] Validate multibyte characters in strings 2024-02-14T01:01:37+00:00 Kevin Newton kddnewton@gmail.com 2024-02-13T22:45:27+00:00 2fa051f627172674b342da26ebe1e671a5e449ec Check that multibyte characters are valid using pm_strpbrk. We need to add a couple of codepaths to ensure all encodings are covered. Importantly this doesn't check regular expressions, because apparently you're allowed to have invalid multibyte characters inside regular expression comment groups/extended mode. https://github.com/ruby/prism/commit/2857d3e1b5 
Check that multibyte characters are valid using pm_strpbrk. We need
to add a couple of codepaths to ensure all encodings are covered.

Importantly this doesn't check regular expressions, because
apparently you're allowed to have invalid multibyte characters
inside regular expression comment groups/extended mode.

https://github.com/ruby/prism/commit/2857d3e1b5
[ruby/prism] Fix KOI-8 char width check 2024-02-13T21:12:10+00:00 Kevin Newton kddnewton@gmail.com 2024-02-13T21:01:06+00:00 dc5191d6952246be6717a76c193abe1bc55128b7 https://github.com/ruby/prism/commit/06d3747c5b 
https://github.com/ruby/prism/commit/06d3747c5b
[ruby/prism] Amend assert in pm_utf_8_codepoint(), n=0 is fine 2024-02-01T22:04:46+00:00 Alan Wu XrXr@users.noreply.github.com 2024-02-01T20:55:49+00:00 770b5499a550956001b879abb5bbd205eabda19f This assert used to trip in the included test: ``` ./miniruby --parser=prism -e ' "%W"\u" ' ``` https://github.com/ruby/prism/commit/8c0f84db4f 
This assert used to trip in the included test:
```
./miniruby --parser=prism -e ' "%W"\u" '
```

https://github.com/ruby/prism/commit/8c0f84db4f
[ruby/prism] Always return the character width for char_is_identifier_start() and char_is_identifier_utf8() 2024-01-31T21:29:16+00:00 Benoit Daloze eregontp@gmail.com 2024-01-31T21:19:36+00:00 9fdfdf4fca22e892e92ad7060abac48a00516d81 * This is also faster than calling pm_encoding_utf_8_alpha_char/pm_encoding_utf_8_alnum_char as those compute the character width and do extra checks. https://github.com/ruby/prism/commit/4cb276ac4c 
* This is also faster than calling pm_encoding_utf_8_alpha_char/pm_encoding_utf_8_alnum_char
  as those compute the character width and do extra checks.

https://github.com/ruby/prism/commit/4cb276ac4c
[ruby/prism] Exclude encodings that are dynamic 2024-01-11T19:14:25+00:00 Kevin Newton kddnewton@gmail.com 2024-01-11T19:14:19+00:00 242dc537f93a0e11d91e729823cf8df2caec6abd https://github.com/ruby/prism/commit/6749146c0e 
https://github.com/ruby/prism/commit/6749146c0e
[ruby/prism] Update documentation for encodings 2023-12-06T19:26:35+00:00 Kevin Newton kddnewton@gmail.com 2023-12-06T19:26:27+00:00 82072254a05c88648caac2bd51069681e37f91aa https://github.com/ruby/prism/commit/18e6df0d4f 
https://github.com/ruby/prism/commit/18e6df0d4f