ruby.git/lib/prism/polyfill/append_as_bytes.rb, branch v4.0.4 The Ruby Programming Language [ruby/prism] Fix parser translator crash for certain octal escapes 2025-03-18T17:36:53+00:00 Earlopain 14981592+Earlopain@users.noreply.github.com 2025-01-19T12:38:17+00:00 a679597547b23259159dd914959ca36409878199 `Integer#chr` performs some validation that we don't want/need. Octal escapes can go above 255, where it will then raise trying to convert. `append_as_bytes` actually allows to pass a number, so we can just skip that call. Although, on older rubies of course we still need to handle this in the polyfill. I don't really like using `pack` but don't know of another way to do so. For the utf-8 escapes, this is not an issue. Invalid utf-8 in these is simply a syntax error. https://github.com/ruby/prism/commit/161c606b1f 
`Integer#chr` performs some validation that we don't want/need. Octal escapes can go above 255, where it will then raise trying to convert.

`append_as_bytes` actually allows to pass a number, so we can just skip that call.
Although, on older rubies of course we still need to handle this in the polyfill.
I don't really like using `pack` but don't know of another way to do so.

For the utf-8 escapes, this is not an issue. Invalid utf-8 in these is simply a syntax error.

https://github.com/ruby/prism/commit/161c606b1f
[ruby/prism] Fix parser translator when unescaping invalid utf8 2025-03-18T17:36:53+00:00 Earlopain 14981592+Earlopain@users.noreply.github.com 2025-01-12T19:08:28+00:00 705bd6fadbd5a784d2c0fd52138007404de6a5c2 1. The string starts out as binary 2. `ち` is appended, forcing it back into utf-8 3. Some invalid byte sequences are tried to append > incompatible character encodings: UTF-8 and BINARY (ASCII-8BIT) This makes use of my wish to use `append_as_bytes`. Unfortunatly that method is rather new so it needs a fallback https://github.com/ruby/prism/commit/e31e94a775 
1. The string starts out as binary
2. `ち` is appended, forcing it back into utf-8
3. Some invalid byte sequences are tried to append

> incompatible character encodings: UTF-8 and BINARY (ASCII-8BIT)

This makes use of my wish to use `append_as_bytes`. Unfortunatly that method is rather new
so it needs a fallback

https://github.com/ruby/prism/commit/e31e94a775