[Bug #20883] Fix coderange for sprintf on binary strings (#12040)

Fix update_coderange for binary strings

Although a binary (aka ASCII-8BIT) string will never have a broken
coderange, it still has to differentiate between "valid" and "7bit".

On Ruby 3.4/trunk this problem is masked because we now clear the
coderange more agressively in rb_str_resize, and we happened to always
be strinking this string, but we should not assume that.

On Ruby 3.3 this created strings where `ascii_only?` was true in cases
it shouldn't be as well as other problems.

Fixes [Bug #20883]

Co-authored-by: Daniel Colson <danieljamescolson@gmail.com>
Co-authored-by: Matthew Draper <matthew@trebex.net>

2 files changed, 9 insertions, 2 deletions

diff --git a/sprintf.c b/sprintf.c
index b2d89617aa..0b7a70a50c 100644
--- a/sprintf.c
+++ b/sprintf.c
@@ -247,8 +247,7 @@ rb_str_format(int argc, const VALUE *argv, VALUE fmt)
     }
 
 #define update_coderange(partial) do { \
-        if (coderange != ENC_CODERANGE_BROKEN && scanned < blen \
-            && rb_enc_to_index(enc) /* != ENCINDEX_ASCII_8BIT */) { \
+        if (coderange != ENC_CODERANGE_BROKEN && scanned < blen) { \
             int cr = coderange; \
             scanned += rb_str_coderange_scan_restartable(buf+scanned, buf+blen, enc, &cr); \
             ENC_CODERANGE_SET(result, \
diff --git a/test/ruby/test_sprintf.rb b/test/ruby/test_sprintf.rb
index c453ecd350..9b972dcbaa 100644
--- a/test/ruby/test_sprintf.rb
+++ b/test/ruby/test_sprintf.rb
@@ -543,4 +543,12 @@ class TestSprintf < Test::Unit::TestCase
       sprintf("%*s", RbConfig::LIMITS["INT_MIN"], "")
     end
   end
+
+  def test_binary_format_coderange
+    1.upto(500) do |i|
+      str = sprintf("%*s".b, i, "\xe2".b)
+      refute_predicate str, :ascii_only?
+      assert_equal i, str.bytesize
+    end
+  end
 end