[ruby/openssl] cipher: fix buffer overflow in Cipher#update

OpenSSL::Cipher#update currently allocates the output buffer with size
(input data length)+(the block size of the cipher). This is insufficient
for the id-aes{128,192,256}-wrap-pad (AES keywrap with padding) ciphers.
They have a block size of 8 bytes, but the output may be up to 15 bytes
larger than the input.

Use (input data length)+EVP_MAX_BLOCK_LENGTH (== 32) as the output
buffer size, instead. OpenSSL doesn't provide a generic way to tell the
maximum required buffer size for ciphers, but this is large enough for
all algorithms implemented in current versions of OpenSSL.

Fixes: https://bugs.ruby-lang.org/issues/20236

https://github.com/ruby/openssl/commit/3035559f54

1 files changed, 16 insertions, 0 deletions

diff --git a/test/openssl/test_cipher.rb b/test/openssl/test_cipher.rb
index 8faa570648..41885fd59b 100644
--- a/test/openssl/test_cipher.rb
+++ b/test/openssl/test_cipher.rb
@@ -331,6 +331,22 @@ class OpenSSL::TestCipher < OpenSSL::TestCase
     assert_equal tag1, tag2
   end
 
+  def test_aes_keywrap_pad
+    # RFC 5649 Section 6; The second example
+    kek = ["5840df6e29b02af1ab493b705bf16ea1ae8338f4dcc176a8"].pack("H*")
+    key = ["466f7250617369"].pack("H*")
+    wrap = ["afbeb0f07dfbf5419200f2ccb50bb24f"].pack("H*")
+
+    begin
+      cipher = OpenSSL::Cipher.new("id-aes192-wrap-pad").encrypt
+    rescue OpenSSL::Cipher::CipherError, RuntimeError
+      omit "id-aes192-wrap-pad is not supported: #$!"
+    end
+    cipher.key = kek
+    ct = cipher.update(key) << cipher.final
+    assert_equal wrap, ct
+  end
+
   def test_non_aead_cipher_set_auth_data
     assert_raise(OpenSSL::Cipher::CipherError) {
       cipher = OpenSSL::Cipher.new("aes-128-cfb").encrypt