digest: remove OpenSSL engine

The OpenSSL engine of Digest uses the low-level API of OpenSSL, whose
use has been discouraged for years for multiple reasons.

A long-standing issue on a FIPS-enabled system is that using ::Digest
results in crashing the Ruby process, because the low-level API lacks
the mechanism to report an error (the policy violation) and thus kills
the process as a last resort[1][2]. Also, the upcoming OpenSSL 3.0 will
deprecate it for future removal[3]. Compiling with
-Wdeprecated-declarations will start to emit warnings.

A proper fix for this is to make it use the EVP API instead. This is a
non-trivial work as it requires backwards-incompatible changes to the
framework interface of Digest::Base and rb_digest_metadata_t.

It is more than 15 years ago that the openssl library became part of the
standard library. It has implemented the exactly same functionality as
OpenSSL::Digest, in fact, as a subclass of Digest::Class. There is not
much point in having an identical code in the digest library. Let's
just get rid of OpenSSL within digest. This leaves the C implementations
and the CommonCrypto engine for Apple systems.

A patch is being prepared for the openssl library to provide ::Digest
constants for better performance[4].

[1] https://bugs.ruby-lang.org/issues/6946
[2] https://bugs.ruby-lang.org/issues/13681
[3] https://www.openssl.org/docs/OpenSSL300Design.html
[4] https://github.com/ruby/openssl/pull/377

1 files changed, 0 insertions, 1 deletions

diff --git a/ext/digest/md5/depend b/ext/digest/md5/depend
index 9c3f6e55e0..987ebcc12d 100644
--- a/ext/digest/md5/depend
+++ b/ext/digest/md5/depend
@@ -329,5 +329,4 @@ md5init.o: $(srcdir)/../defs.h
 md5init.o: $(srcdir)/../digest.h
 md5init.o: md5.h
 md5init.o: md5init.c
-md5init.o: md5ossl.h
 # AUTOGENERATED DEPENDENCIES END