openssl: import v2.1.0.beta1

Import Ruby/OpenSSL 2.1.0.beta1. The full commit log since v2.0.5
(imported by r59567) can be found at:

	https://github.com/ruby/openssl/compare/v2.0.5...v2.1.0.beta1

----------------------------------------------------------------
Antonio Terceiro (1):
      test/test_ssl: explicitly accept TLS 1.1 in corresponding test

Colby Swandale (1):
      document using secure protocol to fetch git master in Bundler

Colton Jenkins (1):
      Add fips_mode_get to return fips_mode

Kazuki Yamaguchi (85):
      Start preparing for 2.1.0
      Remove support for OpenSSL 0.9.8 and 1.0.0
      bn: refine tests
      bn: implement unary {plus,minus} operators for OpenSSL::BN
      bn: implement OpenSSL::BN#negative?
      Don't define main() when built with --enable-debug
      test: let OpenSSL::TestCase include OpenSSL::TestUtils
      test: prepare test PKey instances on demand
      Add OpenSSL.print_mem_leaks
      Enable OSSL_MDEBUG on CI builds
      ssl: move default DH parameters from OpenSSL::PKey::DH
      Make exceptions with the same format regardless of OpenSSL.debug
      ssl: show reason of 'certificate verify error' in exception message
      ssl: remove OpenSSL::ExtConfig::TLS_DH_anon_WITH_AES_256_GCM_SHA384
      ssl: do not confuse different ex_data index registries
      ssl: assume SSL/SSL_CTX always have a valid reference to the Ruby object
      Fix RDoc markup
      ssl: suppress compiler warning
      ext/openssl/deprecation.rb: remove broken-apple-openssl
      extconf.rb: print informative message if OpenSSL can't be found
      Rakefile: compile the extension before test
      kdf: introduce OpenSSL::KDF module
      ossl.h: add NUM2UINT64T() macro
      kdf: add scrypt
      Expand rb_define_copy_func() macro
      Expand FPTR_TO_FD() macro
      Remove SafeGet*() macros
      cipher: rename GetCipherPtr() to ossl_evp_get_cipherbyname()
      digest: rename GetDigestPtr() to ossl_evp_get_digestbyname()
      Add ossl_str_new(), an exception-safe rb_str_new()
      bio: simplify ossl_membio2str() using ossl_str_new()
      Remove unused functions and macros
      Drop support for LibreSSL 2.3
      ocsp: add OpenSSL::OCSP::Request#signed?
      asn1: infinite length -> indefinite length
      asn1: rearrange tests
      ssl: remove a needless NULL check in SSL::SSLContext#ciphers
      ssl: return nil in SSL::SSLSocket#cipher if session is not started
      asn1: remove an unnecessary function prototype
      asn1: require tag information when instantiating generic type
      asn1: initialize 'unused_bits' attribute of BitString with 0
      asn1: check for illegal 'unused_bits' value of BitString
      asn1: disallow NULL to be passed to asn1time_to_time()
      asn1: avoid truncating OID in OpenSSL::ASN1::ObjectId#oid
      asn1: allow constructed encoding with definite length form
      asn1: prohibit indefinite length form for primitive encoding
      asn1: allow tag number to be >= 32 for universal tag class
      asn1: use ossl_asn1_tag()
      asn1: clean up OpenSSL::ASN1::Constructive#to_der
      asn1: harmonize OpenSSL::ASN1::*#to_der
      asn1: prevent EOC octets from being in the middle of the content
      asn1: do not treat EOC octets as part of content octets
      x509name: add 'loc' and 'set' kwargs to OpenSSL::X509::Name#add_entry
      ssl: do not call session_remove_cb during GC
      Backport "Merge branch 'topic/test-memory-leak'" to maint
      cipher: update the documentation for Cipher#auth_tag=
      Rakefile: let sync:to_ruby know about test/openssl/fixtures
      test: fix formatting
      test/utils: remove OpenSSL::TestUtils.silent
      test/utils: add SSLTestCase#tls12_supported?
      test/utils: have start_server yield only the port number
      test/utils: do not set ecdh_curves in start_server
      test/utils: let server_loop close socket
      test/utils: improve error handling in start_server
      test/utils: add OpenSSL::TestUtils.openssl? and .libressl?
      test/utils: do not use DSA certificates in SSL tests
      test/test_ssl: remove test_invalid_shutdown_by_gc
      test/test_ssl: move test_multibyte_read_write to test_pair
      test/test_ssl_session: rearrange tests
      test/test_pair, test/test_ssl: fix for TLS 1.3
      ssl: remove useless call to rb_thread_wait_fd()
      ssl: fix NPN support
      ssl: mark OpenSSL::SSL::SSLContext::DEFAULT_{1024,2048} as private
      ssl: use 2048-bit group in the default tmp_dh_cb
      ssl: ensure that SSL option flags are non-negative
      ssl: update OpenSSL::SSL::OP_* flags
      ssl: prefer TLS_method() over SSLv23_method()
      ssl: add SSLContext#min_version= and #max_version=
      ssl: rework SSLContext#ssl_version=
      test/test_x509name: change script encoding to ASCII-8BIT
      x509name: refactor OpenSSL::X509::Name#to_s
      x509name: add OpenSSL::X509::Name#to_utf8
      x509name: add OpenSSL::X509::Name#inspect
      x509name: update regexp in OpenSSL::X509::Name.parse
      Ruby/OpenSSL 2.1.0.beta1

Marcus Stollsteimer (1):
      Fix rdoc for core Integer class

nobu (4):
      [DOC] {read,write}_nonblock with exception: false
      [DOC] keyword argument _exception_
      [DOC] mark up literals
      Revert r57690 except for read_nonblock

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59734 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

1 files changed, 221 insertions, 0 deletions

diff --git a/ext/openssl/ossl_kdf.c b/ext/openssl/ossl_kdf.c
new file mode 100644
index 0000000000..9fa42e174a
--- /dev/null
+++ b/ext/openssl/ossl_kdf.c
@@ -0,0 +1,221 @@
+/*
+ * Ruby/OpenSSL Project
+ * Copyright (C) 2007, 2017 Ruby/OpenSSL Project Authors
+ */
+#include "ossl.h"
+
+static VALUE mKDF, eKDF;
+
+/*
+ * call-seq:
+ *   KDF.pbkdf2_hmac(pass, salt:, iterations:, length:, hash:) -> aString
+ *
+ * PKCS #5 PBKDF2 (Password-Based Key Derivation Function 2) in combination
+ * with HMAC. Takes _pass_, _salt_ and _iterations_, and then derives a key
+ * of _length_ bytes.
+ *
+ * For more information about PBKDF2, see RFC 2898 Section 5.2
+ * (https://tools.ietf.org/html/rfc2898#section-5.2).
+ *
+ * === Parameters
+ * pass       :: The passphrase.
+ * salt       :: The salt. Salts prevent attacks based on dictionaries of common
+ *               passwords and attacks based on rainbow tables. It is a public
+ *               value that can be safely stored along with the password (e.g.
+ *               if the derived value is used for password storage).
+ * iterations :: The iteration count. This provides the ability to tune the
+ *               algorithm. It is better to use the highest count possible for
+ *               the maximum resistance to brute-force attacks.
+ * length     :: The desired length of the derived key in octets.
+ * hash       :: The hash algorithm used with HMAC for the PRF. May be a String
+ *               representing the algorithm name, or an instance of
+ *               OpenSSL::Digest.
+ */
+static VALUE
+kdf_pbkdf2_hmac(int argc, VALUE *argv, VALUE self)
+{
+    VALUE pass, salt, opts, kwargs[4], str;
+    static ID kwargs_ids[4];
+    int iters, len;
+    const EVP_MD *md;
+
+    if (!kwargs_ids[0]) {
+	kwargs_ids[0] = rb_intern_const("salt");
+	kwargs_ids[1] = rb_intern_const("iterations");
+	kwargs_ids[2] = rb_intern_const("length");
+	kwargs_ids[3] = rb_intern_const("hash");
+    }
+    rb_scan_args(argc, argv, "1:", &pass, &opts);
+    rb_get_kwargs(opts, kwargs_ids, 4, 0, kwargs);
+
+    StringValue(pass);
+    salt = StringValue(kwargs[0]);
+    iters = NUM2INT(kwargs[1]);
+    len = NUM2INT(kwargs[2]);
+    md = ossl_evp_get_digestbyname(kwargs[3]);
+
+    str = rb_str_new(0, len);
+    if (!PKCS5_PBKDF2_HMAC(RSTRING_PTR(pass), RSTRING_LENINT(pass),
+			   (unsigned char *)RSTRING_PTR(salt),
+			   RSTRING_LENINT(salt), iters, md, len,
+			   (unsigned char *)RSTRING_PTR(str)))
+	ossl_raise(eKDF, "PKCS5_PBKDF2_HMAC");
+
+    return str;
+}
+
+#if defined(HAVE_EVP_PBE_SCRYPT)
+/*
+ * call-seq:
+ *   KDF.scrypt(pass, salt:, N:, r:, p:, length:) -> aString
+ *
+ * Derives a key from _pass_ using given parameters with the scrypt
+ * password-based key derivation function. The result can be used for password
+ * storage.
+ *
+ * scrypt is designed to be memory-hard and more secure against brute-force
+ * attacks using custom hardwares than alternative KDFs such as PBKDF2 or
+ * bcrypt.
+ *
+ * The keyword arguments _N_, _r_ and _p_ can be used to tune scrypt. RFC 7914
+ * (published on 2016-08, https://tools.ietf.org/html/rfc7914#section-2) states
+ * that using values r=8 and p=1 appears to yield good results.
+ *
+ * See RFC 7914 (https://tools.ietf.org/html/rfc7914) for more information.
+ *
+ * === Parameters
+ * pass   :: Passphrase.
+ * salt   :: Salt.
+ * N      :: CPU/memory cost parameter. This must be a power of 2.
+ * r      :: Block size parameter.
+ * p      :: Parallelization parameter.
+ * length :: Length in octets of the derived key.
+ *
+ * === Example
+ *   pass = "password"
+ *   salt = SecureRandom.random_bytes(16)
+ *   dk = OpenSSL::KDF.scrypt(pass, salt: salt, N: 2**14, r: 8, p: 1, length: 32)
+ *   p dk #=> "\xDA\xE4\xE2...\x7F\xA1\x01T"
+ */
+static VALUE
+kdf_scrypt(int argc, VALUE *argv, VALUE self)
+{
+    VALUE pass, salt, opts, kwargs[5], str;
+    static ID kwargs_ids[5];
+    size_t len;
+    uint64_t N, r, p, maxmem;
+
+    if (!kwargs_ids[0]) {
+	kwargs_ids[0] = rb_intern_const("salt");
+	kwargs_ids[1] = rb_intern_const("N");
+	kwargs_ids[2] = rb_intern_const("r");
+	kwargs_ids[3] = rb_intern_const("p");
+	kwargs_ids[4] = rb_intern_const("length");
+    }
+    rb_scan_args(argc, argv, "1:", &pass, &opts);
+    rb_get_kwargs(opts, kwargs_ids, 5, 0, kwargs);
+
+    StringValue(pass);
+    salt = StringValue(kwargs[0]);
+    N = NUM2UINT64T(kwargs[1]);
+    r = NUM2UINT64T(kwargs[2]);
+    p = NUM2UINT64T(kwargs[3]);
+    len = NUM2LONG(kwargs[4]);
+    /*
+     * OpenSSL uses 32MB by default (if zero is specified), which is too small.
+     * Let's not limit memory consumption but just let malloc() fail inside
+     * OpenSSL. The amount is controllable by other parameters.
+     */
+    maxmem = SIZE_MAX;
+
+    str = rb_str_new(0, len);
+    if (!EVP_PBE_scrypt(RSTRING_PTR(pass), RSTRING_LEN(pass),
+			(unsigned char *)RSTRING_PTR(salt), RSTRING_LEN(salt),
+			N, r, p, maxmem, (unsigned char *)RSTRING_PTR(str), len))
+	ossl_raise(eKDF, "EVP_PBE_scrypt");
+
+    return str;
+}
+#endif
+
+void
+Init_ossl_kdf(void)
+{
+#if 0
+    mOSSL = rb_define_module("OpenSSL");
+    eOSSLError = rb_define_class_under(mOSSL, "OpenSSLError", rb_eStandardError);
+#endif
+
+    /*
+     * Document-module: OpenSSL::KDF
+     *
+     * Provides functionality of various KDFs (key derivation function).
+     *
+     * KDF is typically used for securely deriving arbitrary length symmetric
+     * keys to be used with an OpenSSL::Cipher from passwords. Another use case
+     * is for storing passwords: Due to the ability to tweak the effort of
+     * computation by increasing the iteration count, computation can be slowed
+     * down artificially in order to render possible attacks infeasible.
+     *
+     * Currently, OpenSSL::KDF provides implementations for the following KDF:
+     *
+     * * PKCS #5 PBKDF2 (Password-Based Key Derivation Function 2) in
+     *   combination with HMAC
+     * * scrypt
+     *
+     * == Examples
+     * === Generating a 128 bit key for a Cipher (e.g. AES)
+     *   pass = "secret"
+     *   salt = OpenSSL::Random.random_bytes(16)
+     *   iter = 20_000
+     *   key_len = 16
+     *   key = OpenSSL::KDF.pbkdf2_hmac(pass, salt: salt, iterations: iter,
+     *                                  length: key_len, hash: "sha1")
+     *
+     * === Storing Passwords
+     *   pass = "secret"
+     *   # store this with the generated value
+     *   salt = OpenSSL::Random.random_bytes(16)
+     *   iter = 20_000
+     *   hash = OpenSSL::Digest::SHA256.new
+     *   len = hash.digest_length
+     *   # the final value to be stored
+     *   value = OpenSSL::KDF.pbkdf2_hmac(pass, salt: salt, iterations: iter,
+     *                                    length: len, hash: hash)
+     *
+     * == Important Note on Checking Passwords
+     * When comparing passwords provided by the user with previously stored
+     * values, a common mistake made is comparing the two values using "==".
+     * Typically, "==" short-circuits on evaluation, and is therefore
+     * vulnerable to timing attacks. The proper way is to use a method that
+     * always takes the same amount of time when comparing two values, thus
+     * not leaking any information to potential attackers. To compare two
+     * values, the following could be used:
+     *
+     *   def eql_time_cmp(a, b)
+     *     unless a.length == b.length
+     *       return false
+     *     end
+     *     cmp = b.bytes
+     *     result = 0
+     *     a.bytes.each_with_index {|c,i|
+     *       result |= c ^ cmp[i]
+     *     }
+     *     result == 0
+     *   end
+     *
+     * Please note that the premature return in case of differing lengths
+     * typically does not leak valuable information - when using PBKDF2, the
+     * length of the values to be compared is of fixed size.
+     */
+    mKDF = rb_define_module_under(mOSSL, "KDF");
+    /*
+     * Generic exception class raised if an error occurs in OpenSSL::KDF module.
+     */
+    eKDF = rb_define_class_under(mKDF, "KDFError", eOSSLError);
+
+    rb_define_module_function(mKDF, "pbkdf2_hmac", kdf_pbkdf2_hmac, -1);
+#if defined(HAVE_EVP_PBE_SCRYPT)
+    rb_define_module_function(mKDF, "scrypt", kdf_scrypt, -1);
+#endif
+}