Replace size check with more meaningful pubkey check

It does not make sense to check the size because this
function can be used in other contexts than in TLS-1.3 and
the value might not be padded to the size of p.

However it makes sense to do the partial pubkey check because
there is no valid reason having the pubkey value outside the
1 < pubkey < p-1 bounds.

Fixes #15465

Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17630)

diff --git a/crypto/dh/dh_key.c b/crypto/dh/dh_key.c
index 6b8cd550f25fd0bb77133ded48cfbe84751bfb36..c78ed618bf83a731f85d79a81e6075d07f9e9965 100644 (file)
--- a/crypto/dh/dh_key.c
+++ b/crypto/dh/dh_key.c
@@ -375,20 +375,17 @@ int ossl_dh_buf2key(DH *dh, const unsigned char *buf, size_t len)
     int err_reason = DH_R_BN_ERROR;
     BIGNUM *pubkey = NULL;
     const BIGNUM *p;
-    size_t p_size;
+    int ret;
 
     if ((pubkey = BN_bin2bn(buf, len, NULL)) == NULL)
         goto err;
     DH_get0_pqg(dh, &p, NULL, NULL);
-    if (p == NULL || (p_size = BN_num_bytes(p)) == 0) {
+    if (p == NULL || BN_num_bytes(p) == 0) {
         err_reason = DH_R_NO_PARAMETERS_SET;
         goto err;
     }
-    /*
-     * As per Section 4.2.8.1 of RFC 8446 fail if DHE's
-     * public key is of size not equal to size of p
-     */
-    if (BN_is_zero(pubkey) || p_size != len) {
+    /* Prevent small subgroup attacks per RFC 8446 Section 4.2.8.1 */
+    if (!ossl_dh_check_pub_key_partial(dh, pubkey, &ret)) {
         err_reason = DH_R_INVALID_PUBKEY;
         goto err;
     }