pubkey: avoid spurious audit messages from _gnutls_pubkey_compatible_with_sig()

When checking in _gnutls_pubkey_compatible_with_sig() whether a public
key is compatible with a signature algorithm, run first
pubkey_supports_sig() before performing weaker checks that can accept
the given algorithm but with an audit-log warning. This avoids an issue
when a weaker check would log an audit message for some signature
algorithm that would then be determined as incompatible by the
pubkey_supports_sig() check anyway.

For instance, a GnuTLS server might have a certificate with a SECP384R1
public key and a client can report that it supports
ECDSA-SECP256R1-SHA256 and ECDSA-SECP384R1-SHA384. In such a case, the
GnuTLS server will eventually find that it must use
ECDSA-SECP384R1-SHA384 with this public key. However, the code would
first run _gnutls_pubkey_compatible_with_sig() to check if SECP384R1 is
compatible with ECDSA-SECP256R1-SHA256. The function would report the
audit warning "The hash size used in signature (32) is less than the
expected (48)" but then reject the signature algorithm in
pubkey_supports_sig() as incompatible because it has a different curve.
Since the algorithm gets rejected it is not necessary to inform about
its hash size difference in the audit log.

Signed-off-by: Petr Pavlu <petr.pavlu@suse.com>

diff --git a/lib/pubkey.c b/lib/pubkey.c
index de95a04c3758850d035c1de9b011f52132fb1463..6f9d54f119ad75e7cfd989a8cb1320edd0a29ede 100644 (file)
--- a/lib/pubkey.c
+++ b/lib/pubkey.c
@@ -2092,10 +2092,16 @@ int _gnutls_pubkey_compatible_with_sig(gnutls_session_t session,
        unsigned int sig_hash_size;
        const mac_entry_st *me;
        const gnutls_sign_entry_st *se;
+       int ret;
 
        se = _gnutls_sign_to_entry(sign);
-       if (se == NULL && _gnutls_version_has_selectable_sighash(ver))
+       if (se != NULL) {
+               ret = pubkey_supports_sig(pubkey, se);
+               if (ret < 0)
+                       return gnutls_assert_val(ret);
+       } else if (_gnutls_version_has_selectable_sighash(ver)) {
                return gnutls_assert_val(GNUTLS_E_INVALID_REQUEST);
+       }
 
        if (pubkey->params.algo == GNUTLS_PK_DSA) {
                me = _gnutls_dsa_q_to_hash(&pubkey->params, &hash_size);
@@ -2158,9 +2164,6 @@ int _gnutls_pubkey_compatible_with_sig(gnutls_session_t session,
                }
        }
 
-       if (se != NULL)
-               return pubkey_supports_sig(pubkey, se);
-
        return 0;
 }