Ensure X509 extensions are hashed & cached, before deciding a cert is CA or EE

If X509_check_ca() fails to cache X509v3 extension values, the return
value may be incorrect, leading to erroneously assuming a given certificate
is a CA or EE cert (while in reality it is the other, or neither).

This failure mode can arise because X509_check_ca() doesn't verify
whether libcrypto's (void)x509v3_cache_extensions(x) flipped the EXFLAG_INVALID
flag in x->ex_flags. Unfortunately, X509_check_ca() doesn't have a return code
to indicate an error, so this can't be fixed in libcrypto - the API is broken.

The workaround is to call X509_check_purpose(3) with a purpose argument of -1,
before calling X509_check_ca(), this ensures the X509v3 extensions are cached.
Since X509_check_purpose() does have a return code to indicate errors, we can
use that to supplement X509_check_ca()'s shortcomings.

OpenBSD's rpki-client also uses the above approach.

diff --git a/src/object/certificate.c b/src/object/certificate.c
index 8966741d4e435c1a207685d4b75713ec802e079a..dfc88996c2e89e98334f6421eca6a0bba4d96424 100644 (file)
--- a/src/object/certificate.c
+++ b/src/object/certificate.c
@@ -1725,6 +1725,9 @@ get_certificate_type(X509 *cert, bool is_ta, enum cert_type *result)
                return 0;
        }
 
+       if (X509_check_purpose(cert, -1, -1) <= 0)
+               goto err;
+
        if (X509_check_ca(cert) == 1) {
                *result = CA;
                return 0;
@@ -1735,6 +1738,7 @@ get_certificate_type(X509 *cert, bool is_ta, enum cert_type *result)
                return 0;
        }
 
+err:
        *result = EE; /* Shuts up nonsense gcc 8.3 warning */
        return pr_val_err("Certificate is not TA, CA nor BGPsec. Ignoring...");
 }