openssl: clear errors after a failed `d2i_X509()`

Without it, subsequent OpenSSL API calls may fail with an error caught
within the OpenSSL `d2i_X509()` (decode) call.

It was seen to happen when importing from the Windows certificate store
(e.g. with `--ca-native`), and any one of the certificates failed while
decoding, then skipped.

Behind the scene (and undocumented), the failed decode call is adding
an error to an internal OpenSSL error queue. This error is picked up
later, at the connect phase, by another OpenSSL API call, which happens
to check the error queue, without clearing it first. It made the connect
fail with the error collected earlier, while decoding the malformed and
discarded certificate.

Fix by explicitly clearing the error queue if the decode call fails.

Ref: https://docs.openssl.org/3.5/man3/d2i_X509/

`-vvvv` output before this patch:
```
[0-0] == Info: successfully imported Windows ROOT store
[0-0] == Info: successfully imported Windows CA store
[0-0] == Info: [SSL] SSL_connect() -> err=-1, detail=1
[0-0] == Info: TLS connect error: error:068000DD:asn1 encoding routines::illegal padding
[0-0] == Info: [SSL] cf_connect() -> 35, done=0
```

Mainline OpenSSL (as of 3.5.2) and quictls (as of 3.3.0) are affected.

LibreSSL is not affected. (I did not test BoringSSL and AWS-LC)

Assisted-by: Stefan Eissing
Reported-by: Michał Petryka
Fixes #18190

Closes #18228

diff --git a/lib/vtls/openssl.c b/lib/vtls/openssl.c
index a2ab831f20c5860e4008b5fdeb97b8a6a646d7c4..dc4a6d122cee9688a9c7f0888a1eb9df00d44ad3 100644 (file)
--- a/lib/vtls/openssl.c
+++ b/lib/vtls/openssl.c
@@ -3327,8 +3327,10 @@ static CURLcode import_windows_cert_store(struct Curl_easy *data,
         continue;
 
       x509 = d2i_X509(NULL, &encoded_cert, (long)pContext->cbCertEncoded);
-      if(!x509)
+      if(!x509) {
+        ERR_clear_error();
         continue;
+      }
 
       /* Try to import the certificate. This may fail for legitimate
          reasons such as duplicate certificate, which is allowed by MS but