EVP_PKEY_up_ref(): fix bad reference count locking.

When enabling client authentication for proxy (SSLProxyMachineCertificateFile),
the client certificate callback function ssl_callback_proxy_cert uses another
reference count locking type then one that is used by the caller function when
trying to free the private key afterwards by using EVP_PKEY_free.

This can lead to a race-condition on pkey->references resulting in a double
free error.

On my system, the error occurs sporadically when threaded health checking
(mod_watchdog) forces two threads competing for the client's private key.

For example, see following two backtraces of a coredump where thread 1 and
thread 15 both run into CRYPTO_free(). Actually, the private key should never
be freed during run-time nor should two threads ever enter CRYPTO_free()
concurrently.

(gdb) t 1
[Switching to thread 1 (Thread 0xb2cfbb40 (LWP 16054))]
#0 0xf7f3f329 in __kernel_vsyscall ()
(gdb) bt
#0 0xf7f3f329 in __kernel_vsyscall ()
#1 0xf7cec9e7 in raise () from /lib32/libc.so.6
#2 0xf7cedfb9 in abort () from /lib32/libc.so.6
#3 0xf7d2a14d in ?? () from /lib32/libc.so.6
#4 0xf7d2fd27 in ?? () from /lib32/libc.so.6
#5 0xf7d3047d in ?? () from /lib32/libc.so.6
#6 0x08499c70 in CRYPTO_free (str=0x93376b0) at mem.c:434
#7 0x084cc063 in EVP_PKEY_free (x=0x93376b0) at p_lib.c:406
#8 0x08463917 in ssl3_send_client_certificate (s=0xad21f070) at s3_clnt.c:3475
#9 0x0845d62c in ssl3_connect (s=0xad21f070) at s3_clnt.c:426
#10 0x08484213 in SSL_connect (s=0xad21f070) at ssl_lib.c:1008
#11 0x0846f9c8 in ssl23_get_server_hello (s=0xad21f070) at s23_clnt.c:832
#12 0x0846ea45 in ssl23_connect (s=0xad21f070) at s23_clnt.c:231
#13 0x08484213 in SSL_connect (s=0xad21f070) at ssl_lib.c:1008
#14 0x08261e73 in ssl_io_filter_handshake (filter_ctx=0xb4d3f450) at ssl_engine_io.c:1245
#15 0x08263ba6 in ssl_io_filter_output (f=0xb4d3f480, bb=0xacc079a0) at ssl_engine_io.c:1760
#16 0x080ea2c9 in ap_pass_brigade (next=0xb4d3f480, bb=0xacc079a0) at util_filter.c:590
#17 0x08263b07 in ssl_io_filter_coalesce (f=0xb4d3f468, bb=0xacc079a0) at ssl_engine_io.c:1728
#18 0x080ea2c9 in ap_pass_brigade (next=0xb4d3f468, bb=0xacc079a0) at util_filter.c:590
#19 0x08251658 in hc_send (r=0xacc069b0, out=0x8c25ec8 "GET /hcheck HTTP/1.0\r\nHost: XXX\r\n\r\n", bb=0xacc079a0) at mod_proxy_hcheck.c:664
#20 0x08251eb3 in hc_check_http (baton=0xacc068d8) at mod_proxy_hcheck.c:806
#21 0x08252653 in hc_check (thread=0x8cc6b10, b=0xacc068d8) at mod_proxy_hcheck.c:870
#22 0x08383185 in thread_pool_func (t=0x8cc6b10, param=0x8c245e0) at misc/apr_thread_pool.c:266
#23 0x083baef6 in dummy_worker (opaque=0x8cc6b10) at threadproc/unix/thread.c:142
#24 0xf7ec615f in start_thread () from /lib32/libpthread.so.0
#25 0xf7da862e in clone () from /lib32/libc.so.6

(gdb) t 15
[Switching to thread 15 (Thread 0xb44feb40 (LWP 16049))]
#0 0xf7dd90a5 in _dl_addr () from /lib32/libc.so.6
(gdb) bt
#0 0xf7dd90a5 in _dl_addr () from /lib32/libc.so.6
#1 0xf7db610c in backtrace_symbols_fd () from /lib32/libc.so.6
#2 0xf7cd89ab in ?? () from /lib32/libc.so.6
#3 0xf7d2a148 in ?? () from /lib32/libc.so.6
#4 0xf7d2fd27 in ?? () from /lib32/libc.so.6
#5 0xf7d3047d in ?? () from /lib32/libc.so.6
#6 0x08499c70 in CRYPTO_free (str=0x93376b0) at mem.c:434
#7 0x084cc063 in EVP_PKEY_free (x=0x93376b0) at p_lib.c:406
#8 0x08463917 in ssl3_send_client_certificate (s=0xacf1baa0) at s3_clnt.c:3475
#9 0x0845d62c in ssl3_connect (s=0xacf1baa0) at s3_clnt.c:426
#10 0x08484213 in SSL_connect (s=0xacf1baa0) at ssl_lib.c:1008
#11 0x0846f9c8 in ssl23_get_server_hello (s=0xacf1baa0) at s23_clnt.c:832
#12 0x0846ea45 in ssl23_connect (s=0xacf1baa0) at s23_clnt.c:231
#13 0x08484213 in SSL_connect (s=0xacf1baa0) at ssl_lib.c:1008
#14 0x08261e73 in ssl_io_filter_handshake (filter_ctx=0xb4d37430) at ssl_engine_io.c:1245
#15 0x08263ba6 in ssl_io_filter_output (f=0xb4d37460, bb=0xad101588) at ssl_engine_io.c:1760
#16 0x080ea2c9 in ap_pass_brigade (next=0xb4d37460, bb=0xad101588) at util_filter.c:590
#17 0x08263b07 in ssl_io_filter_coalesce (f=0xb4d37448, bb=0xad101588) at ssl_engine_io.c:1728
#18 0x080ea2c9 in ap_pass_brigade (next=0xb4d37448, bb=0xad101588) at util_filter.c:590
#19 0x08251658 in hc_send (r=0xad100598, out=0x8c25898 "GET /hcheck HTTP/1.0\r\nHost: XXX\r\n\r\n", bb=0xad101588) at mod_proxy_hcheck.c:664
#20 0x08251eb3 in hc_check_http (baton=0xad1004c0) at mod_proxy_hcheck.c:806
#21 0x08252653 in hc_check (thread=0x8cc6ab0, b=0xad1004c0) at mod_proxy_hcheck.c:870
#22 0x08383185 in thread_pool_func (t=0x8cc6ab0, param=0x8c245e0) at misc/apr_thread_pool.c:266
#23 0x083baef6 in dummy_worker (opaque=0x8cc6ab0) at threadproc/unix/thread.c:142
#24 0xf7ec615f in start_thread () from /lib32/libpthread.so.0
#25 0xf7da862e in clone () from /lib32/libc.so.6

Many thanks to Armin for finding this.

Github: closes #129
Submitted by: Armin Abfalterer (arminabf)
Reviewed by: ylavic

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1879179 13f79535-47bb-0310-9956-ffa450edef68

diff --git a/modules/ssl/ssl_private.h b/modules/ssl/ssl_private.h
index 8723207d21fc9444ca65dec3754d7b291bf19a18..3b11b04ecd9d61aa10b7511297588603bab6bb71 100644 (file)
--- a/modules/ssl/ssl_private.h
+++ b/modules/ssl/ssl_private.h
@@ -245,7 +245,7 @@
 #define BIO_set_shutdown(x,v)      (x->shutdown=v)
 #define DH_bits(x)                 (BN_num_bits(x->p))
 #define X509_up_ref(x)             (CRYPTO_add(&(x)->references, +1, CRYPTO_LOCK_X509))
-#define EVP_PKEY_up_ref(pk)        (CRYPTO_add(&(pk)->references, +1, CRYPTO_LOCK_X509_PKEY))
+#define EVP_PKEY_up_ref(pk)        (CRYPTO_add(&(pk)->references, +1, CRYPTO_LOCK_EVP_PKEY))
 #else
 void init_bio_methods(void);
 void free_bio_methods(void);