Prevent crash with engine using different openssl runtime

This problem happens usually because an application
links libcrypto and/or libssl statically which
installs an atexit handler, but later an engine using
a shared instance of libcrypto is installed.
The problem is in simple words that both instances
of libcrypto have an atexit handler installed,
but both are unable to coordinate with each other,
which causes a crash, typically a use-after-free
in the engine's destroy function.

Work around that by preventing the engine's
libcrypto to install the atexit handler.
This may result in a small memory leak, but that
memory is still reachable.

Fixes #15898

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17112)

diff --git a/include/openssl/engine.h b/include/openssl/engine.h
index 25c3cf7c1985baeeb3857abfea2c4315026d2218..70c856a9cd547401948e412b1d3cf31c6a50d138 100644 (file)
--- a/include/openssl/engine.h
+++ b/include/openssl/engine.h
@@ -801,6 +801,7 @@ typedef int (*dynamic_bind_engine) (ENGINE *e, const char *id,
             CRYPTO_set_mem_functions(fns->mem_fns.malloc_fn, \
                                      fns->mem_fns.realloc_fn, \
                                      fns->mem_fns.free_fn); \
+            OPENSSL_init_crypto(OPENSSL_INIT_NO_ATEXIT, NULL); \
         skip_cbs: \
             if (!fn(e, id)) return 0; \
             return 1; }