Prevent crash with engine using different openssl runtime

This problem happens usually because an application
links libcrypto and/or libssl statically which
installs an atexit handler, but later an engine using
a shared instance of libcrypto is installed.
The problem is in simple words that both instances
of libcrypto have an atexit handler installed,
but both are unable to coordinate with each other,
which causes a crash, typically a use-after-free
in the engine's destroy function.

Work around that by preventing the engine's
libcrypto to install the atexit handler.
This may result in a small memory leak, but that
memory is still reachable.

Fixes #15898

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17541)

diff --git a/include/openssl/engine.h b/include/openssl/engine.h
index 0780f0fb5f327db9782995e9b94440e065555465..756751c6d35d7047631c88b98aeaf7cdb3d192ac 100644 (file)
--- a/include/openssl/engine.h
+++ b/include/openssl/engine.h
@@ -722,6 +722,7 @@ typedef int (*dynamic_bind_engine) (ENGINE *e, const char *id,
             CRYPTO_set_mem_functions(fns->mem_fns.malloc_fn, \
                                      fns->mem_fns.realloc_fn, \
                                      fns->mem_fns.free_fn); \
+            OPENSSL_init_crypto(OPENSSL_INIT_NO_ATEXIT, NULL); \
         skip_cbs: \
             if (!fn(e, id)) return 0; \
             return 1; }