From: Hugo Landau Date: Wed, 24 Apr 2024 12:20:44 +0000 (+0100) Subject: QUIC: Add documentation on concurrency model X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=dfd2e82a188c97a40114c146751fe67cb008ea3a;p=thirdparty%2Fopenssl.git QUIC: Add documentation on concurrency model Reviewed-by: Matt Caswell Reviewed-by: Neil Horman (Merged from https://github.com/openssl/openssl/pull/24971) --- diff --git a/doc/build.info b/doc/build.info index 063caf4459a..edd1f301953 100644 --- a/doc/build.info +++ b/doc/build.info @@ -2331,6 +2331,10 @@ DEPEND[html/man3/SSL_CTX_set_default_passwd_cb.html]=man3/SSL_CTX_set_default_pa GENERATE[html/man3/SSL_CTX_set_default_passwd_cb.html]=man3/SSL_CTX_set_default_passwd_cb.pod DEPEND[man/man3/SSL_CTX_set_default_passwd_cb.3]=man3/SSL_CTX_set_default_passwd_cb.pod GENERATE[man/man3/SSL_CTX_set_default_passwd_cb.3]=man3/SSL_CTX_set_default_passwd_cb.pod +DEPEND[html/man3/SSL_CTX_set_domain_flags.html]=man3/SSL_CTX_set_domain_flags.pod +GENERATE[html/man3/SSL_CTX_set_domain_flags.html]=man3/SSL_CTX_set_domain_flags.pod +DEPEND[man/man3/SSL_CTX_set_domain_flags.3]=man3/SSL_CTX_set_domain_flags.pod +GENERATE[man/man3/SSL_CTX_set_domain_flags.3]=man3/SSL_CTX_set_domain_flags.pod DEPEND[html/man3/SSL_CTX_set_generate_session_id.html]=man3/SSL_CTX_set_generate_session_id.pod GENERATE[html/man3/SSL_CTX_set_generate_session_id.html]=man3/SSL_CTX_set_generate_session_id.pod DEPEND[man/man3/SSL_CTX_set_generate_session_id.3]=man3/SSL_CTX_set_generate_session_id.pod @@ -3594,6 +3598,7 @@ html/man3/SSL_CTX_set_client_hello_cb.html \ html/man3/SSL_CTX_set_ct_validation_callback.html \ html/man3/SSL_CTX_set_ctlog_list_file.html \ html/man3/SSL_CTX_set_default_passwd_cb.html \ +html/man3/SSL_CTX_set_domain_flags.html \ html/man3/SSL_CTX_set_generate_session_id.html \ html/man3/SSL_CTX_set_info_callback.html \ html/man3/SSL_CTX_set_keylog_callback.html \ @@ -4259,6 +4264,7 @@ man/man3/SSL_CTX_set_client_hello_cb.3 \ man/man3/SSL_CTX_set_ct_validation_callback.3 \ man/man3/SSL_CTX_set_ctlog_list_file.3 \ man/man3/SSL_CTX_set_default_passwd_cb.3 \ +man/man3/SSL_CTX_set_domain_flags.3 \ man/man3/SSL_CTX_set_generate_session_id.3 \ man/man3/SSL_CTX_set_info_callback.3 \ man/man3/SSL_CTX_set_keylog_callback.3 \ @@ -4913,6 +4919,10 @@ DEPEND[html/man7/openssl-qlog.html]=man7/openssl-qlog.pod GENERATE[html/man7/openssl-qlog.html]=man7/openssl-qlog.pod DEPEND[man/man7/openssl-qlog.7]=man7/openssl-qlog.pod GENERATE[man/man7/openssl-qlog.7]=man7/openssl-qlog.pod +DEPEND[html/man7/openssl-quic-concurrency.html]=man7/openssl-quic-concurrency.pod +GENERATE[html/man7/openssl-quic-concurrency.html]=man7/openssl-quic-concurrency.pod +DEPEND[man/man7/openssl-quic-concurrency.7]=man7/openssl-quic-concurrency.pod +GENERATE[man/man7/openssl-quic-concurrency.7]=man7/openssl-quic-concurrency.pod DEPEND[html/man7/openssl-quic.html]=man7/openssl-quic.pod GENERATE[html/man7/openssl-quic.html]=man7/openssl-quic.pod DEPEND[man/man7/openssl-quic.7]=man7/openssl-quic.pod @@ -5182,6 +5192,7 @@ html/man7/openssl-core_names.h.html \ html/man7/openssl-env.html \ html/man7/openssl-glossary.html \ html/man7/openssl-qlog.html \ +html/man7/openssl-quic-concurrency.html \ html/man7/openssl-quic.html \ html/man7/openssl-threads.html \ html/man7/openssl_user_macros.html \ @@ -5329,6 +5340,7 @@ man/man7/openssl-core_names.h.7 \ man/man7/openssl-env.7 \ man/man7/openssl-glossary.7 \ man/man7/openssl-qlog.7 \ +man/man7/openssl-quic-concurrency.7 \ man/man7/openssl-quic.7 \ man/man7/openssl-threads.7 \ man/man7/openssl_user_macros.7 \ diff --git a/doc/man3/SSL_CTX_set_domain_flags.pod b/doc/man3/SSL_CTX_set_domain_flags.pod new file mode 100644 index 00000000000..8535f958214 --- /dev/null +++ b/doc/man3/SSL_CTX_set_domain_flags.pod @@ -0,0 +1,114 @@ +=pod + +=head1 NAME + +SSL_CTX_set_domain_flags, SSL_CTX_get_domain_flags, SSL_get_domain_flags, +SSL_DOMAIN_FLAG_SINGLE_THREAD, +SSL_DOMAIN_FLAG_MULTI_THREAD, +SSL_DOMAIN_FLAG_THREAD_ASSISTED, +SSL_DOMAIN_FLAG_BLOCKING, +SSL_DOMAIN_FLAG_LEGACY_BLOCKING +- control the concurrency model used by a QUIC domain + +=head1 SYNOPSIS + + #include + + #define SSL_DOMAIN_FLAG_SINGLE_THREAD + #define SSL_DOMAIN_FLAG_MULTI_THREAD + #define SSL_DOMAIN_FLAG_LEGACY_BLOCKING + #define SSL_DOMAIN_FLAG_BLOCKING + #define SSL_DOMAIN_FLAG_THREAD_ASSISTED + + int SSL_CTX_set_domain_flags(SSL_CTX *ctx, uint64_t flags); + int SSL_CTX_get_domain_flags(SSL_CTX *ctx, uint64_t *flags); + + int SSL_get_domain_flags(SSL *ssl, uint64_t *flags); + +=head1 DESCRIPTION + +SSL_CTX_set_domain_flags() and SSL_CTX_get_domain_flags() set and get the QUIC +domain flags on a B using a QUIC B. These flags determine +the concurrency model which is used for a QUIC domain. A detailed introduction +to these concepts can be found in L. + +The flags comprise zero or more of the following flags: + +=over 4 + +=item B + +Specifying this flag configures the Single-Threaded Concurrency Model (SCM). + +=item B + +Speciyfing this flag configures the Contentive Concurrency Model (CCM) (unless +B is also specified). + +=item B + +Specifying this flag configures the Thread-Assisted Concurrency Modle (TACM). +It implies B. + +=item B + +Enable reliable support for blocking I/O calls, allocating whatever OS resources +are necessary to realise this. If this flag is specified, +B is ignored. + +=item B + +Enables legacy blocking compatibility mode. See +L. + +=back + +Mutually exclusive flag combinations result in an error (for example, combining +B and B). + +Because exactly one concurrency model must be chosen, the domain flags cannot be +set to 0 and attempting to do so will result in an error. + +Changing these flags using SSL_CTX_set_domain_flags() has no effect on QUIC +domains which have already been created. + +The default set of domain flags set on a newly created B may vary by +OpenSSL version, chosen B, and operating environment. See +L for details. An application can retrieve the +default domain flags by calling SSL_CTX_get_domain_flags() immediately after +constructing a B. + +SSL_get_domain_flags() retrieves the domain flags which are effective for a QUIC +domain when called on any QUIC SSL object under that domain. + +=head1 RETURN VALUES + +SSL_CTX_set_domain_flags(), SSL_CTX_get_domain_flags() and +SSL_get_domain_flags() return 1 on success and 0 on failure. + +SSL_CTX_set_domain_flags() fails if called with a set of flags which are +inconsistent or which cannot be supported given the current environment. + +SSL_CTX_set_domain_flags() and SSL_CTX_get_domain_flags() fail if called on a +B which is not using a QUIC B. + +SSL_get_domain_flags() fails if called on a non-QUIC SSL object. + +=head1 SEE ALSO + +L, L + +=head1 HISTORY + +These functions were added in @QUIC_SERVER_VERSION@. + +=head1 COPYRIGHT + +Copyright 2024 The OpenSSL Project Authors. All Rights Reserved. + +Licensed under the Apache License 2.0 (the "License"). You may not use +this file except in compliance with the License. You can obtain a copy +in the file LICENSE in the source distribution or at +L. + +=cut diff --git a/doc/man3/SSL_new_domain.pod b/doc/man3/SSL_new_domain.pod index 7a2233b0f25..41c3f391b7e 100644 --- a/doc/man3/SSL_new_domain.pod +++ b/doc/man3/SSL_new_domain.pod @@ -4,42 +4,32 @@ SSL_new_domain, SSL_is_domain, -SSL_get0_domain, -SSL_get_domain_flags, -SSL_CTX_set_domain_flags, -SSL_CTX_get_domain_flags, -SSL_DOMAIN_FLAG_SINGLE_THREAD, -SSL_DOMAIN_FLAG_MULTI_THREAD, -SSL_DOMAIN_FLAG_THREAD_ASSISTED, -SSL_DOMAIN_FLAG_BLOCKING, -SSL_DOMAIN_FLAG_LEGACY_BLOCKING - SSL object interface for managing QUIC event domains +SSL_get0_domain +- SSL object interface for managing QUIC event domains =head1 SYNOPSIS #include - int SSL_CTX_set_domain_flags(SSL_CTX *ctx, uint64_t domain_flags); - int SSL_CTX_get_domain_flags(SSL_CTX *ctx, uint64_t *domain_flags); - SSL *SSL_new_domain(SSL_CTX *ctx, uint64_t flags); int SSL_is_domain(SSL *ssl); SSL *SSL_get0_domain(SSL *ssl); - int SSL_get_domain_flags(SSL *ssl, uint64_t *domain_flags); - - #define SSL_DOMAIN_FLAG_SINGLE_THREAD - #define SSL_DOMAIN_FLAG_MULTI_THREAD - #define SSL_DOMAIN_FLAG_THREAD_ASSISTED - #define SSL_DOMAIN_FLAG_BLOCKING - #define SSL_DOMAIN_FLAG_LEGACY_BLOCKING - =head1 DESCRIPTION The SSL_new_domain() function creates a new QUIC event domain, represented as an SSL object. This is known as a QUIC domain SSL object (QDSO). The concept of a QUIC event domain is discussed in detail in L. +The I argument to SSL_new_domain() accepts a set of domain flags. If the +I argument to SSL_new_domain() does not specify one of the flags +B, B or +B, the domain flags configured on the +B are used as a default. Otherwise, the domain flags in I +are used. See L for details of the available domain +flags and how they can be configured on a B. + A QUIC domain SSL object can be managed in the same way as any other SSL object, in that it can be refcounted and freed normally. A QUIC domain SSL object is the parent of a number of child objects such as QUIC listener SSL objects. Once a @@ -51,37 +41,9 @@ SSL_is_domain() returns 1 if a SSL object is a QUIC domain SSL object. SSL_get0_domain() obtains a pointer to the QUIC domain SSL object in a SSL object hierarchy (if any). -=head2 Domain Flags - -Domain flags are a set of flags which configure the concurrency model used by -the OpenSSL QUIC implementation. For more information, including the meanings of -the B values, see L. - -SSL_CTX_set_domain_flags() and SSL_CTX_get_domain_flags() can be used to set and -get the domain flags configured on a B which is using a QUIC -B. - -A default set of domain flags are configured on a B when it is -constructed, and these can be determined by calling SSL_CTX_get_domain_flags() -immediately after constructing a B. - -The flags configured on a B are used when an SSL object (such as a QUIC -domain SSL object) is created from an B. Subsequent to the domain flags -on a B do not affect SSL objects which have already been created. - -When calling SSL_new_domain(), if the I argument to -SSL_new_domain() does not specify one of the flags -B, B or -B, the domain flags configured on the -B are used as a default. Otherwise, the domain flags in I -are used. - All SSL objects in a QUIC event domain use the same domain flags, and the domain flags for a QUIC domain cannot be changed after construction. -SSL_get_domain_flags() outputs the effective domain flags being used on any -given QUIC SSL object. - =head2 Supported Operations A QUIC domain SSL object exists to contain other QUIC SSL objects and provide @@ -137,8 +99,8 @@ on which it is called) or NULL. =head1 SEE ALSO -L, L, -L +L L, +L, L =head1 HISTORY diff --git a/doc/man7/openssl-quic-concurrency.pod b/doc/man7/openssl-quic-concurrency.pod new file mode 100644 index 00000000000..ef79eff8127 --- /dev/null +++ b/doc/man7/openssl-quic-concurrency.pod @@ -0,0 +1,326 @@ +=pod + +=head1 NAME + +openssl-quic-concurrency - OpenSSL QUIC Concurrency Model + +=head1 DESCRIPTION + +A QUIC domain is a group of QUIC resources such as listeners (see +L) and connections which share common event processing +resources, such as internal pollers, timers and locks. All usage of OpenSSL QUIC +happens inside a QUIC domain. + +These resources can be accessed and used concurrently depending on the +circumstances. This man page discusses the available concurrency models and how +they can be used. + +=head1 EXPLICIT AND IMPLICIT QUIC DOMAINS + +A QUIC domain is instantiated either explicitly (L) or +implicitly by calling L or L: + +=over 4 + +=item + +An explicit QUIC domain is created by and visible to the application and has +other QUIC SSL objects created underneath it, such as listeners or connections. + +=item + +An implicit QUIC domain is one which is created internally due to the direct +creation of a QUIC connection or listener SSL object; the application does not +explicitly create a QUIC domain SSL object and never directly references the +domain. + +=back + +Explicit creation of a QUIC domain provides the greatest level of control for an +application. Applications can use an implicit QUIC domain for ease of use and to +avoid needing to create a separate QUIC domain SSL object. + +Regardless of whether a QUIC domain is explicitly created, the internal +processing model is the same and the application must choose an appropriate +concurrency model as discussed below. + +=head1 CONCURRENCY MODELS + +The OpenSSL QUIC implementation supports multiple concurrency models to support +a wide variety of usage scenarios. + +The available concurrency models are as follows: + +=over 4 + +=item * + +The B, which supports only +application-synchronised single-threaded usage. + +=item * + +The B, which supports multi-threaded usage. + +=item * + +The B, which also supports +multi-threaded usage and provides assistance to an application for handling QUIC +timer events. + +=back + +The merits of these models are as follows: + +=over 4 + +=item * + +The B performs no locking or +synchronisation. It is entirely up to the application to synchronise access to +the QUIC domain and its subsidary SSL objects. + +This concurrency model is also useful for an application which wants to use the +OpenSSL QUIC implementation as a pure state machine. + +=item * + +The B performs automatic locking when making +API calls to SSL objects in a QUIC domain. This provides automatic +synchronisation for multi-threaded usage of QUIC objects. For example, different +QUIC stream SSL objects in the same QUIC connection can be safely accessed from +different threads. + +This concurrency model adds the overhead of locking over the Single-Threaded +Concurrency Model in order to support multi-threaded usage, but provides limited +performance in highly contended multi-threaded usage due to its simple approach. +However, it may still prove a good solution for a broad class of applications +which spend the majority of their time in application logic and not in QUIC I/O +processing. + +An advantage of this model relative to the more sophisticated concurrency models +below is that it does not create any OS threads. + +=item * + +The B is identical to the Contentive +Concurrency Model except that a thread is spun up in the background to ensure +that QUIC timer events are handled in a timely fashion. This ensures that QUIC +timeout events are handled even if an application does not periodically call +into the QUIC domain to ensure that any outstanding QUIC-related timer or +network I/O events are handled. The assist thread contends for the same +resources like any other thread. However, handshake layer events (TLS) are never +processed by the assist thread. + +=back + +The default concurrency model is CCM or TACM, depending on the B +used with a B. Using L results in a default +concurrency model of CCM, whereas using L +results in a default concurrency model of TACM. + +Additional concurrency models may be offered in future releases of OpenSSL. + +=head1 BLOCKING I/O CAPABILITIES + +All of the supported concurrency models are capable of supporting blocking I/O +calls, where application-level I/O calls (for example, to L or +L on a QUIC stream SSL object) block until the request can be +serviced. This includes the use of L in a blocking fashion. + +Supporting blocking API calls reliably requires the creation of additional OS +resources such as internal file descriptors to allow threads to be woken when +necessary. This creation of internal OS resources is optional and may need to be +explicitly requested by an application depending on the chosen concurrency +model. If this functionality is disabled, depending on the chosen concurrency +model, blocking API calls may not be available and calls to +L attempting to enable blocking mode will fail. + +=head2 Legacy Blocking Support Compatibility + +OpenSSL 3.2 and 3.3 contained a buggy implementation of blocking QUIC I/O calls +which is only reliable under single-threaded usage. This functionality is always +available in the Single-Threaded Concurrency Model, where it works reliably. + +For compatibility reasons, this functionality is also available under the +default concurrency model if the application does not explicitly specify a +concurrency model or disable it. This is known as Legacy Blocking Compatibility +Mode, and its usage is not recommended for multi-threaded applications. + +=head1 RECOMMENDED USAGE + +New applications are advised to choose a concurrency model as follows: + +=over 4 + +=item * + +A purely single-threaded application, or an application which wishes to use +OpenSSL QUIC as a state machine and manage synchronisation itself, should +explicitly select the SCM concurrency model. + +=item * + +An application which wants to engage in multi-threaded usage of different QUIC +connections or streams in the same QUIC domain should a) select the CCM or TACM +concurrency model and b) explicitly opt in or out of blocking I/O support +(depending on whether the application wishes to make blocking I/O calls), +disabling Legacy Blocking Compatibility Mode. + +An application should select the CCM concurrency model if the application can +guarantee that a QUIC domain will be serviced regularly (for example, because +the application can guarantee that the timeout returned by +L will be handled). If an application is unable to do +this, it should select the TACM concurrency model. + +=item * + +Applications should explicitly configure a concurrency model during +initialisation. + +=back + +=head1 CONFIGURING A CONCURRENCY MODEL + +If using an explicit QUIC domain, a concurrency model is chosen when calling +L by specifying zero or more of the following flags: + +=over 4 + +=item B + +Specifying this flag configures the Single-Threaded Concurrency Model (SCM). + +=item B + +Speciyfing this flag configures the Contentive Concurrency Model (CCM) (unless +B is also specified). + +=item B + +Specifying this flag configures the Thread-Assisted Concurrency Modle (TACM). +It implies B. + +=item B + +Enable reliable support for blocking I/O calls, allocating whatever OS resources +are necessary to realise this. If this flag is specified, +B is ignored. + +Details on the allocated OS resources can be found under L below. + +=item B + +Enables legacy blocking compatibility mode. See L. + +=back + +Mutually exclusive flag combinations result in an error (for example, combining +B and B). + +The concurrency model for a domain cannot be changed after the domain is +created. + +=head2 Default Behaviour + +If none of the above flags are provided to L or another +function which can accept the above flags, the default concurrency model set on +the B is used. This default can be set and get using +L and L. + +The default concurrency model set on a newly created B is determined as +follows: + +=over 4 + +=item * + +If an B of L is used, the +Thread-Assisted Concurrency Model (TACM) is used with the +B flag. This provides reliable blocking functionality. + +=item * + +Otherwise, if OpenSSL was built without threading support, the Single-Threaded +Concurrency Model (SCM) is used, with the B +flag. + +=item * + +Otherwise, if an B of L is used, the +Contentive Concurrency Model (CCM) is used with th +B flag. + +=item * + +Otherwise, the Contentive Concurrency Model (CCM) is used. + +=back + +The default concurrency model may vary between releases of OpenSSL. An +application may specify one of more of the domain flags above to ensure +consistent usage of a specific concurrency model between releases. + +=head2 Configuration of Concurrency Models with Implicit QUIC Domains + +If an explicit QUIC domain is not explicitly created using L, +an implicit QUIC domain is created when calling L or +L. Such a domain will use the default domain flags configured on the +B as described above. + +=head1 CONSUMPTION OF OS RESOURCES + +If full blocking I/O support is selected using B, at +least one socket, socket-like OS handle or file descriptor must be allocated to +allow one thread to wake other threads which may be blocking in calls to OS +socket polling interfaces such as select(2) or poll(2). This is allocated +automatically internally by OpenSSL. + +If the Thread-Assisted Concurrency Model (TACM) is selected, a background thread +is spawned. This also implies B and the above. + +The internal consumption by OpenSSL of mutexes, condition variables, spin locks +or other similar thread synchronisation primitives is unspecified under all +concurrency models. + +=head1 BEHAVIOUR OF SSL OBJECTS + +A QUIC SSL object has blocking mode enabled by default where B of the +following criteria are met: + +=over 4 + +=item * + +B or B is enabled; +and + +=item * + +The QUIC connection is being used with network read and write BIOs which expose +supported poll descriptors. See L for details. + +=back + +In all other cases, a QUIC SSL object has blocking mode disabled by default. The +blocking mode can be changed explicitly using L. + +=head1 SEE ALSO + +L, L, L, +L, +L, +L + +=head1 COPYRIGHT + +Copyright 2024 The OpenSSL Project Authors. All Rights Reserved. + +Licensed under the Apache License 2.0 (the "License"). You may not use +this file except in compliance with the License. You can obtain a copy +in the file LICENSE in the source distribution or at +L. + +=cut