Fix 32-bit Windows issues related to QUIC_ACKM / QUIC_CC

The re-occuring surprise is that in Win32, size_t is 32 bits...
Fixed by changing size_t to uint64_t in QUIC_CC

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19345)

Fix definitions of SHUT_RD and SHUT_WR

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19345)

Github Actions: Enable building QUIC on Windows

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19345)

Fix a lock in provider_remove_store_methods()

We were taking a read lock. It should have been a write lock.

Fixes #19474

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19481)

Fix test_tls13_encryption()

This test was disabled during the record write record layer refactor.
We can now enable it again.

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19470)

Fix a compilation failure in bio_comp_test.c

Compiling with clang, --strict-warnings and enable-zlib-dynamic resulted
in a compilation failure. This fixes it.

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/19480)

openssl list: Fix help text about -cipher-algorithms option

Fixes openssl#19133

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19370)

Workaround egd rand source deficiencies

With egd as the rand source the reseed after fork confuses the egd.

Fixes #19396

Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19454)

(cherry picked from commit 0b3fec502298f9a16889d2507948a916416e71cf)

Checking the return of BIO_new_fp(). If it returns NULL, then it is unnecessary to build the BIO chain and better make the caller directly return NULL

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19445)

doc: fix copy/paste error

Fixes #19460

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/19461)

Fix coverity 1516101 deadcode

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19464)

crypto: thread: remove ossl_crypto_thread_native_terminate

Signed-off-by: Čestmír Kalina <ckalina@redhat.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19473)

Errors raised from OPENSSL_sk_set should have ERR_LIB_CRYPTO

Reviewed-by: Todd Short <todd.short@me.com>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19400)

Document the stack functions that are forgiving

I.e., those that can be called with NULL stack parameter or invalid index.

Reviewed-by: Todd Short <todd.short@me.com>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19400)

stack: Do not add error if pop/shift/value accesses outside of the stack

This partially reverts commit 30eba7f35983a917f1007bce45040c0af3442e42.
This is legitimate use of the stack functions and no error
should be reported apart from the NULL return value.

Fixes #19389

Reviewed-by: Todd Short <todd.short@me.com>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19400)

Remove changes entry for RIPEMD160 in 3.2

It is already in 3.0.7.

Reviewed-by: Todd Short <todd.short@me.com>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19438)

Add changes entry for RIPEMD160 in 3.0.7

Reviewed-by: Todd Short <todd.short@me.com>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19438)

Replace BIO_free(bio_err) with BIO_free_all(bio_err)

dup_bio_err() can return a BIO chain when 'OPENSSL_SYS_VMS' is defined.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19446)

test: driver: fix -Wunused-but-set-variable

The value of 'num_failed_inner' isn't ever used.

Fixes this error with Clang 15:
```
test/testutil/driver.c:341:17: error: variable 'num_failed_inner' set but not used [-Werror,-Wunused-but-set-variable]
            int num_failed_inner = 0;
                ^
1 error generated.
```

Signed-off-by: Sam James <sam@gentoo.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19450)

x509: fix -Wunused-but-set-variable

The value of 'l' isn't ever actually used.

Fixes this error with Clang 15:
```
crypto/x509/x_name.c:506:9: error: variable 'l' set but not used [-Werror,-Wunused-but-set-variable]
    int l, i;
        ^
1 error generated.
```

Signed-off-by: Sam James <sam@gentoo.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19450)

txt_db: fix -Wunused-but-set-variable

The loop never uses the value of 'ln'.

Fixes this error with Clang 15:
```
crypto/txt_db/txt_db.c:24:10: error: variable 'ln' set but not used [-Werror,-Wunused-but-set-variable]
    long ln = 0;
         ^
1 error generated.
```

Signed-off-by: Sam James <sam@gentoo.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19450)

pem: fix -Wunused-but-set-variable

The loop never uses the value of 'line'.

Fixes this error with Clang 15:
```
crypto/pem/pem_lib.c:821:14: error: variable 'line' set but not used [-Werror,-Wunused-but-set-variable]
    int len, line, ret = 0, end = 0, prev_partial_line_read = 0, partial_line_read = 0;
             ^
1 error generated.
```

Signed-off-by: Sam James <sam@gentoo.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19450)

CI: add Clang 15

We have to use the PPA provided by LLVM because Clang 15 isn't
officially part of Ubuntu 22.04 (or any other Ubuntu release yet),
see https://apt.llvm.org/ for details.

Signed-off-by: Sam James <sam@gentoo.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19450)

Add af_alg errors to the error queue

If the kernel operation failed the EVP functions
just returned without any error message.

This commit adds them.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19289)

test: threads: replace test_thread_noreturn

While POSIX threads are cancellable and may be asynchronously cancelled,
their cancellation is not guaranteed by the POSIX standard.

test_thread_noreturn, which simulates a long-running possibly
unresponsive thread:

THREAD #1 THREAD #2
LOCK L1
SPAWN #2
LOCK L1

On MacOS, cancelling such thread only queues cancellation request, but
the following pthread_join hangs.

Replace this implementation by an unbounded sequence of sleeps instead.

Signed-off-by: Čestmír Kalina <ckalina@redhat.com>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19433)

crypto: thread: serialize concurrent joins

Multiple concurrent joins with a running thread suffer from a race
condition that allows concurrent join calls to perform concurrent arch
specific join calls, which is UB on POSIX, or to concurrently execute
join and terminate calls.

As soon as a thread T1 exists, one of the threads that joins with T1
is selected to perform the join, the remaining ones await completion.
Once completed, the remaining calls immediately return. If the join
failed, another thread is selected to attempt the join operation.

Forcefully terminating a thread that is in the process of joining
another thread is not supported.

Common code from thread_posix and thread_win was refactored to use
common wrapper that handles synchronization.

Signed-off-by: Čestmír Kalina <ckalina@redhat.com>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19433)

openssl list: add an empty row at the end of each printed list of commands and algorithms

Fixes openssl#19140

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19372)

Fix coverity 1516095 deadcode

Unless multiple compression algorithms are configured, test 3 is
not run, so anything looking at `test == 3` is considered dead code.

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19440)

Fix coverity 1516094 uninit

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19440)

Fix coverity 1516093 tainted scalar

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19440)

update pyca cryptography to 38.0.2

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19439)

link the pyca tests against the correct openssl

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19439)

Fix make update

The recent DTLS write record layer code and the certificate compression
code both added new SSL_R_ reason codes. The numbers are conflicting due
to rebase issues and causing make update to fail.

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19457)

Finer grained error records for provider load/init failures

When a provider is activated, these three cases would record that the
provider init function failed (implying that it was called):

-   failure to load the provider module (in case it's a dynamically
    loadable module)
-   the init function not being present (i.e. being NULL)
-   the init function being called and returning an error indication
    (i.e. returning a false value)

This is confusing.

Separating the three cases so that they record different errors will
make it easier to determine causes of failure.

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19419)

Fix no longer implicitly refresh the cached TBSCertificate

This reverts commit 9249a34b076df9a9d55ab74ab465d336980cae6a.
Fixes #19388

Reviewed-by: Todd Short <todd.short@me.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19393)

Move freeing of BIOs as late as possible

Calling SSL_free() will call BIO_free_all() on the rbio and wbio. We
keep references to the rbio and wbio inside the record layer object.
References to that object are held directly, as well as in fragment
retransmission queues. We need to ensure all record layer objects are
cleaned up before we call BIO_free_all() on rbio/wbio - otherwise the
"top" BIO may not have its reference count drop to 0 when BIO_free_all()
is called. This means that the rest of the BIOs in the chain don't get
freed and a memory leak can occur.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19424)

Remove some redundant code

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19424)

Remove the old buffer management code

We no longer use the old buffer management code now that it has all been
moved to the new record layer.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19424)

Remove some TODO(RECLAYER) comments now that DTLS has been moved

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19424)

Remove dtls_write_records

The dtls_write_records function, after the previous series of commits,
was functionally equivalent to tls_write_records_default - so it can be
removed completely.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19424)

Move sequence increment to post encryption processing

This change make dtls_write_records virtuall the same as
tls_write_records_default, which will enable us to merge them in a
subsequent commit.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19424)

Remove supurious set of the record type

We already set the record type on the SSL3_RECORD structure. We don't
need to do it again (inconsistently).

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19424)

Consolidate sequence counter incrementing code

The sequence counter was incremented in numerous different ways in
numerous different locations. We introduce a single function to do this
inside the record layer.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19424)

Ensure the record layer is responsible for calculating record overheads

Don't calculate the potential record layer expansion outside of the
record layer. We move some code that was doing that into the record
layer.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19424)

Convert dtls_write_records to use standard record layer functions

We have standard functions for most of the work that dtls_write_records
does - so we convert it to use those functions instead.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19424)

Start using WPACKET in the dtls write records code

Previously this was writing to the buffers directly. We use the safer
WPACKET instead

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19424)

Use common tls_write_records() even for DTLS

In practice this just means have a DTLS specific write_records that the
common tls_write_records() just calls. We also replace the use of
ssl3_write_pending() with tls_retry_write_records().

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19424)

Convert dtls_write_records() to return the correct return values

We now use standard record layer return values for this function. We
also convert the code to use RLAYERfatal instead of SSLfatal.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19424)

Use record layer buffers for DTLS rather than the buffers in s->rlayer

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19424)

Move dlts_write_records() function in the record layer

At the this stage we just move the code and don't restructure it to do it
the record layer way yet.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19424)

Create a dlts_write_records() function

In preparation for moving the DTLS code to use the new write record layer
architecture we first restructure the code to create a dtls_write_records()
function that mirrors the functionality that the record layer will provide.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19424)

Remove create_empty_fragment from do_dtls1_write()

do_dtls1_write() was never called with a value for create_empty_fragment
that was ever non-zero - so this is dead code and can be removed. The
equivalent code in the TLS processing is used for TLS1.0/SSLv3 to protect
against known IV weaknesses because those protocol versions do not have
an explicit IV. However DTLS1.0 is based on TLSv1.1 and *does* have an
explicit IV - so this is not useful there.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19424)

c_rehash: Fix file extension matching

For some reason, parenthesis were added 8 years ago in commit
a787c2590e468585a1a19738e0c7f481ec91b762. This essentially removed the
\. and $ constructs from the middle branches. Hence a file called e.g.
cert.key would accidentally match the (cer) rule.

CLA: trivial

Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19427)

add a check for the return of sk_SRP_gN_new_null() so that capture the potential memory error in time

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19435)

ripemd: document as being present in the default provider

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19375)

Avoid putting ripemd_prov.c in libcommon otherwise it is regarded as fips source

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19375)

default provider: include RIPEMD160

Including RIPEMD160 in both the default and legacy providers shouldn't break
anyone and makes the algorithm available more readily.

Fixes #17722

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19375)

Add changes entry for RIPEMD160 being added to the default provider

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19375)

Add `for_comp` flag when retrieving certs for compression

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18186)

Update COMP_METHOD

size_t-ify the COMP_METHOD structure and functions.
Get rid of the non-functional COMP_METHODS and return NULL instead.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18186)

Add CI to build with brotli and zstd

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18186)

Add support for compressed certificates (RFC8879)

* Compressed Certificate extension (server/client)
* Server certificates (send/receive)
* Client certificate (send/receive)

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18186)

Convert ZLIB defines to OPENSSL_NO_ZLIB

Use the normal OPENSSL_NO_ prefix to enable/disable ZLIB
Make `BIO_f_zlib()` always available.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18186)

Add ZSTD compression support (RFC8478bis)

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18186)

Add brotli compression support (RFC7924)

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18186)

Fix sctp compile errors

Fixes #19371

running config with 'enable-sctp' gave compiler errors.

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19398)

Fix documentation for OFB/OCB in the FIPS provider

CLA: trivial

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19408)

Fix typos in doc/designs/ddd/README.md

CLA: trivial

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19416)

Add missing include for DH_get0_priv_key()

Fixes #19410

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/19411)

crypto: add preemptive threading support

Some primitives are designed to be used in a multi-threaded environment,
if supported, e.g., Argon2.

This patch adds support for preemptive threading and basic synchronization
primitives for platforms compliant with POSIX threads or Windows CRT.
Native functions are wrapped to provide a common (internal) API.

Threading support can be disabled at compile time. If enabled, threading
is disabled by default and needs to be explicitly enabled by the user.

Thread enablement requires an explicit limit on the number of threads that
OpenSSL may spawn (non-negative integer/infinity). The limit may be changed.

Signed-off-by: Čestmír Kalina <ckalina@redhat.com>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12255)

ci: add thread-pool and default-thread-pool

Signed-off-by: Čestmír Kalina <ckalina@redhat.com>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12255)

Configure: add thread-pool and default-thread-pool

Signed-off-by: Čestmír Kalina <ckalina@redhat.com>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12255)

Coverity 1515953: negative loop bound

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/19413)

Fix LLVM vs Apple LLVM version numbering confusion, for $avx512ifma

Apple LLVM has a different version numbering scheme than upstream LLVM.
That makes for quite a bit of confusion.

https://en.wikipedia.org/wiki/Xcode#Toolchain_versions to the rescue,
they have collected quite a lot of useful data.

This change is concentrated around the `$avx512ifma` flag

Fixes #16670 for the master branch

Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19361)

Add translation for ECX group parameter

Legacy EVP_PKEY_CTX objects did not support the "group" parameter for X25519
and X448.  The translation of this parameter resulted in an error.  This
caused errors for legacy keys and engines.

Fix this situation by adding a translation that simply checks that the correct
parameter is to be set, but does not actually set anything.  This is correct
since the group name is anyway optional for these two curves.

Fixes #19313

Signed-off-by: Juergen Christ <jchrist@linux.ibm.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19348)

Fix an occasional CI failure due to unaligned access

This happens rarely, but only because very few CI runs
use the exotic CPU type that is necessary to execute
anything within rsaz_exp_x2.c and enable UBSAN at the same time.

crypto/bn/rsaz_exp_x2.c:562:20: runtime error: load of misaligned address 0x612000022cc6 for type 'uint64_t' (aka 'unsigned long'), which requires 8 byte alignment
0x612000022cc6: note: pointer points here
 84 a3 78 e0 8e 8d  4a a5 51 9c 57 d0 d6 41  f3 26 d1 4e e1 98 42 b5  3a 9f 04 f1 73 d2 1d bf  73 44
             ^
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior crypto/bn/rsaz_exp_x2.c:562:20 in
../../util/wrap.pl ../../fuzz/server-test ../../fuzz/corpora/server => 1
not ok 2 - Fuzzing server

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19394)

Improve performance of the encoder collection

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19344)

PKCS12_SAFEBAG_set0_attrs: Remove const from function signature

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19359)

Fix typo in PKCS12_SAFEBAG_set0_attrs

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19359)

optimize ossl_sm4_set_key speed

this optimization comes from libgcrypt, increse about 48% speed

Benchmark on my AMD Ryzen Threadripper 3990X

before:
Did 5752000 SM4 setup operations in 1000151us (5751131.6 ops/sec)
after:
Did 8506000 SM4 setup operations in 1000023us (8505804.4 ops/sec)

Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19270)

Update CHANGES.md and NEWS.md for new release

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19379)

Add a test for TLSv1.3 only client sending a correct key_share

Make sure that a TLSv1.3 only client does not send a TLSv1.3 key_share.

Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/19317)

Add a test for where a client sends a non-TLSv1.3 key share

This should not happen but we should tolerate and send an HRR

Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/19317)

Ensure that the key share group is allowed for our protocol version

We should never send or accept a key share group that is not in the
supported groups list or a group that isn't suitable for use in TLSv1.3

Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/19317)

Fix a return value in tls_default_read_n

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19343)

Introduce a step to prepare the BIO before writing

This removes some KTLS specific code from tls_retry_write_records().

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19343)

Remove ktls specific code from tls_setup_write_buffers

The KTLS code no longer calls this function so this is not necessary.

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19343)

Add a post encryption processing step

For example in this we add the MAC if we are doing encrypt-then-mac.

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19343)

Add a prepare for encryption step

This applies any mac that might be necessary, ensures that we have
enough space in the WPACKET to perform the encryption and sets up the
SSL3_RECORD ready for that encryption.

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19343)

Move record padding out of tls_common.c

Only tls13_meth.c needs to handle adding record padding. All other
*_meth.c files can ignore it.

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19343)

Don't check whether we are using KTLS before calling the cipher function

The KTLS cipher function is a no-op so it doesn't matter if we call it.
We shouldn't special case KTLS in tls_common.c

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19343)

Defer record header preparation to the protocol methods

We introduce a new function to prepare the record header. KTLS has its own
version since this is done by the kernel.

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19343)

Abstract out the record type processing

Remove TLSv1.3 specific processing of the record type out of tls_common.c
and into tls13_meth.c

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19343)

Defer write buffer and WPACKET allocation/initialisation to protocol code

We move some protocol specific code for write buffer and WPACKET allocation
and initialisation out of tls_common.c and into the protocol specific files.

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19343)

Remove an old TODO from the code

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19343)

Use the configured max_send_fragment value in the write record layer

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19343)

Remove enc_write_state

This field was used to track whether a cipher ctx was valid for writing
or not, and also whether we should write out plaintext alerts. With the new
record layer design we no longer need to track whether a cipher ctx is valid
since the whole record layer will be aborted if it is not. Also we have a
different mechanism for tracking whether we should write out plaintext
alerts. Therefore this field is removed from the SSL object.

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19343)

Remove some unneeded usage of the SSL_CONNECTION object

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19343)

Remove most of the DTLS special casing from the write record code

Most of this was unnecessary anyway since DTLS isn't using these codepaths.

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19343)

Convert the TLSv1.3 crypto code to the new write record layer

We also clean up some of the KTLS code while we are doing it now that all
users of KTLS have been moved to the new write record layer.

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19343)