Add generic secret skeymgmt provider, skey abstraction and default skeymgmt

This commits adds an actual skey wrapper structure and skeymgmt
implementation for the default provider

This allows to use fallbacks for any SKEY operation,
and to use it for keys that do not have a specific purpose and
cipher-suite associated to it.

Add a test with a key type that does not have skey support (DES),
to show that the fallback works.

Add raw skey test

Signed-off-by: Simo Sorce <simo@redhat.com>
Signed-off-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/26753)

EVP_SKEY documentation

Signed-off-by: Dmitry Belyavskiy <beldmit@gmail.com>
Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26753)

EVP_SKEY tests

Signed-off-by: Dmitry Belyavskiy <beldmit@gmail.com>
Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26753)

EVP_SKEY implementation for EVP_CIPHER

Signed-off-by: Dmitry Belyavskiy <beldmit@gmail.com>
Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26753)

Design document of using opaque object as symmetric key

Signed-off-by: Dmitry Belyavskiy <beldmit@gmail.com>
Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26753)

Add an initial ML-DSA fuzzer

Add an initial version of an ML-DSA fuzzer.  Exercises various ML-DSA
appropriate APIs. Currently it is able to randomly:

1. Attempt to create raw public private keys of various valid and invalid sizes
2. Generate legitimate keys of various sizes using the keygen api
3. Perform sign/verify operations using real generated keys
4. Perform digest sign/verify operations using real generated keys
5. Do an export and import of a key using todata/fromdata
6. Do a comparison of two equal and unequal keys

Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26685)

When we try to get a group, we need session to be valid

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26722)

dgst commandline support for one shot signing algorithms

Reviewed-by: Paul Dale <ppzgs1@gmail.com>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26697)

ML-DSA: Change ossl_ml_dsa_key_public_from_private() to check that the
decoded value of t0 matches the calculated value of t0.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26681)

doc: fix typo/paste error

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tom Cosgrove <tom.cosgrove@arm.com>
(Merged from https://github.com/openssl/openssl/pull/26650)

Log name of provided peer temp keys

Log the peer's temp key name when it is from a provider.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26734)

rand: avoid property query manipulations

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26714)

sslapitest: include hybrid KEM tests with FIPS

Co-Authored-By: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26714)

fips: include ML-KEM capabilities in FIPS provider

Co-Authored-By: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26714)

fips: add hybrid KEM algorithms to the FIPS provider

Co-Authored-By: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26714)

evp: add additional functions for FIPS internal use to support hybrids KEMs

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26714)

evp: include one shot keygen functions for internal FIPS use

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26714)

mlkem: include hybrid KEM algs in FIPS provider

Co-Authored-By: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26714)

mlkem: include hybrid KEYMGMT algs in FIPS provider

Co-Authored-By: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26714)

fips: FIPS ignores property queries for internal algorithm fetches

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26714)

Refactor squeezing out empty tuples

This is more efficient if multiple empty tuples are present, and may
also help to avoid Coverify false positives.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26732)

SPARC assembly: Don't file aes-cbc on T4 with small sizes.

The "openssl speed -testmode -seconds 1 -bytes 1 aes-128-cbc" test
revealed that the assembly code is crashing if length is less than 16.
The code shifts the provided length by 4 and than subtracts one until
the length hits zero. If it was already zero then it underflows the
counter and continues until it segfaults on reading or writing.

Replace the check against 0 with less than 15.

Signed-off-by: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
Reviewed-by: Saša Nedvědický <sashan@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/25637)

LoongArch: Fix output file name detection for Perl scripts

We were using the first (or second) argument containing a '.' as the
output name file, but it may be incorrect as -march=la64v1.0 may be in
the command line.  If the builder specifies -march=la64v1.0 in the
CFLAGS, the script will write to a file named "-march=la64v1.0" and
cause a build error with cryptic message:

    ld: crypto/pem/loader_attic-dso-pvkfmt.o: in function `i2b_PVK':
    .../openssl-3.4.1/crypto/pem/pvkfmt.c:1070:(.text+0x11a8): undefined reference to `OPENSSL_cleanse'

Adapt the approach of ARM and RISC-V (they have similar flags like
-march=v8.1-a or -misa-spec=2.2) to fix the issue.

Signed-off-by: Xi Ruoyao <xry111@xry111.site>
Reviewed-by: Tom Cosgrove <tom.cosgrove@arm.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26717)

ssl_test.c: Allow using DHE with TLS-1.2 and older versions

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26715)

Make test_ssl_new execute tests with fips provider again

This has regressed with
https://github.com/openssl/openssl/pull/24799

The test configs have to be generated differently based
on the fips provider version.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26715)

Left over doc TODOs

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26715)

Post-merge make update

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26715)

Two more private key checks.

- When a PKCS#8 has both seed and key cross check the implicit
  rejection value |z|

- When an import (EVP_PKEY_fromdata call) provides both a private
  and public key, fail if the redundant public key does not match
  the copy in the private key.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26656)

Reject import of private keys that fail PCT

- Also added a provider "validate" method that wraps the PCT test.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26656)

Update corpora to include ml-kem seed corpus

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26657)

Add an inital ML-KEM fuzzer

Add an inital version of an ML-KEM fuzzer.  Exercises various ML-KEM
appropriate apis, as a fuzzer does.  Currently it is able to randomly:

1) Attempt to create raw public private keys of various valid and
   invalid sizes
2) Generate legitimate keys of various sizes using the keygen api

3) Preform encap/decap operations using real generated keys

4) Do a shared secret derivation using 2 keys

5) Do an export and import of a key using todata/fromdata

6) Do a comparison of two equal and unequal keys

Its not much to start, but it should be fairly extensible

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26657)

ASN.1 format tagging seed, key now octet string

- The main ASN.1 private key syntax is the one from Russ Housley's post
  on the LAMPS list, subsequently amended to tag the seed instead of the
  key (each of the three parameter sets will have a fixed size for the
  `expandedKey`):

    ML-DSA-PrivateKey ::= CHOICE {
      seed [0] IMPLICIT OCTET STRING SIZE (64),
      expandedKey OCTET STRING SIZE (1632 | 2400 | 3168)
      both SEQUENCE {
        seed OCTET STRING SIZE (64),
        expandedKey OCTET STRING SIZE (1632 | 2400 | 3168) } }

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26639)

More polish and renamed codec tests

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Paul Dale <ppzgs1@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/26569)

Implement seed/key preference when decoding

- Moved the codec code out of `ml_kem.c` into its own file in
  the provider tree.  Will be easier to share some code with
  ML-DSA, and possible to use PROV_CTX, to do config lookups
  directly in the functions doing the work.

- Update and fixes of the EVP_PKEY-ML-KEM(8) documentation, which
  had accumulated some stale/inaccurate material, and needed new
  text for the "prefer_seed" parameter.

- Test the "prefer_seed=no" behaviour.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Paul Dale <ppzgs1@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/26569)

Fix code comment typo

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Paul Dale <ppzgs1@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/26569)

Check ML-KEM text encoding

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Paul Dale <ppzgs1@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/26569)

Test ML-KEM in FIPS module only when ML-KEM is enabled

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Paul Dale <ppzgs1@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/26569)

ML-KEM encoder and decoder tests

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26512)

ASN.1 ML-KEM private key format

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26512)

Improved import and export

- On import, if a seed is provided, the keys are regenerated.

- The seed is exported as a separate "seed" parameter, when available.
  The "ml-kem.retain_seed" parameter is also exported, when false.

- The seed is optionally dropped after key generation.
    * When the "ml-kem.retain_seed" keygen parameter is set to zero.
    * When the "ml-kem.retain_seed" keygen parameter is not set to 1,
      and the "ml-kem.retain_seed" provider config property is set
      explictly false.

- The exported private key parameter "priv" is always the FIPS 203 |dk|.

- Private key decoding from PKCS#8 produces a transient "seed-only" form
  of the key, in which "retain_seed" is set to false when the
  "ml-kem.retain_seed" provider config property is set explictly false.
  The full key is generated during "load" and the seed is retained
  or not as specified.

- Import honours the "ml-kem.retain_seed" parameter when specified, or
  otherwise honours the provider's "ml-kem.retain_seed" property.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26512)

Zeroise temporary secrets while doing ML-KEM

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26456)

ML-KEM implementation cleanup/speedup

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26341)

Encoders and Decoders for ML-KEM

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26341)

Temporarily disable oqsprovider testing

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26328)

libssl: Accept any key management which implements the group

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26328)

Update oqs-provider to the latest commit

The oqs-provider testing is fixed so it doesn't fetch OpenSSL
implementations during its testing inadvertently.

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26328)

Provide alias names for ML-KEM algorithms without dashes

Fixes #26326

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26328)

test: add fipsinstall corruption tests for KEMs

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Hugo Landau <hlandau@devever.net>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26338)

fips: add pairwise consistency test for ML-KEM key generation

This is mandated by FIPS 140-3 IG 10.3.A resolution 14

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Hugo Landau <hlandau@devever.net>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26338)

self_test: add ML-KEM test description

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Hugo Landau <hlandau@devever.net>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26338)

doc: update documentation now that ML-KEM is in the FIPS provider

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Hugo Landau <hlandau@devever.net>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26338)

doc: remove ML-KEM line because it doesn't have an indicator associated with it

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Hugo Landau <hlandau@devever.net>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26338)

test: run ML-KEM tests for both default and FIPS providers

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Hugo Landau <hlandau@devever.net>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26338)

tests: run ML-KEM tests in FIPS builds too

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Hugo Landau <hlandau@devever.net>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26338)

fips: add ML-KEM to the FIPS provider

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Hugo Landau <hlandau@devever.net>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26338)

fips: defined for ML-KEM CASTs

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Hugo Landau <hlandau@devever.net>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26338)

update build infrastructure for ML-KEM in the FIPS provider

Also avoid a file name conflict when adding ML-KEM to the FIPS provider.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Hugo Landau <hlandau@devever.net>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26338)

Clear param array to ensure it's initialised properly

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Todd Short <todd.short@me.com>
Reviewed-by: Tom Cosgrove <tom.cosgrove@arm.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26339)

ML-KEM hybrids for TLS

- When used as KEMs in TLS the ECDHE algorithms are NOT subjected to
  HPKE Extract/Expand key derivation.  Instead the TLS HKDF is used
  as usual.

- Consequently these KEMs are just the usual ECDHE key exchange
  operations, be it with the encap ECDH private key unavoidably
  ephemeral.

- A new "MLX" KEM provider is added that supports four hybrids of EC/ECX
  DH with ML-KEM:

    * ML-KEM-768 + X25519
    * ML-KEM-1024 + X448
    * P-256 + ML-KEM-768
    * P-384 + ML-KEM-1024

- Support listing of implemented TLS groups.

  The SSL_CTX_get0_implemented_groups() function and new
  `openssl list -tls-groups` and `openssl list -all-tls-groups`
  commands make it possible to determine which groups are
  implemented by the SSL library for a particular TLS version
  or range of versions matching an SSL_CTX.

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26220)

Prepare to detect side-channels in compiled ML-KEM code

Loosely based on similar code in BoringSSL.

Added the valgrind macros necessary to mark secret inputs as uninitialised on
entry to the ML-KEM keygen, encap and decap functions.  The inputs and outputs
are then untagged before control returns to the caller, where, at least in the
case of tests and protocols that check whether the derived keys succeeded in
decoding a key-confirmation message, there will at some point be a branch based
on the *content* of the compute shared secret.

When a build is configured with `-DOPENSSL_CONSTANT_TIME_VALIDATION`, and
various tests that use ML-KEM are run under:

    $ valgrind --tool=memcheck --error-exitcode=1 --exit-on-first-error=yes cmd [args]

any internal secret-data-dependent branches added by a mis-optimising
compiler, or inadvertently introduced into the source code would cause
the tests to fail, exposing the side channel.

Since the side-channels are liable to depend on the compiler and
selected optimisation flags, tests would need to cover a few combinations.

    * clang vs. gcc
    * debug builds
    * default builds
    * -O2
    * -O3 -fno-vectorise (a problem with clang in "clangover")
    * -Os (was a problem with clang in "clangover")
    ...

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26270)

Remaining batch of ML-KEM-related doc updates

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26217)

Initial batch of ML-KEM doc updates.

With the soon-to-be-merged ML-KEM #26172 as the merge base.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26217)

Fix code and docs of pkeyutl en/decapsulation

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26281)

ML-KEM libcrypto implementation polish

* Core ML_KEM constants in new <openssl/ml_kem.h>

* Renamed variant ordinals to ML_KEM_<bits>_VARIANT, freeing
  up the unadorned ML_KEM_<bits> names.

* Fewer/cleaner macros in <crypto/ml_kem.h>

* Fewer/cleaner macros for setting up the ML_KEM_VINFO table.

* Made (d, z) be separate inputs to the now single key generation
  function.  Both or neither have to be NULL.  This supports potential
  future callers that store them in a different order, or in separate
  buffers.

    - Random values are chosen when both are NULL, we never return the
      generated seeds, rather we may, when/if (d, z) private key support
      is added, store these in the expanded key, and make them available
      for import/export.

* No need for a stand-by keygen encoded public key buffer when the
  caller does not provide one (will ask for it later if needed).
  New `hash_h_pubkey` function can compute the public hash from
  the expanded form in constant space (384 bytes for 12-bit encoded
  scalar).

* Simplified code in `scalar_mult`.

* New `scalar_mult_add` adds the product to an existing scalar.
  Used in new `matrix_mult_transpose_add` replacing `matrix_mult_transpose`.

* Unrolled loop in `encode_12`.

* Folded decompression and inverse NTT into vecode_decode, the three
  were always used together.

* Folded inverse NTT into former `matrix_mult` as `matrix_mult_intt`,
  always used together.

* New gencbd_vector_ntt combines CBD vector generation with inverse NTT
  in one pass.

* All this makes for more readable code in `decrypt_cpa` and especially
  `genkey()`, which no longer requires caller-allocated variant-specific
  temporary storage (just a single EVP_MD_CTX is still needed).

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26236)

Polish ML-KEM kem provider.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26236)

Multi-variant ML-KEM

This introduces support for ML-KEM-512 and ML-KEM-1024 using the same
underlying implementation parameterised by a few macros for the
associated types and constants.

KAT tests are added for ML-KEM 512 and 1024, to complement the previous
tests for ML-KEM-768.

MLKEM{512,768,1024} TLS "group" codepoints are updated to match the
final IANA assigments and to make the additional KEMs known to the TLS
layer.

The pure-QC MLKEMs are not in the default list of supported groups, and
need to be explicitly enabled by the application.  Future work will
introduce support for hybrids, and for more fine-grained policy of
which keyshares a client should send by default, and when a server
should request (HRR) a new mutually-supported group that was not
sent.

Tests for ML-KEM key exchange added to sslapitest to make sure that our
TLS client MLKEM{512,768,1024} implementations interoperate with our TLS
server, and that MLKEM* are not negotiated in TLS 1.2.

Tests also added to excercise non-derandomised ML-KEM APIs, both
directly (bypassing the provider layer), and through the generic EVP KEM
API (exercising the provider).  These make sure that RNG input is used
correctly (KAT tests bypass the RNG by specifying seeds).

The API interface to the provider takes an "const ML_KEM_VINFO" pointer,
(obtained from ossl_ml_kem_get_vinfo()).  This checks input and output
buffer sizes before passing control to internal code that assumes
correctly sized (for each variant) buffers.

The original BoringSSL API was refactored to eliminate the opaque
public/private key structure wrappers, since these structures are an
internal detail between libcrypto and the provider, they are not part of
the public (EVP) API.

New "clangover" counter-measures added, refined with much appreciated
input from David Benjamin (Chromium).

The internal steps of "encrypt_cpa" were reordered to reduce the
working-set size of the algorithm, now needs space for just two
temporary "vectors" rather than three.  The "decap" function now process
the decrypted message in one call, rather than three separate calls to
scalar_decode_1, scalar_decompress and scalar_add.

Some loops were unrolled, improving performance of en/decapsulate
(pre-expanded vectors and matrix) by around 5%.

To handle, however unlikely, the SHA3 primitives not behaving like
"pure" functions and failing, the implementation of `decap` was modifed:

- To use the KDF to compute the Fujisaki-Okamoto (FO) failure secret
  first thing, and if that fails, bail out returning an error, a shared
  secret is still returned at random from the RNG, but it is OK for the
  caller to not use it.

- If any of the subsequently used hash primitives fail, use the computed
  FO failure secret (OK, despite no longer constant-time) and return
  success (otherwise the RNG would replace the result).

- We quite reasonably assume that chosen-ciphertext attacks (of the
  correct length) cannot cause hash functions to fail in a manner the
  depends on the private key content.

Support for ML-KEM-512 required adding a centered binomial distribution
helper function to deal with η_1 == 3 in just that variant.

Some additional comments were added to highlight how the code relates to
the ML-KEM specification in FIPS 203.

Reviewed-by: Paul Dale <ppzgs1@gmail.com>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26172)

Initial ML-KEM documentation

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26037)

Add ML-KEM-768 KATs from BoringSSL

Add KATs for ML-KEM-768 under CCLA from https://boringssl.googlesource.com/boringssl/

These KATs test key generation, encapsulation, and decapsulation for the
ML-KEM-768 algorithm.

Relevant notes:
- Added functionality to the ML-KEM key management to export/import. These may not
  be fully implemented yet (see openssl/openssl#25885)
- Exposed some more low-level ML-KEM API's to the provider implementation to
  allow for deterministic encapsulation/key generation
- Actually run 'mlkem_internal_test' with `make test`

Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/25938)

Add ML-KEM-768 implementation

Based on code from BoringSSL covered under Google CCLA
Original code at https://boringssl.googlesource.com/boringssl/+/HEAD/crypto/mlkem

- VSCode automatic formatting (andrewd@openssl.org)
- Just do some basic formatting to make diffs easier to read later: convert
  from 2 to 4 spaces, add newlines after function declarations, and move
  function open curly brace to new line (andrewd@openssl.org)
- Move variable init to beginning of each function (andrewd@openssl.org)
- Replace CBB API
- Fixing up constants and parameter lists
- Replace BORINGSSL_keccak calls with EVP calls
- Added library symbols and low-level test case
- Switch boringssl constant time routines for OpenSSL ones
- Data type assertion and negative test added
- Moved mlkem.h to include/crypto
- Changed function naming to be in line with ossl convention
- Remove Google license terms based on CCLA
- Add constant_time_lt_32
- Convert asserts to ossl_asserts where possible
- Add bssl keccak, pubK recreation, formatting
- Add provider interface to utilize mlkem768 code enabling TLS1.3 use
- Revert to OpenSSL DigestXOF
- Use EVP_MD_xof() to determine digest finalisation (pauli@openssl.org)
- Change APIs to return error codes; reference new IANA number; move static asserts
  to one place
- Remove boringssl keccak for good
- Fix coding style and return value checks
- ANSI C compatibility changes
- Remove static cache objects
- All internal retval functions used leading to some new retval functions

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/25848)

Post-merge make update

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26715)

Ensure we accept the IANA name for sigalgs

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26654)

ML-DSA: Add TLS certificate test

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26654)

MLDSA: Fix no-ml-dsa configure option.

Added to 'bulk' group and CI

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26654)

Drop the session and PSK test from the clienthellotest

This test doesn't really give us much that the other tests don't already
achieve. Added to that the ClientHello is nearly too long for it to work
reliably. Small changes in the ClientHello length make this test break.
So this test is too brittle with little value - so we drop it.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26654)

Teach SSL_trace() about ML-DSA

Ensure the ML-DSA based sigalgs are recognised by SSL_trace()
Also ensure the test_ssl_trace test passes correctly.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26654)

ML-DSA: Add TLS-SIGALG capability to support ML-DSA signatures

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26654)

ml-dsa test: update ML-DSA key generation tests

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26637)

ml-dsa test: update ML-DSA signature verification tests to cover μ inputs

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26637)

ml-dsa test: update ML-DSA signature generation tests to cover μ inputs

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26637)

test: update the ACVP test data parser to include tests that use μ

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26637)

ml-dsa: allow signature operations to be provided a μ value

The μ value replaces the message and avoids some of the preliminary
processes.  This is part of FIPS 204.

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26637)

params: add OSSL_SIGNATURE_PARAM_MU to param names

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26637)

doc: document OSSL_SIGNATURE_PARAM_MU for ML-DSA

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26637)

Fix docs/comments with ASN.1 private key syntax

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/26674)

Reject private keys with an incorrect pk hash

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/26674)

Make the ML-DSA seed gettable as documented

- Also fix the get_params keymgmt function to always return what's
  available.  Requested, but unavailable, parameters are simply left
  unmodified.  It is not an error to request more than is present.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/26674)

Flexible encoders for ML-DSA

- Same UX as ML-KEM.  The main ASN.1 private key syntax is the one from
  Russ Housley's post on the LAMPS list, subsequently amended to tag the
  seed instead of the key (each of the three parameter sets will have a
  fixed size for the `expandedKey`):

    ML-DSA-PrivateKey ::= CHOICE {
      seed [0] IMPLICIT OCTET STRING SIZE (32),
      expandedKey OCTET STRING SIZE (2560 | 4032 | 4896)
      both SEQUENCE {
        seed OCTET STRING SIZE (32),
        expandedKey OCTET STRING SIZE (2560 | 4032 | 4896) } }

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26638)

 ML_DSA: Make apps.c do_X509_REQ_verify() call work correctly.

- Added sigid_algs for ML_DSA such that OBJ_find_sigid_algs() works.
- OBJ_sn2nid() was also being called, so the SN form of ML_DSA
  algorithms needed to be added to the provider dispatch tables.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Paul Dale <ppzgs1@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/26636)

endecoders: make ML-KEM endecoders have fips=yes property

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26548)

doc: ML-DSA is in the FIPS provider too, so update docs

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26548)

test: update endecode test in light of ML-DSA being in the FIPS provider

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26548)

ml-dsa(fips): add ML-DSA key generation self test

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26548)

ml-dsa(fips): add power up signature self tests for ML-DSA-65

This added self tests for ML-DSA.  IG 10.3.A.15 requires known answer
tests for both signing and verify.  This adds them.

The signature generation is constructed to encounter all three of the rejection
loop tests that are relevant for ML-DSA-65.  The message has been generated
so that:

    * it fails the z_max rejection test on iteration one
    * it fails the r0_max rejection test on iteration two
    * it fails the h_ones rejection test on iteration three
    * it successfully generates the signature on iteration four

It is thus an optimal self test in terms of iterations and coverage.

Key generation self tests will be dealt with separately.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26548)

ml-dsa: document self test names

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26548)

ml-dsa: add FIPS self test macro

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26548)

test: run ML-DSA tests on FIPS provider

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26548)

ml-dsa: add PCT for FIPS provider

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26548)

ml-dsa: add to FIPS provider

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26548)

ml-dsa: add more to internal header

The ossl_ml_dsa_key_get0_libctx() and the various size macros are better in the intneral header

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26548)

fips: build ML-DSA for FIPS provider

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26548)