ASN.1 ML-KEM private key format

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26512)

Improved import and export

- On import, if a seed is provided, the keys are regenerated.

- The seed is exported as a separate "seed" parameter, when available.
  The "ml-kem.retain_seed" parameter is also exported, when false.

- The seed is optionally dropped after key generation.
    * When the "ml-kem.retain_seed" keygen parameter is set to zero.
    * When the "ml-kem.retain_seed" keygen parameter is not set to 1,
      and the "ml-kem.retain_seed" provider config property is set
      explictly false.

- The exported private key parameter "priv" is always the FIPS 203 |dk|.

- Private key decoding from PKCS#8 produces a transient "seed-only" form
  of the key, in which "retain_seed" is set to false when the
  "ml-kem.retain_seed" provider config property is set explictly false.
  The full key is generated during "load" and the seed is retained
  or not as specified.

- Import honours the "ml-kem.retain_seed" parameter when specified, or
  otherwise honours the provider's "ml-kem.retain_seed" property.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26512)

Zeroise temporary secrets while doing ML-KEM

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26456)

ML-KEM implementation cleanup/speedup

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26341)

Encoders and Decoders for ML-KEM

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26341)

Temporarily disable oqsprovider testing

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26328)

libssl: Accept any key management which implements the group

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26328)

Update oqs-provider to the latest commit

The oqs-provider testing is fixed so it doesn't fetch OpenSSL
implementations during its testing inadvertently.

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26328)

Provide alias names for ML-KEM algorithms without dashes

Fixes #26326

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26328)

test: add fipsinstall corruption tests for KEMs

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Hugo Landau <hlandau@devever.net>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26338)

fips: add pairwise consistency test for ML-KEM key generation

This is mandated by FIPS 140-3 IG 10.3.A resolution 14

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Hugo Landau <hlandau@devever.net>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26338)

self_test: add ML-KEM test description

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Hugo Landau <hlandau@devever.net>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26338)

doc: update documentation now that ML-KEM is in the FIPS provider

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Hugo Landau <hlandau@devever.net>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26338)

doc: remove ML-KEM line because it doesn't have an indicator associated with it

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Hugo Landau <hlandau@devever.net>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26338)

test: run ML-KEM tests for both default and FIPS providers

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Hugo Landau <hlandau@devever.net>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26338)

tests: run ML-KEM tests in FIPS builds too

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Hugo Landau <hlandau@devever.net>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26338)

fips: add ML-KEM to the FIPS provider

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Hugo Landau <hlandau@devever.net>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26338)

fips: defined for ML-KEM CASTs

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Hugo Landau <hlandau@devever.net>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26338)

update build infrastructure for ML-KEM in the FIPS provider

Also avoid a file name conflict when adding ML-KEM to the FIPS provider.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Hugo Landau <hlandau@devever.net>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26338)

Clear param array to ensure it's initialised properly

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Todd Short <todd.short@me.com>
Reviewed-by: Tom Cosgrove <tom.cosgrove@arm.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26339)

ML-KEM hybrids for TLS

- When used as KEMs in TLS the ECDHE algorithms are NOT subjected to
  HPKE Extract/Expand key derivation.  Instead the TLS HKDF is used
  as usual.

- Consequently these KEMs are just the usual ECDHE key exchange
  operations, be it with the encap ECDH private key unavoidably
  ephemeral.

- A new "MLX" KEM provider is added that supports four hybrids of EC/ECX
  DH with ML-KEM:

    * ML-KEM-768 + X25519
    * ML-KEM-1024 + X448
    * P-256 + ML-KEM-768
    * P-384 + ML-KEM-1024

- Support listing of implemented TLS groups.

  The SSL_CTX_get0_implemented_groups() function and new
  `openssl list -tls-groups` and `openssl list -all-tls-groups`
  commands make it possible to determine which groups are
  implemented by the SSL library for a particular TLS version
  or range of versions matching an SSL_CTX.

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26220)

Prepare to detect side-channels in compiled ML-KEM code

Loosely based on similar code in BoringSSL.

Added the valgrind macros necessary to mark secret inputs as uninitialised on
entry to the ML-KEM keygen, encap and decap functions.  The inputs and outputs
are then untagged before control returns to the caller, where, at least in the
case of tests and protocols that check whether the derived keys succeeded in
decoding a key-confirmation message, there will at some point be a branch based
on the *content* of the compute shared secret.

When a build is configured with `-DOPENSSL_CONSTANT_TIME_VALIDATION`, and
various tests that use ML-KEM are run under:

    $ valgrind --tool=memcheck --error-exitcode=1 --exit-on-first-error=yes cmd [args]

any internal secret-data-dependent branches added by a mis-optimising
compiler, or inadvertently introduced into the source code would cause
the tests to fail, exposing the side channel.

Since the side-channels are liable to depend on the compiler and
selected optimisation flags, tests would need to cover a few combinations.

    * clang vs. gcc
    * debug builds
    * default builds
    * -O2
    * -O3 -fno-vectorise (a problem with clang in "clangover")
    * -Os (was a problem with clang in "clangover")
    ...

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26270)

Remaining batch of ML-KEM-related doc updates

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26217)

Initial batch of ML-KEM doc updates.

With the soon-to-be-merged ML-KEM #26172 as the merge base.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26217)

Fix code and docs of pkeyutl en/decapsulation

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26281)

ML-KEM libcrypto implementation polish

* Core ML_KEM constants in new <openssl/ml_kem.h>

* Renamed variant ordinals to ML_KEM_<bits>_VARIANT, freeing
  up the unadorned ML_KEM_<bits> names.

* Fewer/cleaner macros in <crypto/ml_kem.h>

* Fewer/cleaner macros for setting up the ML_KEM_VINFO table.

* Made (d, z) be separate inputs to the now single key generation
  function.  Both or neither have to be NULL.  This supports potential
  future callers that store them in a different order, or in separate
  buffers.

    - Random values are chosen when both are NULL, we never return the
      generated seeds, rather we may, when/if (d, z) private key support
      is added, store these in the expanded key, and make them available
      for import/export.

* No need for a stand-by keygen encoded public key buffer when the
  caller does not provide one (will ask for it later if needed).
  New `hash_h_pubkey` function can compute the public hash from
  the expanded form in constant space (384 bytes for 12-bit encoded
  scalar).

* Simplified code in `scalar_mult`.

* New `scalar_mult_add` adds the product to an existing scalar.
  Used in new `matrix_mult_transpose_add` replacing `matrix_mult_transpose`.

* Unrolled loop in `encode_12`.

* Folded decompression and inverse NTT into vecode_decode, the three
  were always used together.

* Folded inverse NTT into former `matrix_mult` as `matrix_mult_intt`,
  always used together.

* New gencbd_vector_ntt combines CBD vector generation with inverse NTT
  in one pass.

* All this makes for more readable code in `decrypt_cpa` and especially
  `genkey()`, which no longer requires caller-allocated variant-specific
  temporary storage (just a single EVP_MD_CTX is still needed).

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26236)

Polish ML-KEM kem provider.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26236)

Multi-variant ML-KEM

This introduces support for ML-KEM-512 and ML-KEM-1024 using the same
underlying implementation parameterised by a few macros for the
associated types and constants.

KAT tests are added for ML-KEM 512 and 1024, to complement the previous
tests for ML-KEM-768.

MLKEM{512,768,1024} TLS "group" codepoints are updated to match the
final IANA assigments and to make the additional KEMs known to the TLS
layer.

The pure-QC MLKEMs are not in the default list of supported groups, and
need to be explicitly enabled by the application.  Future work will
introduce support for hybrids, and for more fine-grained policy of
which keyshares a client should send by default, and when a server
should request (HRR) a new mutually-supported group that was not
sent.

Tests for ML-KEM key exchange added to sslapitest to make sure that our
TLS client MLKEM{512,768,1024} implementations interoperate with our TLS
server, and that MLKEM* are not negotiated in TLS 1.2.

Tests also added to excercise non-derandomised ML-KEM APIs, both
directly (bypassing the provider layer), and through the generic EVP KEM
API (exercising the provider).  These make sure that RNG input is used
correctly (KAT tests bypass the RNG by specifying seeds).

The API interface to the provider takes an "const ML_KEM_VINFO" pointer,
(obtained from ossl_ml_kem_get_vinfo()).  This checks input and output
buffer sizes before passing control to internal code that assumes
correctly sized (for each variant) buffers.

The original BoringSSL API was refactored to eliminate the opaque
public/private key structure wrappers, since these structures are an
internal detail between libcrypto and the provider, they are not part of
the public (EVP) API.

New "clangover" counter-measures added, refined with much appreciated
input from David Benjamin (Chromium).

The internal steps of "encrypt_cpa" were reordered to reduce the
working-set size of the algorithm, now needs space for just two
temporary "vectors" rather than three.  The "decap" function now process
the decrypted message in one call, rather than three separate calls to
scalar_decode_1, scalar_decompress and scalar_add.

Some loops were unrolled, improving performance of en/decapsulate
(pre-expanded vectors and matrix) by around 5%.

To handle, however unlikely, the SHA3 primitives not behaving like
"pure" functions and failing, the implementation of `decap` was modifed:

- To use the KDF to compute the Fujisaki-Okamoto (FO) failure secret
  first thing, and if that fails, bail out returning an error, a shared
  secret is still returned at random from the RNG, but it is OK for the
  caller to not use it.

- If any of the subsequently used hash primitives fail, use the computed
  FO failure secret (OK, despite no longer constant-time) and return
  success (otherwise the RNG would replace the result).

- We quite reasonably assume that chosen-ciphertext attacks (of the
  correct length) cannot cause hash functions to fail in a manner the
  depends on the private key content.

Support for ML-KEM-512 required adding a centered binomial distribution
helper function to deal with η_1 == 3 in just that variant.

Some additional comments were added to highlight how the code relates to
the ML-KEM specification in FIPS 203.

Reviewed-by: Paul Dale <ppzgs1@gmail.com>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26172)

Initial ML-KEM documentation

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26037)

Add ML-KEM-768 KATs from BoringSSL

Add KATs for ML-KEM-768 under CCLA from https://boringssl.googlesource.com/boringssl/

These KATs test key generation, encapsulation, and decapsulation for the
ML-KEM-768 algorithm.

Relevant notes:
- Added functionality to the ML-KEM key management to export/import. These may not
  be fully implemented yet (see openssl/openssl#25885)
- Exposed some more low-level ML-KEM API's to the provider implementation to
  allow for deterministic encapsulation/key generation
- Actually run 'mlkem_internal_test' with `make test`

Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/25938)

Add ML-KEM-768 implementation

Based on code from BoringSSL covered under Google CCLA
Original code at https://boringssl.googlesource.com/boringssl/+/HEAD/crypto/mlkem

- VSCode automatic formatting (andrewd@openssl.org)
- Just do some basic formatting to make diffs easier to read later: convert
  from 2 to 4 spaces, add newlines after function declarations, and move
  function open curly brace to new line (andrewd@openssl.org)
- Move variable init to beginning of each function (andrewd@openssl.org)
- Replace CBB API
- Fixing up constants and parameter lists
- Replace BORINGSSL_keccak calls with EVP calls
- Added library symbols and low-level test case
- Switch boringssl constant time routines for OpenSSL ones
- Data type assertion and negative test added
- Moved mlkem.h to include/crypto
- Changed function naming to be in line with ossl convention
- Remove Google license terms based on CCLA
- Add constant_time_lt_32
- Convert asserts to ossl_asserts where possible
- Add bssl keccak, pubK recreation, formatting
- Add provider interface to utilize mlkem768 code enabling TLS1.3 use
- Revert to OpenSSL DigestXOF
- Use EVP_MD_xof() to determine digest finalisation (pauli@openssl.org)
- Change APIs to return error codes; reference new IANA number; move static asserts
  to one place
- Remove boringssl keccak for good
- Fix coding style and return value checks
- ANSI C compatibility changes
- Remove static cache objects
- All internal retval functions used leading to some new retval functions

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/25848)

Post-merge make update

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26715)

Ensure we accept the IANA name for sigalgs

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26654)

ML-DSA: Add TLS certificate test

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26654)

MLDSA: Fix no-ml-dsa configure option.

Added to 'bulk' group and CI

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26654)

Drop the session and PSK test from the clienthellotest

This test doesn't really give us much that the other tests don't already
achieve. Added to that the ClientHello is nearly too long for it to work
reliably. Small changes in the ClientHello length make this test break.
So this test is too brittle with little value - so we drop it.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26654)

Teach SSL_trace() about ML-DSA

Ensure the ML-DSA based sigalgs are recognised by SSL_trace()
Also ensure the test_ssl_trace test passes correctly.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26654)

ML-DSA: Add TLS-SIGALG capability to support ML-DSA signatures

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26654)

ml-dsa test: update ML-DSA key generation tests

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26637)

ml-dsa test: update ML-DSA signature verification tests to cover μ inputs

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26637)

ml-dsa test: update ML-DSA signature generation tests to cover μ inputs

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26637)

test: update the ACVP test data parser to include tests that use μ

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26637)

ml-dsa: allow signature operations to be provided a μ value

The μ value replaces the message and avoids some of the preliminary
processes.  This is part of FIPS 204.

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26637)

params: add OSSL_SIGNATURE_PARAM_MU to param names

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26637)

doc: document OSSL_SIGNATURE_PARAM_MU for ML-DSA

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26637)

Fix docs/comments with ASN.1 private key syntax

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/26674)

Reject private keys with an incorrect pk hash

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/26674)

Make the ML-DSA seed gettable as documented

- Also fix the get_params keymgmt function to always return what's
  available.  Requested, but unavailable, parameters are simply left
  unmodified.  It is not an error to request more than is present.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/26674)

Flexible encoders for ML-DSA

- Same UX as ML-KEM.  The main ASN.1 private key syntax is the one from
  Russ Housley's post on the LAMPS list, subsequently amended to tag the
  seed instead of the key (each of the three parameter sets will have a
  fixed size for the `expandedKey`):

    ML-DSA-PrivateKey ::= CHOICE {
      seed [0] IMPLICIT OCTET STRING SIZE (32),
      expandedKey OCTET STRING SIZE (2560 | 4032 | 4896)
      both SEQUENCE {
        seed OCTET STRING SIZE (32),
        expandedKey OCTET STRING SIZE (2560 | 4032 | 4896) } }

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26638)

 ML_DSA: Make apps.c do_X509_REQ_verify() call work correctly.

- Added sigid_algs for ML_DSA such that OBJ_find_sigid_algs() works.
- OBJ_sn2nid() was also being called, so the SN form of ML_DSA
  algorithms needed to be added to the provider dispatch tables.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Paul Dale <ppzgs1@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/26636)

endecoders: make ML-KEM endecoders have fips=yes property

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26548)

doc: ML-DSA is in the FIPS provider too, so update docs

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26548)

test: update endecode test in light of ML-DSA being in the FIPS provider

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26548)

ml-dsa(fips): add ML-DSA key generation self test

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26548)

ml-dsa(fips): add power up signature self tests for ML-DSA-65

This added self tests for ML-DSA.  IG 10.3.A.15 requires known answer
tests for both signing and verify.  This adds them.

The signature generation is constructed to encounter all three of the rejection
loop tests that are relevant for ML-DSA-65.  The message has been generated
so that:

    * it fails the z_max rejection test on iteration one
    * it fails the r0_max rejection test on iteration two
    * it fails the h_ones rejection test on iteration three
    * it successfully generates the signature on iteration four

It is thus an optimal self test in terms of iterations and coverage.

Key generation self tests will be dealt with separately.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26548)

ml-dsa: document self test names

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26548)

ml-dsa: add FIPS self test macro

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26548)

test: run ML-DSA tests on FIPS provider

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26548)

ml-dsa: add PCT for FIPS provider

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26548)

ml-dsa: add to FIPS provider

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26548)

ml-dsa: add more to internal header

The ossl_ml_dsa_key_get0_libctx() and the various size macros are better in the intneral header

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26548)

fips: build ML-DSA for FIPS provider

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26548)

fips: build ML-DSA for FIPS provider

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26548)

build: fix copy/paste mistake

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26548)

Add ML-DSA to the FIPS provider

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26548)

ML-DSA Add digestsign tests - The digest must be NULL

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26575)

ML_DSA Add support for generation of X509 certificates using the openssl
commandline.

In order to support this gettables are required in both the key and
signature.:

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26575)

ML-DSA Updates to encoders required because of changes on the master
branch.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26575)

Add ML_DSA encoders

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Paul Dale <ppzgs1@gmail.com>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26575)

ml-dsa: avoid param builder

Using param builder consumes more resources and it is only beneficial
when dealing with bignums.  Directly using the param helpers is a better
alternative.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26529)

ML-DSA: coverage testing fixups

Reviewed-by: Paul Dale <ppzgs1@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26505)

ML-DSA Add coverage tests

Also remove some ACVP test data from ml_dsa.inc since this is now
also done using evp_test.

Reviewed-by: Paul Dale <ppzgs1@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26505)

Add ML-DSA design document.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26400)

Add ML-DSA documentation

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26400)

ML-DSA encoder fixups

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26483)

ML-DSA: Add python utility used to parse ACVP test data.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Paul Dale <ppzgs1@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/26451)

ML-DSA: Fix endian issue in ossl_ml_dsa_poly_sample_in_ball().

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Paul Dale <ppzgs1@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/26451)

ML-DSA: Add evp_test support for ML-DSA ACVP test vectors.

The evp_test line buffer was increased to 32K to deal with the large
lines required for PQ messages and signatures.
The test data files were generated by parsing AVCP test files using
a python script.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Paul Dale <ppzgs1@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/26451)

ML-DSA: Add support for dup.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Paul Dale <ppzgs1@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/26451)

ML_DSA: Fix partially uninitialised hint buffer.
The encoded hint data consists of omega + k bytes.
The bytes at the end of omega section of the buffer may be 0,
so the buffer must be cleared initially.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Paul Dale <ppzgs1@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/26451)

Add ML_DSA encoders

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Paul Dale <ppzgs1@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/26451)

ML-DSA fixups

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26127)

Add ML-DSA-44 and ML-DSA-87, fix endian issues & add fixups

- Make data encoding work on big-endian systems.

- Fix some ML-DSA-44 specific bugs related to w1-vector bits
  per-coefficient, overall size and high-bits rounding.

- Use "do { ... } while (pointer < end)" style consistently.

- Drop redundant reference counting of provided keys.

- Add parameter blocks for ML-DSA-44 and ML-DSA-87 and turn on
  associated provider glue.  These now pass both keygen and
  siggen tests (to be added separately).

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26127)

Add ML-DSA sign/verify

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26127)

ML_DSA: Use pointers instead of arrays for polynomials in Vectors and Matrix.

A DSA_KEY when created will alloc enough space to hold its k & l
vectors and then just set the vectors to point to the allocated blob.

Local Vectors and Matricies can then be initialised in a similar way by
passing them an array of Polnomials that are on the local stack.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26127)

Add ML-DSA Keygen support

The key generation algorithm requires a significant portion of the many
algorithms present in FIPS 204.

This work is derived from the BoringSSL code located at
https://boringssl.googlesource.com/boringssl/+/refs/heads/master/crypto/mldsa/mldsa.cc

Instead of c++ templates it uses an ML_DSA_PARAMS object to store constants such as k & l.
To perform hash operations a temporary EVP_MD_CTX object is used, which is supplied with a
prefetched EVP_MD shake128 or shake256 object that reside in the ML_DSA_KEY object.

The ML_DSA_KEY object stores the encoded public and/or private key
whenever a key is loaded or generated. A public  key is always present
if the private key component exists.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26127)

Rework and simplify RCU code

Use __ATOMIC_RELAXED where possible.
Dont store additional values in the users field.

Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26690)

Revert "rcu: Ensure that updates to the ID field of a qp don't lose refs"

This reverts commit fbd34c03e3ca94d3805e97a01defdf8b6037f61c.

Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26690)

Test openssl ecparam -list_curves

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26525)

ecparam: process -list_curves before reading input

Fixes #26519

This is a regression from commit 1dbb67c4

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26525)

Fix arithmetic in a comment

CLA: trivial

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Paul Dale <ppzgs1@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26707)

[vpsm4_ex][aarch64] Move constant to .rodata section

Reviewed-by: Saša Nedvědický <sashan@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26509)

[vpsm4][aarch64] Move constant to .rodata section

Reviewed-by: Saša Nedvědický <sashan@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26509)

[sm3][aarch64] Move constant to .rodata section

Reviewed-by: Saša Nedvědický <sashan@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26509)

[poly1305][aarch64] Extend address range by adrp + add

Reviewed-by: Saša Nedvědický <sashan@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26509)

[bsaes][aarch64] Move constant to .rodata section

Reviewed-by: Saša Nedvědický <sashan@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26509)

[chacha_sve][aarch64] Move constant to .rodata section

Reviewed-by: Saša Nedvědický <sashan@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26509)

[sm4][aarch64] Move constant to .rodata section

Reviewed-by: Saša Nedvědický <sashan@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26509)

Address coverity reports

Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26703)

ossl_rand_ctx_new(): Use OPENSSL_strdup instead of strdup

Fixes Coverity 1642620

Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Paul Dale <ppzgs1@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/26641)