build.info: Make it possible to use compiled programs as generators

Our goal is to be able to produce fipsmodule.cnf with the help of
'openssl fipsinstall', using the openssl program that we build.

This refactors the generatesrc code in all the build file templates to
replace $generator and $generator_incs with $gen0, $gen_args and $gen_incs,
which makes it easier and more consistent to manipulate different bits
of the generator command, and also keeps the variable names consistent
while not overly long.

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14320)

build.info: Add the possibility to add dependencies on raw targets

We need to add something for the 'tests' target to depend on, so a
special syntax for those is introduced:

    DEPEND[|tests|]=fipsmodule.cnf

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14320)

DOCS: Fix provider-mac.pod and the docs of our implementations

The idea being that doc/man7/provider-mac.pod is for provider authors,
while provider users find the documentation for each implementation in
doc/man7/EVP_MAC-*.pod, the documentation of parameters wasn't quite
aligned.  This change re-arranges the parameter documentation to be
more aligned with this idea.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14380)

doc: add a note to the RAND_get0_ calls indicating how to set the DRBG type.

The type needs to be set before the DRBGs are created.

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14386)

ossl_rsa_sp800_56b_check_public: Be more lenient with small keys

Fixes #13995

For small keys the MR test on the modulus can return
BN_PRIMETEST_COMPOSITE_WITH_FACTOR status although the modulus
is correct.

Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14389)

rand: remove FIPS mode conditional code.

The FIPS provider no longer has seeding sources inside the boundary, the
related conditional code can therefore be removed.

Fixes #14358

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14382)

EVP_PKEY_CTX_get/settable_params: pass provider operation context

This allows making the signature operations return different
settable params when the context is initialized with
EVP_DigestSign/VerifyInit.

Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14338)

Fix NULL access in ssl_build_cert_chain() when ctx is NULL.

Fixes #14294

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14295)

Remove todos in decode_der2key.c and decode_ms2key.c

Those TODOs do not really apply to 3.0 as the legacy internal
keys will stay.

Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14367)

Improve error reporting in key exchange provider implementations

Added some error reporting in dh_exch.c and unified error reporting
with it in other key exchange methods.

Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14367)

Remove unused MAX_TLS_MAC_SIZE define

Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14367)

Remove todos in providers/implementations/include/prov

Those TODOs are not relevant anymore as the headers
are now in providers.

Also make the header guard defines better reflect the
header placement.

Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14367)

Resolve TODOs in signature implementations.

The DER writing errors can be ignored safely.

Document that the EVP_MAX_MD_SIZE is a hardcoded limit
for digest sizes.

Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14367)

statem_lib.c: Remove TODOs that are unnecessary

If the EVP_MD_CTX_ctrl is deprecated the code will
generate deprecation warnings. So there is no point in marking
all EVP_MD_CTX_ctrl() calls with TODOs.

Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14367)

test_ssl_new: X448, X25519, and EdDSA are supported with fips

Removed the related TODOs.

Also adjusted the DH parameters used for the DH test to be acceptable
for FIPS as that now allows only known safe prime parameters.

Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14367)

evp_extra_test2: Remove TODO 3.0

The TODO marks optional cleanup that can be done any time in future.

Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14367)

evp_extra_test: Remove TODO comment as setting the curve is mandatory

Even with the SM2 algorithm the curve is needed for the paramgen.

Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14367)

Fix a copy&paste error in evp_extra_test

test_EC_priv_pub fails to test the case where both a private and public
key have been supplied.

Fixes #14349

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14351)

Fix compiling error on arm

Fixes #14313

Change-Id: I0dc9dd475a1ed1331738355fbbec0c51fbcb37f1

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14346)

openssl-cmp.pod.in: replace the term 'verify' by the more correct 'validate'

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14018)

CMP: On NULL-DN subject or issuer input omit field in cert template

Also improve diagnostics on inconsistent cert request input in apps/cmp.c,
add trace output for transactionIDs on new sessions,
and update the documentation in openssl-cmp.pod.in.

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14018)

test: use the new set public and private together call

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14341)

test: add utility function to set the fake random callback on both the public and private instances

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14341)

test: update ECDSA and SM2 internal tests in line with the fake_random change

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14341)

test: update test_random to create real contexts instead of sharing one

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14341)

make update

Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14352)

Fix build of /dev/crypto engine with no-dynamic-engine option

CLA: trivial

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/14329)

Remove RSA SSLv23 padding mode

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14248)

Generalize schmeme parsing of OSSL_HTTP_parse_url() to OSSL_parse_url()

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14009)

OSSL_HTTP_parse_url(): Handle any userinfo, query, and fragment components

Now handle [http[s]://][userinfo@]host[:port][/path][?query][#frag]
by optionally providing any userinfo, query, and frag components.

All usages of this function, which are client-only,
silently ignore userinfo and frag components,
while the query component is taken as part of the path.
Update and extend the unit tests and all affected documentation.
Document and deprecat OCSP_parse_url().

Fixes an issue that came up when discussing FR #14001.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14009)

apps/x509.c: Fix mem leaks in processing of -next_serial in print loop

Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14340)

apps/x509.c: Improve print_name() and coding style of large print loop in x509_main()

Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14340)

apps/x509.c: Improve indentation of the large print loop in x509_main()

Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14340)

apps/x509.c: Fix too eager call to X509_set_issuer_name() introduced recently

Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14340)

rand: use params argument on instantiate call

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

core: add params argument to DRBG instantiate call

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

doc: update documenation with params argument on DRBG instantiate calls

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

test: update tests to allow for params argument for the instantiate call on EVP_RAND_CTXs

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

prov: update rand implementations to have a params argument for the instantiate call

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

fips: update DRBG KATs for the extra instantiate argument

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

evp: add params argument to EVP_RAND_instantiate()

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

doc: document additional argument to KDF derive calls

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

prov: add additional argument to KDF derive call in key exchange

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

fips: add additional argument to KDF derive call in self test

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

crypto: add additional argument to KDF derive calls

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

apps: add addition argument to KDF derive call

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

prov: add extra params argument to KDF implementations

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

tls: adjust for extra argument to KDF derive call

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

test: adjust tests to include extra argument to KDF derive call

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

evp: add param argument to KDF derive call

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

core: add param argument to KDF derive call

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

doc: update provider-mac documentation to account for the additional init() arguments

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

doc: update KMAC doc to not say that the `KEY\' parameter needs to be set before the init call

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

apps: update speed to use the additional arguments to MAC_init

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

doc: note the additional parameters to EVP_MAC_init()

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

update poly1305 to have additional init arguments

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

update BLAKE2 to have additional init arguments

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

prov: update kmac to have additional init arguments

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

prov: update hmac to have additional init arguments

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

prov: update gmac to have additional init arguments

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

prov: update cmac to have additional init arguments

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

prov: use new MAC_init arguments in HMAC-DRBG

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

prov: use new MAC_init arguments in signature legacy code

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

prov: update provider util to be less agressive about changing things unnecessarily

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

fips: update to use the extra MAC init arguments

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

core: update to use the extra MAC init arguments

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

test: updates for the new additional MAC_init arguments

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

evp_test: updates for the new additional MAC_init arguments

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

tls: updates for the new additional MAC_init arguments

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

evp: updates for the new additional MAC_init arguments

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

crmf: updates for the new additional MAC_init arguments

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

apps: updates for the new additional MAC_init arguments

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

apps: update mac to work with additional MAC_init arguments.  This doesn't include the creation of new 'key' arguments.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

apps: update fipsinstall to work with additional MAC_init arguments

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

prov kdf: update to use the extra MAC init arguments

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

prov: update SipHash to new init function

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

siphash: Add the C and D round parameters for SipHash.

This represents a gap in functionality from the low level APIs.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14310)

crypto/asn1/i2d_evp.c: Fix i2d_provided() to return a proper length

Fixes #14258

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/14291)

PROV: Implement an EC key -> blob encoder, to get the public key

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/14291)

Modify i2d_PublicKey() so it can get an EC public key as a blob

This introduces the encoder output type "blob", to be used for
anything that outputs an unstructured blob of data.

Fixes #14258

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/14291)

test_ecpub: test that we can decode the DER we encoded

We should be able to round-trip through the encoded DER form of the
EC public key and get back something that compares as equal to the
original key.

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14291)

test_ecpub: verify returned length after encoding

Save the length we got from querying how much space was needed, and
check that the actual encoding call returned the same length.

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14291)

Add test for EC pubkey export/import

There seems to be an issue with i2d_provided() in i2d_evp.c that causes
us to fail to construct a valid chain of encoders for the "type-specific"
output when it's an EC pubkey.  This test is designed to exercise that
codepath for a variety of curves.

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14291)

Code cleanup mostly in crypto/x509/v3_purp.c

Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14328)

Check ASN1_item_ndef_i2d() return value.

Return an error instead of trying to malloc a negative number.
The other usage in this file already had a similar check, and the caller
should have put an entry on the error stack already.

Note that we only check the initial calls to obtain the encoded length,
and assume that the follow-up call to actually encode to the allocated
storage will succeed if the first one did.

Fixes: #14177
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/14308)

evp_pkey_provided_test: Improve diagnostic output

Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14316)

tests: Always print errors before test verdict

Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14316)

fuzzer: add ctx gettable/settable to the fuzzer RNG

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14240)

test: add ctx gettable/settable to the generic fake random number generator

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14240)

core: support modified gettable/settable ctx calls for ciphers

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14240)

changes to match the updated context gettable/settable calls for ciphers

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14240)

evp: upport modified gettable/settable ctx calls for ciphers

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14240)

prov: upport modified gettable/settable ctx calls for ciphers

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14240)

evp: support modified gettable/settable ctx calls for MACs

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14240)

doc: changes to match the updated context gettable/settable calls for MACs

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14240)

core: core: support modified gettable/settable ctx calls for MACs

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14240)

prov: support modified gettable/settable ctx calls for MACs

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14240)

prov: support modified gettable/settable ctx calls for KDFs

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14240)

core: support modified gettable/settable ctx calls for KDFs

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14240)

evp: support modified gettable/settable ctx calls for KDFs

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14240)