Fix change in behaviour of EVP_PKEY_CTRL_RSA_KEYGEN_BITS

Fix change in behaviour of EVP_PKEY_CTRL_RSA_KEYGEN_BITS

In 1.1.1 the ctrl EVP_PKEY_CTRL_RSA_KEYGEN_BITS would fail immediately
if the number of bits was too small. In 3.0 it always succeeds, and only
fails later during the key generation stage.

We fix that so that it fails early like it used to in 1.1.1.

Note that in 1.1.1 it fails with a -2 return code. That is not the case
in 3.0 and has not been addressed here (see #14442)

Fixes #14443

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14702)