BN_priv_rand_range_ex() and BN_add() both return a 0 on failure and a 1
on success. In case of failure, the algorithm should fail. However, the
branch that it goes through on failure is "goto end", not "goto err".
Therefore, the algorithm will return 1 which indicates success instead
of 0 for failure, leading to potential problems for the callers.
Fix it by changing the goto to "goto err" instead of "goto end".
CLA: trivial
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Todd Short <todd.short@me.com>
(Merged from https://github.com/openssl/openssl/pull/20279)
* sqrt(2) * 2^(nlen/2-1) <= Random X <= (2^(nlen/2)) - 1.
*/
if (!BN_priv_rand_range_ex(X, range, 0, ctx) || !BN_add(X, X, base))
- goto end;
+ goto err;
}
/* (Step 4) Y = X + ((R - X) mod 2r1r2) */
if (!BN_mod_sub(Y, R, X, r1r2x2, ctx) || !BN_add(Y, Y, X))