From: Chaitanya Tata Date: Mon, 6 Nov 2023 17:19:15 +0000 (+0530) Subject: dragonfly: Fix legendre symbol calculation failure handling X-Git-Tag: hostap_2_11~821 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=3e1fb2dec781c816e07b1a91ab096e69a12647e4;p=thirdparty%2Fhostap.git dragonfly: Fix legendre symbol calculation failure handling In case of low-memory conditions, the computation for legendre symbol can fail and return -2 as per documentation, but the check for that was missed here. And this can can cause an infinite loop searching for qr and qnr if the error repeats for each attempt. Break the loop if calculation fails, we can leave retry to the callers or user. This is similar to the way allocation and generation of a new random number was handled in this loop. Signed-off-by: Chaitanya Tata --- diff --git a/src/common/dragonfly.c b/src/common/dragonfly.c index 1e8427166..d039e5f9e 100644 --- a/src/common/dragonfly.c +++ b/src/common/dragonfly.c @@ -67,12 +67,15 @@ int dragonfly_get_random_qr_qnr(const struct crypto_bignum *prime, } res = crypto_bignum_legendre(tmp, prime); - if (res == 1 && !(*qr)) + if (res == 1 && !(*qr)) { *qr = tmp; - else if (res == -1 && !(*qnr)) + } else if (res == -1 && !(*qnr)) { *qnr = tmp; - else + } else { crypto_bignum_deinit(tmp, 0); + if (res == -2) + break; + } } if (*qr && *qnr)