OpenSSL: Handle EC_POINT_is_on_curve() error case

Even though this OpenSSL function is documented as returning "1 if point
if on the curve and 0 otherwise", it can apparently return -1 on some
error cases. Be prepared for that and check explicitly against 1 instead
of non-zero.

Signed-off-by: Jouni Malinen <j@w1.fi>

diff --git a/src/crypto/crypto_openssl.c b/src/crypto/crypto_openssl.c
index 3923b8be41576758ac2d292b000d92c8f1ace9eb..3703b9360261a937c40ee26dbddd1530fe4e9826 100644 (file)
--- a/src/crypto/crypto_openssl.c
+++ b/src/crypto/crypto_openssl.c
@@ -1412,7 +1412,8 @@ int crypto_ec_point_is_at_infinity(struct crypto_ec *e,
 int crypto_ec_point_is_on_curve(struct crypto_ec *e,
                                const struct crypto_ec_point *p)
 {
-       return EC_POINT_is_on_curve(e->group, (const EC_POINT *) p, e->bnctx);
+       return EC_POINT_is_on_curve(e->group, (const EC_POINT *) p,
+                                   e->bnctx) == 1;
 }