openssl: properly check return value of RAND_bytes()

openssl: properly check return value of RAND_bytes()

This patch is in response to an off-list report by Sebastian Krahmer of
the SuSE security team.  Sebastian noticed we do not check the return
value of RAND_bytes() correctly.

The RAND_bytes() man page first says "RAND_bytes() returns 1 on success,
0 otherwise.", but then a bit later "Both functions return -1 if they are
not supported by the current RAND method.".  This second case was not
covered by our return value checking.

Note that if RAND_bytes() would return -1, it would *always* return -1 and
fail to generate random.

Also note that if RAND_bytes() would return -1, it would do so too in the
openssl internal ssl funtions.  The openssl internal function do check the
return value properly, and connection setup would fail all together.  If
that would be at least somewhat common, we would have received a *lot* of
bug reports.  In other words, the error affects static key setups only,
and seems highly unlikely to occur in actual setups.

Only builds using OpenSSL as the crypto backend are affected.

This patch:
1. Changes the behaviour of rand_bytes() in openssl builds to match what
   the doxygen claims (and polarssl builds already do).
2. Adds error reporting for RAND_bytes() failures.

Note: crypto_msg() was changed to msg() for 2.3

Signed-off-by: Steffan Karger <steffan@karger.me>
Acked-by: Arne Schwabe <arne@rfc2549.org>
Acked-by: Gert Doering <gert@greenie.muc.de>
Message-Id: <1448707105-10753-1-git-send-email-steffan@karger.me>
URL: http://article.gmane.org/gmane.network.openvpn.devel/10637
Signed-off-by: Gert Doering <gert@greenie.muc.de>
(cherry picked from commit 756602e7da11362f25be04743cd09f798b6f528a)