Prefer TLS libraries TLS PRF function, fix OpenVPN in FIPS mode

Prefer TLS libraries TLS PRF function, fix OpenVPN in FIPS mode

This moves from using our own copy of the TLS1 PRF function to using
TLS library provided function where possible. This includes currently
OpenSSL 1.1.0+ and mbed TLS 2.18+.

For the libraries where it is not possible to use the library's own
function, we still use our own implementation. mbed TLS will continue
to use our own old PRF function while for OpenSSL we will use a
adapted version from OpenSSL 1.0.2t code. The version allows to be
used in a FIPS enabled environment.

The old OpenSSL and mbed TLS implementation could have shared some
more code but as we will eventually drop support for older TLS
libraries, the separation makes it easier it remove that code
invdidually.

In FIPS mode MD5 is normally forbidden, the TLS1 PRF1 function we
use, makes uses of MD5, which in the past has caused OpenVPN to segfault.
The new implementation for OpenSSL version of our custom implementation
has added the special flags that tell OpenSSL that this specific use
of MD5 is allowed in FIPS mode.

No FIPS conformitiy testing etc has been done, this is only about
allowing OpenVPN on a system where FIPS mode has been enabled system
wide (e.g. on RHEL derivates).

Patch v4: Handle the unlikely case that PRF generation fails. More
formatting
          fixes.
Patch v5: v4 with the formatting fixes actually commited. sigh.

Patch v6: More formatting fixes, make OpenSSL fucntion return bool instead
          of int.

Signed-off-by: Arne Schwabe <arne@rfc2549.org>
Acked-by: Antonio Quartulli <antonio@openvpn.net>
Message-Id: <20210305141352.21847-1-arne@rfc2549.org>
URL: https://www.mail-archive.com/openvpn-devel@lists.sourceforge.net/msg21612.html
Signed-off-by: Gert Doering <gert@greenie.muc.de>