Pass the hash without the DigestInfo header to NCryptSignHash()

Pass the hash without the DigestInfo header to NCryptSignHash()

In case of TLS 1.2 signatures, the callback rsa_priv_enc() gets
the hash with the DigestInfo prepended. Signing this using
NCryptSignHash() with hash algorithm id set to NULL works in most cases.
But when using some hardware tokens, the data gets interpreted as the pre
TLS 1.2 MD5+SHA1 hash and is silently truncated to 36 bytes.
Avoid this by passing the raw hash to NCryptSignHash() and let it
add the DigestInfo.

To get the raw hash we set the RSA_sign() method in the rsa_method
structure. This callback bypasses rsa_priv_enc() and gets called with
the hash type and the hash.

Fixes Trac #1050

Cherry-picked from master 6b495dc4c5cfc118091ddc9c19330b3c9e3e3dff
and conflicts resolved manually

Changes:

- Move setting RSA_sign method from ssl_ctx_set_rsakey() to its
  right place in SSL_CTX_use_CryptoAPI_certificate(). The former
  function is only in master and appeared when the code was
  refactored for EC cert support.

- Remove the stale comment about NULL hash algorithm pointed out by Gert.

Except for the context change of one hunk, the patch is the same as
for master and nothing extra is pulled-in.

Signed-off-by: Selva Nair <selva.nair@gmail.com>
Acked-by: Steffan Karger <steffan.karger@fox-it.com>
Acked-by: Gert Doering <gert@greenie.muc.de>
Message-Id: <1538784495-24988-1-git-send-email-selva.nair@gmail.com>
URL: https://www.mail-archive.com/openvpn-devel@lists.sourceforge.net/msg17579.html
Signed-off-by: Gert Doering <gert@greenie.muc.de>