Pass the hash without the DigestInfo header to NCryptSignHash()

Pass the hash without the DigestInfo header to NCryptSignHash()

In case of TLS 1.2 signatures, the callback rsa_priv_enc() gets
the hash with the DigestInfo prepended. Signing this using
NCryptSignHash() with hash algorithm id set to NULL works in most cases.
But when using some hardware tokens, the data gets interpreted as the pre
TLS 1.2 MD5+SHA1 hash and is silently truncated to 36 bytes.
Avoid this by passing the raw hash to NCryptSignHash() and let it
add the DigestInfo.

To get the raw hash we set the RSA_sign() method in the rsa_method
structure. This callback bypasses rsa_priv_enc() and gets called with
the hash type and the hash.

Fixes Trac #1050
Acked-by: Steffan Karger <steffan.karger@fox-it.com>
Message-Id: <1524752664-27946-1-git-send-email-selva.nair@gmail.com>
URL: https://www.mail-archive.com/openvpn-devel@lists.sourceforge.net/msg16840.html

Signed-off-by: Gert Doering <gert@greenie.muc.de>