resolved: replace assert() with error return in DNSSEC verify functions

resolved: replace assert() with error return in DNSSEC verify functions

dnssec_rsa_verify_raw() asserts that RSA_size(key) matches the RRSIG
signature size, and dnssec_ecdsa_verify_raw() asserts that
EC_KEY_check_key() succeeds. Both conditions depend on parsed DNS
record content. Replace with proper error returns.

The actual crypto verify calls (EVP_PKEY_verify / ECDSA_do_verify)
handle mismatches fine on their own, so the asserts were also redundant.

While at it, fix the misleading "EC_POINT_bn2point failed" log message
that actually refers to an EC_KEY_set_public_key() failure.

Fixes: https://github.com/systemd/systemd/issues/41569