bn: Properly error out if aliasing return value with modulus

bn: Properly error out if aliasing return value with modulus

Test case amended from code initially written by Bernd Edlinger.

Fixes #21110

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/22421)