kernel-pfkey: Fix replay window size on FreeBSD and Mac OS X

The FreeBSD and Mac OS X kernels interpret sadb_sa_replay as the size of the
replay window in bytes.  Linux on the other hand does the same for PF_KEY it
does for XFRM so sadb_sa_replay denotes the number of packets/bits in the
window.  Similarly, the window size on Linux is limited to 32 by the four
byte default bitmap used for IPsec SAs (may only be changed with
XFRMA_REPLAY_ESN_VAL), which is not the case on the other platforms.

diff --git a/src/libhydra/plugins/kernel_pfkey/kernel_pfkey_ipsec.c b/src/libhydra/plugins/kernel_pfkey/kernel_pfkey_ipsec.c
index 00ab5ab5a69e62ea5552ef076c3b911136ec42c8..6b5678270a1a64fb2a298dcaa80bd45ce300d64f 100644 (file)
--- a/src/libhydra/plugins/kernel_pfkey/kernel_pfkey_ipsec.c
+++ b/src/libhydra/plugins/kernel_pfkey/kernel_pfkey_ipsec.c
@@ -1680,7 +1680,13 @@ METHOD(kernel_ipsec_t, add_sa, status_t,
        }
        else
        {
+               /* Linux interprets sadb_sa_replay as number of packets/bits in the
+                * replay window, whereas on BSD it's the size of the window in bytes */
+#ifdef __linux__
                sa->sadb_sa_replay = min(replay_window, 32);
+#else
+               sa->sadb_sa_replay = (replay_window + 7) / 8;
+#endif
                sa->sadb_sa_auth = lookup_algorithm(INTEGRITY_ALGORITHM, int_alg);
                sa->sadb_sa_encrypt = lookup_algorithm(ENCRYPTION_ALGORITHM, enc_alg);
        }