From f4ec94c524895c7bdceb418a1a7dae80b8496f10 Mon Sep 17 00:00:00 2001 From: Jim Jagielski Date: Mon, 22 Jun 2026 17:28:42 +0000 Subject: [PATCH] Make ProxyBeaconSecret a required option now, not Optional. Thus, a passphrase secret between beacons is always required. git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1935556 13f79535-47bb-0310-9956-ffa450edef68 --- docs/manual/mod/mod_proxy_beacon.xml | 29 ++++---- modules/proxy/README.beacon | 16 +++-- modules/proxy/mod_proxy_beacon-guide.md | 13 ++-- modules/proxy/mod_proxy_beacon.c | 93 +++++++++++++------------ 4 files changed, 81 insertions(+), 70 deletions(-) diff --git a/docs/manual/mod/mod_proxy_beacon.xml b/docs/manual/mod/mod_proxy_beacon.xml index c4ac7c70db..413e8b952c 100644 --- a/docs/manual/mod/mod_proxy_beacon.xml +++ b/docs/manual/mod/mod_proxy_beacon.xml @@ -73,13 +73,13 @@ to the reverse proxy over unicast UDP datagrams Authentication

Any host that can reach the proxy's receive port could otherwise announce an arbitrary backend URL and cause the proxy to send client traffic to it - (and a UDP source address is trivially spoofable). Set - ProxyBeaconSecret to the same value on the proxy and on - every backend so that announcements are authenticated with a keyed - message-authentication code (MAC) and a timestamp. When a secret is - configured the proxy drops any announcement that is not validly signed and - recent. If no secret is configured the channel is unauthenticated - and the proxy logs a warning at startup.

+ (and a UDP source address is trivially spoofable). + ProxyBeaconSecret is therefore required: + it must be set to the same value on the proxy and on every backend, and the + server fails to start if any participating server omits it. Announcements are + authenticated with a keyed message-authentication code (MAC) and a timestamp, + and the proxy drops any announcement that is not validly signed and recent. + There is no unauthenticated mode.

 Confidentiality @@ -321,16 +321,19 @@ is taken out of rotation advance, so a captured-and-resent message (for example, one replayed to keep a dead backend from being evicted) is dropped.

-

If ProxyBeaconSecret is set on the proxy, every - announcement must carry a valid, recent MAC or it is rejected. If the +

This directive is required on every server that + participates in the beacon channel — the proxy + (ProxyBeaconListen) and every backend + (ProxyBeaconAddress). If any such server omits it, the + server fails to start; there is no unauthenticated mode.

+ +

Every announcement must carry a valid, recent MAC or it is rejected. If the secrets on the proxy and a backend differ, that backend's announcements are silently rejected (and logged), which appears as the backend never joining the balancer.

-

If no secret is configured the channel is unauthenticated and the proxy - emits a warning when it starts listening. Because the secret is stored in - the configuration file, restrict that file's permissions as you would for a - private key.

+

Because the secret is stored in the configuration file, restrict that + file's permissions as you would for a private key.

Clock synchronisation

The timestamp-based replay protection compares the announcement's time diff --git a/modules/proxy/README.beacon b/modules/proxy/README.beacon index 143befd1b7..adf60bf1b6 100644 --- a/modules/proxy/README.beacon +++ b/modules/proxy/README.beacon @@ -93,19 +93,21 @@ Message format: BEACON url=http://host:port host= pid= seq= ts= mac= url= is the routable backend origin the proxy adds as a BalancerMember. ts= - (microseconds since the epoch) and mac= are present only when a shared secret - is configured (see below). host=/pid=/seq= are informational. + (microseconds since the epoch) and mac= authenticate the message and are always + present, since a shared secret is required (see below). host=/pid=/seq= are + informational. A UDP datagram is delivered whole, so the receiver reads each message into a fixed stack buffer and NUL-terminates it (buf[len] = '\0') before parsing. The sender transmits strlen(msg) bytes (no trailing NUL on the wire). -Authentication (ProxyBeaconSecret, optional but recommended): - Without a secret the channel is unauthenticated: anyone who can reach the - receiver port could announce a URL and hijack client traffic (the proxy logs a - warning at startup in this case). A UDP source address is trivially spoofable, - so authentication matters at least as much here as it would over a connection. +Authentication (ProxyBeaconSecret, REQUIRED): + ProxyBeaconSecret must be set on every participating server (the receiver and + every sender); startup fails otherwise. There is no unauthenticated mode: + without a MAC, anyone who can reach the receiver port could announce a URL and + hijack client traffic, and a UDP source address is trivially spoofable, so + authentication matters at least as much here as it would over a connection. With ProxyBeaconSecret set identically on the proxy and all backends, each announcement is signed with a SipHash-2-4 MAC over the message prefix (the key diff --git a/modules/proxy/mod_proxy_beacon-guide.md b/modules/proxy/mod_proxy_beacon-guide.md index 930fb0000e..4495dd0cc2 100644 --- a/modules/proxy/mod_proxy_beacon-guide.md +++ b/modules/proxy/mod_proxy_beacon-guide.md @@ -124,7 +124,7 @@ as an enabled member of `balancer://cluster` on the proxy. Stop one, and after | `ProxyBeaconListen [addr][:port]` | — | **Marks this server as the receiver.** Binds a UDP socket. `addr`/`port` are optional and inherited from the server's own `Listen`/`ServerName` when omitted (see note below). | | `ProxyBeaconBalancer name` | — | Balancer that announced backends are added to. Bare name (`cluster`); a leading `balancer://` is stripped. Must already exist with spare `growth`. | | `ProxyBeaconTimeout interval` | `0` (no eviction) | Seconds of silence before a member is disabled. `0` = add-only, never auto-remove. Set to a small multiple of the backends' interval to get self-healing. | -| `ProxyBeaconSecret secret` | — (unauthenticated) | Shared cluster secret; **same value on proxy and all backends.** | +| `ProxyBeaconSecret secret` | — (**required**) | Shared cluster secret; **same value on proxy and all backends.** The server fails to start if a participating server omits it. | | `ProxyBeaconMaxSkew interval` | `30` | Anti-replay freshness window: reject announcements whose signed timestamp is more than this far from now (either direction). | **Inheriting the listen address:** because UDP and TCP are separate port spaces, @@ -141,7 +141,7 @@ this way — use an explicit high port there.) | `ProxyBeaconAddress addr:port` | — | **Marks this server as a sender.** UDP target = the proxy's `ProxyBeaconListen` address. | | `ProxyBeaconAdvertise url` | — | The routable `scheme://host[:port]` the proxy adds as a member. Omit it and the backend beacons but advertises nothing (logged, never added). | | `ProxyBeaconInterval interval` | `5` | How often this backend announces. Must be **meaningfully smaller** than the proxy's `ProxyBeaconTimeout`. | -| `ProxyBeaconSecret secret` | — | Same shared secret as the proxy. | +| `ProxyBeaconSecret secret` | — (**required**) | Same shared secret as the proxy. | > `ProxyBeaconListen` and `ProxyBeaconAddress` are **mutually exclusive** on the > same server — a server is either a receiver or a sender, not both. @@ -160,8 +160,9 @@ unauthenticated channel is a route-hijack / SSRF risk: anyone who can reach the receive port could announce an arbitrary backend URL, and **UDP source addresses are trivially spoofable.** -**Always set `ProxyBeaconSecret`** in production (identical on proxy and every -backend). With it: +`ProxyBeaconSecret` is therefore **required** (identical on proxy and every +backend) — the server refuses to start if a participating server omits it, so +there is no unauthenticated mode to fall into. With it: - Each announcement is signed with a **SipHash-2-4 MAC** plus a timestamp; the proxy recomputes the MAC (constant-time compare) and drops anything forged or @@ -177,8 +178,6 @@ Operational notes: - **Clocks must be roughly in sync** (NTP). The timestamp check compares the announcement's time against the proxy's clock; widen `ProxyBeaconMaxSkew` if your hosts drift. -- With **no secret**, the channel is unauthenticated and the proxy logs a - one-time `UNAUTHENTICATED` warning at startup. - The secret lives in your config file — **restrict its permissions like a private key.** - Announcements are **authenticated, not encrypted.** The payload is operational @@ -233,7 +232,7 @@ Useful log signals (grep the error log): | `re-enabled backend ...` | a previously-evicted backend came back | | `dropped ... mac mismatch` | wrong/missing secret on a sender (or a forged datagram) | | `replayed/reordered ts` | a stale/duplicate datagram was rejected | -| `UNAUTHENTICATED` (startup) | no `ProxyBeaconSecret` — channel is open | +| `ProxyBeaconSecret is required` (startup) | a participating server has no `ProxyBeaconSecret` — startup aborts | **Common gotchas:** diff --git a/modules/proxy/mod_proxy_beacon.c b/modules/proxy/mod_proxy_beacon.c index 3cbbd0dd35..6adb468aa5 100644 --- a/modules/proxy/mod_proxy_beacon.c +++ b/modules/proxy/mod_proxy_beacon.c @@ -72,8 +72,9 @@ * a keyed MAC (SipHash-2-4 via APR-util -- the same primitive mod_session_crypto * uses) plus a timestamp; the receiver recomputes the MAC and checks timestamp * freshness (anti-replay), dropping any forged, tampered, or stale message - * before it is parsed/acted on. Authentication is opt-in: with no secret the - * channel behaves as before but logs a one-time "UNAUTHENTICATED" warning. We + * before it is parsed/acted on. Authentication is REQUIRED: ProxyBeaconSecret + * must be set on every server that participates in the channel (sender or + * listener), or startup fails -- there is no unauthenticated mode. We * authenticate, not encrypt -- the payload (backend URLs) is not secret; for * transport confidentiality DTLS would be a separate, orthogonal future layer. * @@ -150,7 +151,6 @@ typedef struct { apr_int64_t max_skew; /* ProxyBeaconMaxSkew, seconds (replay window) */ apr_uint64_t reject_count; /* listener: dropped-since-last-log counter */ apr_time_t last_reject_log; /* listener: reject-log rate limiter */ - int warned_insecure; /* listener: emitted the one-time warning */ int open_failed; /* STARTING: socket open/listen/dial failed permanently */ } beacon_ctx_t; @@ -166,9 +166,9 @@ typedef struct { * announcement -- back off this long between attempts for a given url. */ #define BEACON_RETRY_BACKOFF apr_time_from_sec(60) -/* Cap on tracked backend urls, to bound memory against an unauthenticated - * channel announcing unboundedly many distinct urls. Once reached, unknown - * urls are dropped (rate-limited log) rather than tracked/added. */ +/* Cap on tracked backend urls, to bound memory against a buggy or compromised + * (authenticated) backend announcing unboundedly many distinct urls. Once + * reached, unknown urls are dropped (rate-limited log) rather than tracked/added. */ #define BEACON_MAX_MEMBERS 256 /* Process-wide watchdog handle, like mod_proxy_hcheck's static watchdog. */ @@ -383,9 +383,10 @@ static const command_rec beacon_cmds[] = { "proxy: seconds without an announcement after which a backend " "is disabled (taken out of rotation); 0 disables eviction"), AP_INIT_TAKE1("ProxyBeaconSecret", beacon_set_secret, NULL, RSRC_CONF, - "pre-shared cluster secret; set on both proxy and backends to " - "authenticate announcements (SipHash MAC). Keep the conf file " - "readable only by the server user."), + "pre-shared cluster secret (REQUIRED); set to the same value on " + "the proxy and all backends to authenticate announcements " + "(SipHash MAC). Keep the conf file readable only by the server " + "user."), AP_INIT_TAKE1("ProxyBeaconMaxSkew", beacon_set_maxskew, NULL, RSRC_CONF, "proxy: max allowed seconds between an announcement's timestamp " "and now (anti-replay window; requires NTP-synced clocks). " @@ -544,15 +545,6 @@ static void beacon_cb_starting(beacon_ctx_t *ctx) "Set ProxyBeaconBalancer to add announced backends.", ctx->addr); } - /* Phase 4: warn once if we'll add members from an unauthenticated - * channel (anyone who can reach this port could announce a backend). */ - if (ctx->balancer_name && !ctx->has_secret && !ctx->warned_insecure) { - ap_log_error(APLOG_MARK, APLOG_WARNING, 0, s, - APLOGNO(10572) "mod_proxy_beacon: beacon channel on %pI is " - "UNAUTHENTICATED; set ProxyBeaconSecret on the proxy and " - "all backends to require signed beacons", ctx->addr); - ctx->warned_insecure = 1; - } } else if (ctx->role == BEACON_ROLE_SEND) { /* Sender: the destination host:port targets the proxy and cannot be @@ -763,8 +755,8 @@ static apr_status_t beacon_try_add(beacon_ctx_t *ctx, apr_pool_t *pool, * - a previously-evicted member is re-enabled when it announces again; * - an add that failed (e.g. balancer full) is retried only after a backoff, * not on every announcement. - * msg_ts is the signed timestamp (microseconds) from the verified message, or 0 - * when the channel is unauthenticated. + * msg_ts is the signed timestamp (microseconds) from the verified message; this + * path is only reached on the authenticated balancer path, so it is always set. */ static void beacon_handle_announce(beacon_ctx_t *ctx, apr_pool_t *pool, const char *url, apr_time_t now, @@ -773,8 +765,8 @@ static void beacon_handle_announce(beacon_ctx_t *ctx, apr_pool_t *pool, beacon_member_t *m = apr_hash_get(ctx->seen, url, APR_HASH_KEY_STRING); if (!m) { - /* Bound memory: don't track unboundedly many distinct urls (a concern - * on an unauthenticated channel). */ + /* Bound memory: don't track unboundedly many distinct urls (defense in + * depth against a buggy or compromised authenticated backend). */ if (apr_hash_count(ctx->seen) >= BEACON_MAX_MEMBERS) { beacon_log_throttled(ctx, now, "member table full"); return; @@ -788,12 +780,12 @@ static void beacon_handle_announce(beacon_ctx_t *ctx, apr_pool_t *pool, return; } - /* Anti-replay (2): on an authenticated channel, each url's signed ts must - * strictly increase. A replayed (byte-identical) announcement carries a ts - * we've already accepted, so reject it -- this closes the in-window replay - * the freshness check alone allows (e.g. replaying a dead backend's last - * announcement to keep it from being evicted). */ - if (ctx->has_secret && msg_ts <= m->last_ts) { + /* Anti-replay (2): each url's signed ts must strictly increase. A replayed + * (byte-identical) announcement carries a ts we've already accepted, so + * reject it -- this closes the in-window replay the freshness check alone + * allows (e.g. replaying a dead backend's last announcement to keep it from + * being evicted). */ + if (msg_ts <= m->last_ts) { beacon_log_throttled(ctx, now, "replayed/reordered ts"); return; } @@ -912,14 +904,13 @@ static void beacon_mac_hex(const beacon_ctx_t *ctx, const char *base, ap_bin2hex(mac, sizeof(mac), out); /* writes BEACON_MAC_HEXLEN + NUL */ } -/* sender: return " mac=" (signed) when a secret is set, else base. */ +/* sender: return " mac=", appending the keyed MAC. A secret is + * required on every participating server (enforced at config time), so this + * always signs. */ static const char *beacon_sign(beacon_ctx_t *ctx, apr_pool_t *pool, const char *base) { char hex[BEACON_MAC_HEXLEN + 1]; - if (!ctx->has_secret) { - return base; - } beacon_mac_hex(ctx, base, strlen(base), hex); return apr_psprintf(pool, "%s mac=%s", base, hex); } @@ -1087,20 +1078,22 @@ static void beacon_cb_running(beacon_ctx_t *ctx, apr_pool_t *pool) buf[sz] = '\0'; msg_str = buf; - /* Escape the untrusted payload before it can reach the log: a - * publisher -- or, on an unauthenticated channel, anyone who can - * reach the listen port -- could embed newline/control/ANSI - * sequences to forge or corrupt error-log lines. */ + /* Escape the untrusted payload before it can reach the log: on the + * log-only path (no ProxyBeaconBalancer) messages are logged without + * MAC verification, so anyone who can reach the listen port could + * embed newline/control/ANSI sequences to forge or corrupt error-log + * lines. (On the balancer path, forged messages are dropped at + * verification below and `safe` is only logged after it passes.) */ safe = ap_escape_logitem(pool, msg_str); - /* Phase 4: when a secret is set, verify the MAC and freshness - * BEFORE logging or acting on the payload, so forged/tampered - * content is dropped (throttled) and never reaches the INFO log. + /* Phase 4: verify the MAC and freshness BEFORE logging or acting on + * the payload, so forged/tampered content is dropped (throttled) and + * never reaches the INFO log. A secret is required (enforced at + * config time), so this always runs in the active (balancer) path. * msg_ts (microseconds) is the signed timestamp, used below for the - * per-url monotonic replay check. Verification only matters in the - * active (balancer) path; with no balancer we take no action and - * the escaped payload is safe to log for diagnostics. */ - if (ctx->balancer_name && ctx->has_secret) { + * per-url monotonic replay check. With no balancer we take no action + * and the escaped payload is safe to log for diagnostics. */ + if (ctx->balancer_name) { const char *reason = "?"; if (!beacon_verify(ctx, msg_str, msg_now, &msg_ts, &reason)) { beacon_log_throttled(ctx, msg_now, reason); @@ -1210,6 +1203,20 @@ static int beacon_post_config(apr_pool_t *pconf, apr_pool_t *plog, if (ctx->role == BEACON_ROLE_NONE) { continue; } + /* ProxyBeaconSecret is required on every participating server (sender or + * listener): the channel is authenticated, with no unauthenticated mode. + * A UDP source address is trivially spoofable, so an unsigned channel + * would let anyone who can reach the listen port announce an arbitrary + * backend url and hijack client traffic. Fail startup rather than run + * insecurely. */ + if (!ctx->has_secret) { + ap_log_error(APLOG_MARK, APLOG_CRIT, 0, s, + APLOGNO(10572) "mod_proxy_beacon: ProxyBeaconSecret is " + "required but not set; the beacon channel must be " + "authenticated. Set ProxyBeaconSecret to the same value " + "on the proxy and all backends."); + return !OK; + } rv = beacon_register_callback(beacon_watchdog, AP_WD_TM_SLICE, ctx, beacon_watchdog_callback); if (rv) { -- 2.47.3