[Feature] rspamadm: add memstat command and pretty-printer
Add the memstat (alias mem_stat) subcommand to rspamadm control: the
help text gains a new entry, the command name maps to /memstat, and
the response is fed through lualib/rspamadm/memstat.lua for table
output. The Lua module supports --top, --no-callsites, --no-jemalloc
and -n (raw numbers); JSON / compact JSON modes still bypass the
formatter as for other commands.
Introduce src/libserver/memory_stat.{cxx,h} that gathers a UCL dump for
a worker process: OS-level RSS/VmSize breakdown, mempool aggregate plus
per-callsite suggestions, Lua heap usage, and (when WITH_JEMALLOC is
defined) jemalloc mallctl counters and the textual malloc_stats_print
dump. The document is serialized to a tempfile and the descriptor is
passed back over the control pipe with SCM_RIGHTS, mirroring the
existing fuzzy_stat pattern.
Wire the collector into rspamd_control_default_cmd_handler so any
worker registered with the default control handlers transparently
answers RSPAMD_CONTROL_MEMORY_STAT without per-worker boilerplate.
[Feature] rspamd_control: wire /memstat command and reply union
Add RSPAMD_CONTROL_MEMORY_STAT to the enum, a fixed-size summary slot
in the cmd/reply unions (status, rss_kb, lua_kb, mempool_bytes,
jemalloc_allocated), the /memstat URL mapping, and the per-worker UCL
emission and totals aggregation in rspamd_control_write_reply().
The actual collector and the dispatch through default_cmd_handler are
introduced in the following commit; with this change in isolation the
command is reachable end-to-end but returns only zero summaries.
Add rspamd_lua_get_memory_used() that combines LUA_GCCOUNT and
LUA_GCCOUNTB into a byte count. Used by the memstat control command;
also a convenient single entry point for any future per-worker Lua
heap diagnostics.
Add struct rspamd_proc_mem_info and rspamd_get_process_memory_info()
that fills it in from OS-specific sources: /proc/self/status on Linux
(VmSize/VmRSS/VmData/RssAnon/etc.), task_info(MACH_TASK_BASIC_INFO) on
macOS, and getrusage(RUSAGE_SELF) as a portable fallback. Will be used
by the memstat control command to expose worker-process footprint.
Add rspamd_mempool_entry_stat_t and rspamd_mempool_entries_foreach() so
callers can introspect the per-location mempool registry (suggestion,
preallocated counts, average fragmentation/leftover) without reaching
into mem_pool_internal.h. Used by the upcoming memstat control command.
[Fix] dmarc: floor connect timestamp before os.date for PUC Lua
task:get_date returns a fractional double; PUC-Rio Lua 5.3+ rejects
non-integer floats as the second argument to os.date with "number has
no integer representation". LuaJIT accepts it, so the bug only fires
on the Fedora CI build.
[Minor] lua_upstream: pack acquired/retired into bitfields
Two gboolean (gint) fields cost 8 padded bytes plus alignment per
wrapper. Each wrapper only needs two bits, so use unsigned:1
bitfields instead. struct rspamd_lua_upstream shrinks from 24 to
16 bytes on 64-bit targets.
[Fix] lua_upstream: retire inflight on __gc when caller forgets ok/fail
Lua plugin code can drop a get_upstream_*() wrapper without ever
calling :ok or :fail (e.g. when an async callback never fires or is
written incorrectly). Without retirement, the C-side inflight counter
introduced for P2C scoring leaks indefinitely and biases selection
away from the affected upstream.
Add acquired/retired bookkeeping on the Lua wrapper:
- lua_push_upstream() takes an explicit acquired flag. The three
get_upstream_* bindings pass TRUE; all_upstreams() inserter passes
FALSE since it returns a view, not a fresh inflight reference.
- The watcher path inlines lua_newuserdata; explicitly zero the new
fields there so uninitialised stack memory doesn't trigger spurious
retire calls.
- :ok and :fail set retired = TRUE so the destructor doesn't double
retire when the caller did pair properly.
- The __gc destructor calls rspamd_upstream_release when
acquired && !retired, decrementing inflight without affecting error
counts or latency.
Lua GC is non-deterministic, so retirement may lag for some time;
that's acceptable noise for a load comparator and strictly better
than an unbounded leak.
Tests in test/lua/unit/upstream.lua cover smoke-level API usage,
the abandoned-wrapper path, view safety from all_upstreams(), and
double-retirement protection.
[Fix] upstream: add release() for non-success/failure paths
The new inflight counter introduced for P2C exposed several pre-existing
leaks where a get_* selection had no matching ok()/fail() call. ok() was
unsuitable as a generic retire because it also clears the error count.
Add rspamd_upstream_release() — decrement inflight without touching
errors, latency, or watchers — and apply at four call sites:
- rspamd_proxy.c mirror loop: copy_msg failure after upstream selection
- rspamd_proxy.c master loop: copy_msg failure after upstream selection
- fuzzy_check.c PING: fire-and-forget address lookup
- http_connection.c proxy: hand-off path where new_common drops the
upstream pointer (per-request tracking left for a follow-up)
Two more leak classes remain for separate PRs: Lua-side retire fallback
via __gc, and librdns retransmit/select pairing in dns.c.
Track an exponentially-weighted moving average of per-request latency
on each upstream, with a configurable half-life (default 60s) so older
samples decay and a once-slow-now-recovered backend isn't permanently
penalised. Updates are time-weighted: alpha = 1 - exp(-dt/tau) where
tau = half_life / ln(2). Setting half_life to 0 falls back to a flat
moving average where every sample has equal weight.
Wire it into the P2C load score:
score = latency * (inflight + 1) + errors * 5 * latency
when at least one sample exists; fall back to the existing
inflight + errors*2 form otherwise. This is a lightweight approximation
of PeakEWMA — a slow backend with low load loses to a fast one with
comparable load, but a fast backend can still lose if it gets too busy.
New public API:
rspamd_upstream_record_latency(up, seconds)
rspamd_upstream_get_latency(up)
rspamd_upstreams_set_latency_half_life(ups, seconds)
Callers opt in by recording observed RTT alongside their existing
ok()/fail() calls. The score function falls back gracefully to Phase 1
behaviour for upstream lists where no caller has wired up sampling
yet, so this commit is a no-op for current users.
Newly revived upstreams previously rejoined the alive list at full
weight, producing a thundering herd that would land on a backend that
just came back up and was still warming caches/connection pools — the
same backend that had been failing minutes before. This often caused
immediate re-failure and a flap loop.
Add an opt-in slow_start_ms window (default 0 = disabled) configurable
via rspamd_upstreams_set_slow_start. While the window is open, both
round-robin (effective weight = weight * factor) and P2C (effective
load score = base / factor + warmup penalty) bias selection away from
the warming upstream linearly over time.
Hashed (Ketama) intentionally not integrated: scaling vnode counts
during the window would defeat the consistency property that hashed
selection exists for. Token bucket likewise unaffected — its
inflight-based fairness already handles cold buckets gracefully.
revived_at is set in the two real revive paths: the timer-based
revive_cb and the half-open probe success path in ok(). The initial
add_upstream activation is left unmarked so cold starts after a
config reload aren't artificially throttled.
[Feature] upstream: add Power of Two Choices (P2C) selection
P2C samples two alive upstreams uniformly at random and chooses the
one with the lower load score (inflight + errors*2). Provably within
a constant factor of optimal max-load and the modern default for
load-aware random selection (Envoy LEAST_REQUEST, Finagle, NGINX
least_conn).
A passive in-flight counter on struct upstream is incremented on every
selection in get_common and in get_token_bucket, decremented in ok()
and fail(); the existing caller contract (every get pairs with one
ok or fail) is preserved without any new public API.
RSPAMD_UPSTREAM_RANDOM callers are silently upgraded to P2C since it
strictly dominates uniform random with no extra cost. The token-bucket
fallback when message size is unavailable also uses P2C now.
Tests: new upstream_p2c suite (7 cases, 800+ assertions) covers
single-upstream cases, the silent RANDOM upgrade, load-aware bias
toward idle upstreams, and balanced inflight tracking under mixed
ok/fail outcomes.
[Fix] upstream: drop pool-less branch in set_token_bucket
The fallback that g_malloc'd a fresh limits struct when no pool was
available leaked it on the next call and on destroy. The function is
only ever invoked with a real ctx; assert that explicitly. Also keep
the new refill rate proportional to max_tokens when it's overridden,
so users tuning the bucket size don't get a stale default refill.
[Fix] upstream: lazy time-based refill for token bucket
return_tokens with success=false decremented inflight but never
returned tokens to available_tokens, so a flapping upstream's bucket
drained monotonically toward zero and never recovered. Selection
then permanently fell into the least-inflight fallback path,
defeating the cost signal.
Add a real refill rate (token_bucket_refill_per_s, default = max/60
so a quiet bucket fully regenerates in 60s of wall time). Call lazy
refill from get_token_bucket and return_tokens; failure no longer
permanently penalises the bucket. Within-tick test workloads see dt
small enough that floor(dt * rate) == 0, so existing assertions are
unaffected.
[Rework] upstream: drop token-bucket heap, use flat scan
The intrusive min-heap stored entries by value; swim/sink swaps
mutated the slot pointer's contents, so up->heap_idx went stale after
every update. The cache-miss workaround was a linear scan, making
each get/return effectively O(n) anyway. Alive sets are typically
2-10 upstreams, where a flat scan is faster in practice than a heap
with by-value repair.
Replaces the heap with a single pass over alive[] that tracks both
the lowest-inflight eligible upstream and the absolute least-loaded
one as a fallback for the exhausted-bucket case. Removes
upstream_token_heap_entry, the RSPAMD_HEAP_DECLARE, three helper
functions, the heap_idx field on struct upstream, and the
token_bucket_initialized/token_heap fields on struct upstream_list.
[Fix] upstream: preserve backoff for pending-resolve
set_active stopped the timer and re-armed at INITIAL_DELAY (~1s),
discarding the exponential backoff lazy_resolve_cb had accumulated.
Snapshot ev.repeat before stopping and reuse it when the upstream is
still PENDING_RESOLVE so repeated DNS failures actually back off.
[Fix] upstream: bail out of get_random when only candidate is excluded
rspamd_upstream_get_random looped forever when alive->len == 1 and the
single survivor matched the 'except' argument. Front-gate the empty and
single-survivor cases explicitly; the unbounded loop only runs for
n >= 2 where it is guaranteed to terminate.
[CritFix] mime_parser: avoid NULL deref on SMIME with empty pkcs7-data
When an S/MIME signed message wraps an inner pkcs7-data with a zero-length
OCTET STRING, the SMIME inner-content extraction in rspamd_mime_parse_normal_part
allocated a zero-length buffer and recursed into rspamd_mime_process_multipart_node
with start/end pointing at NULL (g_malloc(0) returns NULL under always_malloc
mempool mode), causing a SIGSEGV at the first byte check.
Fix:
- Skip the SMIME inner recursion when the encapsulated OCTET STRING is empty
or has a NULL data pointer.
- Add a defensive guard at the top of rspamd_mime_process_multipart_node to
return RSPAMD_MIME_PARSE_NO_PART for NULL or empty buffers, protecting any
other caller from the same UB.
Add a Lua regression test that exercises the SMIME-empty path through
rspamd_message_parse. With VALGRIND=1 (forcing always_malloc) the test
reliably reproduced the crash before the fix.
[Fix] Honor mime_utf8 option in INVALID_MSGID rule
Two related issues caused INVALID_MSGID false positives on valid
EAI/SMTPUTF8 Message-IDs (RFC 6532):
* The sane_msgid regexp unconditionally rejected bytes \x80-\xff,
even when mime_utf8 was enabled. Relax the regexp in that case
while keeping structural checks intact.
* The configuration option was registered only as enable_mime_utf,
but the corresponding Lua API is rspamd_config:is_mime_utf8(),
so users naturally try enable_mime_utf8. That spelling silently
had no effect because the parser did not bind it to any field.
Register enable_mime_utf8 as an alias mapped to the same struct
field so configs using it actually take effect.
Add a functional test (configs/mid_utf8.conf, messages/mid_eai_utf8.eml,
cases/107_mid_utf8.robot) that exercises both fixes via the new
option name and verifies that structurally invalid Message-IDs are
still flagged.
fix(rspamadm/vault): write formatted output to stdout directly (#6005)
Closes #6005.
`rspamadm vault list` produced completely empty output (no stdout,
no stderr, exit code 0) when the Vault held 356+ DKIM entries.
Deleting one entry made it work again.
Root cause: `maybe_print_vault_data` passed the formatted payload
through `printf`, which calls `rspamd_logger.slog(fmt, ...)`. slog
treats its first argument as a format string. When the formatted
UCL/JSON body contained anything slog interprets as a format
specifier (`%` characters in keys, escaped strings, etc.) — or
simply exceeded slog's internal buffer — the output was silently
dropped and the user saw nothing.
The same path is hit by every other handler that already worked
(`show`, etc.) only because their payloads were smaller and didn't
trigger the silent-drop edge case.
Write the formatted payload to stdout directly via `io.write`. No
format-string interpretation, no buffer limit, no surprise. Append
a trailing newline only when the formatted output didn't already
end with one (UCL output usually does).
The empty table caused a spurious warning in lua_maps when no
whitelist was configured. Since settings.whitelist defaults to nil,
the else-branch was a no-op. User-configured whitelists via
link_affiliation { whitelist = ... } continue to work as before.
[Minor] lua_maps: handle empty table as static empty map
When map_add_from_ucl receives an empty Lua table, it fell through
to the C map infrastructure, which logged a spurious error-level
message with no map name. Return a lightweight empty map object
directly in Lua, cache it for consistency with other code paths,
and log a warning since an empty table is likely a misconfiguration.
[Minor] maps: include map description in load error messages
Without a map description in the log, users had no way to identify
which map triggered the error, forcing unnecessary investigation.
All 'no urls to be loaded' and 'invalid type' error sites in
rspamd_map_add_from_ucl now include the description; rspamd_printf
handles NULL safely.
[Fix] upstream consumers: make NULL/nil branches sound
A NULL guard is only useful if the branch behind it logs the failure,
propagates it correctly to the caller, and leaves internal state
consistent. Re-audited every NULL/nil-upstream branch (pre-existing
and newly added by this branch) and tightened the silent or
state-corrupting ones:
* fuzzy_backend_redis: the three rspamd_upstream_get NULL branches in
read / count / version paths invoked the caller's callback with an
empty result and returned silently. Admins had no signal that fuzzy
was being skipped because every backend was dead or pending DNS.
Each branch now also msg_err_redis_session's the reason.
* libserver/http_connection.c: when ctx->http_proxies is configured
but every proxy upstream is unavailable, the code silently fell
back to a direct connection - a security/privacy footgun for
configs that meant to force traffic through a proxy. Added an
msg_info to surface the fallback so the admin notices.
* lua_redis prepare_redis_call: the previous patch in this branch
marked skipped servers as "tempfail" but did not insert a
placeholder into `options`, so the load_script_task /
load_script_taskless consumer loop's iteration index no longer
matched the original servers_ready index. A successful upload to
one server would then write "done" into the wrong slot of
servers_ready (the slot for a different, possibly skipped server),
corrupting the script-load state machine. Insert a `{ skip = true,
upstream = s }` placeholder so the indexes stay aligned, and skip
the placeholder in both consumer loops.
[Fix] upstream: make addr accessors and all_upstreams pending-safe
The PENDING_RESOLVE upstream state introduced earlier kept pending
entries out of the alive list, but `:all_upstreams()` walks the full
`ups` array and exposes them to Lua callers - which then crashed in
`s:get_addr()` because `rspamd_upstream_addr_next/cur/port` indexed
a NULL `addrs.addr`.
Defensive fix at the C accessor layer:
* rspamd_upstream_addr_next / _cur now return NULL when the upstream
has no addresses (NULL or empty array). This is the safe layer that
every other consumer eventually goes through.
* rspamd_upstream_port returns the parsed `deferred_port` for pending
upstreams (so callers that just want a port get a sensible answer)
and -1 if even that is unknown.
* lua_upstream:get_addr() pushes nil when the C side has no address.
Audit of `:all_upstreams()` callers, all updated to skip pending:
* lua_redis prepare_redis_call (SCRIPT LOAD broadcast): if
`s:get_addr()` is nil, mark the slot as "tempfail" so the next
retry will pick it up once DNS comes back, log, and skip it.
* rspamadm statistics_dump connect_to_upstream: log and return early
before opening a redis connection with a nil host.
* clickhouse plugin check_clickhouse_upstream: skip with an info log
so the periodic check tries again next tick.
The DKIM Vault helper already passes `upstream = ... or nil` to
http.request and lets the HTTP layer fall back to URL-based connect,
which remains the right behaviour.
[Fix] fuzzy_check: handle NULL upstream in lua_ping_storage
fuzzy_lua_ping_storage selected an upstream from rule->read_servers
without checking the result, then dereferenced the NULL pointer in
rspamd_upstream_addr_next(). With the new deferred-DNS upstream layer
this becomes reachable in normal operation (every upstream still
pending), and was already reachable before whenever the alive list
was empty.
Audit of other rspamd_upstream_get / _forced / _except / _token_bucket
call sites in C/C++ (rspamd_proxy.c, libserver/dns.c,
fuzzy_backend_redis.c, http/http_connection.c, libstat http_backend,
the other fuzzy_check sites) confirms they already guard the result
with `if (up)` or a `while (up = ...)` loop; only this site was
unchecked.
Return (false, "no fuzzy storage upstream available for rule X") to
the Lua caller instead of crashing.
[Fix] lua: tolerate nil upstream in transport, plugins, rspamadm
Audit of every Lua caller of upstream_list:get_upstream_round_robin /
:get_upstream_master_slave / :get_upstream_by_hash that is not a
scanner. Each one now reacts to a nil result instead of dereferencing
it and crashing the call site:
* lua_redis.lua: all four selection sites already logged "cannot
select server" but then continued into addr:get_addr() and crashed.
They now `return false, nil, nil` after the log, so callers see a
proper failure. The sentinel watcher tick logs and skips this round.
* lua_maps.lua: the external-map HTTP path logs and invokes the
caller's callback with (false, "no upstream available", 502, ctx)
so map consumers see a normal lookup failure.
* aws_s3.lua: lifts the upstream selection out of the http.request
table so it can warn before letting the HTTP layer fall back to
URL-based connect (the request still goes out).
* clickhouse.lua, elastic.lua, gpt.lua: each get_upstream_round_robin
site now logs and returns from its enclosing function (send,
retention, distro detect, geoip pipeline, index policy/template,
GPT/Ollama model dispatch).
* rspamadm/clickhouse.lua and rspamadm/statistics_dump.lua: print to
stderr and exit / abort the redistribute scan.
[Fix] lua_scanners: emit fail symbol on nil upstream
After the upstream layer learnt to defer DNS resolution, every
scanner's :get_upstream_round_robin() may return nil while the host
is still pending (or when every backend has been marked dead). The
scanners then crashed in the scan callback on `upstream:get_addr()`,
which silently aborted the scan instead of surfacing the failure.
Add a shared lua_scanners/common.get_upstream_or_fail(task, rule,
maybe_part, reason) helper that selects an upstream and, on nil,
emits the configured *_FAIL symbol via yield_result with reason
"no upstream available (DNS pending or all dead)". Update every
scanner under lualib/lua_scanners/ to use the helper for the initial
selection, and to add an inline yield_result+return guard around the
retry-on-error sites that pick a different upstream.
Cloudmark also has a config-time preload that picks an upstream to
detect max message size; that path now logs an error and returns,
since there is no task context to report against.
[Feature] upstream: defer DNS resolution for unreachable hosts
Previously rspamd_upstreams_add_upstream() returned FALSE whenever
rspamd_parse_host_port_priority() could not resolve a hostname, so a
single DNS hiccup at config time would drop the upstream list and
cascade into module init failures (issue #6000 was one symptom).
Introduce RSPAMD_UPSTREAM_FLAG_PENDING_RESOLVE: when DNS fails for a
hostname-style input we now keep the parsed host and port on the
upstream, mark it pending, and let the existing async lazy-resolve
machinery retry. Pending upstreams are deliberately kept out of the
`alive` list so existing selectors (round-robin, hashed, master/slave)
and the ring/heap rotators do not need to learn a new state - they
continue to see only usable upstreams. The probe-mode fallback that
walks `ups` directly skips pending entries explicitly.
set_active() schedules a fast initial resolve (1s with jitter) for
pending upstreams; lazy_resolve_cb() backs off exponentially up to
60s while the upstream stays pending. update_addrs() handles the
empty-initial case by reading the port from the stashed
`deferred_port` field, and on success rspamd_upstream_promote_pending
clears the flag and inserts the upstream into `alive` (initialising
token-bucket state if needed) and fires the WATCH_ONLINE event.
This changes the failure mode for every consumer of upstreams (Redis,
ClamAV, ICAP, ClickHouse, ...): a misconfigured or briefly-down DNS
no longer prevents the daemon from starting, and recovers automa-
tically without a restart. Callers that pick from an alive-empty list
already had to handle nil from rspamd_upstream_get; later commits
audit and tighten the Lua callers that did not.
[Fix] external_services: register fail-only stub when configure() fails
Issue #6000: a hostname that does not resolve at startup made
add_scanner_rule() crash on a nil-deref ("rule.symbol or sym:upper()"
where rule is already nil), which disabled the whole external_services
module and silently dropped malware scans.
Resolve the symbol names from opts up-front so the error path no longer
touches the nil rule. When cfg.configure(opts) returns nil (bad type,
missing servers, etc.) build a degraded rule whose callback always
emits the configured *_FAIL symbol with reason
"<type>: configuration failed (see startup log)", so the misconfigura-
tion is visible on every scan instead of being silent.
The upstream-layer changes that follow make DNS failures no longer
land in this branch, but the stub remains the right behaviour for
genuine configure() failures.
[Fix] fuzzy_storage: drop per-command state caching from TCP session (issue #6001)
Each TCP frame allocates a per-frame `cmd_session` whose key, extensions
buffer, ip_stat, and decrypt nm are owned and released by
`fuzzy_session_destroy`. The TCP I/O loop also pre-populated those fields
from a `fuzzy_common_session` cache and "transferred ownership" back at
the end of every frame — but the transfer overwrites without releasing,
so on a persistent TCP connection every frame after the first leaked:
- one g_malloc'd extensions buffer (sent by every scanning request),
- one fuzzy_key REF_RETAIN from the pre-populate path,
- and, for encrypted frames, a second fuzzy_key REF_RETAIN from
rspamd_fuzzy_decrypt_command's own unguarded `s->key = key`.
Stop caching per-command state on the TCP session entirely. cmd_session
starts zeroed each frame, decrypt populates `key`/`nm`, the wire parser
allocates `extensions`, process_command retains `ip_stat`, and
fuzzy_session_destroy frees the lot when the frame is done. The decrypt
overwrite is also no longer a leak because cmd_session->key starts NULL.
Also delete the never-instantiated `fuzzy_udp_session` /
`fuzzy_udp_session_destroy` / `rspamd_fuzzy_udp_write_reply` — UDP has
been using the legacy `fuzzy_session` path all along.
[Test] checkv3: use spam_message for inline-settings regression test
GTUBE short-circuits the Rspamd filter pipeline (forced reject before
SETTINGS_CHECK runs), so the inline metadata.settings test from the
previous commit silently never exercised settings.lua at all. Switch
to messages/spam_message.eml so the prefilter chain actually runs and
apply_settings_side_effects gets a chance to inject the symbol.
Also drop the actions.reject override case: with GTUBE it was a false
positive (GTUBE forces reject regardless of thresholds) and a proper
rewrite needs a second non-GTUBE message wired into the suite, which
can come in a follow-up.
Use the array form `symbols = [..]` matching the v2 INJECT SYMBOL test
in 108_settings.robot - it is the documented apply spelling.
[Fix] protocol: apply inline metadata.settings on /checkv3
Inline metadata.settings on /checkv3 was stashed on task->settings
directly, skipping the apply pipeline. As a result every documented
"apply" effect (action thresholds, symbols / symbols_enabled /
symbols_disabled, subject, variables, add/remove headers) silently
no-op'd, while the same payload sent via the /checkv2 Settings HTTP
header worked. Fixes #5999.
Serialize the inline UCL object to compact JSON and synthesize a
Settings request header so the existing settings.lua
check_query_settings -> apply_settings -> {task:set_settings,
apply_settings_side_effects} pipeline runs uniformly with v2. One
source of truth, no duplicated apply logic in C. Also update the
"settings will be merged" log to "inline settings will take
precedence" since that is what settings.lua actually does when both
settings_id and inline settings are present.
Audit-companion fix: metadata.deliver_to now goes through
rspamd_protocol_escape_braces too, matching the v2 Deliver-To header
which has stripped <...> braces since the beginning.
Add two regression tests in 430_checkv3.robot covering the Lua-side
side effect path (settings.symbols injection) and the C-side action
threshold path (actions.reject override).
[Fix] protocol: populate request headers in /checkv3
/checkv3 was reading only a fixed set of HTTP headers and never passed
them through rspamd_task_add_request_header, so task:get_request_header()
returned nil for arbitrary client-supplied headers under v3 — a regression
vs. v2. Iterate msg->headers at the top of rspamd_protocol_handle_v3_request
and register every HTTP header. Skip Shm / Shm-Offset / Shm-Length, which
at the HTTP level are the proxy body-transfer mechanism and would collide
with the metadata-synthesized "shm" zero-copy message pointer consumed by
rspamd_task_load_message.
[Fix] neural: retarget training vectors to best-known profile
Workers kept writing training data to the last-loaded ANN's redis key
(set.ann.redis_key) even when a newer, more specific profile was
already registered in the ZLIST. If that new profile had no ANN data
yet (e.g. controller created a placeholder after a symbols change),
load_new_ann silently left set.ann untouched, so vectors kept piling
up under the old key while the controller waited for the new key's
spam/ham sets to fill — a deadlock visible as repeated "more specific
ann is available" log lines without any retrain ever happening.
Track the best-known profile in set.training_profile, updated on every
process_existing_ann tick whenever sel_elt is picked. ann_push_task_result
now routes target_key and the vectors_len script call through
(set.training_profile or set.ann).redis_key, so training data lands on
the newest profile immediately. set.ann is left alone so inference
keeps using the previously loaded ANN until a fresh one is trained.
- lua_redis: args[0] is allocated as g_malloc(arglens[0]) + memcpy and
is NOT NUL-terminated, so reusing it as a diagnostic label (read
later via %s) could read past the buffer. Store a dedicated
g_strdup'd copy of the command in sp_ud->cmd and use that for the
event label; free it alongside args in the dtor.
- async_session: the overflow counter increments once per event that
could not be placed into the first 32 dump groups, not once per
lost group. Rename overflow_groups -> overflow_events and reword
the suffix to "(+N events not shown)" so the message matches
reality (the lost events may belong to any number of groups).
[Feature] async_session: add labels for redis/fuzzy/lua_http events
Propagate per-subsystem identifying info as event labels so timeout
logs spell out what each stalled event is waiting on:
- lua_redis.c: use the redis command (args[0], e.g. "HGET" / "SET").
- fuzzy_check.c: use rule->name in all four add_event sites so the
stalled backend rule is visible (the plugin may be running many
named fuzzy rules in parallel, and the symcache item is just
FUZZY_CHECK for all of them).
- lua_http.c: use the resolved hostname (cbd->host).
Example timeout line for a task stuck on two Redis calls and a
hostname that won't connect:
[Feature] async_session: annotate DNS and TCP events with operation labels
Add rspamd_session_event_update_label() for mutating a live event's
label field, and use it to give operators a better idea of what a
long-lived event is waiting on at timeout time:
- dns.c: pass rdns_strtype(type) ("A" / "AAAA" / "TXT" / ...) as the
label at add time, so timeout logs show which record type stalled.
- lua_tcp.c: update the label as the per-connection event transitions
between operations ("tcp connect" / "tcp write" / "tcp read").
Timeout summary now reads
"rspamd dns[RBL_FOO/A]=3, rspamd lua tcp[RULE/tcp read]=1"
instead of just "rspamd dns=3, rspamd lua tcp=1".
[Feature] task: route all task sessions through single constructor
Only worker.c and 1 of 8 controller.c task sessions had the item-name
resolver wired, so timeout logs for scans from other entry points
(check, learn, scan-lua, worker_util startup, proxy, lua.task_from_mime)
lost symbol context entirely: the summary said only "rspamd lua http=1"
with no hint of which rule stalled.
Introduce rspamd_task_create_session() that wraps session_create and
set_item_name_resolver, and switch all 11 task-scoped session creation
sites to use it. The two controller sites with cbdata user_data (stats,
metrics) stay on rspamd_session_create — they do not execute symcache.
While here, collapse rspamd_session_describe_pending into a single
line that folds symbol/label info into each group key, e.g.
"total=5; rspamd dns[RBL_FOO]=3, rspamd lua http[X]=1". Previously the
symbol detail was a separate line that was silently omitted whenever
the resolver was not wired — exactly the breakage this fixes.
[Feature] async_session: replace G_STRLOC event_source with item_name + label
event_source was always G_STRLOC, which carries no useful information
for operators: "src/libserver/dns.c:274" tells nothing about which rule
stalled the task. Drop it entirely. The event now carries:
- item_name: auto-snapshot of the currently-running symcache symbol,
resolved through a new session-level callback (wired into task
sessions in worker.c / controller.c to return the current symbol);
- label: optional human annotation (e.g. "tcp write"), NULL by default.
rspamd_session_describe_pending groups by (item_name, label), so the
timeout detail line now reads
"[rspamd dns: RBL_DNSBL_FOO x5, SURBL_CHECK x2]" instead of file:line.
Events that carry neither field are still counted in the subsystem
summary but omitted from the detail line.
As a knock-on simplification, rspamd_session_remove_event_full loses
its G_STRLOC parameter (now just rspamd_session_remove_event), and the
four direct add_event_full callers in lua_udp / lua_http / lua_task /
lua_tcp no longer need to pass rspamd_symcache_dyn_item_name manually —
the auto-resolver handles it uniformly.
When a task hits the timeout the log now includes a grouped summary of
the still-pending async events (DNS / Redis / Lua HTTP / fuzzy check /
...) and the list of symbols that started but have not finished. This
gives operators an immediate picture of which subsystem or rule stalled
the scan, instead of only seeing "forced processing".
The per-event "forced removed event on destroy" lines that previously
printed at info level during rspamd_session_cleanup are demoted to
debug — the new summary replaces them and keeps the log compact.
Patrick C. [Sat, 18 Apr 2026 08:09:43 +0000 (10:09 +0200)]
[Feature] Add --batch-wait option to dmarc_report (#5985)
* [Feature] Add --batch-wait option to dmarc_report
- Add -w/--batch-wait option to rspamadm dmarc_report to wait n
seconds between sending batches. This can be used to avoid
overloading SMTP server with too many messages in a short timeframe.
* replaced os.execute(sleep) with rspamadm_ev_base:sleep
* also delay generation of reports if batch-wait is given
this reduces the peak of DNS requests to avoid overwelming a weak resolver
* fixed indentation
* fix: 0 is true in case of lua, so the correct thing is to compare with 0
* skip last wait when preparing reports
---------
Co-authored-by: Patrick Cernko <pcernko@mpi-klsb.mpg.de>
[Fix] html: preserve verbatim href as url->raw (issue #5986)
When the same URL appears in both the HTML and plain-text parts of a
multipart/alternative message, the HTML variant wins deduplication (HTML
parts are processed first). html_process_url builds a partially
percent-decoded scratch buffer — decoding %40, %3A, %7C, %2F, %3F, %5C
to defeat URL obfuscation — and rspamd_url_parse sets url->raw to that
buffer. Consumers of url:get_raw() then saw a partially decoded form
instead of the bytes that actually appeared in the message.
Re-point url->raw to a mempool-owned copy of the verbatim (trimmed)
href after a successful parse. Length is clamped to G_MAXUINT16 / 2
with RSPAMD_URL_FLAG_TRUNCATED on overflow, matching rspamd_url_parse's
own truncation policy.
Adds test/lua/unit/url_raw_preserve.lua covering plain-text-only,
HTML-only, same-URL-in-both-parts (the issue reproduction), and a
GC-stability check for the mempool-backed raw buffer.
Add 32 HTML5 tags used in modern email (sectioning, media, text-level,
interactive, forms, web components). Notably adds video/audio/source/
track/picture/svg recognition so their URLs and structure are visible
to the parser. Also fixes latent bug where Tag_KEYGEN existed in the
enum but was missing from the defs array.
New tag IDs are appended after Tag_NEXTID so existing tag IDs remain
stable.
[Minor] Preserve real_charset in set_part_binary()
Early exit via set_part_binary() skipped the text_part->real_charset
assignment, leaving it NULL and causing downstream consumers to report
#cs:unk instead of #cs:x-binaryenc for binary parts.
[Fix] Skip ICU conversion for x-binaryenc charset in all detection paths
x-binaryenc is a synthetic name returned by CED (Google Compact
Encoding Detection) to signal binary content, not a real text
encoding. ICU has no converter for it and always fails with
U_FILE_ACCESS_ERROR, producing a misleading warning in the logs.
Add an early exit in rspamd_mime_text_part_maybe_convert() for all
three detection paths: announced-charset missing, announced-charset
unknown, and rspamd_mime_charset_utf_check content-heuristic. When
the detected charset is x-binaryenc, mark the part as raw binary
immediately, consistent with what the existing fallback already does.
Extract set_part_binary() helper and RSPAMD_BINARYENC_CHARSET constant
to eliminate the resulting code duplication.
Fixes the spurious "cannot open converter for x-binaryenc" warning
seen when processing messages with binary MIME parts.
* [Minor] selectors: deduplicate helpers and reuse cheaper task APIs
Extract the "find largest text part" loop into common.largest_text_part
shared by fuzzy_digest and fuzzy_shingles. Drop the pcall wrappers
around get_fuzzy_hashes since the C API does not throw. Use
get_header_count('Received') instead of allocating the full received
headers table just to take its length. Add selector smoke tests for
authenticated and received_count.
* [Minor] selectors: use lua_mime.get_displayed_text_part
Switch fuzzy_digest and fuzzy_shingles to lua_mime.get_displayed_text_part,
which already handles the right MUA-display semantics: prefer non-attachment
HTML, fall back to plain text, then to attachment HTML/text within size
limits, with a minimum-words threshold. Drops the largest_text_part helper
added in the previous commit since it picked the wrong part (e.g. a verbose
text/plain alternative or a .txt attachment).
[Test] lua_feedback_parsers: add unit tests for DSN and ARF
Cover the pure helpers (strip_angles, parse_field_blocks) directly via
internal exports and exercise parse_dsn / parse_arf end-to-end on
synthetic tasks built with rspamd_task.load_from_string. Tests cover
header folding, repeated fields, CRLF normalisation, original-message
extraction, and detection of non-report messages.
Replace local trim/split_lines/normalize_eol with lua_util.str_trim and
lua_util.rspamd_str_split. Drop the safe_get_type_full/safe_get_content
pcall wrappers since the mime_part C API does not throw, and remove the
internal _-prefixed exports that nothing consumes.
[Fix] regexp: do not discard all capture groups that follow an empty one (#5974)
* [Test] Test empty capture groups in the middle of the match
Tests #5973
* [Fix] Empty capture groups in the middle of a match don't discard all remaining groups (PCRE2)
Do not stop on first empty capture group, rather lookup from the end for the last non-empty capture group, and truncate there.
By the way the results array thus gets correctly dimensioned from the start, instead of being truncated afterwards.
Fixes #5973
* [Fix] Empty capture groups in the middle of a match don't discard all remaining groups (PCRE1)
Do not stop on first empty capture group, rather lookup from the end for the last non-empty capture group, and truncate there.
By the way the results array thus gets correctly dimensioned from the start, instead of being truncated afterwards.
Fixes #5973
[Fix] Propagate redis_timeout into nested greylist.redis{} block
When Redis is configured via greylist { redis { ... } }, parse_redis_server
reads opts.redis and ignores opts.timeout, so redis_timeout had no effect.
Shallow-copy the nested redis subtable and set its timeout when not
explicitly configured by the operator.
[Fix] Avoid mutating cached redis_params in greylist
Pass a shallow copy of opts with redis_timeout set before calling
parse_redis_server, so the corrected timeout is baked into the cached
table. The previous post-assignment mutated the shared cached object,
potentially affecting other modules reusing the same Redis config.
[Fix] Check per-worker task_timeout overrides in configtest
The previous fix only validated cfg->task_timeout, missing cases where
individual workers (normal, controller) override task_timeout in their
own config section. Iterate cfg->workers and check each worker's
effective timeout via its UCL options object.
[Fix] Warn on task_timeout less than symcache symbol timeout
Call rspamd_worker_check_and_adjust_timeout during configtest so
misconfigured plugin timeouts are reported at configuration validation
time. Elevate the diagnostic from info to warning level.
Fix rspamd_symcache_add_symbol_augmentation to parse "key=value" format
in augmentation strings, allowing numeric timeout augmentations from
Lua plugins to be stored and compared correctly.
Clarify mx_check timeout comment to explain that the effective symbol
timeout includes dns.timeout in addition to the configured value.
[Fix] Separate greylisting period from Redis connection timeout in greylist
settings.timeout (greylisting period, 5 min) was being picked up by
lua_redis as the Redis connection timeout, inflating the symbol's
augmentation timeout to 300s. Add redis_timeout (default 1.0s) and
explicitly set redis_params.timeout after parse_redis_server.
[Fix] Fix infinite loop and OOB read in archive processing
- RAR v5 extra area parsing: fix wrong loop boundary (used advanced `p`
instead of `ex + extra_sz`), wrong advancement (`cur_sz` alone didn't
account for the size vint bytes), and wrong remain calculation for the
second vint read. Add bounds validation for `cur_sz`.
- 7zip codec ID: validate `p + sz <= end` before reading codec ID bytes
to prevent out-of-bounds read with malformed archives.
[Feature] Allow custom response codes for fuzzy dynamic blocks
Dynamic blocks always returned 503, making them indistinguishable
from static blacklist blocks on the client side. Ratelimit scripts
need to return 403 so that fuzzy_check clients can map the response
to FUZZY_RATELIMITED instead of FUZZY_FORBIDDEN.
Add response_code field to rspamd_fuzzy_dynamic_ban and propagate
it through rspamd_fuzzy_check_client (now returns int: 0=allowed,
>0=block code) and rspamd_fuzzy_block_addr. The Lua API
block_fuzzy_client accepts an optional 6th argument for the code.
[Test] Add functional test for rspamadm autolearnstats command
Scans sample messages through rspamd to generate autolearn logs dynamically,
then validates autolearnstats output parsing, column headers, and filtering.
Replace the naive split-by-semicolon + regex approach in
parse_arc_header with a proper LPeg-based parser that respects
RFC 8601 comments (...) and quoted strings "..." when splitting
AAR elements. The old unanchored pattern matched "i=N" inside
comments like arc=pass ("domain:s=sel:i=2"), misclassifying
the element as an ARC instance index. This left cur_aar.ar nil,
crashing gen_arc_seal_cb and producing cv=fail on valid chains.
Add parse_aar_header() to lua_auth_results.lua with shared LPeg
atoms for comment/quote-aware semicolon splitting, and unit tests
covering the #5963 scenario and edge cases.
[Fix] Keep original fd in proxy session after milter dup
Revert the client_sock = -1 added in 2c56437e7 which incorrectly
assumed the milter library owned the accepted fd. The milter library
calls dup() internally (since 4a75d59cc) and works with the copy,
so the original fd must stay in session->client_sock for
proxy_session_dtor to close it. Without this, every milter connection
leaked one fd permanently, causing unbounded CLOSE_WAIT accumulation.
This restores the pre-2c56437e7 behaviour where proxy_session_dtor
closed the original fd and the milter dtor closed the dup'd copy.
[Fix] Use task_timeout instead of upstream timeout for proxy self-scan
The proxy self-scan path was using default_upstream->timeout (the
milter/HTTP wire timeout, typically 120s) as the task processing
timeout. This caused tasks to linger for up to 2 minutes before
being reaped, leading to massive CLOSE_WAIT fd accumulation under
load.
Use ctx->task_timeout (derived from cfg->task_timeout, default 8s)
instead, matching how the normal worker handles task timeouts. The
previous else-if branch also had a bug where the repeat interval
used upstream timeout while the initial used cfg->task_timeout.
[Feature] Expose ratelimit_whitelist check to Lua via worker method
Add worker:is_ratelimit_whitelisted(addr) Lua method that checks the
fuzzy worker's existing ratelimit_whitelist radix map, so Lua scripts
can skip provisional blocks for whitelisted IPs (e.g. spamtraps).
[Fix] Use ev_now() instead of rspamd_get_ticks() for dynamic ban expiry
rspamd_get_ticks(FALSE) returns CLOCK_MONOTONIC (seconds since boot),
while rspamd_util.get_time() in Lua returns ev_time() (wall-clock unix
epoch). Using mismatched clocks made expire_ts comparisons nonsensical.
Switch to ev_now(ctx->event_loop) which matches the ev_time() epoch used
by Lua callers of block_fuzzy_client.
[Feature] Add dynamic block API to fuzzy storage worker
Adds worker:block_fuzzy_client(addr, prefix_len[, expire_ts[, reason]])
Lua method that inserts a CIDR network block into a per-worker btrie
(dynamic_blocked_nets). Checked in rspamd_fuzzy_check_client() after
the static blocked_ips map.
expire_ts is an absolute monotonic timestamp (0 = permanent). Callers
compute get_time() once and pass it to a batch of block calls — C never
calls get_time() itself. Re-inserting an existing prefix updates the
expire_ts and reason in-place (idempotent).
Each fuzzy worker independently applies bans via Redis polling; no new
IPC commands or control socket extensions are needed.
Vsevolod Stakhov [Tue, 31 Mar 2026 15:57:12 +0000 (16:57 +0100)]
[Rework] Make hard deps opt-in, weak deps default
Invert the dependency semantics for backward compatibility:
- Default (no flag) = weak/soft dep, no cascade-disable (old behavior)
- hard=true = cascade-disable when dep is disabled by settings
This preserves backward compat: all existing register_dependency
calls without flags behave exactly as before. Hard cascade-disable
is an explicit opt-in feature.
Lua API: register_dependency('A', 'B', true) now means hard dep.
Remove redundant weak annotations from plugins (default is weak).
Vsevolod Stakhov [Tue, 31 Mar 2026 11:08:09 +0000 (12:08 +0100)]
[Rework] Replace GHashTable with id_list for force-enabled IDs
Move force_enabled_ids from GHashTable mempool variable to a
native id_list* field in symcache_runtime. Uses lazy allocation
and cleanup in savepoint_dtor. Avoids GLib in C++ internals
and mempool variable indirection.
Vsevolod Stakhov [Tue, 31 Mar 2026 10:45:33 +0000 (11:45 +0100)]
[Fix] Use finished status in disable_all_symbols
disable_all_symbols implements "disable all, then enable some"
for symbols_enabled/groups_enabled. Using `disabled` status here
caused cascade-disable of hard dependents: if an enabled symbol
depended on a non-enabled one, the enabled symbol was wrongly
cascade-disabled.
Revert to `finished` for disable_all_symbols. Reserve `disabled`
status for explicit symbol/group disable via settings rules
(disable_symbol), where cascade-disable IS the intended behavior.
projects / thirdparty / rspamd.git / log
