Register the historyreset/history response listeners before the click
via Promise.all. waitForResponse only catches responses arriving after
it is set up, and the fast local historyreset round-trip could beat a
listener attached post-click, causing a flaky 10s timeout under CI load.
Remove jQuery from the RequireJS config (paths, app/bootstrap shims) and
delete the vendored jquery-3.7.1.min.js. All app modules now load without
jQuery; the bootstrap shim's stale "Popovers require jQuery" comment and
deps entry are removed.
Drive-by: rename the modalDialog data-backdrop attribute to its Bootstrap
5 name data-bs-backdrop, so the static-backdrop behaviour is actually
applied (the BS4 attribute has been silently inert).
Migrate libft.js off jQuery. $("<tag>", {hash}) element construction
becomes common.el, $.extend becomes spread, $.each becomes forEach.
The column-options dropdown's namespaced hidden.bs.dropdown.rspamd
rebind (jQuery namespaces) becomes a common.data-tracked
addEventListener that removes the previous handler before re-binding;
the delegated fuzzy-hash buttons (re-bound every render) use a cleanup
map so common.delegate doesn't stack. $(arr).not(arr) set-diffs become
Array.some/ includes, and .siblings()/.attr()/.html()/.val() move to
native DOM.
Migrate rspamd.js off jQuery. The /stat connect probe moves to a raw
XHR (avoiding common.query's error logging on the normal "not logged
in" path); $.ajaxSetup and $(document).ajaxStart/Complete are dropped
since no $.ajax calls remain. tabClick disables/enables the nav bar
controls via attribute+class helpers (li/a need both for Bootstrap);
:radio + .val([v]) and .data("value") become checkRadio / activeMenuValue
helpers; delegated handlers move to common.delegate; BS5 events move to
addEventListener (with {once:true} replacing self-removing .on/.off);
the popover content uses cloneNode and the outside-click dismiss uses
.contains(). Form/input re-binds become onsubmit/oninput properties.
Migrate stats.js off jQuery. The delegated mouseenter/mouseleave hover
highlighting (which doesn't bubble natively) is reimplemented on
mouseover/mouseout with a relatedTarget-containment check; the jQuery
sibling-traversal chains (prevAll/nextAll/slice/find) move to native
sibling walks. $.each becomes Object.entries/forEach, $(html).appendTo
becomes insertAdjacentHTML, and $.wrapAll becomes common.el DOM
construction. The legacy /auth fan-out keeps a raw XHR to preserve its
custom error message and alerted-session dedup; $.when().always becomes
Promise.all().finally. Adds common.getAjaxTimeout so direct XHRs honour
the configured timeout.
Migrate upload.js off jQuery: $.Deferred/$.when to Promise/Promise.all,
$.trim to trim, :selected to selectedIndex, multi-arg :not() to CSS,
.val/.prop/.attr/.find/.append/.empty/.children/.closest/.parent to
native DOM, .animate({scrollTop}) to scrollTo, and $("<option>") to
common.el. Option metadata stored via .data() (objects/arrays) moves
to common.data; the fuzzy-picker change handler is bound once instead
of re-bound on every refresh.
Migrate config.js off jQuery: HTML-string/$("<tag>") element building
to common.el, $.each to forEach, the :input/:checked pseudos to CSS
selectors, .data() round-trips (object values) to common.data, the
delegated .map-link handler to common.delegate, and .attr()/.val()/
.on()/.html()/.remove() to native DOM and addEventListener.
Map cells and the modal title now escape server data via textContent
(consistent with the leaf-modules hardening).
Migrate graph.js, selectors.js, history.js, and symbols.js off
jQuery: $.extend/$.each/$.trim and $().val/prop/attr/hasClass/is/
addClass/removeClass/change/click/on to native DOM and the shared
common.el / common.delegate helpers added in the common.js stage.
Delegated handlers on Tabulator-rendered DOM (.scorebar) move to
common.delegate; the repeated button disable/enable patterns
collapse into small per-module helpers.
jQuery remains loaded for the remaining modules (config, libft,
stats, upload, rspamd) — removed in later stages.
Rewrite the AJAX layer (query/queryServer) on XMLHttpRequest,
preserving jQuery's semantics: neighbours fan-out and aggregation,
per-server error dispatch, statusCode, timeout, GET data appended
to the URL as a query string, default Content-Type for POST bodies,
NProgress download progress, and jqXHR-compatible callback arguments.
Add a spinner pub/sub (onAjaxStart/onAjaxComplete with an in-flight
counter honoring global:false) and setAjaxTimeout, plus shared
vanilla helpers: el, delegate, data. Migrate alertMessage, hide,
show, toggle (Web Animations API), the error-log UI and fileUtils
to plain DOM.
Bridge rspamd.js to the new APIs (setAjaxTimeout, onAjaxStart/
onAjaxComplete) alongside the jQuery hooks, which stay for the
remaining direct $.ajax calls until a later stage. jQuery is still
loaded; this is stage 1 of the staged removal.
Replace the Bootstrap 5 jQuery bridge (.modal/.popover/.trigger
"click.bs.dropdown") with the vanilla constructor API
(bootstrap.{Modal,Popover,Dropdown}.getOrCreateInstance) in
rspamd.js, config.js and libft.js, capturing the bootstrap module
where needed.
Prerequisite for removing jQuery: the bridge only activates when
jQuery is loaded, while the vanilla API works regardless. With jQuery
still present the existing $(el).on("*.bs.*") subscriptions keep
firing, so those stay jQuery for now and move to addEventListener
together with the rest of the jQuery removal. The main.js shim
deps:["jquery"] becomes redundant and is dropped in the final stage.
Replace the jquery.stickytabs plugin with an inline vanilla
implementation using the BS5 Tab API, removing the last jQuery
bridge dependency of that plugin. Behavior (URL hash persistence
and initial-tab fallback) is preserved.
D3Evolution 2.0.4 -> 2.1.0, D3Pie 1.1.2 -> 1.2.0. Both libraries
dropped the jQuery dependency (replaced $.extend with an internal
deep-merge), so drop the stale "jquery" shim deps in main.js.
Removing footable.standalone.min.css dropped its .form-group
{margin-bottom:15px} rule, collapsing the gap above the buttons on
the selectors tab (and the action-score rows on the configuration
tab). .form-group is a Bootstrap 3 class with no meaning in BS5;
replace it with the mb-3 spacing utility that provides the same gap.
installScrollPreservation bound listeners on the mount element and
window without removing the previous set. Each column-options rebuild
(destroy + re-init on the persistent mount element) stacked a new set
that was never released — a listener/memory leak on the history and
scan tables. Mirror bindRowClickToggle's WeakMap guard: key the set by
mount element and tear it down before re-binding.
FooTable was superseded by Tabulator across all tables (Phases 0-3).
Remove the vendored library/CSS, the footable-fontawesome icon module,
and every remaining reference:
- index.html: drop the two <link> tags
- main.js: drop the requirejs path and shim
- rspamd.js: drop the app/footable-fontawesome require; stickyTabs
runs directly (declared module dependency)
- common.js: delete dead appendButtonsToFtFilterDropdown (no callers)
- rspamd.css: drop all .footable rules and the dead .footable-details /
.footable-filtering-search selectors
Rename column-options dropdown hooks ft-columns-* -> tab-columns-*
(Tabulator; consistent with tab-utils.js). Legacy history path kept.
Add a per-table boolean search box (history/scan) with a query
language ported from FooTable's built-in filtering: whitespace =
AND, OR, - (exclude), "exact phrase". The predicate ANDs with the
per-column header filters.
Also add headerFilter:"input" to the text columns (matching the
errors-table convention) and align all header-filter inputs on a
shared bottom baseline (title-holder flex-grow) regardless of how
many lines a wrapped title occupies.
Migrate the last two FooTable tables, #historyTable_history and
#historyTable_scan, to Tabulator. Both share rendering code in libft.js
(process_history_v2/columns_v2, initHistoryTable, the column-options
dropdown and the symbol-order toggle).
- Action column uses a custom headerFilter (select + "not" checkbox)
with headerFilterFunc for exact matching.
- Column-options dropdown keeps Visible/Hidden plus "Row", emulated
via responsive:100 and persisted to localStorage.
- Symbol-order "Sort by:" buttons are embedded in the symbols column
title, so they are recreated on every render; the active state is
reapplied on tableBuilt/renderComplete.
- columnDefaults uses the "html" formatter so upstream-escaped values
decode instead of being re-escaped by "plaintext".
- Shared helpers (scroll preservation, etc.) live in tab-utils.js.
Update the scan/symbols Playwright specs for the Tabulator DOM and
replace the racy "disabled during reload" check with response waits.
- Extract reusable Tabulator helpers (scroll-prevention, tabindex
removal, footer-hide, row-click-toggle, scrollIntoView guard) from
history.js into a new tab-utils.js module — no footable dependency,
so graph.js no longer loads footable.min.js for the throughput tab
- Refactor history.js initErrorsTable to use tab-utils helpers
- Migrate the rrd summary table (graph.js) from FooTable to Tabulator:
row coloring via CSS variable, in-place updates via updateData,
dynamic column titles via updateColumnDefinition, header sorting
- Update rspamd.css: rrd-table selectors for Tabulator DOM, row color
via --rrd-row-color (visible in dark mode), row hover, layout tweaks
Replace FooTable with Tabulator (v6.4.0) for the errors log table — the
first step of the FooTable→Tabulator migration. Tabulator is vanilla-JS and
jQuery-free, so it survives the planned jQuery removal; FooTable is
unmaintained (GPLv3, last release 2017).
- Vendored tabulator.min.js + tabulator_bs5.min.css; wired via RequireJS
- Rewrote initErrorsTable on the Tabulator API: columns, local pagination,
responsive collapse, per-column header filters
- rspamd.css: map the bs5 theme to Bootstrap CSS variables so light/dark
flip automatically with data-bs-theme
- Row click toggles responsive collapse (only when collapse is active);
pagination footer hidden on a single page (FooTable parity)
- Scroll prevention: Tabulator scrolls the page internally on interactions;
the call is elusive, so window.scrollY is preserved/restored around
clicks/Update. Row hover backgrounds disabled to avoid the resulting
expand-scroll flicker (documented inline for future maintainers).
Stefan Benten [Tue, 30 Jun 2026 20:02:26 +0000 (22:02 +0200)]
[Feature]: src/rspamadm: allow reading password from env (#6114)
* src/rspamadm: allow reading password from stdin
Currently the cli only allows reading the password from tty.
This makes it challenging to check the password via automation,
ie. used by continous deployment to ensure a clean state.
This change adds the functionality to read from stdin, so that
passwords can be piped in.
* test/functional/cases: test for stdin password checks
Vsevolod Stakhov [Tue, 23 Jun 2026 07:58:48 +0000 (08:58 +0100)]
[Feature] rspamadm: discover command modules from external directories
rspamadm discovered Lua sub-commands by globbing only the built-in
LUALIBDIR/rspamadm directory, so a third-party or premium package could
not ship a rspamadm command without writing into the OSS-owned lualib
tree.
Scan two additional sources, in order, after the built-in directory:
- $CONFDIR/rspamadm.d/*.lua, a drop-in dir consistent with local.d
- every directory in the colon-separated RSPAMADM_COMMAND_PATH env var
(mirrors how PATH works)
Duplicate command names are skipped (first wins), so built-in commands
can never be shadowed. Externally loaded modules run with the same
globals and lua_path as built-in ones, so they can require premium
lualibs and use lua_redis. No behavior change when the env var and the
.d directory are absent.
Factor the per-file loader and per-directory scan into helpers, and fix
a latent out-of-bounds in the basename fallback (it searched .lua in the
full path but indexed the basename buffer). Document discovery order and
the new env var in the man page.
Vsevolod Stakhov [Tue, 23 Jun 2026 07:57:22 +0000 (08:57 +0100)]
[Fix] lua_kann: correct load arg index and save return value
rspamd_kann.load is a module function, not a method, so the options
table is at arg 1. The lua_istable(L, 1) branch incorrectly read the
filename field from arg 2 (a copy-paste from lua_kann_save where arg 1
is self), so the documented load({filename = ...}) form always failed
with 'missing filename'.
Also fix lua_kann_save: the trailing lua_pop popped the boolean result
it had just pushed, so save({filename = ...}) returned the filename
string instead of true. Pop the filename field before pushing the
result.
Add a regression test covering the file round-trip via the single-table
form.
[Test] Run push-triggered CI on all branches in forks
rspamd/rspamd restricts push to master (71605c25) to avoid the duplicate
push+pull_request run on feature branches that leaves a spurious *cancelled*
status. This trade-off is acceptable for the main repo where contributors
open PRs, but inconvenient in forks where CI is needed on every push without
opening a PR.
Vsevolod Stakhov [Sat, 20 Jun 2026 19:20:19 +0000 (20:20 +0100)]
[Feature] neural: forced-learn fast path and first-class freeze
Two training controls plus a supporting task primitive:
* train.forced_learn_minimal_scan (default on when disable_symbols_input):
a high-priority neural prefilter disables every non-neural symbol on an
ANN-Train scan, so a symbols-independent training vector is built without
issuing RBL/DNS, fuzzy, bayes, ClickHouse or capture/cluster work. The
stored vector and the profile key are byte-for-byte identical to the live
full-scan path (asserted in tests). For symbol-dependent rules it stays off,
and if any applicable neural rule needs symbols the whole task falls back to
a full scan.
* train.frozen: stops automatic training and auto-storing of live vectors so a
frozen model's pools cannot drift into an imbalanced live set, while
inference keeps serving the current ANN unchanged. An explicit ANN-Train
still stores and retrains on demand (gated by a per-profile retrain marker).
Supersedes the auto-learn side of store_set_only/store_pool_only; both keep
working when frozen is unset.
* task:disable_all_symbols([skip_mask]): Lua binding over the existing
rspamd_symcache_disable_all_symbols "process only these" primitive (defaults
to keeping explicit_disable symbols), used by the prefilter.
Functional coverage in test/functional/cases/330_neural/006_forced_learn_minimal
and 007_frozen.
Vsevolod Stakhov [Fri, 19 Jun 2026 11:31:37 +0000 (12:31 +0100)]
[Minor] lua_text stats: use 1-based byte indexing
The byte-distribution statistics methods (entropy, byte_mean,
byte_deviation, serial_correlation, monte_carlo_pi) took a 0-based
offset, which was inconsistent with the rest of the rspamd_text API
(span/sub/at) and with text:crc32. Switch the optional range argument
to a 1-based start index so all rspamd_text slicing uses the same
convention; len semantics and the lenient out-of-range-yields-0
behaviour are unchanged. Docs and tests updated accordingly.
Vsevolod Stakhov [Fri, 19 Jun 2026 11:28:15 +0000 (12:28 +0100)]
[Feature] lua: zlib/YARA-compatible crc32
Expose standard CRC-32 (poly 0xEDB88320, init/final 0xFFFFFFFF XOR),
computed via zlib crc32() so it is bit-exact with YARA hash.crc32:
- rspamd_cryptobox_hash.create_specific("crc32"): streaming
update + final, big-endian out so :hex() yields e.g. cbf43926
- rspamd_text:crc32([start[, len]]) and
rspamd_util.crc32(input[, start, len]): return the checksum as a
Lua integer over a zero-copy 1-based slice (no buffer copy)
Tests cover golden values ("" -> 0, "123456789" -> 0xCBF43926),
1-based slicing, streaming-vs-one-shot, reset, and a real
attachment-like buffer cross-checked against zlib/YARA crc32.
Vsevolod Stakhov [Fri, 19 Jun 2026 09:37:44 +0000 (10:37 +0100)]
[Feature] lua_archive: extraction limits vs zip bombs
archive.unpack/unzip/untar read whole members into memory via
libarchive, so a tiny hostile archive could expand to gigabytes
(OOM). Add an optional opts table that caps extraction, enforced
while reading so memory stays bounded:
- max_output total uncompressed bytes across all members
- max_file_size per-member uncompressed cap (truncated at cap)
- max_files member count cap
- max_ratio per-member uncompressed/compressed ratio cap
Ratio uses real compressed bytes consumed via
archive_filter_bytes(a, -1), checked per chunk above a 64 KiB
floor, so a bomb is stopped early even with no size cap set. All
limits are opt-in (0/absent = unlimited), preserving the previous
behaviour. A second boolean return flags truncation so a capped
extraction is never mistaken for a complete one.
Vsevolod Stakhov [Thu, 18 Jun 2026 18:16:54 +0000 (19:16 +0100)]
[Minor] CI: trigger push only on master to dedup PR runs
A commit on a branch with an open PR fired both a push and a
pull_request run; concurrency cancel-in-progress reaped one, leaving a
spurious *cancelled* run that reads like a CI failure. Restrict push to
master so feature branches run a single pull_request workflow; key the
concurrency group on PR number / ref to still cancel superseded runs.
Add byte-distribution statistics as methods on the rspamd_text class,
implemented in C++20 under src/lua (lua_text_stats.{hxx,cxx}); lua_text.c
is left untouched and the rspamd{text} metatable is augmented at load.
Methods (each takes an optional 0-based (off, len) range, defaulting to
the whole buffer):
- text:entropy([off[, len]]) Shannon entropy, bits/byte
- text:byte_mean([off[, len]]) mean of unsigned byte values
- text:byte_deviation(mean[, off[, len]]) mean abs deviation from mean
- text:serial_correlation([off[, len]]) ENT serial correlation
- text:monte_carlo_pi([off[, len]]) ENT Monte-Carlo Pi deviation
The core is header-only, allocation-free and O(n) (a single histogram
pass shared by entropy/mean/deviation) and produces deterministic,
bit-reproducible results. Offsets are byte offsets, 0-based; the range is
clamped to the buffer and an out-of-range or empty range yields 0.
Add C++ doctest golden-vector tests (analytically-derived exact values)
and Lua unit tests covering empty/single-byte/uniform/two-symbol buffers,
overlapping groups, slicing and edge cases.
Vsevolod Stakhov [Thu, 18 Jun 2026 13:22:25 +0000 (14:22 +0100)]
[Feature] multipattern: explicit SOM flag and offset docs
SOM (start-of-match) reporting already exists on master as the default
(hyperscan compiles every pattern with HS_FLAG_SOM_LEFTMOST), but there
was no explicit way to request it and the offset convention was
undocumented.
- Add RSPAMD_MULTIPATTERN_SOM (rspamd_trie.flags.som): an explicit
opt-in for start offsets that also overrides no_start/single_match
(forces SOM and drops the incompatible SINGLEMATCH).
- Document the offset convention: pattern id is 1-based; match start
and end are byte offsets, 0-based, start inclusive and end exclusive
(one past the last matched byte), so end - start is the match length.
- Fix the regex (flags.re) fallback used when hyperscan is unavailable:
it discarded the real PCRE start and reported end - strlen(pattern),
which is bogus for variable-length matches. It now reports the true
start/end from rspamd_regexp_search.
Add C++ (rspamd_cxx_unit_multipattern.hxx) and Lua (trie.lua) unit
tests asserting (id, start, end) against hand-computed positions:
multiple/overlapping occurrences, icase, literal vs regex, no-match,
SOM-overrides-single_match and a large buffer. Existing rspamd_trie
behaviour and its callers (url.c, lang_detection, lua plugins) are
unchanged.
Vsevolod Stakhov [Wed, 17 Jun 2026 17:52:16 +0000 (18:52 +0100)]
[Fix] monitored: alphanumeric-only random DNS prefixes
random_monitored RBL checks built random labels from an alphabet that
included '-' and '_'. That produced names like '_Q8...0-' (leading
underscore, trailing hyphen) or '-7d0...' (leading hyphen) which are
not valid DNS labels (RFC 952/1123: no leading/trailing hyphen, no
underscore in hostnames). Authoritative DNSBL servers such as
spfbl.net reject these with SERVFAIL.
Restrict the alphabet to alphanumerics, which always forms a valid
label regardless of position while keeping ample entropy.
Vsevolod Stakhov [Tue, 16 Jun 2026 19:05:26 +0000 (20:05 +0100)]
[Fix] lua: refcount coroutine thread entries
The state/generation guards from the previous commit read the entry on
resume, so they only help while the entry is still allocated. If the
owning task is torn down while an async request is in flight, the entry
could be freed before the late completion fires, turning the guard into
a use-after-free.
Make thread_entry refcounted (ref.h, non-atomic - workers are single
threaded). The pool holds the initial reference; terminate_thread() and
the pool-full path now drop it via REF_RELEASE instead of freeing
directly, so the struct is destroyed only once the last reference goes
away. Every async library that stashes an entry for a later completion
now takes its own reference and drops it when done:
- dns/util: retain at the request, release in the one-shot callback.
- http/redis: retain at yield, release in the cbdata/ctx destructor
(and, for redis, at each point that consumes ctx->thread).
- tcp: retain at each yield, release at the matching resume, since the
tcp cbdata has several direct-free error paths that bypass its
destructor; pairing with yield/resume keeps the balance exact and
leaves bad-argument paths (which return before yielding) untouched.
Combined with the generation guard, a completion that races task
teardown now finds the entry alive but DEAD/recycled and refuses the
resume, instead of dereferencing freed memory.
Vsevolod Stakhov [Tue, 16 Jun 2026 15:35:34 +0000 (16:35 +0100)]
[Fix] lua: state management for coroutine thread pool
Async libraries (dns/redis/tcp/http/util) capture the "currently
running" coroutine at a yield point and resume it later from a C
completion callback. Nothing previously guaranteed the entry resumed
was still the one captured: a double-fired event, a completion racing
task teardown, or an entry recycled into another task would resume the
wrong (or freed) coroutine and corrupt memory. These failures are
interleaving-dependent and invisible in isolation.
Give each pooled thread an explicit lifecycle (FREE/RUNNING/YIELDED/
DEAD) plus a generation counter, both carried in the existing
thread_entry and per-request cbdata structs - no new allocations on any
hot path:
- get/return/terminate/yield/resume enforce legal state transitions,
so returning a suspended thread or resuming a non-suspended one now
aborts at the exact violation instead of corrupting a core later.
- lua_thread_resume_checked() refuses to resume unless the thread is
still YIELDED and its generation matches the value snapshotted at
the yield point; a stale/duplicate completion becomes a logged
no-op rather than a wrong-coroutine resume. All five async libs are
migrated to it.
- lua_tcp keeps cbd->thread pointing at the coroutine actually
yielded by sync read/write, so the resume always targets it.
generation is bumped on every acquire and release, so an entry that
goes back to the pool and is handed out again no longer matches a
completion that was already in flight.
Vsevolod Stakhov [Tue, 16 Jun 2026 07:54:45 +0000 (08:54 +0100)]
[Fix] dns: do not defer resolver nameservers (fixes #6096)
A nameserver that failed to resolve at config time was turned into a
zero-address PENDING_RESOLVE upstream by 904fd6218, then dereferenced as
NULL in rspamd_dns_server_init -> SIGSEGV at worker startup (regression
in 4.1.0). DNS resolver nameservers are consumed synchronously by
rdns_resolver_add_server and can never be promoted async, so never defer
them; also NULL-guard rspamd_dns_server_init as defense in depth.
Vsevolod Stakhov [Mon, 15 Jun 2026 08:59:47 +0000 (09:59 +0100)]
[Fix] mx_check: loopback-only MX is LOCAL, not bogon
A domain whose MX resolves only to loopback is hosted on the scanning
host itself -- a self-MX, typically the host's own FQDN mapped to
127.0.0.1 in /etc/hosts, which rspamd's resolver honours as a fake reply
that shadows public DNS. That made fully DMARC-aligned self-hosted mail
score MX_BOGON_ONLY (+8.0): the strongest "not spam infrastructure"
signal treated as the strongest spam signal.
Move 127.0.0.0/8 and ::1/128 from BOGON_CIDRS to LOCAL_CIDRS so a
loopback-only MX emits MX_LOCAL_ONLY (3.0) instead. test_mode now lifts
loopback out of the LOCAL set (was: bogon) so the probe path stays
exercisable against local listeners.
Add a regression test (170_mx_check_selfmx.robot, test_mode = false):
the existing suites run test_mode = true and cannot cover the production
loopback-classification path.
Vsevolod Stakhov [Sat, 13 Jun 2026 12:18:18 +0000 (13:18 +0100)]
[Feature] neural: sequence output mode and SIF word selection in fasttext_embed
Add a generic word-vector sequence output to the fasttext_embed provider
so that custom ANN architectures (e.g. attention pooling) can learn their
own pooling instead of receiving a pre-pooled vector:
* output_mode = "sequence": emits the first max_words word vectors
flattened word-major and zero-padded to max_words * channels.
* word_selection = "sif": since order-invariant poolers do not need a
prefix, optionally fill the sequence with the max_words most
distinctive (highest SIF weight) words from anywhere in the message
instead of the leading ones. Default stays "prefix".
This is the data half only; how the sequence is consumed is left to the
ANN architecture. Both modes are bounded by max_words.
Vsevolod Stakhov [Sat, 13 Jun 2026 12:18:01 +0000 (13:18 +0100)]
[Feature] neural: pluggable feature-provider and ANN-architecture registries
Turn the neural plugin into an extension point so third-party (including
closed-source) modules can add feature providers and network topologies
without patching the core.
* register_architecture(name, builder) / get_architecture(name): a
registry of ANN builders, function(n_inputs, rule) -> kann network.
The built-in 'symbol', 'embedding' and 'conv1d' architectures are now
registered through it; create_ann() dispatches on rule.architecture
and falls back to the historical auto-selection when it is unset, so
existing configs are unaffected.
* register_provider (already present) and register_architecture are
exported from the neural module, so a module that does
require 'plugins/neural' can register a custom provider or
architecture and select it with provider type / rule.architecture.
An unknown rule.architecture now fails loudly with a hint that the
providing module may not be loaded, instead of silently falling back.
Vsevolod Stakhov [Sat, 13 Jun 2026 12:17:42 +0000 (13:17 +0100)]
[Feature] lua_kann: expose slice and concat transforms
kad_slice and kad_concat_array were already implemented and serialized
in kautodiff but not reachable from Lua. Exposed as
rspamd_kann.transform.slice(node, axis, start, end) (0-based, end
exclusive, batch is axis 0) and rspamd_kann.transform.concat(axis,
node1, node2, ...). These make split/bypass/merge graph topologies
buildable from Lua, e.g. routing different parts of a fused input
vector through different sub-networks (needed by custom ANN
architectures such as attention pooling with late fusion).
New kad operator attn_pool (op 38, appended to preserve model
serialization compatibility): multi-head dot-product attention pooling
over a zero-padded sequence of word vectors with learned query vectors.
All-zero positions are treated as padding and masked out of the
softmax; attention weights are stashed in gtmp between the forward and
backward passes. Exposed as kann_layer_attn_pool() and
rspamd_kann.layer.attn_pool(node, n_words[, n_heads]).
Verified: converges on a needle-in-haystack task unsolvable by a flat
dense net (0.985 vs 0.715 accuracy), exact word-order invariance of the
pooled output, padding determinism and save/load roundtrip.
Vsevolod Stakhov [Fri, 12 Jun 2026 14:35:59 +0000 (15:35 +0100)]
[Fix] neural: stabilize training on dense embedding inputs
Training an ANN on dense provider features (fasttext_embed, text_hash)
could silently produce a degenerate model: with the historical
learning_rate=0.01 default, RMSprop drives the net into tanh saturation
depending on weight init luck - the loss freezes, yet the constant
all-one-class model is saved and classifies every message as spam (or
ham) until the next retrain. On a real corpus this happened in roughly
one of three weight inits.
Fixes:
* use the embedding (funnel) architecture for any rule with dense
feature providers, not only LLM ones: the simple symbol architecture
applies ReLU directly to the input, clipping the negative half of the
embedding space, and is the least stable option on such vectors
(it is also less accurate; layernorm in the funnel fixes the
conditioning)
* resolve the learning_rate default by input type: 0.01 for symbol
vectors as before, 0.001 for dense embeddings, which converges
reliably with equal accuracy; an explicit config value still wins
* add a quality gate to the training child: a model with constant or
single-class output on its own training set is rejected instead of
saved; the lock is released and training retries on the next cycle
with a different weight init, which converges in practice
* return an explicit msgpack rejection marker from the training child
instead of nil on the gate/NaN paths: a nil return used to deadlock
the controller against the training subprocess (see the lua_worker
fix) and stalled training forever
Vsevolod Stakhov [Fri, 12 Jun 2026 14:35:41 +0000 (15:35 +0100)]
[Fix] lua_worker: do not deadlock when subprocess returns an invalid value
When a function run via worker:spawn_process returned nil (or any
non-string value), the child logged an error but wrote nothing to the
result pipe. The parent then kept waiting for a reply while the child
blocked forever on the post-reply ack read, deadlocking both processes
and anything serialised behind them (e.g. the neural training lock,
which got extended indefinitely so training never retried until a full
restart).
Report invalid return values to the parent as a regular error reply so
on_complete fires and the caller can recover.
Vsevolod Stakhov [Fri, 12 Jun 2026 10:46:34 +0000 (11:46 +0100)]
[Fix] protocol: use case preserving boundary for HTTP multipart parsing
RFC 2046 boundaries are case sensitive, but the v3 HTTP multipart
callers used ct->boundary, which is lowercased for MIME clients quirks,
so requests with uppercase characters in the boundary failed to parse.
Use ct->orig_boundary in protocol.c, rspamd_proxy.c and rspamdclient.c.
Vsevolod Stakhov [Fri, 12 Jun 2026 00:15:50 +0000 (01:15 +0100)]
[Feature] css: detect more text hiding tricks
Extend the invisible-text detection with several common CSS hiding
techniques used to dilute visible content with hidden ham text:
- off-screen positioning: position:absolute|fixed with a large negative
left/top
- image-replacement text-indent: a large negative text-indent
- clip / clip-path collapsing the element to a zero area, e.g.
rect(0,0,0,0), inset(100%), circle(0)
- visibility:collapse (treated as hidden)
- tiny font sizes (<= 3px), not only font-size:0
These are modelled as a hidden display in compile_to_block so the
hiding correctly propagates to descendants. The previous off-screen
heuristic was a fragile substring match on the raw style that only
incremented a feature counter and never actually hid the text; it is
replaced by structured parsing of the position/left/top/text-indent/
clip properties, with the offscreen feature counter now driven by a
flag set on the compiled block.
Vsevolod Stakhov [Thu, 11 Jun 2026 22:02:56 +0000 (23:02 +0100)]
[Fix] css: detect text hidden via overflow clipping, opacity and max-* sizes
Phishing messages dilute the visible content with hidden ham text using
CSS hiding techniques that the parser did not understand:
- 'max-width:0; max-height:0; overflow:hidden' was fully ignored as
max-width/max-height/overflow were not parsed at all
- 'opacity:0' was parsed but the value was silently discarded in
compile_to_block
- 'height:0' was applied to the block width due to a copy-paste bug,
and zero dimensions were never considered by compute_visibility
Fixes:
- parse max-width/max-height (clamping width/height) and overflow
- treat a block with zero height or width and overflow:hidden as
invisible, propagating it to descendants via the display value
- treat opacity < 0.1 as a hidden display, as descendants cannot reset
the ancestor opacity
- do not allow a child display value to resurrect content of a hidden
ancestor in propagate_block (display:none is not resettable in CSS)
- fix the height->width copy-paste bug and a missing break that made
the font-size case fall through into the opacity case
Vsevolod Stakhov [Thu, 11 Jun 2026 17:20:02 +0000 (18:20 +0100)]
[Fix] mime: fix build with OpenSSL 4.0 opaque ASN1_STRING
OpenSSL 4.0 made ASN1_STRING (and thus ASN1_OCTET_STRING) opaque, so
direct access to its length/data fields no longer compiles. Use
ASN1_STRING_length()/ASN1_STRING_get0_data() which are available since
OpenSSL 1.1.0 and LibreSSL 2.7.
Also move the legacy OpenSSL init calls (ERR_load_crypto_strings,
SSL_load_error_strings, OpenSSL_add_all_*) under the pre-1.1.0 guard:
they are redundant on modern OpenSSL and break no-deprecated builds.
Vsevolod Stakhov [Thu, 11 Jun 2026 14:05:13 +0000 (15:05 +0100)]
[Fix] milter: send QUARANTINE even with a custom reply
The QUARANTINE command was sent inside the `if (!reply)` guard that
synthesises a default quarantine reason, so a caller-supplied SMTP
message (e.g. task:set_pre_result('quarantine', 'reason')) suppressed
the command entirely and the message was accepted instead of
quarantined. Affects both METRIC_ACTION_QUARANTINE and reject converted
via quarantine_on_reject. Regression from fbc6e35db (3.10.0).
The reply now becomes the quarantine reason, matching the reject and
tempfail branches where a caller-supplied message takes precedence over
the configured default.
Reported by @johnmosli, who also attached the fix.
Vsevolod Stakhov [Wed, 10 Jun 2026 19:02:34 +0000 (20:02 +0100)]
[Fix] url: scan bare query parameters containing '=' as a whole
A bare embedded URL is its own query parameter and can contain '='
itself (e.g. base64 padding in the path). The query-embedded scan
treated everything before the first '=' as a parameter key, so such
URLs were discarded. Treat the prefix as a key only when it has no
URL structure characters (':' or '/'); otherwise scan the whole
parameter.
[Fix] symcache: fix timeout inflation in pre_postfilter_iter grouping
The `saved_priority` initialization to -1 caused the first item in each
phase vector (prefilters/postfilters/idempotent) to be split off from
its priority group and counted individually, inflating the computed
maximum symbols cache timeout. Initialize to the first item's priority
instead so items at the same priority are correctly grouped.
[Test] checkv3: drop requests_toolbelt from form-data parse
The /checkv3 content negotiation tests parsed the multipart/form-data
reply with requests_toolbelt, a third-party module not present in every
test pipeline (e.g. the rspamd-docker functional run installs
python3-msgpack but not requests-toolbelt), so they failed with
ModuleNotFoundError.
Replace it with a self-contained stdlib HTTP-multipart splitter: split
on the boundary delimiter (HTTP-multipart style, deliberately not the
email/MIME parser used for the message/rfc822 case) and trim only the
single CRLF framing each part so binary (zstd) payloads stay byte-exact.
Drop the now-unneeded requests-toolbelt from the CI pip install.
A trained ANN could become unreachable to workers even though
training succeeded: NEURAL_SPAM/NEURAL_HAM stopped firing while the
controller logged "ann ... is changed, our version = N, remote
version = M" forever.
Root cause is a version regression, not a missing zset registration.
The new version was seeded from the in-memory set.ann, and
fill_set_ann resets set.ann.version to 0 whenever a worker never
loaded an ANN (restart, or the selected profile's blob was missing).
A worker that trained from the _4 profile then saved version 1.
process_existing_ann selects the highest version among compatible
profiles, so the live version-1 blob was shadowed by the stale
version-4 zset entry whose key was empty. The profile zset has no
TTL, so the dead high-version tombstone was immortal and the
condition self-perpetuated (the _4 blob was never rewritten).
Three fixes:
1. Version monotonicity (lualib/plugins/neural.lua): seed the new
version from the profile actually trained from (the trained-from
key encodes it as the trailing _<n>), max'd with
training_profile/set.ann, so the new entry always outranks the
profile it supersedes.
2. Liveness-aware selection (src/plugins/lua/neural.lua,
neural_maybe_invalidate.lua): when the selected profile's blob is
missing, fall back to the next compatible profile with a live blob
instead of going dark, and emit a throttled warning (was a silent
debug line). The invalidate script also GCs profile entries that
have no blob and no training data and are older than a grace
window.
3. Lifetime coupling (neural_save_unlock.lua,
src/plugins/lua/neural.lua): give the profile zset a TTL refreshed
each check_anns cycle, and refresh the blob TTL on every reload,
so an actively used ANN never expires out from under its entry.
Adds 330_neural/005_stale_version.robot, which injects a
higher-version tombstone and asserts inference recovers.
[Minor] url_redirector: distinguish direct URL errors from redirect errors
When http_callback reports an error on the first hop (orig_url == url),
no redirect has occurred yet, but the old message "found redirect error
from X to X" implied one. Split the message: "error checking URL" for
direct failures and "redirect error: A -> B" for mid-chain failures.
[Minor] checkv3: trim verbose comments to house style
Condense the multi-line explanatory blocks added with the negotiation
reply to single-line notes, and drop a dangling 'see contract above'
reference that pointed to nothing in the source.
The /checkv3 reply ignored Accept beyond a json-vs-msgpack toggle and
always emitted a hard-coded multipart/mixed body (with form-data part
headers). There was no way to ask for a plain v2-style json/msgpack
body, no true multipart/form-data reply for HTTP multipart parsers,
and Accept-Encoding had no defined default.
Negotiate the representation solely from Accept and compression solely
from Accept-Encoding on the single chokepoint all three workers (normal
scan worker, rspamd_proxy, controller) share, the reply_v3 helper:
Inside the multipart envelopes the result-part serialization mirrors
the input metadata serialization (json or msgpack); the two envelopes
differ only in the top-level Content-Type. Compression honours
Accept-Encoding: zstd and defaults to identity. Vary: Accept,
Accept-Encoding is always advertised. Negotiation reuses the existing
http_content_negotiation parser (q-values + wildcards), extended with
two media types; the input metadata serialization is recorded on the
task via a new protocol flag.
rspamc previously sent Accept: application/json|msgpack for v3, which
now selects a single-body reply it does not expect; it now requests
multipart/form-data and accepts any multipart/ subtype, with the result
serialization carried by the metadata Content-Type.
Tested by a new C++ content_negotiation suite, multipart envelope-mode
unit tests, and a functional negotiation suite run against both the
normal worker and the controller (json/msgpack/email-MIME/HTTP-multipart
parsers). Adds msgpack/requests/requests-toolbelt to functional CI deps.
The scan worker and rspamd_proxy both handle /checkv3 (multipart
metadata + message in, multipart/mixed results out), but the
controller's HTTP path router never registered it. Since rspamc
defaults to the controller port (11334) for localhost, a plain
`rspamc --protocol-v3` returned 404 with
"rspamd_http_router_finish_handler: path: /checkv3 not found".
Register /checkv3 on the controller (routed to the existing scan
handler) and branch on CMD_CHECK_V3 in both directions:
parse the body via rspamd_protocol_handle_v3_request() on input and
emit the multipart reply via rspamd_protocol_http_reply_v3() on
output, mirroring the proxy. Auth posture matches /check and
/checkv2 (read command, no enable password).
[Minor] url_redirector: clarify log messages for successful HTTP responses
The phrases "err code 200" and "err code <N>" are misleading since
they refer to HTTP status codes, not errors. Successful resolutions
(HTTP 200) and intermediate redirects (30x) now use unambiguous
wording that clearly separates the action from the status code.
[Fix] url: canonicalise bare emails to slash-less mailto:
A bare email in text/HTML and the same address inside an explicit
mailto: URL were extracted as two separate emails. The '@' matcher (and
the HTML bare-email path) injected a literal "mailto://" prefix, while a
parsed mailto: URL is non-hierarchical and drops the // (RFC 6068, a4ae51536). The URL/email khash dedup keys on the full url->string
(hash + a urllen guard ahead of rspamd_emails_cmp), so the two string
forms landed in different buckets and never collapsed -> the address was
duplicated.
Canonicalise both bare-email injection sites to the slash-less "mailto:"
form so every path yields the identical mailto:user@host string and the
existing dedup works:
- src/libserver/url.c: '@' matcher prefix "mailto://" -> "mailto:"
- src/libserver/html/html_url.cxx: same for bare emails in HTML
Adjust the extract_specific_urls scheme-strip helper to tolerate the
slash-less form, and add a regression test in test/lua/unit/url.lua.
[Fix] url: keep query nesting cap a fixed functional limit
f068a1156 derived RSPAMD_URL_QUERY_MAX_NESTING from the multipattern
scratch budget (MAX_REENTRANCY - 2), which silently bumped the
redirect/wrapper unwrap depth from 5 to 8 and broke the get_html_urls
unit test that pins the cap at 5.
The nesting depth is a functional/product decision, not a function of
the scratch pool size. Restore the fixed cap of 5 and instead assert at
compile time that it stays within the scratch budget (plus the
enclosing scan and the leaf TLD lookup); rspamd_multipattern_lookup()
still degrades gracefully if the bound is ever exceeded.
Schedule every external service under a stable <RULE>_CHECK callback
symbol so it is a predictable dependency target, regardless of how the
scan result symbols are named. This generalises the pattern vadesecure
and cloudmark already follow (VADE_CHECK, CLOUDMARK_CHECK).
A scanner whose main symbol is already a *_CHECK keeps it as the
callback. Otherwise the callback is named <KEY>_CHECK (from the config
block key -- unique per rule, so instances never collide) and the
scanner's result symbol (e.g. DCC_REJECT, RAZOR) becomes a virtual
child of it: its score and emitted results are unchanged, and an
existing dependency on the old name still resolves (virtual -> parent).
This lets register_dependency('<SERVICE>_CHECK', X) order any external
check after another symbol uniformly, e.g. after a sender-bypass rule.
[Fix] external_services: honour scanner default symbol name
Commit f654ec35d resolved the rule symbol names up-front as
`opts.symbol or sym:upper()` so the configure()-failed stub could still
register a fail symbol, then assigned them unconditionally - which
discarded the symbol the scanner resolves in configure(). A scanner
such as vadesecure defines `symbol = 'VADE_CHECK'` as its default main
symbol, but a rule keyed `vadesecure { ... }` without an explicit
`symbol =` was then registered as VADESECURE instead of VADE_CHECK.
Besides the surprising rename, this broke dependency registration
against the documented symbol:
Restore the pre-regression precedence: prefer the symbol the scanner
resolved (its own default, or the user's `symbol =` applied via
override_defaults) and fall back to the key-derived names only when the
scanner left them unset. The up-front names are still used for the
configure()-failed stub. Applies to every scanner with a default
symbol (vadesecure, dcc, razor, cloudmark, ...).
A URL whose query embeds a percent-escaped URL is unwrapped
recursively (PR #6066): rspamd_url_find_in_query re-enters
rspamd_multipattern_lookup on the URL trie while the enclosing scan
is still on the stack. Each scan borrows one of MAX_SCRATCH hyperscan
scratch contexts; once the recursion nests deeper than the pool, the
slot loop leaves scr == NULL and g_assert(scr != NULL) aborts the
worker. A crafted message with a few levels of nested query URLs thus
crashes a normal worker (DoS).
The peak number of simultaneously-held scratch contexts on the
deepest chain is RSPAMD_URL_QUERY_MAX_NESTING + 2: one for the
enclosing text/subject scan and one for the per-URL TLD lookup that
rspamd_url_parse runs on each freshly extracted leaf. The old pool of
4 with a nesting cap of 5 needed 7 -> assertion.
- Introduce RSPAMD_MULTIPATTERN_MAX_REENTRANCY (10) and size the
scratch stack from it; a scratch context is ~2.5-4 KiB, so the
deeper stack costs only tens of KiB per multipattern.
- Tie RSPAMD_URL_QUERY_MAX_NESTING to that budget (minus the two
implicit levels) so normal nesting stays on the fast path.
- Make scratch exhaustion non-fatal: allocate a one-off scratch for
the scan instead of aborting the worker on attacker input.
- Guard the unsigned-int scratch bitmask with a static assert.
Add functional regression test 170_url_query_nesting.
The two-phase composite evaluation classified a composite as second-pass
if any referenced symbol carried SYMBOL_TYPE_NOSTAT. NOSTAT is auto-set
on nearly every virtual/callback symbol (regexp rules, multimap, rbl,
...), so most composites that depend only on ordinary filter rules were
wrongly deferred to the COMPOSITES_POST stage, which runs after
post-filters. As a result task:get_groups()/get_symbols() called from a
postfilter no longer saw those composite symbols and their groups
(regression vs 3.5); they only became visible from idempotent rules.
Classifiers run before composites, so keying off SYMBOL_TYPE_CLASSIFIER
was also unnecessary.
Only postfilter-stage symbols are genuinely unavailable during the first
composites pass. Add rspamd_symcache_get_symbol_stage(), which resolves
virtual symbols to their parent and returns the processing stage as a
SYMBOL_TYPE_* bit, and defer a composite only when a dependency resolves
to the postfilter stage. NEURAL_SPAM (a virtual child of the
NEURAL_CHECK postfilter) still resolves to POSTFILTER, so the original
#5674 fix keeps working.
Add a functional regression test covering a composite that depends only
on a filter-stage NOSTAT symbol and must be visible from a postfilter.
[Fix] ratelimit: Track all buckets in selector rules
When a rule defines several buckets (e.g. "200 / 1h" plus "30 / 1m")
and uses a selector, limit_to_prefixes keyed the prefixes table by the
selector value alone. Every bucket therefore mapped to the same Redis
key and only the last bucket in the array was ever tracked, so the
other limits were silently ignored.
Give each bucket a distinct key by prefixing the selector value with a
per-bucket id (burst + rate), mirroring the burst component that
gen_rate_key already prepends for the non-selector path. The
non-selector path is unchanged, so its existing Redis keys are kept.
Adds a functional regression test (two buckets, burst 2 + burst 20)
that fails before the fix because the restrictive bucket is ignored.
Alexandra Parker [Sun, 31 May 2026 20:52:35 +0000 (13:52 -0700)]
[Fix] mime_string const iterator
mime_string's iterator is just a value iterator anyway. it's
intrinsically const. drop reference to const_iterator, let iterator_base
take a const pointer, and mark begin() and end() as const.
doctest 2.5.0 receives a const reference and can't use a mutable iterator,
which leads to compile error.
[Feature] protocol: Expose custom metadata for /checkv3
Add two complementary ways to read custom fields sent with a /checkv3
multipart scan request, both free of the 80KB HTTP header limit that v2
hits, since the metadata travels in the multipart body:
* A "headers" sub-object in the metadata part is injected into the
task request headers, so task:get_request_header() works for custom
fields exactly like v2 HTTP request headers. Reserved control-header
names (shm/file/path/dictionary/Content-Encoding...) are skipped so
client metadata cannot collide with the message-loading channel, and
a repeated name (collapsed by UCL into an array) expands to a
multi-valued request header.
* The parsed metadata object is kept on task->meta and exposed to Lua
via task:get_metadata() and task:get_metadata_field(key), mirroring
get_settings()/lookup_settings(). The task now owns the object and
frees it once in rspamd_task_free instead of via a pool destructor.
rspamc gains a repeatable --metadata-header KEY=VALUE option that builds
the metadata "headers" sub-object for v3 requests. Also drop a dead
is_msgpack variable in the v3 request handler.
Tests: functional cases in 430_checkv3.robot plus a checkv3_meta.lua
plugin exercising both options via raw multipart and rspamc.
[Fix] functional: move test server ports below the ephemeral range
The real root cause of the 440_ssl_server flake (and the family of
intermittent "bind 98 / Address already in use" failures): the test
server ports sat INSIDE Linux's default ephemeral range
(net.ipv4.ip_local_port_range = 32768..60999). Bases were 56379 (redis),
56380 (nginx) and 567xx (rspamd normal/controller/proxy/fuzzy + the two
TLS listeners), all squarely in that window.
So any outbound client socket in the test environment -- a redis client,
monitored URIBL DNS lookups, an upstream connection, a dummy-helper
connection -- could be handed one of those numbers by the kernel as its
EPHEMERAL SOURCE PORT on connect(). When rspamd later tried to bind() a
LISTENER on that exact port it got EADDRINUSE. rspamd sets SO_REUSEADDR,
which does nothing against a live socket already bound by another
process. The controller's SSL socket is the LAST of its five ports to
bind -- by then the controller has already opened many client sockets --
so it lost this race most often and surfaced as "SSL controller never
came up" -> HTTPS connection-refused for the whole retry budget. It was
probabilistic (depends which ephemeral ports were in use at bind time),
hence flaky and distro-dependent.
Move the whole rspamd/redis/nginx block down by 31000 (e.g. normal
56789 -> 25789, controller-SSL 56796 -> 25796, redis 56379 -> 25379,
nginx 56380 -> 25380). This preserves every relative offset, so the
carefully spaced, collision-free per-worker layout (base + slot*100) is
unchanged: across 64 worker slots the dummy_* helpers stay <= 24383,
this block spans 25379..32097, and the ephemeral floor 32768 is never
reached. Verified by importing vars.py for slots 0 and 63 (max port
32097 < 32768, zero cross-family collisions) and a serial 001_merged run
(all six 440_ssl_server tests pass on the relocated ports).
Also bump the two cosmetic fallbacks that mirrored the old bases:
test_redis_client.lua's getenv default and a port_is_free docstring.
A commit on a branch with an open PR triggers two full ci runs at once:
one for the push event (ref refs/heads/<branch>) and one for the
pull_request event (ref refs/pull/<n>/merge). Besides wasting runner
time they share GitHub's hosted runners and double the CPU load, which
is enough to push the heavy 001_merged rspamd's controller startup past
the functional suites' fixed readiness timeouts -- the residual
440_ssl_server flake reproduced only on whichever of the two same-SHA
runs lost the CPU race (the other passed).
Add a top-level concurrency group keyed on the head commit SHA with
cancel-in-progress. push and pull_request expose the head differently
(github.sha vs github.event.pull_request.head.sha -- the latter is the
real head on PR events, where github.sha is the merge commit), so the
group key uses pull_request.head.sha when present and falls back to
github.sha, collapsing both events for one commit into a single run.
[Test] functional: also wait for SSL/proxy ports in teardown
The teardown port-release wait added in the previous commit only covered
the normal + controller plain ports. The 440_ssl_server flake is the
same race on a port it missed: a test rspamd binds up to five sockets
(normal, controller, proxy, controller-SSL, normal-SSL), and a previous
suite's controller-SSL listener could still hold its port when the next
rspamd on that pabot worker started. The CI log shows it exactly:
rspamd_fork_worker: prepare to fork process controller (0);
listen on: 127.0.0.1:57190
rspamd_inet_address_listen: bind 127.0.0.1:57196 failed: 98,
'Address already in use'
spawn_workers: cannot listen on normal socket 127.0.0.1:57196
57196 is PORT_CONTROLLER_SSL for that worker slot. main carried on and
forked the controller with only its plain socket, so the SSL listener
never came up and every HTTPS test hit connection-refused for the full
retry budget -- the "slow SSL controller" the two prior band-aids tried
to wait out.
Extend Wait For Rspamd Ports Released to loop over all five ports. All
RSPAMD_PORT_* vars are always defined in vars.py, and a port the current
config never bound refuses connection immediately, so Port Is Free
passes at once -- waiting on an unused port is a cheap no-op. Verified
001_merged (which owns 440_ssl_server) still passes serially with the
SSL ports now checked in teardown; the SSL bind race is timing-dependent
under CI contention, so the fedora/ubuntu runs are the real check.
projects / thirdparty / rspamd.git / log
