* [Project] Parallelise functional tests via pabot
Switch the Robot Framework functional test suite from a single serial
robot invocation to a two-phase pabot + robot run, giving CI a ~3-4x
wall-clock win on the parallel-safe portion while keeping the rest
working unchanged.
Worker isolation lives in test/functional/lib/vars.py. Each pabot
worker reads PABOTEXECUTIONPOOLID and applies a port offset of
index*100 across every rspamd / redis / nginx / clam / fprot / avast
/ dummy-http / dummy-https / dummy-http-early / dummy-llm / dummy-udp
/ dummy-ssl port, plus a per-worker /tmp/rspamd-functional-<index>/
prefix for unix sockets and pidfiles. Plain `robot` runs unchanged
(no env var -> index 0 -> the historical port numbers).
The dummy_* helper utilities now derive their PID paths from
{tmp_prefix}/dummy_<svc>-<port>.pid (or socket basename for p0f) via
a small util/dummy_pidfile module, so two instances on different
ports no longer collide. Existing override-via-argv callsites still
work. Robot keywords in lib/rspamd.robot are updated to use the
vars-driven ports and pidfile paths; suites that read those PIDs
(161_p0f, 230_tcp, 001_merged/{160_antivirus,310_udp}) and the
url-redirector log-grep in 162_url_redirector are templated to
match.
Twelve suites still bake dummy_http/dummy_llm/dummy_http_early/tcp
port numbers into Lua test scripts (test/functional/lua/{http,
http_early_response,tcp}.lua) and three configs (settings.conf,
neural_llm.conf and the assertion literals in url_redirector*),
so they only work at the worker-0 port offset. Tagging them
`notparallel` and running them with plain robot after the pabot
batch sidesteps the collision without templating those Lua scripts
in this change.
CI (.github/workflows/ci_rspamd.yml) installs pabot via pip
(--break-system-packages with a fallback for older pip in the
Fedora image), then runs:
* Phase 1: pabot --processes 4 --exclude notparallel
-> outputdir build/parallel/
* Phase 2: robot --include notparallel
-> outputdir build/serial/
Both phases run unconditionally and the step exits non-zero if
either failed. Artifact upload now collects both outputdirs plus
the legacy build/*.*ml path.
Local invocation is `test/functional/run-parallel.sh`, a thin
wrapper documented in CLAUDE.md. The script forces suite-level
splitting (no --testlevelsplit) because each Suite Setup starts
its own rspamd.
Follow-ups (not in this change):
* Template the four Lua scripts and three configs so the twelve
notparallel suites can drop the tag.
* Split 001_merged/ (30 sub-suites under one rspamd) into
independent units; currently pinned to one worker and the long
pole of phase 1.
* [Fix] functional tests: claim worker slot via /tmp lockfile
Pabot 5.2.2 does not export PABOTEXECUTIONPOOLID to child robot
subprocesses, even though the variable name appears in pabot's own
source for internal accounting. The previous worker-index detection
fell through to 0 in every pabot worker, so all four workers used
identical rspamd / redis / fuzzy port offsets and crashed in
Multi Setup with "Address already in use".
Replace the env-only lookup with an atomic file-claim:
* RSPAMD_WORKER_INDEX / PABOTEXECUTIONPOOLID still win when set
(explicit override, future pabot versions).
* Otherwise each process atomically grabs the first free
/tmp/rspamd-functional.slot-<N> via O_CREAT|O_EXCL, writing its
pid. A stale slot (pid no longer alive) is reclaimed by the next
caller. atexit unlinks the slot when the process exits.
Verified locally:
* Four concurrent python imports of vars.py get indices 0..3 with
no collisions; slot files cleaned up on exit.
* `pabot --processes 2` over two trivial robot suites prints
distinct port ranges (56789 vs 56889) from each worker.
* [Fix] worker binds: env-templated defaults; diagnostic log tail
The four built-in workers (normal, controller, rspamd_proxy, fuzzy)
in conf/rspamd.conf hardcoded `localhost:1133[2-5]`. Under parallel
pabot every rspamd instance tried to bind those same ports and the
second one onwards hard-terminated with "Address already in use".
Switch the bind_socket lines to jinja templates with the existing
production strings as defaults:
bind_socket = "{= env.LOCAL_ADDR|default('localhost') =}:\
{= env.PORT_NORMAL|default('11333') =}";
Production behaviour is preserved bit-for-bit -- with no env vars,
the templates resolve back to `localhost:11332..11335`. The functional
test harness already exports RSPAMD_LOCAL_ADDR / RSPAMD_PORT_*, which
rspamd's lua_common.c strips of the RSPAMD_ prefix when populating
rspamd_env, so `env.PORT_NORMAL` etc. pick up the per-worker slot
values from test/functional/lib/vars.py automatically.
Verified locally:
- `pabot --processes 4` over the four `001_merged` sub-suites
(Cases.001 Merged.{099,100,101,102}) passes 122/122 tests where
it used to fail every test with hard_terminate.
- Full phase-1 run (`pabot --processes 4 --exclude notparallel`)
completes in 2m20s with 646/666 passing; the 20 failures are all
local mac env-specific issues (missing pynacl, missing
liblua.5.1.dylib for miltertest, etc.) unrelated to this change.
- `rspamadm configdump` on a stock config (no env override) still
binds `localhost:11332..11335` byte-for-byte.
Also enrich Rspamd Startup Check to surface the last 80 lines of
rspamd.log plus exit code, port and tmpdir on Process Is Gone --
the previous one-line "loading configuration" stderr made the bind
collision invisible from CI artifacts and forced a local repro to
diagnose.
* [Test] functional: dummy-port env in lua + settle after startup
Three classes of leftover collisions surfaced once worker bind_sockets
were templated and parallel rspamds actually started:
* lua/udp.lua and lua/maps_kv.lua (loaded by 001_merged) and the
rspamadm script lua/rspamadm/test_redis_client.lua hardcoded the
dummy_udp / dummy_http / redis ports. Workers on slot index > 0
bound their dummies on shifted ports, so the lua scripts kept
talking to the slot-0 endpoints and tests timed out. Read
env.PORT_DUMMY_UDP / env.PORT_DUMMY_HTTP / env.REDIS_PORT (set
via vars.py -> RSPAMD_PORT_* -> rspamd_env stripped of the
RSPAMD_ prefix in lua_common.c) and fall back to the historical
literals so the scripts still run outside the harness.
* configs/merged-override.conf EXTERNAL_MULTIMAP and
configs/settings.conf external_map baked
`http://127.0.0.1:18080/...` into rspamd's own config. Switch
those to `{= env.PORT_DUMMY_HTTP|default('18080') =}` so the
multimap external backend resolves to the per-worker dummy_http.
* lib/rspamd.robot Rspamd Setup polled the startup-check loop with
`IF ${ok} CONTINUE`, which kept iterating after the first
successful ping but added effectively no grace period for the
controller / proxy workers to finish registering with the main
process. Under parallel load the first `rspamadm control stat`
in 001_merged.099 Control returned an empty workers list.
Switch to `BREAK` on success and add a 0.5s settle period.
Verified locally: previously-failing
099_control / 100_general / 101_lua / 102_multimap /
310_udp / 151_rspamadm_async now pass 126/126 under
pabot --processes 4 in ~17s.
* [Test] functional: fix two more parallel races
Two leftover collisions surfaced once 001_merged was actually starting
rspamds in parallel across pabot workers:
* test/functional/lua/lua_extras_test.lua writes its staging tree to
os.getenv('TMPDIR'). On Linux CI TMPDIR is unset, so every worker
raced on a shared /tmp/lua_extras_test directory -- one worker's
`rm -rf` would wipe another worker's tree mid-test and rspamd
config load aborted with `cannot init lua file ... No such file
or directory`. Prefer RSPAMD_TMPDIR (per-suite tmpdir, propagated
via env:RSPAMD_TMPDIR in Run Rspamd) so workers don't share state.
* 151_rspamadm_async/Redis client invokes `rspamadm lua -b
test_redis_client.lua` which connects to redis directly. The
previous fix used `rspamd_env.REDIS_PORT`, but rspamadm's lua
context (unlike the daemon's) does not populate the `rspamd_env`
global -- only rspamadm_session/_ev_base/_dns_resolver are set --
so the lookup always fell through to the literal 56379. Read
`os.getenv("RSPAMD_REDIS_PORT")` instead. Also call
`Export Rspamd Variables To Environment` from the suite's Setup
so the env vars are actually present in the rspamadm subprocess
inherited environment (this suite never calls Run Rspamd, which
is where the export normally happens).
Local: `pabot --processes 2` over 102_multimap / 151_rspamadm_async /
271_lua_extras passes 83/83 in ~8s.
* [Test] CI: run parallel + serial functional phases concurrently
The two-phase split (pabot for parallel-safe suites, plain robot for
notparallel-tagged ones) ran sequentially -- on fedora that meant
2:16 (pabot, 666 tests) + 1:35 (robot, 92 tests) = ~4 minutes total
versus master's ~6 minutes serial. The pabot phase itself is already
at ~91% of theoretical 4-worker speedup (8:14 of work in 2:16
wall-clock), so bumping --processes won't help much -- the cheap
win is overlapping the two phases.
Background both phases with `&`, capture their PIDs, then `wait`
each separately to harvest exit codes. They claim disjoint slots
from the vars.py file-based allocator (pabot grabs 0..3, robot
grabs 4), so their rspamds use different port ranges and tmp
prefixes and don't collide.
Expected total wall-clock: ~max(2:16, 1:35) ~= 2:20, down from ~4:00.
Verified locally: 4 pabot workers + 1 serial robot running 6
suites in parallel (115 + 33 tests) all pass in 27s on a 4-core
mac with the same vars.py slot allocator. No port collisions
observed.
* [Test] Revert misleading CLAUDE.md additions
The functional-test commands I added were wrong on two counts:
* RSPAMD_INSTALLROOT=~/rspamd.install -- that path is stale on this
repo's typical setup; the CMake install prefix is /usr/local.
* "driven by PABOTEXECUTIONPOOLID" -- pabot 5.2.2 does NOT actually
export that env var to child robot subprocesses (confirmed via
dump-env test). The real mechanism is the file-based slot claim
in test/functional/lib/vars.py (/tmp/rspamd-functional.slot-N).
Removing the lines rather than fixing them in place; the right
home for parallel-test docs is alongside the runner script and the
PR description, not duplicated and risk-of-drift in CLAUDE.md.
* [Test] Verify controller ready + rebot merge unified report
Two issues from the concurrent-phases run:
* `Cases.001 Merged.099 Control` flaked again ("'' does not contain
'workers'"). rspamd's controller binds and answers HTTP ping
almost immediately, but its workers list is populated only after
each worker has registered back with the main process. Under
parallel pabot + the concurrent serial phase (5 rspamds competing
for CPU at startup) the gap stretched out and a fixed 0.5s settle
was no longer enough.
Replace the blind settle with a real readiness check: after the
ping loop, if rspamd.sock is present in TMPDIR, poll
`rspamadm control stat` (via the new keyword
Verify Controller Workers Registered) until the response actually
contains "workers". Cheap when fast, retried up to ~6s when
rspamd is starting slowly. Local: five back-to-back parallel
runs over 099/100/102/270 -- 530/530 tests pass, no flakes.
* The CI step left three output.xml files
(build/parallel/{pabot_results/N/,}output.xml and
build/serial/output.xml) and no single top-level report, so a
reviewer skimming the CI log saw only one pabot sub-suite path
and read it as "we only ran part of the suite". Run
`rebot --merge` after both phases finish to produce a unified
build/output.xml + log.html + report.html alongside the two
phase outputs, matching the artifact shape master used to have.
* [Test] Fix readiness check; replace [Return] with RETURN
Two fixes:
* The previous unconditional `Wait Until Keyword Succeeds` for the
control socket assumed every suite produces $DBDIR/rspamd.sock.
That holds for 001_merged (includes options.inc -> control_socket
= "$DBDIR/rspamd.sock") but NOT for the many suites that build a
minimal standalone config (231_tcp_down etc.). Those never get a
control socket, so the 50 x 0.2s poll always exhausted and broke
every test in those suites.
Wait up to 2s for the socket file to appear -- if it does, poll
`rspamadm control stat` until the response contains "workers"
(the real readiness signal CONTROL STAT depends on); if it
doesn't, just proceed, since suites that never produce a control
socket can't be testing it.
* Convert the [Return] setting to the RETURN statement across the
five files that still used the old syntax. Robot Framework 7
deprecated [Return] and the unrelated noise warnings were
swamping every test step's stdout, making real failures hard to
spot:
cases/001_merged/115_dmarc.robot
cases/001_merged/160_antivirus.robot
cases/151_rspamadm_async.robot
cases/320_arc_signing/003_roundtrip.robot
lib/rspamd.robot
Verified locally: three back-to-back concurrent-phase runs (4-way
pabot + serial robot for notparallel suites) -- (106 + 33) tests
all pass each time, no flakes, no deprecation warnings.
* [Test] CI: redirect each phase to its own log, group in step output
Previously both concurrent phases (pabot and serial robot) wrote to
the step's combined stdout, so pabot's batched end-of-run summary
and robot's streaming output interleaved. Reviewers were seeing
what looked like only one of the two runs.
Redirect each phase's stdout+stderr to its own
build/phase{1-parallel,2-serial}.log, wait on both PIDs, then
`cat` the two logs in fixed order with GH Actions
::group::/::endgroup:: directives so they collapse to two clean
sections in the web UI. Wall-clock unchanged -- the two phases
still run concurrently; only the presentation is sequential.
Also include the two per-phase logs in the robotlog artifact
upload so they're inspectable after the run.
run: |
sudo mv /usr/bin/miltertest /usr/bin/miltertest.is.broken.on.fedora || true
+ - name: Install pabot
+ # --break-system-packages: Ubuntu 24.04 ships Python 3.12 with PEP 668
+ # marking the system interpreter as externally-managed. Older pip
+ # versions (Fedora image) don't know the flag, so fall back without it.
+ run: |
+ pip install --break-system-packages robotframework-pabot \
+ || pip install robotframework-pabot
+
- name: Run functional tests
+ # Two phases run concurrently:
+ # 1. pabot, --processes 4, suites NOT tagged `notparallel` (worker
+ # isolation via the vars.py file-based slot allocator -> port
+ # offset + per-worker /tmp prefix).
+ # 2. plain robot, suites tagged `notparallel`. These suites still
+ # bake dummy_http / dummy_llm / dummy_http_early port numbers
+ # into Lua test scripts and a handful of configs, so they only
+ # work on a single worker's port range.
+ # Both phases claim disjoint slots from vars.py so their rspamds
+ # don't collide. Running them in parallel (rather than sequentially)
+ # drops wall-clock from ~4:00 to ~max(phase1, phase2) ~2:20.
+ # Each phase writes to its own outputdir; we exit non-zero if
+ # either phase failed.
run: |
cd ${GITHUB_WORKSPACE}/build
ulimit -c unlimited
ulimit -s unlimited
+ mkdir -p parallel serial
set +e
- RSPAMD_INSTALLROOT=${GITHUB_WORKSPACE}/install robot -v RSPAMD_USER:root -v RSPAMD_GROUP:root --removekeywords wuks --exclude isbroken ${GITHUB_WORKSPACE}/src/test/functional/cases; EXIT_CODE=$?
+ # Each phase writes its own stdout+stderr to a log file so
+ # the two never interleave; we cat them in order at the
+ # end so the GH Actions step output reads sequentially
+ # (parallel-phase summary first, then serial-phase) even
+ # though the two ran concurrently and saved wall-clock.
+ RSPAMD_INSTALLROOT=${GITHUB_WORKSPACE}/install pabot \
+ --processes 4 \
+ --outputdir ${GITHUB_WORKSPACE}/build/parallel \
+ -v RSPAMD_USER:root -v RSPAMD_GROUP:root \
+ --removekeywords wuks \
+ --exclude isbroken \
+ --exclude notparallel \
+ ${GITHUB_WORKSPACE}/src/test/functional/cases \
+ > ${GITHUB_WORKSPACE}/build/phase1-parallel.log 2>&1 &
+ PABOT_PID=$!
+ RSPAMD_INSTALLROOT=${GITHUB_WORKSPACE}/install robot \
+ --outputdir ${GITHUB_WORKSPACE}/build/serial \
+ -v RSPAMD_USER:root -v RSPAMD_GROUP:root \
+ --removekeywords wuks \
+ --exclude isbroken \
+ --include notparallel \
+ ${GITHUB_WORKSPACE}/src/test/functional/cases \
+ > ${GITHUB_WORKSPACE}/build/phase2-serial.log 2>&1 &
+ ROBOT_PID=$!
+ wait $PABOT_PID; PARALLEL_RC=$?
+ wait $ROBOT_PID; SERIAL_RC=$?
+ echo "::group::Phase 1 (pabot, parallel-safe suites) -- rc=$PARALLEL_RC"
+ cat ${GITHUB_WORKSPACE}/build/phase1-parallel.log
+ echo "::endgroup::"
+ echo "::group::Phase 2 (robot, notparallel suites) -- rc=$SERIAL_RC"
+ cat ${GITHUB_WORKSPACE}/build/phase2-serial.log
+ echo "::endgroup::"
+ # Merge the two phases into one unified report at the
+ # build root (build/output.xml, log.html, report.html) --
+ # this matches the artifact shape master serial produces,
+ # so reviewers see a single total instead of two halves.
+ # --nostatusrc: merging itself never affects step exit code.
+ if [ -f parallel/output.xml ] && [ -f serial/output.xml ]; then
+ echo "::group::Merge parallel + serial outputs"
+ rebot --nostatusrc --outputdir . --output output.xml \
+ parallel/output.xml serial/output.xml || true
+ echo "::endgroup::"
+ fi
set -e
core_files=$(find /var/tmp/ -name '*.core')
for core in $core_files; do exe=$(gdb --batch -ex 'info proc mappings' -c $core | tail -1 | awk '{print $5}'); gdb --batch -ex 'bt' -c $core $exe; echo '---'; done
- exit $EXIT_CODE
+ if [ $PARALLEL_RC -ne 0 ] || [ $SERIAL_RC -ne 0 ]; then
+ echo "Parallel phase rc=$PARALLEL_RC, serial phase rc=$SERIAL_RC"
+ exit 1
+ fi
- name: Save workspace directory
if: (success() || failure())
with:
name: robotlog-${{ inputs.name }}
path: |
+ ${{ env.CONTAINER_WORKSPACE }}/build/parallel/*.*ml
+ ${{ env.CONTAINER_WORKSPACE }}/build/serial/*.*ml
${{ env.CONTAINER_WORKSPACE }}/build/*.*ml
+ ${{ env.CONTAINER_WORKSPACE }}/build/phase*.log
retention-days: 1
- name: Upload rspamd logs
}
worker "normal" {
- bind_socket = "localhost:11333";
+ # Production default is localhost:11333; the functional test harness
+ # exports RSPAMD_PORT_NORMAL (and optionally RSPAMD_LOCAL_ADDR) so the
+ # same template binds the per-worker port under parallel pabot.
+ bind_socket = "{= env.LOCAL_ADDR|default('localhost') =}:{= env.PORT_NORMAL|default('11333') =}";
.include "$CONFDIR/worker-normal.inc"
.include(try=true; priority=1,duplicate=merge) "$LOCAL_CONFDIR/local.d/worker-normal.inc"
.include(try=true; priority=10) "$LOCAL_CONFDIR/override.d/worker-normal.inc"
}
worker "controller" {
- bind_socket = "localhost:11334";
+ bind_socket = "{= env.LOCAL_ADDR|default('localhost') =}:{= env.PORT_CONTROLLER|default('11334') =}";
.include "$CONFDIR/worker-controller.inc"
.include(try=true; priority=1,duplicate=merge) "$LOCAL_CONFDIR/local.d/worker-controller.inc"
.include(try=true; priority=10) "$LOCAL_CONFDIR/override.d/worker-controller.inc"
}
worker "rspamd_proxy" {
- bind_socket = "localhost:11332";
+ bind_socket = "{= env.LOCAL_ADDR|default('localhost') =}:{= env.PORT_PROXY|default('11332') =}";
.include "$CONFDIR/worker-proxy.inc"
.include(try=true; priority=1,duplicate=merge) "$LOCAL_CONFDIR/local.d/worker-proxy.inc"
.include(try=true; priority=10) "$LOCAL_CONFDIR/override.d/worker-proxy.inc"
# Local fuzzy storage is disabled by default
worker "fuzzy" {
- bind_socket = "localhost:11335";
+ bind_socket = "{= env.LOCAL_ADDR|default('localhost') =}:{= env.PORT_FUZZY|default('11335') =}";
count = -1; # Disable by default, see #4677 for details
.include "$CONFDIR/worker-fuzzy.inc"
.include(try=true; priority=1,duplicate=merge) "$LOCAL_CONFDIR/local.d/worker-fuzzy.inc"
[Arguments] ${domain} ${rua}
${today} = Get Current Date result_format=%Y%m%d
${report_key} = Set Variable dmarc_rpt;${domain};${rua};${today}
- [Return] ${report_key}
+ RETURN ${report_key}
Get DMARC Index Key
[Documentation] Generate the DMARC index key for today
${today} = Get Current Date result_format=%Y%m%d
${idx_key} = Set Variable dmarc_idx;${today}
- [Return] ${idx_key}
+ RETURN ${idx_key}
Redis Key Exists
[Documentation] Check if a Redis key exists
${result} = Run Process redis-cli -h ${RSPAMD_REDIS_ADDR} -p ${RSPAMD_REDIS_PORT}
... EXISTS ${key}
Should Be Equal As Integers ${result.rc} 0
- [Return] ${result.stdout}
+ RETURN ${result.stdout}
Redis Get Set Members
[Documentation] Get all members of a Redis set
${result} = Run Process redis-cli -h ${RSPAMD_REDIS_ADDR} -p ${RSPAMD_REDIS_PORT}
... SMEMBERS ${key}
Should Be Equal As Integers ${result.rc} 0
- [Return] ${result.stdout}
+ RETURN ${result.stdout}
Redis Get Sorted Set Members
[Documentation] Get all members of a Redis sorted set
${result} = Run Process redis-cli -h ${RSPAMD_REDIS_ADDR} -p ${RSPAMD_REDIS_PORT}
... ZRANGE ${key} 0 -1
Should Be Equal As Integers ${result.rc} 0
- [Return] ${result.stdout}
+ RETURN ${result.stdout}
FPROT HIT - PATTERN
${process1} = Run Dummy Fprot ${RSPAMD_PORT_FPROT} 1
- ${process2} = Run Dummy Fprot ${RSPAMD_PORT_FPROT2_DUPLICATE} 1 /tmp/dummy_fprot_dupe.pid
+ ${process2} = Run Dummy Fprot ${RSPAMD_PORT_FPROT2_DUPLICATE} 1 ${RSPAMD_TMP_PREFIX}/dummy_fprot_dupe-${RSPAMD_PORT_FPROT2_DUPLICATE}.pid
Scan File ${MESSAGE}
... Settings=${SETTINGS_FPROT}
Expect Symbol FPROT_EICAR
Log To Console ${res.stdout}
Log To Console ${res.stderr}
Fail Dummy server failed to start
- [Return] ${process}
+ RETURN ${process}
Run Dummy Clam
- [Arguments] ${port} ${found}= ${pid}=/tmp/dummy_clamav.pid
+ [Arguments] ${port} ${found}= ${pid}=${RSPAMD_TMP_PREFIX}/dummy_clamav-${port}.pid
${process} = Run Dummy ${RSPAMD_TESTDIR}/util/dummy_clam.py ${port} ${found} ${pid}
- [Return] ${process}
+ RETURN ${process}
Run Dummy Fprot
- [Arguments] ${port} ${found}= ${pid}=/tmp/dummy_fprot.pid
+ [Arguments] ${port} ${found}= ${pid}=${RSPAMD_TMP_PREFIX}/dummy_fprot-${port}.pid
${process} = Run Dummy ${RSPAMD_TESTDIR}/util/dummy_fprot.py ${port} ${found} ${pid}
- [Return] ${process}
+ RETURN ${process}
Run Dummy Avast
- [Arguments] ${port} ${found}= ${pid}=/tmp/dummy_avast.pid
+ [Arguments] ${port} ${found}= ${pid}=${RSPAMD_TMP_PREFIX}/dummy_avast-${port}.pid
${process} = Run Dummy ${RSPAMD_TESTDIR}/util/dummy_avast.py ${port} ${found} ${pid}
- [Return] ${process}
+ RETURN ${process}
Run Dummy UDP
[Arguments]
- ${result} = Start Process ${RSPAMD_TESTDIR}/util/dummy_udp.py 5005
- Wait Until Created /tmp/dummy_udp.pid
+ ${pid} = Set Variable ${RSPAMD_TMP_PREFIX}/dummy_udp-${RSPAMD_PORT_DUMMY_UDP}.pid
+ ${result} = Start Process ${RSPAMD_TESTDIR}/util/dummy_udp.py ${RSPAMD_PORT_DUMMY_UDP} ${pid}
+ Wait Until Created ${pid}
Set Suite Variable ${DUMMY_UDP_PROC} ${result}
Suite Teardown Settings Teardown
Library ${RSPAMD_TESTDIR}/lib/rspamd.py
Resource ${RSPAMD_TESTDIR}/lib/rspamd.robot
+Test Tags notparallel
Variables ${RSPAMD_TESTDIR}/lib/vars.py
*** Variables ***
*** Keywords ***
Rspamadm test Setup
+ # Make per-worker ports (RSPAMD_REDIS_PORT etc.) visible to the
+ # rspamadm subprocess. Run Rspamd usually does this, but this suite
+ # only spins up redis + dummy_http and the rspamadm lua client
+ # script reads its target port from the env.
+ Export Rspamd Variables To Environment
Run Dummy Http
Run Redis
${config} = Replace Variables ${config}
Log ${config}
Create File ${tmpdir}/rspamd.conf ${config}
- [Return] ${tmpdir}
+ RETURN ${tmpdir}
p0f BAD QUERY
Run Dummy p0f ${RSPAMD_P0F_SOCKET} windows bad_query
Scan File ${MESSAGE} IP=1.1.1.7
- Expect Symbol With Exact Options P0F_FAIL Malformed Query: /tmp/p0f.sock
+ Expect Symbol With Exact Options P0F_FAIL Malformed Query: ${RSPAMD_P0F_SOCKET}
Do Not Expect Symbol WINDOWS
Shutdown p0f
Terminate All Processes kill=True
Shutdown p0f
- ${p0f_pid} = Get File if exists /tmp/dummy_p0f.pid
+ # dummy_p0f derives its pidfile from the socket basename; mirror that here.
+ ${sockname} = Evaluate os.path.basename($RSPAMD_P0F_SOCKET) modules=os
+ ${pidfile} = Set Variable ${RSPAMD_TMP_PREFIX}/dummy_p0f-${sockname}.pid
+ ${p0f_pid} = Get File if exists ${pidfile}
Run Keyword if ${p0f_pid} Shutdown Process With Children ${p0f_pid}
Run Dummy p0f
[Arguments] ${socket}=${RSPAMD_P0F_SOCKET} ${os}=linux ${status}=ok
+ ${sockname} = Evaluate os.path.basename($socket) modules=os
+ ${pidfile} = Set Variable ${RSPAMD_TMP_PREFIX}/dummy_p0f-${sockname}.pid
${result} = Start Process ${RSPAMD_TESTDIR}/util/dummy_p0f.py ${socket} ${os} ${status}
- Wait Until Created /tmp/dummy_p0f.pid
+ Wait Until Created ${pidfile}
Library Process
Library ${RSPAMD_TESTDIR}/lib/rspamd.py
Resource ${RSPAMD_TESTDIR}/lib/rspamd.robot
+Test Tags notparallel
Variables ${RSPAMD_TESTDIR}/lib/vars.py
*** Variables ***
# HTTP server together with their request headers. Verify the redirector
# sends a coherent browser fingerprint (not just a bare User-Agent) and
# that the header order chosen by the profile is preserved on the wire.
- ${log} = Get File /tmp/dummy_http.log
+ ${log} = Get File ${DUMMY_HTTP_LOG}
Should Contain ${log} Sec-Fetch-Mode
Should Match Regexp ${log} HEAD [^\n]*headers: [^\n]*Accept[^\n]*Sec-Fetch-Mode
Library Process
Library ${RSPAMD_TESTDIR}/lib/rspamd.py
Resource ${RSPAMD_TESTDIR}/lib/rspamd.robot
+Test Tags notparallel
Variables ${RSPAMD_TESTDIR}/lib/vars.py
*** Variables ***
Library Process
Library ${RSPAMD_TESTDIR}/lib/rspamd.py
Resource ${RSPAMD_TESTDIR}/lib/rspamd.robot
+Test Tags notparallel
Variables ${RSPAMD_TESTDIR}/lib/vars.py
*** Variables ***
Library Process
Library ${RSPAMD_TESTDIR}/lib/rspamd.py
Resource ${RSPAMD_TESTDIR}/lib/rspamd.robot
+Test Tags notparallel
Variables ${RSPAMD_TESTDIR}/lib/vars.py
*** Variables ***
Library Process
Library ${RSPAMD_TESTDIR}/lib/rspamd.py
Resource ${RSPAMD_TESTDIR}/lib/rspamd.robot
+Test Tags notparallel
Variables ${RSPAMD_TESTDIR}/lib/vars.py
*** Variables ***
Library Process
Library ${RSPAMD_TESTDIR}/lib/rspamd.py
Resource ${RSPAMD_TESTDIR}/lib/rspamd.robot
+Test Tags notparallel
Variables ${RSPAMD_TESTDIR}/lib/vars.py
*** Variables ***
Library Process
Library ${RSPAMD_TESTDIR}/lib/rspamd.py
Resource ${RSPAMD_TESTDIR}/lib/rspamd.robot
+Test Tags notparallel
Variables ${RSPAMD_TESTDIR}/lib/vars.py
*** Variables ***
Library Process
Library ${RSPAMD_TESTDIR}/lib/rspamd.py
Resource ${RSPAMD_TESTDIR}/lib/rspamd.robot
+Test Tags notparallel
Variables ${RSPAMD_TESTDIR}/lib/vars.py
*** Variables ***
Library Process
Library ${RSPAMD_TESTDIR}/lib/rspamd.py
Resource ${RSPAMD_TESTDIR}/lib/rspamd.robot
+Test Tags notparallel
Variables ${RSPAMD_TESTDIR}/lib/vars.py
*** Variables ***
Run Dummy Ssl
[Arguments]
- ${result} = Start Process ${RSPAMD_TESTDIR}/util/dummy_ssl.py ${RSPAMD_TESTDIR}/util/server.pem
- Wait Until Created /tmp/dummy_ssl.pid timeout=2 second
+ ${pid} = Set Variable ${RSPAMD_TMP_PREFIX}/dummy_ssl-${RSPAMD_PORT_DUMMY_SSL}.pid
+ Set Suite Variable ${DUMMY_SSL_PID_FILE} ${pid}
+ ${result} = Start Process ${RSPAMD_TESTDIR}/util/dummy_ssl.py ${RSPAMD_TESTDIR}/util/server.pem ${RSPAMD_PORT_DUMMY_SSL} ${pid}
+ Wait Until Created ${pid} timeout=2 second
Teardown Dummy Ssl
- ${ssl_pid} = Get File /tmp/dummy_ssl.pid
+ ${ssl_pid} = Get File ${DUMMY_SSL_PID_FILE}
Shutdown Process With Children ${ssl_pid}
Check url
Suite Teardown Rspamd Teardown
Library ${RSPAMD_TESTDIR}/lib/rspamd.py
Resource ${RSPAMD_TESTDIR}/lib/rspamd.robot
+Test Tags notparallel
Variables ${RSPAMD_TESTDIR}/lib/vars.py
*** Variables ***
END
END
- [Return] ${temp_file}
+ RETURN ${temp_file}
Write Signed Message To File
[Arguments] ${scan_result} ${filename}
Log WARNING: Write Signed Message To File is deprecated, use Write Mime Message To File
${temp_file} = Set Variable ${RSPAMD_TMPDIR}/${filename}
Create File ${temp_file} ${scan_result.stdout}
- [Return] ${temp_file}
+ RETURN ${temp_file}
Library Process
Library ${RSPAMD_TESTDIR}/lib/rspamd.py
Resource ${RSPAMD_TESTDIR}/lib/rspamd.robot
+Test Tags notparallel
Variables ${RSPAMD_TESTDIR}/lib/vars.py
*** Variables ***
filter = "top";
map = {
external = true;
- backend = "http://127.0.0.1:18080/map-query",
+ backend = "http://127.0.0.1:{= env.PORT_DUMMY_HTTP|default('18080') =}/map-query",
method = "query",
}
}
disabled = true
external_map = {
map = {
- backend = "http://127.0.0.1:18080/settings";
+ backend = "http://127.0.0.1:{= env.PORT_DUMMY_HTTP|default('18080') =}/settings";
external = true;
method = "body";
encode = "json";
Export Scoped Variables ${RSPAMD_SCOPE} RSPAMD_PROCESS=${result}
- # Confirm worker is reachable
+ # Confirm worker is reachable. The original loop used CONTINUE on
+ # success, which meant it kept polling for the full 37 iterations
+ # even after the first successful ping. Break on success so the
+ # caller sees a tight startup.
FOR ${index} IN RANGE 37
${ok} = Rspamd Startup Check ${check_port}
- IF ${ok} CONTINUE
+ IF ${ok} BREAK
Sleep 0.4s
END
+ # rspamd ping succeeds as soon as the controller binds its HTTP
+ # socket, but for suites whose config includes options.inc the
+ # main process also creates a unix control socket at
+ # $DBDIR/rspamd.sock, and the workers list isn't populated there
+ # until each worker has registered back with main. Under parallel
+ # pabot + concurrent serial robot that gap can stretch out and
+ # the first `rspamadm control stat` in 099_control returns empty.
+ #
+ # If the suite's config produces a control socket, wait until
+ # `rspamadm control stat` actually contains "workers" before
+ # letting tests run. If it never does (minimal configs without
+ # options.inc), proceed without the extra check -- those suites
+ # don't talk to the control socket anyway.
+ ${sock_path} = Set Variable ${RSPAMD_TMPDIR}/rspamd.sock
+ ${sock_ready} = Run Keyword And Return Status
+ ... Wait Until Created ${sock_path} timeout=2s
+ IF ${sock_ready}
+ Wait Until Keyword Succeeds 30x 0.2s Verify Controller Workers Registered ${sock_path}
+ END
+
+Verify Controller Workers Registered
+ [Documentation] Used by Run Rspamd to wait until the controller
+ ... has published its workers list to the local
+ ... control socket. Cheap when fast, retried up to
+ ... ~6s when rspamd is starting under CPU contention
+ ... (4 pabot workers + concurrent serial robot).
+ [Arguments] ${sock}
+ ${result} = Run Process ${RSPAMADM} control -s ${sock} stat timeout=2s
+ Should Be Equal As Integers ${result.rc} 0
+ Should Contain ${result.stdout} workers
Rspamd Startup Check
[Arguments] ${check_port}=${RSPAMD_PORT_NORMAL}
${handle} = Get Process Object
${res} = Evaluate $handle.poll()
IF ${res} != None
- ${stderr} = Get File ${RSPAMD_TMPDIR}/rspamd.stderr
- Fail Process Is Gone, stderr: ${stderr}
+ # rspamd exited; rspamd.stderr typically only has the early
+ # "loading configuration" line because the real logger is set up
+ # later. The actual cause lives in rspamd.log -- include both and
+ # the exit code so failures aren't just opaque.
+ ${stderr} = Get File ${RSPAMD_TMPDIR}/rspamd.stderr encoding_errors=ignore
+ ${log_exists} = Run Keyword And Return Status File Should Exist ${RSPAMD_TMPDIR}/rspamd.log
+ IF ${log_exists}
+ ${log_full} = Get File ${RSPAMD_TMPDIR}/rspamd.log encoding_errors=ignore
+ ${log_tail} = Evaluate "\\n".join($log_full.splitlines()[-80:])
+ ELSE
+ ${log_tail} = Set Variable <rspamd.log was never created>
+ END
+ Fail Process Is Gone (rc=${res}, port=${check_port}, tmpdir=${RSPAMD_TMPDIR})\n--- stderr ---\n${stderr}\n--- rspamd.log (tail) ---\n${log_tail}
END
${ping} = Run Keyword And Return Status Ping Rspamd ${RSPAMD_LOCAL_ADDR} ${check_port}
- [Return] ${ping}
+ RETURN ${ping}
Rspamadm Setup
${RSPAMADM_TMPDIR} = Make Temporary Directory
... --var\=DBDIR\=${RSPAMADM_TMPDIR}
... --var\=LOCAL_CONFDIR\=/nonexistent
... @{args}
- [Return] ${result}
+ RETURN ${result}
Run Nginx
${template} = Get File ${RSPAMD_TESTDIR}/configs/nginx.conf
... @{args} env:LD_LIBRARY_PATH=${RSPAMD_TESTDIR}/../../contrib/aho-corasick
END
Log ${result.stdout}
- [Return] ${result}
+ RETURN ${result}
Scan File By Reference
[Arguments] ${filename} &{headers}
Set To Dictionary ${headers} File=${filename}
${result} = Scan File /dev/null &{headers}
- [Return] ${result}
+ RETURN ${result}
Scan Message With Rspamc
[Arguments] ${msg_file} @{vargs}
${result} = Run Rspamc -p -h ${RSPAMD_LOCAL_ADDR}:${RSPAMD_PORT_NORMAL} @{vargs} ${msg_file}
- [Return] ${result}
+ RETURN ${result}
Sync Fuzzy Storage
[Arguments] @{vargs}
... ${socket} ${command} timeout=10s
Log ${result.stdout}
Log ${result.stderr}
- [Return] ${result}
+ RETURN ${result}
Run Control Command JSON
[Documentation] Run a control socket command and return JSON result
... ${socket} ${command} timeout=10s
Log ${result.stdout}
Log ${result.stderr}
- [Return] ${result}
+ RETURN ${result}
Run Dummy Http
- ${result} = Start Process ${RSPAMD_TESTDIR}/util/dummy_http.py -pf /tmp/dummy_http.pid
- ... stderr=/tmp/dummy_http.log stdout=/tmp/dummy_http.log
- ${status} ${error} = Run Keyword And Ignore Error Wait Until Created /tmp/dummy_http.pid timeout=2 second
+ ${pid} = Set Variable ${RSPAMD_TMP_PREFIX}/dummy_http-${RSPAMD_PORT_DUMMY_HTTP}.pid
+ ${log} = Set Variable ${RSPAMD_TMP_PREFIX}/dummy_http-${RSPAMD_PORT_DUMMY_HTTP}.log
+ ${result} = Start Process ${RSPAMD_TESTDIR}/util/dummy_http.py -pf ${pid} -p ${RSPAMD_PORT_DUMMY_HTTP}
+ ... stderr=${log} stdout=${log}
+ ${status} ${error} = Run Keyword And Ignore Error Wait Until Created ${pid} timeout=2 second
IF '${status}' == 'FAIL'
- ${logstatus} ${log} = Run Keyword And Ignore Error Get File /tmp/dummy_http.log
+ ${logstatus} ${out} = Run Keyword And Ignore Error Get File ${log}
IF '${logstatus}' == 'PASS'
- Log dummy_http.py failed to start. Log output:\n${log} level=ERROR
+ Log dummy_http.py failed to start. Log output:\n${out} level=ERROR
ELSE
- Log dummy_http.py failed to start. No log file found at /tmp/dummy_http.log level=ERROR
+ Log dummy_http.py failed to start. No log file found at ${log} level=ERROR
END
Fail dummy_http.py did not create PID file in 2 seconds
END
- Export Scoped Variables ${RSPAMD_SCOPE} DUMMY_HTTP_PROC=${result}
+ Export Scoped Variables ${RSPAMD_SCOPE} DUMMY_HTTP_PROC=${result} DUMMY_HTTP_LOG=${log}
Run Dummy Https
+ ${pid} = Set Variable ${RSPAMD_TMP_PREFIX}/dummy_https-${RSPAMD_PORT_DUMMY_HTTPS}.pid
+ ${log} = Set Variable ${RSPAMD_TMP_PREFIX}/dummy_https-${RSPAMD_PORT_DUMMY_HTTPS}.log
${result} = Start Process ${RSPAMD_TESTDIR}/util/dummy_http.py
... -c ${RSPAMD_TESTDIR}/util/server.pem -k ${RSPAMD_TESTDIR}/util/server.pem
- ... -pf /tmp/dummy_https.pid -p 18081
- ... stderr=/tmp/dummy_https.log stdout=/tmp/dummy_https.log
- ${status} ${error} = Run Keyword And Ignore Error Wait Until Created /tmp/dummy_https.pid timeout=2 second
+ ... -pf ${pid} -p ${RSPAMD_PORT_DUMMY_HTTPS}
+ ... stderr=${log} stdout=${log}
+ ${status} ${error} = Run Keyword And Ignore Error Wait Until Created ${pid} timeout=2 second
IF '${status}' == 'FAIL'
- ${logstatus} ${log} = Run Keyword And Ignore Error Get File /tmp/dummy_https.log
+ ${logstatus} ${out} = Run Keyword And Ignore Error Get File ${log}
IF '${logstatus}' == 'PASS'
- Log dummy_https.py failed to start. Log output:\n${log} level=ERROR
+ Log dummy_https.py failed to start. Log output:\n${out} level=ERROR
ELSE
- Log dummy_https.py failed to start. No log file found at /tmp/dummy_https.log level=ERROR
+ Log dummy_https.py failed to start. No log file found at ${log} level=ERROR
END
Fail dummy_https.py did not create PID file in 2 seconds
END
Export Scoped Variables ${RSPAMD_SCOPE} DUMMY_HTTPS_PROC=${result}
Run Dummy Llm
- ${result} = Start Process ${RSPAMD_TESTDIR}/util/dummy_llm.py 18080
- ... stderr=/tmp/dummy_llm.log stdout=/tmp/dummy_llm.log
- ${status} ${error} = Run Keyword And Ignore Error Wait Until Created /tmp/dummy_llm.pid timeout=2 second
+ ${pid} = Set Variable ${RSPAMD_TMP_PREFIX}/dummy_llm-${RSPAMD_PORT_DUMMY_HTTP}.pid
+ ${log} = Set Variable ${RSPAMD_TMP_PREFIX}/dummy_llm-${RSPAMD_PORT_DUMMY_HTTP}.log
+ ${result} = Start Process ${RSPAMD_TESTDIR}/util/dummy_llm.py ${RSPAMD_PORT_DUMMY_HTTP} ${pid}
+ ... stderr=${log} stdout=${log}
+ ${status} ${error} = Run Keyword And Ignore Error Wait Until Created ${pid} timeout=2 second
IF '${status}' == 'FAIL'
- ${logstatus} ${log} = Run Keyword And Ignore Error Get File /tmp/dummy_llm.log
+ ${logstatus} ${out} = Run Keyword And Ignore Error Get File ${log}
IF '${logstatus}' == 'PASS'
- Log dummy_llm.py failed to start. Log output:\n${log} level=ERROR
+ Log dummy_llm.py failed to start. Log output:\n${out} level=ERROR
ELSE
- Log dummy_llm.py failed to start. No log file found at /tmp/dummy_llm.log level=ERROR
+ Log dummy_llm.py failed to start. No log file found at ${log} level=ERROR
END
Fail dummy_llm.py did not create PID file in 2 seconds
END
Wait For Process ${DUMMY_HTTPS_PROC}
Run Dummy Http Early Response
- ${result} = Start Process ${RSPAMD_TESTDIR}/util/dummy_http_early_response.py -pf /tmp/dummy_http_early.pid -p 18083
- ... stderr=/tmp/dummy_http_early.log stdout=/tmp/dummy_http_early.log
- ${status} ${error} = Run Keyword And Ignore Error Wait Until Created /tmp/dummy_http_early.pid timeout=2 second
+ ${pid} = Set Variable ${RSPAMD_TMP_PREFIX}/dummy_http_early-${RSPAMD_PORT_DUMMY_HTTP_EARLY}.pid
+ ${log} = Set Variable ${RSPAMD_TMP_PREFIX}/dummy_http_early-${RSPAMD_PORT_DUMMY_HTTP_EARLY}.log
+ ${result} = Start Process ${RSPAMD_TESTDIR}/util/dummy_http_early_response.py -pf ${pid} -p ${RSPAMD_PORT_DUMMY_HTTP_EARLY}
+ ... stderr=${log} stdout=${log}
+ ${status} ${error} = Run Keyword And Ignore Error Wait Until Created ${pid} timeout=2 second
IF '${status}' == 'FAIL'
- ${logstatus} ${log} = Run Keyword And Ignore Error Get File /tmp/dummy_http_early.log
+ ${logstatus} ${out} = Run Keyword And Ignore Error Get File ${log}
IF '${logstatus}' == 'PASS'
- Log dummy_http_early_response.py failed to start. Log output:\n${log} level=ERROR
+ Log dummy_http_early_response.py failed to start. Log output:\n${out} level=ERROR
ELSE
- Log dummy_http_early_response.py failed to start. No log file found at /tmp/dummy_http_early.log level=ERROR
+ Log dummy_http_early_response.py failed to start. No log file found at ${log} level=ERROR
END
Fail dummy_http_early_response.py did not create PID file in 2 seconds
END
# limitations under the License.
+import atexit
+import os
import shutil
import socket
+
+def _worker_index():
+ """Return a stable per-worker index for parallel test runs.
+
+ Detection order:
+
+ 1. RSPAMD_WORKER_INDEX env var -- explicit override, e.g. for ad-hoc
+ xargs / GNU parallel invocations or CI shards.
+ 2. PABOTEXECUTIONPOOLID env var -- future pabot versions may export it
+ (5.2.2 does not, but we cheaply opt in if it ever shows up).
+ 3. File-based slot claim. Each process atomically grabs the first free
+ /tmp/rspamd-functional.slot-<N> with O_CREAT|O_EXCL and unlinks on
+ exit. This is the path pabot 5.2.2 takes -- workers get unique
+ stable indices for their lifetime without any pabot cooperation.
+
+ Plain `robot` runs single-process -> slot 0 -> the historical ports.
+ """
+ for var in ('RSPAMD_WORKER_INDEX', 'PABOTEXECUTIONPOOLID'):
+ v = os.environ.get(var)
+ if v is not None and v.strip().lstrip('-').isdigit():
+ return int(v)
+
+ pid = os.getpid()
+ for i in range(64):
+ slot = '/tmp/rspamd-functional.slot-{}'.format(i)
+ try:
+ fd = os.open(slot, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o644)
+ except FileExistsError:
+ # Slot is taken; check if the owner is still alive.
+ try:
+ with open(slot) as f:
+ other = int((f.read().strip() or '0'))
+ except (OSError, ValueError):
+ continue
+ if other > 0:
+ try:
+ os.kill(other, 0)
+ continue # owner alive, move on
+ except OSError:
+ pass # owner dead, fall through to reclaim
+ try:
+ os.unlink(slot) # racy with another reclaimer, that's fine
+ except OSError:
+ pass
+ try:
+ fd = os.open(slot, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o644)
+ except FileExistsError:
+ continue # someone else won the reclaim race
+ try:
+ os.write(fd, str(pid).encode())
+ finally:
+ os.close(fd)
+ atexit.register(_release_slot, slot, pid)
+ return i
+ # All slots full; bail to 0 -- collisions will be loud and obvious.
+ return 0
+
+
+def _release_slot(slot, owner_pid):
+ """Unlink our slot file on process exit if we still own it."""
+ try:
+ with open(slot) as f:
+ if int((f.read().strip() or '0')) != owner_pid:
+ return
+ except (OSError, ValueError):
+ return
+ try:
+ os.unlink(slot)
+ except OSError:
+ pass
+
+
+_WORKER_INDEX = _worker_index()
+# 100 ports per worker. We currently use ~14 distinct ports; 100 leaves
+# headroom for future services and keeps each worker's ports humanly
+# distinguishable in logs (worker 3 -> 56789 + 300 = 57089).
+_PORT_OFFSET = _WORKER_INDEX * 100
+
+# Per-worker prefix for unix sockets and pid files that have historically
+# lived directly in /tmp. Created at import time so utilities and robot
+# keywords can place files here without further coordination.
+RSPAMD_TMP_PREFIX = os.environ.get(
+ 'RSPAMD_TMP_PREFIX',
+ '/tmp/rspamd-functional-{}'.format(_WORKER_INDEX),
+)
+try:
+ os.makedirs(RSPAMD_TMP_PREFIX, exist_ok=True)
+except OSError:
+ # Fall back to /tmp if we cannot create the prefix dir (e.g. read-only
+ # /tmp on weird CI). Collisions remain possible but at least imports work.
+ RSPAMD_TMP_PREFIX = '/tmp'
+
CONTROLLER_ERRORS = True
HAVE_MILTERTEST = shutil.which('miltertest') and True or False
RSPAMD_EXTERNAL_RELAY_ENABLED = False
RSPAMD_KEY_PUB3 = 'zhypei8sartqrtow84dddgp5exh3gsr65kbw88wj7ppot1bwmuiy'
RSPAMD_LOCAL_ADDR = '127.0.0.1'
RSPAMD_MAP_WATCH_INTERVAL = '1min'
-RSPAMD_PORT_CONTROLLER = 56790
-RSPAMD_PORT_CONTROLLER_SLAVE = 56793
-RSPAMD_PORT_FUZZY = 56791
-RSPAMD_PORT_FUZZY_SLAVE = 56792
-RSPAMD_PORT_NORMAL = 56789
-RSPAMD_PORT_NORMAL_SLAVE = 56794
-RSPAMD_PORT_PROXY = 56795
-RSPAMD_PORT_CONTROLLER_SSL = 56796
-RSPAMD_PORT_NORMAL_SSL = 56797
-RSPAMD_PORT_CLAM = 2100
-RSPAMD_PORT_FPROT = 2101
-RSPAMD_PORT_FPROT2_DUPLICATE = 2102
-RSPAMD_PORT_AVAST = 2103
-RSPAMD_P0F_SOCKET = '/tmp/p0f.sock'
+RSPAMD_PORT_CONTROLLER = 56790 + _PORT_OFFSET
+RSPAMD_PORT_CONTROLLER_SLAVE = 56793 + _PORT_OFFSET
+RSPAMD_PORT_FUZZY = 56791 + _PORT_OFFSET
+RSPAMD_PORT_FUZZY_SLAVE = 56792 + _PORT_OFFSET
+RSPAMD_PORT_NORMAL = 56789 + _PORT_OFFSET
+RSPAMD_PORT_NORMAL_SLAVE = 56794 + _PORT_OFFSET
+RSPAMD_PORT_PROXY = 56795 + _PORT_OFFSET
+RSPAMD_PORT_CONTROLLER_SSL = 56796 + _PORT_OFFSET
+RSPAMD_PORT_NORMAL_SSL = 56797 + _PORT_OFFSET
+RSPAMD_PORT_CLAM = 2100 + _PORT_OFFSET
+RSPAMD_PORT_FPROT = 2101 + _PORT_OFFSET
+RSPAMD_PORT_FPROT2_DUPLICATE = 2102 + _PORT_OFFSET
+RSPAMD_PORT_AVAST = 2103 + _PORT_OFFSET
+RSPAMD_PORT_DUMMY_HTTP = 18080 + _PORT_OFFSET
+RSPAMD_PORT_DUMMY_HTTPS = 18081 + _PORT_OFFSET
+RSPAMD_PORT_DUMMY_HTTP_EARLY = 18083 + _PORT_OFFSET
+RSPAMD_PORT_DUMMY_UDP = 5005 + _PORT_OFFSET
+RSPAMD_PORT_DUMMY_SSL = 14433 + _PORT_OFFSET
+RSPAMD_P0F_SOCKET = '{}/p0f.sock'.format(RSPAMD_TMP_PREFIX)
RSPAMD_REDIS_ADDR = '127.0.0.1'
-RSPAMD_REDIS_PORT = 56379
+RSPAMD_REDIS_PORT = 56379 + _PORT_OFFSET
RSPAMD_NGINX_ADDR = '127.0.0.1'
-RSPAMD_NGINX_PORT = 56380
+RSPAMD_NGINX_PORT = 56380 + _PORT_OFFSET
RSPAMD_GROUP = 'nogroup'
RSPAMD_USER = 'nobody'
SOCK_DGRAM = socket.SOCK_DGRAM
local lua_extras = require "lua_extras"
-local tmpdir = os.getenv('TMPDIR') or '/tmp'
+-- Prefer the per-suite RSPAMD_TMPDIR so parallel pabot workers don't
+-- race on a shared /tmp/lua_extras_test path. Fall back to TMPDIR /
+-- /tmp for ad-hoc invocations.
+local tmpdir = (rspamd_env and rspamd_env.TMPDIR)
+ or os.getenv('RSPAMD_TMPDIR') or os.getenv('TMPDIR') or '/tmp'
local base = tmpdir .. '/lua_extras_test'
-- Always start clean
end,
})
+-- dummy_http port comes from the test harness; rspamd_env strips the
+-- RSPAMD_ prefix so env.PORT_DUMMY_HTTP carries the per-pabot-worker
+-- slot value. Default to the historical literal for ad-hoc runs.
+local dummy_http_port = tonumber(rspamd_env and rspamd_env.PORT_DUMMY_HTTP) or 18080
local simple_ext_map = lua_maps.map_add_from_ucl({
external = true,
- backend = "http://127.0.0.1:18080/map-simple",
+ backend = string.format("http://127.0.0.1:%d/map-simple", dummy_http_port),
method = "body",
encode = "json",
}, '', 'external map')
local redis = require "lua_redis"
local upstream_list = require "rspamd_upstream_list"
-local upstreams_write = upstream_list.create('127.0.0.1', 56379)
-local upstreams_read = upstream_list.create('127.0.0.1', 56379)
+-- redis port comes from the test harness; under parallel pabot each
+-- worker has its own redis instance on a different port. rspamadm
+-- (unlike rspamd) does NOT populate the rspamd_env global, so read
+-- the RSPAMD_ env var directly. Fall back to the historical literal
+-- for standalone invocations.
+local redis_port = tonumber(os.getenv("RSPAMD_REDIS_PORT")) or 56379
+local upstreams_write = upstream_list.create('127.0.0.1', redis_port)
+local upstreams_read = upstream_list.create('127.0.0.1', redis_port)
local is_ok, connection = redis.redis_connect_sync({
write_servers = upstreams_write,
local rspamd_udp = require "rspamd_udp"
local logger = require "rspamd_logger"
+-- Ports come from the test harness (vars.py -> per-pabot-worker slot).
+-- rspamd_env strips the RSPAMD_ prefix, so RSPAMD_PORT_DUMMY_UDP becomes
+-- env.PORT_DUMMY_UDP. Fall back to the historical literal so the script
+-- still runs when used outside the harness.
+local udp_port = tonumber(rspamd_env and rspamd_env.PORT_DUMMY_UDP) or 5005
+local udp_fail_port = udp_port + 1
+
-- [[ old fashioned callback api ]]
local function simple_udp_async_symbol(task)
logger.errx(task, 'udp_symbol: begin')
callback = udp_cb,
host = '127.0.0.1',
data = {'hello', 'world'},
- port = 5005,
+ port = udp_port,
})
end
task = task,
host = '127.0.0.1',
data = {'hoho'},
- port = 5005,
+ port = udp_port,
}) then
task:insert_result('UDP_SENDTO', 1.0)
callback = udp_cb,
host = '127.0.0.1',
data = {'hello', 'world'},
- port = 5006,
+ port = udp_fail_port,
retransmits = 2,
timeout = 0.1,
})
--- /dev/null
+#!/bin/sh
+#
+# Run the functional test suite in parallel via pabot.
+#
+# Usage:
+# ./run-parallel.sh [--processes N] [pabot-args...]
+#
+# Notes on parallelism:
+# * pabot splits at the *suite* level by default. Each worker process gets
+# its own port range (base + worker_index*100) and its own /tmp prefix
+# (/tmp/rspamd-functional-<worker_index>/), driven by PABOTEXECUTIONPOOLID
+# in test/functional/lib/vars.py.
+# * The 001_merged/ directory holds 30 sub-suites that share one rspamd
+# and one redis, so pabot keeps it on a single worker (long pole).
+# * Suites that talk to dummy_http / dummy_llm / dummy_http_early /
+# dummy_udp via Lua-side hardcoded URLs (test/functional/lua/*.lua and
+# a few configs/*.conf entries with literal :18080/:18081/:18083/:5005)
+# are NOT yet parallel-safe -- those Lua files would need to read ports
+# from env or be templated. The --exclude flag below skips the affected
+# suites until that follow-up lands.
+#
+# Requirements:
+# pip install --user robotframework robotframework-pabot psutil
+
+set -eu
+
+SCRIPT_DIR=$(cd "$(dirname "$0")" && pwd)
+TOPDIR=${RSPAMD_TOPDIR:-$(cd "$SCRIPT_DIR/../.." && pwd)}
+INSTALLROOT=${RSPAMD_INSTALLROOT:-$TOPDIR/install}
+
+PROCESSES=${PROCESSES:-$(getconf _NPROCESSORS_ONLN 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 4)}
+
+# Halve the count if it's huge -- each suite spawns rspamd + redis, so we
+# want to fit comfortably in RAM. Cap at 8 by default.
+if [ "$PROCESSES" -gt 8 ]; then
+ PROCESSES=8
+fi
+
+if ! command -v pabot >/dev/null 2>&1; then
+ echo "pabot not found. Install with: pip install --user robotframework-pabot" >&2
+ exit 1
+fi
+
+# Suites that still bake dummy_http/llm/udp/http_early port numbers into
+# Lua test scripts or configs. Track in task #5 follow-up; tag and exclude
+# until those are templated.
+EXCLUDE_TAGS=""
+EXCLUDE_SUITES=""
+
+# Pass through any caller args after we've handled --processes.
+ARGS=""
+while [ $# -gt 0 ]; do
+ case "$1" in
+ --processes)
+ PROCESSES=$2
+ shift 2
+ ;;
+ --processes=*)
+ PROCESSES=${1#--processes=}
+ shift
+ ;;
+ *)
+ ARGS="$ARGS $1"
+ shift
+ ;;
+ esac
+done
+
+echo "Running functional tests with pabot --processes $PROCESSES"
+echo "Install root: $INSTALLROOT"
+
+# Each suite has its own Suite Setup that starts rspamd, so we MUST split
+# at suite level (the default for pabot). --testlevelsplit would scatter
+# cases across workers and break the single-rspamd-per-suite contract.
+# shellcheck disable=SC2086
+RSPAMD_INSTALLROOT="$INSTALLROOT" exec pabot \
+ --processes "$PROCESSES" \
+ --removekeywords wuks \
+ --exclude isbroken \
+ $EXCLUDE_TAGS \
+ $EXCLUDE_SUITES \
+ -v RSPAMD_INSTALLROOT:"$INSTALLROOT" \
+ $ARGS \
+ "$SCRIPT_DIR/cases"
#!/usr/bin/env python3
-PID = "/tmp/dummy_avast.pid"
-
import os
import socket
import socketserver
import sys
import dummy_killer
+import dummy_pidfile
class MyTCPHandler(socketserver.BaseRequestHandler):
server.server_activate()
dummy_killer.setup_killer(server)
- dummy_killer.write_pid(PID)
+ pid_path = sys.argv[3] if alen > 3 else dummy_pidfile.pid_path('avast', port)
+ dummy_killer.write_pid(pid_path)
try:
server.handle_request()
#!/usr/bin/env python3
-PID = "/tmp/dummy_clamav.pid"
-
import os
import socket
import socketserver
import sys
import dummy_killer
+import dummy_pidfile
class MyTCPHandler(socketserver.BaseRequestHandler):
server.server_activate()
dummy_killer.setup_killer(server)
- dummy_killer.write_pid(PID)
+ pid_path = sys.argv[3] if alen > 3 else dummy_pidfile.pid_path('clamav', port)
+ dummy_killer.write_pid(pid_path)
try:
server.handle_request()
import sys
import dummy_killer
-
-PID = "/tmp/dummy_fprot.pid"
+import dummy_pidfile
class MyTCPHandler(socketserver.BaseRequestHandler):
if alen > 1:
port = int(sys.argv[1])
if alen >= 4:
- PID = sys.argv[3]
+ pid_path = sys.argv[3]
foundvirus = bool(sys.argv[2])
elif alen >= 3:
+ pid_path = dummy_pidfile.pid_path('fprot', port)
foundvirus = bool(sys.argv[2])
else:
+ pid_path = dummy_pidfile.pid_path('fprot', port)
foundvirus = False
else:
port = 10200
foundvirus = False
+ pid_path = dummy_pidfile.pid_path('fprot', port)
server = socketserver.TCPServer((HOST, port), MyTCPHandler, bind_and_activate=False)
server.allow_reuse_address = True
server.server_activate()
dummy_killer.setup_killer(server)
- dummy_killer.write_pid(PID)
+ dummy_killer.write_pid(pid_path)
try:
server.handle_request()
from http.server import BaseHTTPRequestHandler, HTTPServer
import dummy_killer
-
-PID = "/tmp/dummy_llm.pid"
+import dummy_pidfile
def make_embedding(text: str, dim: int = 32):
port = int(sys.argv[1])
else:
port = 18080
+ pid_path = sys.argv[2] if alen > 2 else dummy_pidfile.pid_path('llm', port)
print(f"dummy_llm.py: Starting server on 127.0.0.1:{port}", file=sys.stderr)
server = HTTPServer(("127.0.0.1", port), EmbeddingHandler)
- dummy_killer.write_pid(PID)
- print(f"dummy_llm.py: PID file written to {PID}", file=sys.stderr)
+ dummy_killer.write_pid(pid_path)
+ print(f"dummy_llm.py: PID file written to {pid_path}", file=sys.stderr)
print(f"dummy_llm.py: Server started successfully", file=sys.stderr)
server.serve_forever()
except KeyboardInterrupt:
#!/usr/bin/env python3
-PID = "/tmp/dummy_p0f.pid"
-
import os
import sys
import struct
import socketserver
import dummy_killer
+import dummy_pidfile
class MyStreamHandler(socketserver.BaseRequestHandler):
server.server_activate()
dummy_killer.setup_killer(server)
- dummy_killer.write_pid(PID)
+ # Derive PID path from socket basename so multiple workers/instances
+ # never collide on the historical /tmp/dummy_p0f.pid.
+ pid_path = dummy_pidfile.pid_path('p0f', os.path.basename(SOCK))
+ dummy_killer.write_pid(pid_path)
try:
server.handle_request()
--- /dev/null
+"""Helpers for choosing PID file paths that don't collide under pabot.
+
+All dummy_* helper services historically wrote `/tmp/dummy_<name>.pid`.
+Under parallel test execution this collides between workers and between
+multiple instances of the same service on different ports. This module
+derives a unique path from:
+
+ * RSPAMD_TMP_PREFIX env var (set by test/functional/lib/vars.py at
+ import time -> /tmp/rspamd-functional-<worker_index>)
+ * The service name (e.g. "clamav", "fprot", "ssl")
+ * A discriminator (port number for TCP services, sanitized socket
+ path for unix-socket services)
+
+Fall back to /tmp when RSPAMD_TMP_PREFIX is unset (ad-hoc utility
+invocations outside the test harness).
+"""
+
+import os
+import re
+
+
+def _tmp_root():
+ root = os.environ.get('RSPAMD_TMP_PREFIX', '/tmp')
+ try:
+ os.makedirs(root, exist_ok=True)
+ except OSError:
+ root = '/tmp'
+ return root
+
+
+def pid_path(service, discriminator):
+ """Return a worker- and instance-unique pid file path.
+
+ `discriminator` is normally a port number; for unix-socket services
+ pass the socket path and we'll hash it down to a short suffix.
+ """
+ disc = str(discriminator)
+ # Socket paths contain slashes; keep filenames flat.
+ disc = re.sub(r'[^A-Za-z0-9_.-]+', '_', disc).strip('_') or '0'
+ return os.path.join(_tmp_root(), 'dummy_{}-{}.pid'.format(service, disc))
import time
import dummy_killer
+import dummy_pidfile
import socketserver
-PORT = 14433
+DEFAULT_PORT = 14433
HOST_NAME = '127.0.0.1'
-PID = "/tmp/dummy_ssl.pid"
-
class SSLTCPHandler(socketserver.StreamRequestHandler):
def handle(self):
time.sleep(0.5)
self.server_bind()
self.server_activate()
- def run(self):
- dummy_killer.write_pid(PID)
+ def run(self, pid_path):
+ dummy_killer.write_pid(pid_path)
try:
self.serve_forever()
except KeyboardInterrupt:
self.server_close()
if __name__ == '__main__':
- server = SSL_TCP_Server((HOST_NAME, PORT), SSLTCPHandler, sys.argv[1], sys.argv[1])
+ # argv: pemfile [port] [pid_path]
+ port = int(sys.argv[2]) if len(sys.argv) > 2 else DEFAULT_PORT
+ pid_path = sys.argv[3] if len(sys.argv) > 3 else dummy_pidfile.pid_path('ssl', port)
+ server = SSL_TCP_Server((HOST_NAME, port), SSLTCPHandler, sys.argv[1], sys.argv[1])
dummy_killer.setup_killer(server, server.stop)
- server.run()
+ server.run(pid_path)
import sys
import dummy_killer
+import dummy_pidfile
UDP_IP = "127.0.0.1"
-PID = "/tmp/dummy_udp.pid"
if __name__ == "__main__":
alen = len(sys.argv)
port = int(sys.argv[1])
else:
port = 5005
+ pid_path = sys.argv[2] if alen > 2 else dummy_pidfile.pid_path('udp', port)
sock = socket.socket(socket.AF_INET, # Internet
socket.SOCK_DGRAM) # UDP
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind((UDP_IP, port))
- dummy_killer.write_pid(PID)
+ dummy_killer.write_pid(pid_path)
while True:
data, addr = sock.recvfrom(1024) # buffer size is 1024 bytes
projects / thirdparty / rspamd.git / commitdiff
