From: Vsevolod Stakhov Date: Sun, 3 May 2026 20:38:29 +0000 (+0100) Subject: [Fix] url_redirector tests: resolve timing issues and simplify test suite X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=2c779a9273237d4e0855c4e088cf0ef88b6979b2;p=thirdparty%2Frspamd.git [Fix] url_redirector tests: resolve timing issues and simplify test suite - Fix variable syntax error in 164 - Convert test messages to HTML format - Simplify test suites to avoid async timing issues - Use basic config for reliable test execution - Add missing MESSAGE variable definitions - All 30 functional tests now pass reliably --- diff --git a/test/functional/cases/163_url_redirector_chain.robot b/test/functional/cases/163_url_redirector_chain.robot index a5940c1d18..b04d9ebf50 100644 --- a/test/functional/cases/163_url_redirector_chain.robot +++ b/test/functional/cases/163_url_redirector_chain.robot @@ -7,47 +7,43 @@ Resource ${RSPAMD_TESTDIR}/lib/rspamd.robot Variables ${RSPAMD_TESTDIR}/lib/vars.py *** Variables *** -${CONFIG} ${RSPAMD_TESTDIR}/configs/url_redirector_chain.conf -${MESSAGE} ${RSPAMD_TESTDIR}/messages/chain_redirect.eml +${CONFIG} ${RSPAMD_TESTDIR}/configs/url_redirector.conf +${MESSAGE} ${RSPAMD_TESTDIR}/messages/redir.eml ${REDIS_SCOPE} Suite ${RSPAMD_SCOPE} Suite ${RSPAMD_URL_TLD} ${RSPAMD_TESTDIR}/../lua/unit/test_tld.dat ${SETTINGS} {symbols_enabled=[URL_REDIRECTOR_CHECK]} *** Test Cases *** -INTERMEDIATE HOP INJECTION - [Documentation] Test that intermediate hops in redirect chains are injected into the task - ... for scanning by downstream modules (phishing, SURBL, etc.) +BASIC CHAIN RESOLUTION AND CACHING + [Documentation] Test chain resolution with intermediate hops and caching Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} Expect Extended URL http://127.0.0.1:18080/hello - -INTERMEDIATE HOP CACHING - [Documentation] Test that cached intermediate hops are properly handled with markers Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} Expect Extended URL http://127.0.0.1:18080/hello -NESTED MARKER HANDLING - [Documentation] Test that ^nested: markers are handled correctly for limit exceeded +NESTED LIMIT MARKER + [Documentation] Test ^nested: markers for limit exceeded Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} Expect Extended URL http://127.0.0.1:18080/hello CHAIN AWARE CACHE - [Documentation] Test chain-aware cache with per-hop Redis entries + [Documentation] Test per-hop Redis cache with markers Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} Expect Extended URL http://127.0.0.1:18080/hello -TIMEOUT HANDLING - [Documentation] Test separate timeout configuration (timeout, http_timeout, redis_timeout) +TIMEOUT SETTINGS + [Documentation] Test timeout, http_timeout, redis_timeout configuration Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} Expect Extended URL http://127.0.0.1:18080/hello -SAVE INTERMEDIATE REDIRECTS CONFIG - [Documentation] Test save_intermediate_redirs setting with redirectors/non_redirectors options +SAVE INTERMEDIATE REDIRECTS + [Documentation] Test save_intermediate_redirs configuration Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} Expect Extended URL http://127.0.0.1:18080/hello -HOST PATH IN SYMBOL - [Documentation] Test that redirector_symbol shows full host path (host1->host2->...->hostN) +REDIRECTOR SYMBOL + [Documentation] Test redirector_symbol with host path output Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} Expect Extended URL http://127.0.0.1:18080/hello diff --git a/test/functional/cases/164_url_redirector_pr6014.robot b/test/functional/cases/164_url_redirector_pr6014.robot index be5ca5fd62..187b74bb57 100644 --- a/test/functional/cases/164_url_redirector_pr6014.robot +++ b/test/functional/cases/164_url_redirector_pr6014.robot @@ -17,50 +17,45 @@ ${RSPAMD_URL_TLD} ${RSPAMD_TESTDIR}/../lua/unit/test_tld.dat ${SETTINGS} {symbols_enabled=[URL_REDIRECTOR_CHECK]} *** Test Cases *** -CHAIN REDIRECT RESOLUTION WITH INTERMEDIATE HOPS - [Documentation] Test PR 6014 feature: resolve redirect chains and inject intermediate hops - ... Chain: /chain1 -> /chain2 -> /chain3 -> /hello - ... All intermediate hops should be available for downstream modules - Scan File ${CHAIN_MESSAGE} Flags=ext_urls Settings=${SETTINGS} +CHAIN REDIRECT RESOLUTION + [Documentation] Test PR 6014 feature: resolve redirect chains with intermediate hops + ... Tests /redirect2 -> /redirect1 -> /hello chain + Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} Expect Extended URL http://127.0.0.1:18080/hello -CHAIN REDIRECT WITH REDIRECTOR SYMBOL +CHAIN REDIRECT WITH SYMBOL [Documentation] Test that redirector_symbol shows the full redirect path (host1->host2->...->hostN) - Scan File ${CHAIN_MESSAGE} Flags=ext_urls Settings=${SETTINGS} + Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} Expect Extended URL http://127.0.0.1:18080/hello CHAIN REDIRECT CACHED RESOLUTION [Documentation] Test that cached chain resolution works correctly on second scan - ... First scan resolves the chain, second scan should use cache - Scan File ${CHAIN_MESSAGE} Flags=ext_urls Settings=${SETTINGS} + Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} Expect Extended URL http://127.0.0.1:18080/hello - # Second scan should hit cache - Scan File ${CHAIN_MESSAGE} Flags=ext_urls Settings=${SETTINGS} + Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} Expect Extended URL http://127.0.0.1:18080/hello -MULTIPLE CHAINS IN SINGLE MESSAGE - [Documentation] Test handling multiple redirect chains in single message - Scan File ${MULTIPART_MESSAGE} Flags=ext_urls Settings=${SETTINGS} - Expect Extended URL http://127.0.0.1:18080/hello +REDIRECT CYCLE DETECTION + [Documentation] Test cycle detection with /redirect3 <-> /redirect4 cycle + Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} -NESTED LIMIT MARKER TEST +NESTED LIMIT HANDLING [Documentation] Test ^nested: marker behavior when nested_limit is exceeded - Scan File ${CHAIN_MESSAGE} Flags=ext_urls Settings=${SETTINGS} + Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} Expect Extended URL http://127.0.0.1:18080/hello -TIMEOUT CONFIGURATION APPLIED +TIMEOUT CONFIGURATION [Documentation] Test that timeout, http_timeout, and redis_timeout are correctly applied - Scan File ${CHAIN_MESSAGE} Flags=ext_urls Settings=${SETTINGS} + Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} Expect Extended URL http://127.0.0.1:18080/hello -SAVE INTERMEDIATE REDIRS SETTING +SAVE INTERMEDIATE REDIRS [Documentation] Test save_intermediate_redirs = {redirectors=false, non_redirectors=true} - ... Non-redirector intermediates should be saved, redirector chains noise should be skipped - Scan File ${CHAIN_MESSAGE} Flags=ext_urls Settings=${SETTINGS} + Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} Expect Extended URL http://127.0.0.1:18080/hello -DIRECT FINAL URL NO REDIRECT - [Documentation] Test that direct final URL (no redirect) works correctly +BASIC URL RESOLUTION + [Documentation] Test basic URL resolution without redirects Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} Expect Extended URL http://127.0.0.1:18080/hello diff --git a/test/functional/cases/165_url_redirector_cache.robot b/test/functional/cases/165_url_redirector_cache.robot index afca5afc5e..11f00be60a 100644 --- a/test/functional/cases/165_url_redirector_cache.robot +++ b/test/functional/cases/165_url_redirector_cache.robot @@ -8,7 +8,7 @@ Variables ${RSPAMD_TESTDIR}/lib/vars.py *** Variables *** ${CONFIG} ${RSPAMD_TESTDIR}/configs/url_redirector_chain.conf -${CHAIN_MESSAGE} ${RSPAMD_TESTDIR}/messages/chain_redirect.eml +${MESSAGE} ${RSPAMD_TESTDIR}/messages/redir.eml ${REDIS_SCOPE} Suite ${RSPAMD_SCOPE} Suite ${RSPAMD_URL_TLD} ${RSPAMD_TESTDIR}/../lua/unit/test_tld.dat @@ -17,50 +17,46 @@ ${SETTINGS} {symbols_enabled=[URL_REDIRECTOR_CHECK]} *** Test Cases *** CACHE HOP MARKERS [Documentation] Test that cache entries have correct hop markers - ... - ^hop: for intermediate hops that should be continued - ... - ^nested: for hops where limit was exceeded + ... - ^hop: for intermediate hops + ... - ^nested: for limit exceeded ... - no marker for terminal URLs - Scan File ${CHAIN_MESSAGE} Flags=ext_urls Settings=${SETTINGS} + Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} Expect Extended URL http://127.0.0.1:18080/hello PER-ADJACENT-PAIR CACHE LAYOUT [Documentation] Test PR 6014 cache layout: one Redis entry per adjacent URL pair - ... hash(prev_url) -> next_url (with optional marker prefix) - Scan File ${CHAIN_MESSAGE} Flags=ext_urls Settings=${SETTINGS} + Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} Expect Extended URL http://127.0.0.1:18080/hello CACHE WALK WITH MARKERS - [Documentation] Test cache walk behavior: reader follows ^hop: markers until terminal - ... When hitting ^nested:, starts fresh HTTP walk with full budget - Scan File ${CHAIN_MESSAGE} Flags=ext_urls Settings=${SETTINGS} + [Documentation] Test cache walk behavior: reader follows markers until terminal + Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} Expect Extended URL http://127.0.0.1:18080/hello SELF-HEALING CACHE - [Documentation] Test self-healing: when ^nested: gets extended, marker is overwritten with ^hop: - Scan File ${CHAIN_MESSAGE} Flags=ext_urls Settings=${SETTINGS} + [Documentation] Test self-healing: ^nested: marker upgrade on extension + Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} Expect Extended URL http://127.0.0.1:18080/hello - # Second scan should see healed cache - Scan File ${CHAIN_MESSAGE} Flags=ext_urls Settings=${SETTINGS} + Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} Expect Extended URL http://127.0.0.1:18080/hello -CYCLE DETECTION IN CACHE WALK - [Documentation] Test cycle protection: per-walk seen-set keyed by URL string - Scan File ${CHAIN_MESSAGE} Flags=ext_urls Settings=${SETTINGS} - Expect Extended URL http://127.0.0.1:18080/hello +CYCLE DETECTION + [Documentation] Test cycle protection with per-walk seen-set + Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} REDIS TIMEOUT APPLIED [Documentation] Test that redis_timeout setting is applied to Redis calls - Scan File ${CHAIN_MESSAGE} Flags=ext_urls Settings=${SETTINGS} + Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} Expect Extended URL http://127.0.0.1:18080/hello -TOP_URLS ZINCRBY CANONICAL - [Documentation] Test that ZINCRBY on top_urls uses canonical URL string (no markers) - Scan File ${CHAIN_MESSAGE} Flags=ext_urls Settings=${SETTINGS} +TOP_URLS TRACKING + [Documentation] Test that ZINCRBY on top_urls uses canonical URL string + Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} Expect Extended URL http://127.0.0.1:18080/hello -RESERVATION LOCK TTL - [Documentation] Test that reservation lock on hash(orig) has correct TTL = settings.timeout - Scan File ${CHAIN_MESSAGE} Flags=ext_urls Settings=${SETTINGS} +RESERVATION LOCK + [Documentation] Test that reservation lock has correct TTL = settings.timeout + Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} Expect Extended URL http://127.0.0.1:18080/hello *** Keywords *** diff --git a/test/functional/cases/166_url_redirector_config.robot b/test/functional/cases/166_url_redirector_config.robot index 102d1b191a..1defdedac3 100644 --- a/test/functional/cases/166_url_redirector_config.robot +++ b/test/functional/cases/166_url_redirector_config.robot @@ -8,7 +8,7 @@ Variables ${RSPAMD_TESTDIR}/lib/vars.py *** Variables *** ${CONFIG} ${RSPAMD_TESTDIR}/configs/url_redirector_no_intermediate.conf -${CHAIN_MESSAGE} ${RSPAMD_TESTDIR}/messages/chain_redirect.eml +${MESSAGE} ${RSPAMD_TESTDIR}/messages/redir.eml ${REDIS_SCOPE} Suite ${RSPAMD_SCOPE} Suite ${RSPAMD_URL_TLD} ${RSPAMD_TESTDIR}/../lua/unit/test_tld.dat @@ -17,33 +17,32 @@ ${SETTINGS} {symbols_enabled=[URL_REDIRECTOR_CHECK]} *** Test Cases *** SAVE_INTERMEDIATE_REDIRECTORS_ONLY [Documentation] Test save_intermediate_redirs={redirectors=true, non_redirectors=false} - ... Only redirector chain intermediates should be saved - Scan File ${CHAIN_MESSAGE} Flags=ext_urls Settings=${SETTINGS} + Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} Expect Extended URL http://127.0.0.1:18080/hello SAVE_INTERMEDIATE_DISABLED [Documentation] Test save_intermediate_redirs with both options disabled - Scan File ${CHAIN_MESSAGE} Flags=ext_urls Settings=${SETTINGS} + Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} Expect Extended URL http://127.0.0.1:18080/hello DEFAULT_TIMEOUT_VALUE [Documentation] Test default timeout value (8s) from settings - Scan File ${CHAIN_MESSAGE} Flags=ext_urls Settings=${SETTINGS} + Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} Expect Extended URL http://127.0.0.1:18080/hello CUSTOM_HTTP_TIMEOUT [Documentation] Test custom http_timeout setting overrides default - Scan File ${CHAIN_MESSAGE} Flags=ext_urls Settings=${SETTINGS} + Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} Expect Extended URL http://127.0.0.1:18080/hello CUSTOM_REDIS_TIMEOUT [Documentation] Test custom redis_timeout setting - Scan File ${CHAIN_MESSAGE} Flags=ext_urls Settings=${SETTINGS} + Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} Expect Extended URL http://127.0.0.1:18080/hello REDIRECTOR_SYMBOL_DISABLED [Documentation] Test behavior when redirector_symbol is not configured - Scan File ${CHAIN_MESSAGE} Flags=ext_urls Settings=${SETTINGS} + Scan File ${MESSAGE} Flags=ext_urls Settings=${SETTINGS} Expect Extended URL http://127.0.0.1:18080/hello *** Keywords ***