]>
git.ipfire.org Git - thirdparty/rspamd.git/log
Jan Smutny [Fri, 25 Apr 2025 09:36:49 +0000 (11:36 +0200)]
fix(lua_redis): Improve Redis script loading
This change modifies how Rspamd handles Redis script loading across multiple
servers to improve resilience during server failures and restarts.
Previously, the system required ALL Redis servers to successfully load a script
before considering it ready for use. This caused modules to get stuck in a waiting
state when any Redis server was down, reporting errors like:
"redis script is not ready, waiting it to be loaded"
even when most servers were operational.
Key changes:
- Replace is_all_servers_ready() with is_any_server_ready() to allow operation
when at least one server has successfully loaded the script
- Reset all servers to "unsent" status when NOSCRIPT errors are encountered to
properly handle server restarts
- Ensure script loading is retried appropriately on reconnection
This fix allows Rspamd to continue operating when some Redis servers are temporarily
unavailable and to recover gracefully when servers rejoin the pool.
Vsevolod Stakhov [Tue, 22 Apr 2025 08:53:03 +0000 (09:53 +0100)]
[Fix] Arc: Use tonumber when comparing
Issue: #4044
Vsevolod Stakhov [Fri, 18 Apr 2025 07:29:33 +0000 (13:29 +0600)]
Merge pull request #5436 from SuperSandro2000/patch-1
Fix writing rules tutorial link
Vsevolod Stakhov [Thu, 17 Apr 2025 09:29:20 +0000 (15:29 +0600)]
Merge pull request #5434 from fatalbanana/preresult_modules
[Minor] Use module name when calling `set_pre_result`
Sandro [Wed, 16 Apr 2025 23:47:56 +0000 (01:47 +0200)]
Fix writing rules tutorial link
Andrew Lewis [Wed, 16 Apr 2025 10:45:19 +0000 (12:45 +0200)]
[Minor] Use module name when calling `set_pre_result`
Vsevolod Stakhov [Mon, 14 Apr 2025 07:52:57 +0000 (13:52 +0600)]
Merge pull request #5407 from dragoangel/patch-10
Add Sieve and cPanel forwarding symbols
Vsevolod Stakhov [Mon, 14 Apr 2025 07:52:51 +0000 (13:52 +0600)]
Merge branch 'master' into patch-10
Vsevolod Stakhov [Wed, 9 Apr 2025 19:36:27 +0000 (01:36 +0600)]
Merge pull request #5426 from fatalbanana/recontextal
[Minor] contextal: conditionally disable timer, minor changes
Andrew Lewis [Tue, 8 Apr 2025 10:02:25 +0000 (12:02 +0200)]
[Minor] contextal: conditionally disable timer, minor changes
Vsevolod Stakhov [Mon, 7 Apr 2025 13:05:30 +0000 (19:05 +0600)]
Merge pull request #5423 from moisseev/webui
[WebUI] Update Bootstrap, D3 and PrismJS libraries
Alexander Moisseev [Sun, 6 Apr 2025 13:23:54 +0000 (16:23 +0300)]
[WebUI] Update Bootstrap, D3 and PrismJS libraries
- Bootstrap 5.3.3 → 5.3.5
- D3 7.8.5 → 7.9.0
- PrismJS 1.29.0 → 1.30.0
Vsevolod Stakhov [Fri, 4 Apr 2025 13:28:32 +0000 (19:28 +0600)]
Merge pull request #5422 from KIMDONGYEON00/master
fix(lua_bit) : Prevent lua-bit stack buffer overflow
KIMDONGYEON00 [Fri, 4 Apr 2025 07:32:49 +0000 (16:32 +0900)]
fix(lua_bit) : Prevent lua-bit stack buffer overflow
Vsevolod Stakhov [Thu, 3 Apr 2025 08:42:20 +0000 (14:42 +0600)]
Merge pull request #5419 from rspamd/vstakhov-redis-version
[Feature] Allow to specify Redis version
Vsevolod Stakhov [Wed, 2 Apr 2025 15:30:57 +0000 (16:30 +0100)]
[Feature] Allow to specify Redis version
Some Redis versions introduce important changes to the commands, for
example `EVAL` vs `EVAL_RO`. Rspamd should be able to use these commands
where possible.
Issue: #5418
Vsevolod Stakhov [Tue, 1 Apr 2025 08:53:29 +0000 (14:53 +0600)]
Merge pull request #5417 from rspamd/vstakhov-bh-greylisting
[Fix] Greylist: Improve body hash calculations
Vsevolod Stakhov [Tue, 1 Apr 2025 08:53:19 +0000 (14:53 +0600)]
Merge pull request #5412 from rspamd/vstakhov-cmake-rework
[Project] Modernize cmake
Vsevolod Stakhov [Mon, 31 Mar 2025 17:46:30 +0000 (18:46 +0100)]
[Fix] Greylist: Improve body hash calculations
Vsevolod Stakhov [Mon, 31 Mar 2025 12:59:32 +0000 (13:59 +0100)]
[Minor] Improve sanitizers stuff style
Vsevolod Stakhov [Sat, 29 Mar 2025 10:59:14 +0000 (10:59 +0000)]
[Minor] Fix tests invocation
Vsevolod Stakhov [Sat, 29 Mar 2025 10:37:03 +0000 (10:37 +0000)]
[Project] Rework OSDep
Vsevolod Stakhov [Sat, 29 Mar 2025 10:17:13 +0000 (10:17 +0000)]
[Project] Modernize cmake
Dmitriy Alekseev [Fri, 28 Mar 2025 17:48:51 +0000 (18:48 +0100)]
Merge branch 'master' into patch-10
Vsevolod Stakhov [Fri, 28 Mar 2025 16:45:04 +0000 (22:45 +0600)]
Merge pull request #5406 from dragoangel/patch-11
Add R_HTTP_URL_IN_FROM
Vsevolod Stakhov [Fri, 28 Mar 2025 13:04:53 +0000 (13:04 +0000)]
[Rework] Use locks/loaded per backend for all maps
Dmitriy Alekseev [Thu, 27 Mar 2025 19:40:01 +0000 (20:40 +0100)]
Merge branch 'master' into patch-10
Dmitriy Alekseev [Thu, 27 Mar 2025 19:39:56 +0000 (20:39 +0100)]
Merge branch 'master' into patch-11
Vsevolod Stakhov [Thu, 27 Mar 2025 16:49:46 +0000 (22:49 +0600)]
Merge pull request #5408 from moisseev/maps
[WebUI] Update Maps table
Dmitriy Alekseev [Wed, 26 Mar 2025 17:49:05 +0000 (18:49 +0100)]
Update rules/regexp/headers.lua
Co-authored-by: Vsevolod Stakhov <vsevolod@rspamd.com>
Alexander Moisseev [Tue, 25 Mar 2025 16:42:18 +0000 (19:42 +0300)]
[WebUI] Grey out not loaded maps in the Maps table
Dmitriy Alekseev [Tue, 25 Mar 2025 12:37:37 +0000 (13:37 +0100)]
Update forwarding.lua
Dmitriy Alekseev [Tue, 25 Mar 2025 11:05:28 +0000 (12:05 +0100)]
Merge branch 'master' into patch-11
Alexander Moisseev [Tue, 25 Mar 2025 11:00:21 +0000 (14:00 +0300)]
[WebUI] Update Maps table
- Display new map flags.
- Rename "Lists" heading to "Maps".
- Add table header.
Dmitriy Alekseev [Tue, 25 Mar 2025 02:03:15 +0000 (03:03 +0100)]
Add Sieve and cPanel forwarding symbols
Dmitriy Alekseev [Mon, 24 Mar 2025 19:19:37 +0000 (20:19 +0100)]
Update headers.lua
Dmitriy Alekseev [Mon, 24 Mar 2025 16:53:40 +0000 (17:53 +0100)]
Update headers.lua
Vsevolod Stakhov [Mon, 24 Mar 2025 16:49:27 +0000 (22:49 +0600)]
Merge pull request #5386 from rspamd/vstakhov-maps-stuff
Improve maps status visibility
Dmitriy Alekseev [Mon, 24 Mar 2025 16:40:21 +0000 (17:40 +0100)]
Update headers.lua
Dmitriy Alekseev [Mon, 24 Mar 2025 16:33:04 +0000 (17:33 +0100)]
Update headers.lua
Vsevolod Stakhov [Mon, 24 Mar 2025 15:17:09 +0000 (15:17 +0000)]
[Minor] Set loaded flag when we load map from cache
Vsevolod Stakhov [Mon, 24 Mar 2025 15:02:37 +0000 (21:02 +0600)]
Merge pull request #5360 from fatalbanana/contextal
[Feature] Plugin to integrate with Contextal platform
Dmitriy Alekseev [Mon, 24 Mar 2025 14:42:31 +0000 (15:42 +0100)]
Add R_HTTP_URL_IN_FROM
Vsevolod Stakhov [Mon, 24 Mar 2025 13:42:34 +0000 (13:42 +0000)]
[Minor] Static maps have pre-defined content, so use it to get id
Vsevolod Stakhov [Mon, 24 Mar 2025 13:39:27 +0000 (13:39 +0000)]
[Minor] Hfilter: add maps with description
Vsevolod Stakhov [Mon, 24 Mar 2025 12:34:32 +0000 (12:34 +0000)]
[Fix] Fix maps ids
Andrew Lewis [Mon, 24 Mar 2025 11:53:15 +0000 (13:53 +0200)]
[Minor] Minor changes to contextal plugin
- Abandon prefilter
- Abandon postfilter in favour of wait if we have request_ttl
- Reformat table
Andrew Lewis [Tue, 4 Mar 2025 10:55:47 +0000 (12:55 +0200)]
[Minor] Add stock config
Andrew Lewis [Tue, 4 Mar 2025 10:52:45 +0000 (12:52 +0200)]
[Minor] Add config schema
Andrew Lewis [Mon, 3 Mar 2025 15:21:38 +0000 (17:21 +0200)]
[Minor] Support defering messages if results are unavailable
Andrew Lewis [Mon, 3 Mar 2025 12:28:09 +0000 (14:28 +0200)]
[Minor] Satisfy luacheck
Andrew Lewis [Mon, 3 Mar 2025 12:11:55 +0000 (14:11 +0200)]
[Minor] Use lua_cache
Andrew Lewis [Fri, 28 Feb 2025 09:05:29 +0000 (11:05 +0200)]
[Minor] Deal with unfinished work properly
Andrew Lewis [Fri, 28 Feb 2025 08:22:34 +0000 (10:22 +0200)]
[Minor] Forgot to use `SETEX`
Andrew Lewis [Fri, 28 Feb 2025 08:09:52 +0000 (10:09 +0200)]
[Minor] Fix luacheck complaints
Andrew Lewis [Thu, 27 Feb 2025 20:23:32 +0000 (22:23 +0200)]
[Feature] Plugin to integrate with Contextal platform
Vsevolod Stakhov [Mon, 24 Mar 2025 07:29:06 +0000 (07:29 +0000)]
[Fix] Fix static maps description passing
Vsevolod Stakhov [Sat, 22 Mar 2025 14:44:33 +0000 (20:44 +0600)]
Merge pull request #5400 from williamdes/patch-1
Fix typo on "destionation" -> "destination"
Vsevolod Stakhov [Sat, 22 Mar 2025 14:27:11 +0000 (14:27 +0000)]
[Feature] Output content for all maps
Vsevolod Stakhov [Fri, 21 Mar 2025 15:16:36 +0000 (21:16 +0600)]
Merge pull request #5383 from amulet1/lua_logger_improvements
[Minor] Lua logging improvements
Vsevolod Stakhov [Fri, 21 Mar 2025 14:57:49 +0000 (20:57 +0600)]
Merge pull request #5387 from alex081512/master
[Minor] gpt.lua improvements and one fix for reason_header
Vsevolod Stakhov [Fri, 21 Mar 2025 14:21:29 +0000 (20:21 +0600)]
Merge pull request #5388 from moisseev/webui
[WebUI] Fix map editor modal handling
William Desportes [Thu, 20 Mar 2025 18:45:36 +0000 (18:45 +0000)]
Fix typo on "destionation" -> "destination"
Vsevolod Stakhov [Mon, 17 Mar 2025 10:24:47 +0000 (16:24 +0600)]
Merge pull request #5390 from rspamd/vstakhov-cdb-maps
Allow CDB external maps
Dmitry Petrov [Sat, 15 Mar 2025 22:57:14 +0000 (18:57 -0400)]
Add support for complex table keys in logging output
Dmitry Petrov [Fri, 14 Mar 2025 17:18:30 +0000 (13:18 -0400)]
More lua logging improvements
* Do not skip numeric indexes that are non-integer or outside of ipairs range when logging table contents
* Simplify the code, drop redundant checks
* Allow negative stack indexes in lua_logger_out_*() functions
Dmitry Petrov [Tue, 11 Mar 2025 00:27:28 +0000 (20:27 -0400)]
[Minor] Lua logging improvements
* Simplify lua_logger_out_str(), fix length check when doing hex output
* Add lua_logger_out() with embedded lua_logger_trace struct
* Replace high level lua_logger_out_type() calls with lua_logger_out()
* Remove unused lua_logger_trace struct references
* Include string terminator into buffer length checks, adjust affected places accordingly
Vsevolod Stakhov [Sun, 16 Mar 2025 16:15:05 +0000 (22:15 +0600)]
Merge pull request #5389 from fatalbanana/openphish
[Fix] Update default URL for openphish
Vsevolod Stakhov [Sun, 16 Mar 2025 16:14:54 +0000 (22:14 +0600)]
Merge pull request #5391 from ertyu/patch-1
Update AUTHORS.md
ertyu [Sat, 15 Mar 2025 20:51:20 +0000 (15:51 -0500)]
Update AUTHORS.md
Vsevolod Stakhov [Sat, 15 Mar 2025 15:49:00 +0000 (15:49 +0000)]
[Test] Add tests for CDB maps
Vsevolod Stakhov [Sat, 15 Mar 2025 14:47:26 +0000 (14:47 +0000)]
[Feature] Allow CDB files as external maps
alex081512 [Fri, 14 Mar 2025 12:32:38 +0000 (13:32 +0100)]
[Minor] C&P issue in last commit
alex081512 [Fri, 14 Mar 2025 12:29:59 +0000 (13:29 +0100)]
[Minor] Reverted commit for result history
Andrew Lewis [Fri, 14 Mar 2025 11:26:57 +0000 (13:26 +0200)]
[Fix] Update default URL for openphish
Alexander Moisseev [Fri, 14 Mar 2025 10:21:21 +0000 (13:21 +0300)]
[WebUI] Fix map editor modal handling
- Prevent WebUI crash on `jar.destroy` when map query fails
- Do not open the map editor when map query fails
- Ensure proper cleanup of the editor modal
alex081512 [Thu, 13 Mar 2025 19:13:56 +0000 (20:13 +0100)]
[Minor] Remove whitespace for lua check
alex081512 [Thu, 13 Mar 2025 18:35:52 +0000 (19:35 +0100)]
[Minor] Insert Reason to result to show gpt answer in history
alex081512 [Thu, 13 Mar 2025 18:33:48 +0000 (19:33 +0100)]
[Fix] Add header with reason everytime (not only for ham) and use correct value for header
alex081512 [Thu, 13 Mar 2025 18:28:18 +0000 (19:28 +0100)]
[Minor] Improve gpt prompt - Only ask for red flag when it's spam
Vsevolod Stakhov [Thu, 13 Mar 2025 14:36:36 +0000 (14:36 +0000)]
[Feature] Show all maps status
Vsevolod Stakhov [Thu, 13 Mar 2025 14:03:02 +0000 (14:03 +0000)]
[Rework] Store shared maps data separately
Vsevolod Stakhov [Thu, 13 Mar 2025 09:58:56 +0000 (15:58 +0600)]
Merge pull request #5385 from JensErat/fix-ollama-nil
[Fix] initialize ollama result table
Jens Erat [Wed, 12 Mar 2025 23:45:36 +0000 (00:45 +0100)]
[Fix] initialize ollama result table
similar to gpt.lua:704 for the chatgpt code path, the ollama code path needs to initialize the result table. Without, rspamd fails gpt requests with
> lua_http_finish_handler: callback call failed: /usr/share/rspamd/plugins/gpt.lua:740: attempt to index a nil value
Vsevolod Stakhov [Mon, 10 Mar 2025 19:47:32 +0000 (01:47 +0600)]
Merge pull request #5378 from amulet1/lua_logger_log_format
[Minor] Fix and improve format string processing in lua_logger_log_format()
Vsevolod Stakhov [Mon, 10 Mar 2025 12:54:50 +0000 (18:54 +0600)]
Merge pull request #5380 from rspamd/vstakhov-fmt-embedding
[Fix] Use bundled libfmt everywhere
Vsevolod Stakhov [Mon, 10 Mar 2025 10:52:51 +0000 (10:52 +0000)]
[Fix] Use bundled libfmt everywhere
Issue: #5258
Dmitry Petrov [Sat, 8 Mar 2025 20:44:28 +0000 (15:44 -0500)]
[Minor] Fix and improve format string processing in lua_logger_log_format()
* Fix handling of %s after % with numeric argument
* Fix argument number check
* Preserve unhandled % in the output
* Redesign to simplify the code
Vsevolod Stakhov [Sat, 8 Mar 2025 14:13:16 +0000 (14:13 +0000)]
[Minor] Update version
Vsevolod Stakhov [Sat, 8 Mar 2025 14:11:50 +0000 (14:11 +0000)]
Release 3.11.1
* [Feature] Add 'noop' redis backend for scripts running
* [Feature] Add Redis caching framework
* [Feature] Add UTF8 `sub` and `len` variants for rspamd_text
* [Feature] Allow adding timers to task (respecting symbols)
* [Feature] Allow additional categories to be defined in GPT
* [Feature] Allow fine-grained control on keys permissions
* [Feature] Allow individual `read_only` flag per key
* [Feature] Allow multiple lua scripts for fuzzy storage
* [Feature] Allow to add periodic functions in Lua API
* [Feature] Allow to disable rbls from map
* [Feature] Allow to hash any Lua types
* [Feature] Allow to store shingles as opaque Lua data
* [Feature] Cache LLM replies
* [Feature] GPT: Add ollama support
* [Feature] GPT: Support reason adding
* [Feature] Improve prompt and use plaintext instead of JSON
* [Feature] Lua_task: Allow to load data into the existing task
* [Feature] More additions
* [Feature] Pass shingles to Lua scripts
* [Feature] Preliminary implementation of LLM based anonymizing
* [Feature] Support LLM models consensus
* [Feature] Try to check maps earlier if their expires is too long
* [Feature] Use debug module name from caller in lua_cache
* [Fix] Add timer update before timer setting
* [Fix] Allow to work with no ratelimits
* [Fix] Always create ratelimit buckets
* [Fix] Avoid collision hacks in mempool variables hash
* [Fix] Expire neural ham and spam sets so they not hangup in redis indefinitely
* [Fix] Fix crash on FreeBSD when Rspamd is built without hyperscan
* [Fix] Make table digests consistent
* [Fix] RBL: fix use of `content_urls` and `images` inside `checks`
* [Fix] Verify key type to match DKIM signature type
* [Fix] connIP is not correctly added to request
* [Fix] properly close multipart/related boundary when adding text footer
* [Rework] GPT: Use cache framework
Vsevolod Stakhov [Fri, 7 Mar 2025 17:07:18 +0000 (23:07 +0600)]
Merge pull request #5377 from rspamd/vstakhov-maps-caching
[Feature] Try to check maps earlier if their expires is too long
Vsevolod Stakhov [Fri, 7 Mar 2025 16:50:55 +0000 (22:50 +0600)]
Merge pull request #5376 from dragoangel/patch-10
[Fix] Properly expire neural ham and spam sets
Vsevolod Stakhov [Fri, 7 Mar 2025 16:49:58 +0000 (16:49 +0000)]
[Feature] Try to check maps earlier if their expires is too long
Issue: #5351
Dmitriy Alekseev [Fri, 7 Mar 2025 14:02:32 +0000 (15:02 +0100)]
Update neural_save_unlock.lua
Vsevolod Stakhov [Fri, 7 Mar 2025 12:52:24 +0000 (18:52 +0600)]
Merge pull request #5375 from moisseev/webui
[Test] Update JavaScript linters
Alexander Moisseev [Fri, 7 Mar 2025 08:18:42 +0000 (11:18 +0300)]
[Test] Update JavaScript linters
- @stylistic/eslint-plugin 2.8.0 -> 4.2.0
- eslint 9.10.0 -> 9.21.0
- postcss-html 1.7.0 -> 1.8.0
- stylelint-config-standard 36.0.1 -> 37.0.0
- stylelint 16.9.0 -> 16.15.0
Vsevolod Stakhov [Thu, 6 Mar 2025 14:09:36 +0000 (20:09 +0600)]
Merge pull request #5374 from rspamd/vstakhov-cache-gpt
Use caching framework in gpt module
Vsevolod Stakhov [Thu, 6 Mar 2025 14:00:31 +0000 (20:00 +0600)]
Merge pull request #5372 from amulet1/map_improvements
Better support for maps and IP-related fixes/improvements in settings
Vsevolod Stakhov [Thu, 6 Mar 2025 13:59:51 +0000 (13:59 +0000)]
[Minor] Fix invocation
Vsevolod Stakhov [Thu, 6 Mar 2025 12:39:22 +0000 (12:39 +0000)]
[Rework] GPT: Use cache framework