]> git.ipfire.org Git - thirdparty/rspamd.git/log
thirdparty/rspamd.git
4 weeks agoMerge branch 'master' into be_kind_to_comcast 5570/head
Vsevolod Stakhov [Fri, 15 Aug 2025 08:25:40 +0000 (14:25 +0600)] 
Merge branch 'master' into be_kind_to_comcast

4 weeks ago[Minor] Drop overzealous regex from hfilter
Andrew Lewis [Wed, 13 Aug 2025 10:16:07 +0000 (12:16 +0200)] 
[Minor] Drop overzealous regex from hfilter

4 weeks agoMerge pull request #5569 from moisseev/e2e-playwright
Vsevolod Stakhov [Tue, 12 Aug 2025 19:06:53 +0000 (01:06 +0600)] 
Merge pull request #5569 from moisseev/e2e-playwright

[Test] Add WebUI E2E workflow with Playwright

4 weeks ago[Test] Add WebUI E2E workflow with Playwright 5569/head
Alexander Moisseev [Tue, 12 Aug 2025 16:09:07 +0000 (19:09 +0300)] 
[Test] Add WebUI E2E workflow with Playwright

Add a GitHub Actions workflow to run WebUI E2E tests
with Playwright on legacy and latest browser versions
against rspamd binaries built in the pipeline.

4 weeks ago[Minor] Add specific calculations for binary classification case
Vsevolod Stakhov [Tue, 12 Aug 2025 15:38:37 +0000 (16:38 +0100)] 
[Minor] Add specific calculations for binary classification case

4 weeks agoMerge pull request #5566 from fatalbanana/el10rpm
Vsevolod Stakhov [Tue, 12 Aug 2025 13:09:25 +0000 (19:09 +0600)] 
Merge pull request #5566 from fatalbanana/el10rpm

[Minor] Build on EL10

5 weeks ago[Minor] Use clang for build on EL10 5566/head
Andrew Lewis [Tue, 12 Aug 2025 11:24:49 +0000 (13:24 +0200)] 
[Minor] Use clang for build on EL10

5 weeks ago[Minor] Use embedded vectorscan on EL10
Andrew Lewis [Mon, 11 Aug 2025 12:53:41 +0000 (14:53 +0200)] 
[Minor] Use embedded vectorscan on EL10

5 weeks ago[Minor] Fix implicit declaration
Andrew Lewis [Mon, 11 Aug 2025 12:53:25 +0000 (14:53 +0200)] 
[Minor] Fix implicit declaration

5 weeks ago[Fix] Try to fix learned order
Vsevolod Stakhov [Sat, 9 Aug 2025 09:42:41 +0000 (10:42 +0100)] 
[Fix] Try to fix learned order

5 weeks agoMerge pull request #5562 from rspamd/vstakhov-proxy-compression
Vsevolod Stakhov [Thu, 7 Aug 2025 11:06:22 +0000 (17:06 +0600)] 
Merge pull request #5562 from rspamd/vstakhov-proxy-compression

[Fix] Fix end-to-end proxy compression

5 weeks ago[Fix] Fix double free in the client... 5562/head
Vsevolod Stakhov [Wed, 6 Aug 2025 16:06:48 +0000 (17:06 +0100)] 
[Fix] Fix double free in the client...

5 weeks ago[Minor] Fix 'Compression' header logic
Vsevolod Stakhov [Wed, 6 Aug 2025 15:58:58 +0000 (16:58 +0100)] 
[Minor] Fix 'Compression' header logic

5 weeks ago[Minor] Some more logic fixes
Vsevolod Stakhov [Tue, 5 Aug 2025 13:10:11 +0000 (14:10 +0100)] 
[Minor] Some more logic fixes

6 weeks ago[Minor] More cleanups for compression stuff
Vsevolod Stakhov [Mon, 4 Aug 2025 19:46:33 +0000 (20:46 +0100)] 
[Minor] More cleanups for compression stuff

6 weeks ago[Fix] Fix end-to-end proxy compression
Vsevolod Stakhov [Mon, 4 Aug 2025 14:40:24 +0000 (15:40 +0100)] 
[Fix] Fix end-to-end proxy compression

Issue: #5561

6 weeks agoMerge pull request #5547 from rspamd/vstakhov-multi-class-bayes
Vsevolod Stakhov [Mon, 4 Aug 2025 11:58:44 +0000 (17:58 +0600)] 
Merge pull request #5547 from rspamd/vstakhov-multi-class-bayes

[Project] Multi-class classification

6 weeks agoMerge pull request #5559 from rspamd/vstakhov-arc-fixes
Vsevolod Stakhov [Fri, 1 Aug 2025 15:50:33 +0000 (21:50 +0600)] 
Merge pull request #5559 from rspamd/vstakhov-arc-fixes

[Fix] Fix whitelist options in the arc module

6 weeks agoMerge pull request #5556 from rspamd/vstakhov-skip-hashes-fuzzy
Vsevolod Stakhov [Fri, 1 Aug 2025 15:49:34 +0000 (21:49 +0600)] 
Merge pull request #5556 from rspamd/vstakhov-skip-hashes-fuzzy

[Fix] Check skip_hashes for the returned hashes

6 weeks ago[Fix] Fix whitelist options in the arc module 5559/head
Vsevolod Stakhov [Fri, 1 Aug 2025 11:41:57 +0000 (12:41 +0100)] 
[Fix] Fix whitelist options in the arc module

Issue: #5558

6 weeks ago[Fix] Check skip_hashes for the returned hashes 5556/head
Vsevolod Stakhov [Thu, 31 Jul 2025 15:51:46 +0000 (16:51 +0100)] 
[Fix] Check skip_hashes for the returned hashes

6 weeks agoMerge pull request #5555 from heptalium/meissner-fix-dmarc-reports
Vsevolod Stakhov [Thu, 31 Jul 2025 11:03:41 +0000 (17:03 +0600)] 
Merge pull request #5555 from heptalium/meissner-fix-dmarc-reports

Use Redis write servers for write commands while generating DMARC reports

6 weeks agoUse Redis write servers for write commands while generating DMARC reports. 5555/head
Jens Meißner [Wed, 30 Jul 2025 13:53:33 +0000 (15:53 +0200)] 
Use Redis write servers for write commands while generating DMARC reports.

6 weeks ago[Fix] Use a more straightforward approach for learn cache 5547/head
Vsevolod Stakhov [Wed, 30 Jul 2025 09:01:10 +0000 (10:01 +0100)] 
[Fix] Use a more straightforward approach for learn cache

6 weeks ago[Fix] Fix various corner cases and tests
Vsevolod Stakhov [Tue, 29 Jul 2025 19:01:36 +0000 (20:01 +0100)] 
[Fix] Fix various corner cases and tests

6 weeks ago[Test] Add logic to match test id and logs id
Vsevolod Stakhov [Tue, 29 Jul 2025 13:17:00 +0000 (14:17 +0100)] 
[Test] Add logic to match test id and logs id

7 weeks ago[Minor] Add --log-tag option for rspamc
Vsevolod Stakhov [Tue, 29 Jul 2025 09:06:35 +0000 (10:06 +0100)] 
[Minor] Add --log-tag option for rspamc

7 weeks ago[Minor] Fix single class fallback
Vsevolod Stakhov [Tue, 29 Jul 2025 08:16:42 +0000 (09:16 +0100)] 
[Minor] Fix single class fallback

7 weeks ago[Project] Apply changes to bayes_expiry plugin
Vsevolod Stakhov [Mon, 28 Jul 2025 18:22:56 +0000 (19:22 +0100)] 
[Project] Apply changes to bayes_expiry plugin

7 weeks ago[Test] Some more adjustments to the tests
Vsevolod Stakhov [Mon, 28 Jul 2025 18:07:35 +0000 (19:07 +0100)] 
[Test] Some more adjustments to the tests

7 weeks ago[Project] Fix unlearn stuff
Vsevolod Stakhov [Mon, 28 Jul 2025 14:06:06 +0000 (15:06 +0100)] 
[Project] Fix unlearn stuff

7 weeks ago[Project] Fix more calculation issues
Vsevolod Stakhov [Mon, 28 Jul 2025 09:41:33 +0000 (10:41 +0100)] 
[Project] Fix more calculation issues

7 weeks ago[Feature] Add some convenience options to rspamc
Vsevolod Stakhov [Mon, 28 Jul 2025 08:53:43 +0000 (09:53 +0100)] 
[Feature] Add some convenience options to rspamc

7 weeks ago[Minor] Further adjustments
Vsevolod Stakhov [Sun, 27 Jul 2025 15:04:29 +0000 (16:04 +0100)] 
[Minor] Further adjustments

7 weeks ago[Fix] Fix statfiles ordering
Vsevolod Stakhov [Sat, 26 Jul 2025 11:28:07 +0000 (12:28 +0100)] 
[Fix] Fix statfiles ordering

7 weeks ago[Fix] Fix probabilities overflow
Vsevolod Stakhov [Sat, 26 Jul 2025 11:27:50 +0000 (12:27 +0100)] 
[Fix] Fix probabilities overflow

7 weeks ago[Minor] Fix stupid change to call Redis for each class
Vsevolod Stakhov [Fri, 25 Jul 2025 14:40:06 +0000 (15:40 +0100)] 
[Minor] Fix stupid change to call Redis for each class

7 weeks ago[Project] Fix various issues
Vsevolod Stakhov [Fri, 25 Jul 2025 10:58:36 +0000 (11:58 +0100)] 
[Project] Fix various issues

7 weeks ago[Test] Add multiclass tests
Vsevolod Stakhov [Fri, 25 Jul 2025 07:48:48 +0000 (08:48 +0100)] 
[Test] Add multiclass tests

7 weeks ago[Minor] Reduce debug level verbosity
Vsevolod Stakhov [Wed, 23 Jul 2025 17:28:46 +0000 (18:28 +0100)] 
[Minor] Reduce debug level verbosity

7 weeks ago[Doc] Add architecture document
Vsevolod Stakhov [Wed, 23 Jul 2025 17:06:34 +0000 (18:06 +0100)] 
[Doc] Add architecture document

7 weeks agoMerge pull request #5550 from moisseev/webui
Vsevolod Stakhov [Wed, 23 Jul 2025 16:38:10 +0000 (22:38 +0600)] 
Merge pull request #5550 from moisseev/webui

[WebUI] Add history load range controls

7 weeks ago[Minor] Use modern bayes configuration
Vsevolod Stakhov [Wed, 23 Jul 2025 14:17:25 +0000 (15:17 +0100)] 
[Minor] Use modern bayes configuration

7 weeks ago[Project] Fix binary classification and lua scripts
Vsevolod Stakhov [Wed, 23 Jul 2025 13:33:04 +0000 (14:33 +0100)] 
[Project] Fix binary classification and lua scripts

7 weeks ago[Project] Further updates
Vsevolod Stakhov [Tue, 22 Jul 2025 21:30:03 +0000 (22:30 +0100)] 
[Project] Further updates

7 weeks ago[WebUI] Add history load range controls 5550/head
Alexander Moisseev [Tue, 22 Jul 2025 16:25:05 +0000 (19:25 +0300)] 
[WebUI] Add history load range controls

- Add 'Offset' and 'Count' inputs to History tab for selecting
  the range of history rows to load
- Add corresponding default setting in Settings tab

8 weeks ago[Project] Fix other classification and learning issues
Vsevolod Stakhov [Mon, 21 Jul 2025 09:55:59 +0000 (10:55 +0100)] 
[Project] Fix other classification and learning issues

8 weeks ago[Minor] Fix various issues
Vsevolod Stakhov [Mon, 21 Jul 2025 09:07:27 +0000 (10:07 +0100)] 
[Minor] Fix various issues

8 weeks ago[Project] Multi-class classification project baseline
Vsevolod Stakhov [Sun, 20 Jul 2025 16:11:52 +0000 (17:11 +0100)] 
[Project] Multi-class classification project baseline

2 months agoMerge pull request #5545 from rspamd/vstakhov-html-tags-overload
Vsevolod Stakhov [Thu, 17 Jul 2025 11:09:06 +0000 (17:09 +0600)] 
Merge pull request #5545 from rspamd/vstakhov-html-tags-overload

[Project] Rework system of html tags to allow more tag types

2 months ago[Test] Write unit tests for new functions 5545/head
Vsevolod Stakhov [Thu, 17 Jul 2025 10:49:28 +0000 (11:49 +0100)] 
[Test] Write unit tests for new functions

2 months ago[Fix] Store html attributes that are empty
Vsevolod Stakhov [Thu, 17 Jul 2025 10:46:08 +0000 (11:46 +0100)] 
[Fix] Store html attributes that are empty

If we see something like <div class=""> we were not storing attribute at all!

2 months ago[Minor] Add method to parse html into html context
Vsevolod Stakhov [Thu, 17 Jul 2025 08:53:04 +0000 (09:53 +0100)] 
[Minor] Add method to parse html into html context

2 months ago[Project] Fix Lua API and some constexpr compatibility
Vsevolod Stakhov [Thu, 17 Jul 2025 08:39:54 +0000 (09:39 +0100)] 
[Project] Fix Lua API and some constexpr compatibility

2 months ago[Project] Support more common html attributes
Vsevolod Stakhov [Thu, 17 Jul 2025 08:16:50 +0000 (09:16 +0100)] 
[Project] Support more common html attributes

2 months agoMerge pull request #5532 from rheoli/icap_ipv6_fix
Vsevolod Stakhov [Wed, 16 Jul 2025 13:57:33 +0000 (19:57 +0600)] 
Merge pull request #5532 from rheoli/icap_ipv6_fix

icap.lua: ipv6 pseudo hostname in GET request

2 months agoMerge pull request #5544 from moisseev/webui
Vsevolod Stakhov [Wed, 16 Jul 2025 13:47:35 +0000 (19:47 +0600)] 
Merge pull request #5544 from moisseev/webui

[Minor] Update Bootstrap 5.3.5 → 5.3.7

2 months ago[Project] Rework system of html tags to allow more tag types
Vsevolod Stakhov [Wed, 16 Jul 2025 11:31:26 +0000 (12:31 +0100)] 
[Project] Rework system of html tags to allow more tag types

2 months ago[Minor] Update Bootstrap 5.3.5 → 5.3.7 5544/head
Alexander Moisseev [Tue, 15 Jul 2025 10:58:46 +0000 (13:58 +0300)] 
[Minor] Update Bootstrap 5.3.5 → 5.3.7

2 months agoMerge pull request #5543 from rspamd/vstakhov-nary-ops-opts
Vsevolod Stakhov [Tue, 15 Jul 2025 10:40:42 +0000 (16:40 +0600)] 
Merge pull request #5543 from rspamd/vstakhov-nary-ops-opts

[Rework] Return back N-ary optimizations for arithmetic-alike expressions

2 months ago[Rules] Make bitcoin expression to use explicit flags 5543/head
Vsevolod Stakhov [Tue, 15 Jul 2025 10:10:09 +0000 (11:10 +0100)] 
[Rules] Make bitcoin expression to use explicit flags

2 months ago[Feature] Allow to pass expression flags in the regexp plugin
Vsevolod Stakhov [Tue, 15 Jul 2025 10:09:45 +0000 (11:09 +0100)] 
[Feature] Allow to pass expression flags in the regexp plugin

2 months ago[Rework] Return back N-ary optimizations for arithmetic-alike expressions
Vsevolod Stakhov [Mon, 14 Jul 2025 20:22:06 +0000 (21:22 +0100)] 
[Rework] Return back N-ary optimizations for arithmetic-alike expressions

2 months agoMerge pull request #5530 from rspamd/vstakhov-more-maps-features
Vsevolod Stakhov [Mon, 14 Jul 2025 15:05:51 +0000 (21:05 +0600)] 
Merge pull request #5530 from rspamd/vstakhov-more-maps-features

Create rule based maps

2 months ago[Test] Add functional tests for regexp maps 5530/head
Vsevolod Stakhov [Mon, 14 Jul 2025 13:50:36 +0000 (14:50 +0100)] 
[Test] Add functional tests for regexp maps

2 months ago[Project] Fix symbols finalisation
Vsevolod Stakhov [Mon, 14 Jul 2025 09:46:32 +0000 (10:46 +0100)] 
[Project] Fix symbols finalisation

2 months ago[Minor] Add some more debug logs
Vsevolod Stakhov [Sun, 13 Jul 2025 21:03:50 +0000 (22:03 +0100)] 
[Minor] Add some more debug logs

2 months ago[Minor] Improve logging
Vsevolod Stakhov [Sun, 13 Jul 2025 14:56:58 +0000 (15:56 +0100)] 
[Minor] Improve logging

2 months ago[Fix] Fix DL lists initialisations
Vsevolod Stakhov [Sun, 13 Jul 2025 14:28:09 +0000 (15:28 +0100)] 
[Fix] Fix DL lists initialisations

2 months agoMerge pull request #5540 from 1358/clean-gpt-response-for-openai
Vsevolod Stakhov [Sat, 12 Jul 2025 13:21:43 +0000 (19:21 +0600)] 
Merge pull request #5540 from 1358/clean-gpt-response-for-openai

Cleanup GPT response for openai-provider, too (<think>)

2 months ago[Project] Store regexp rules state to avoid incomplete/orphaned rules
Vsevolod Stakhov [Sat, 12 Jul 2025 13:21:00 +0000 (14:21 +0100)] 
[Project] Store regexp rules state to avoid incomplete/orphaned rules

2 months agoclean gpt response for openai, too 5540/head
Tom Mittelstädt [Thu, 10 Jul 2025 08:31:18 +0000 (10:31 +0200)] 
clean gpt response for openai, too

2 months ago[Minor] Use more efficient approach for iterations
Vsevolod Stakhov [Mon, 7 Jul 2025 14:45:43 +0000 (15:45 +0100)] 
[Minor] Use more efficient approach for iterations

2 months ago[Project] Fix scoped compilation again
Vsevolod Stakhov [Mon, 7 Jul 2025 11:10:07 +0000 (12:10 +0100)] 
[Project] Fix scoped compilation again

2 months ago[Minor] Another rework of the notifications
Vsevolod Stakhov [Mon, 7 Jul 2025 10:03:44 +0000 (11:03 +0100)] 
[Minor] Another rework of the notifications

2 months ago[Fix] Fix order of descriptor closing
Vsevolod Stakhov [Mon, 7 Jul 2025 10:02:13 +0000 (11:02 +0100)] 
[Fix] Fix order of descriptor closing

2 months agoMerge pull request #5534 from moisseev/webui
Vsevolod Stakhov [Sat, 5 Jul 2025 07:16:29 +0000 (13:16 +0600)] 
Merge pull request #5534 from moisseev/webui

[WebUI] Update classifiers on server or tab switch

2 months agoMerge pull request #5537 from moisseev/test
Vsevolod Stakhov [Sat, 5 Jul 2025 07:16:12 +0000 (13:16 +0600)] 
Merge pull request #5537 from moisseev/test

[Test] Update ESLint and Stylelint linters

2 months ago[Test] Update ESLint and Stylelint linters 5537/head
Alexander Moisseev [Wed, 2 Jul 2025 10:46:29 +0000 (13:46 +0300)] 
[Test] Update ESLint and Stylelint linters

2 months ago[Minor] Replace legacy rgba() alias
Alexander Moisseev [Wed, 2 Jul 2025 10:42:10 +0000 (13:42 +0300)] 
[Minor] Replace legacy rgba() alias

with modern rgb() notation

2 months ago[Feature] Add a signal from main to workers for workers ready state
Vsevolod Stakhov [Tue, 1 Jul 2025 20:27:23 +0000 (21:27 +0100)] 
[Feature] Add a signal from main to workers for workers ready state

2 months ago[WebUI] Update classifiers on server or tab switch 5534/head
Alexander Moisseev [Mon, 30 Jun 2025 14:05:49 +0000 (17:05 +0300)] 
[WebUI] Update classifiers on server or tab switch

Ensure the Bayes classifiers dropdown is refreshed whenever the user
changes the selected server or navigates to the Scan/Learn tab, so it
always reflects the selected server’s configuration.

2 months agoMerge pull request #5524 from moisseev/fuzzy-config
Vsevolod Stakhov [Mon, 30 Jun 2025 11:44:55 +0000 (17:44 +0600)] 
Merge pull request #5524 from moisseev/fuzzy-config

[WebUI] Add fuzzy flag selectors to Scan/Learn tab

2 months ago[Project] Implement scoped compilation
Vsevolod Stakhov [Sat, 28 Jun 2025 12:32:17 +0000 (13:32 +0100)] 
[Project] Implement scoped compilation

2 months ago[Project] Use re_cache scopes for maps
Vsevolod Stakhov [Fri, 27 Jun 2025 16:38:28 +0000 (17:38 +0100)] 
[Project] Use re_cache scopes for maps

2 months ago[Project] Implement scoped regexp cache system
Vsevolod Stakhov [Fri, 27 Jun 2025 16:16:00 +0000 (17:16 +0100)] 
[Project] Implement scoped regexp cache system

2 months agoMerge pull request #5533 from moisseev/sys-user-h
Vsevolod Stakhov [Fri, 27 Jun 2025 08:38:03 +0000 (14:38 +0600)] 
Merge pull request #5533 from moisseev/sys-user-h

[Minor] Fix FreeBSD build by including sys/user.h

2 months ago[Minor] Fix FreeBSD build by including sys/user.h 5533/head
Alexander Moisseev [Fri, 27 Jun 2025 06:46:42 +0000 (09:46 +0300)] 
[Minor] Fix FreeBSD build by including sys/user.h

2 months agoicap.lua: ipv6 pseudo hostname in GET request 5532/head
Stephan Toggweiler [Thu, 26 Jun 2025 13:07:12 +0000 (15:07 +0200)] 
icap.lua: ipv6 pseudo hostname in GET request

ipv6 addresses need to be changed to a pseudo hostname in GET request who is encapsulated in ICAP.
A change to [::] did not work in SkyHigh Web Gateway (aka McAfee)

Didn't use Lua often maybe the code can be optimized.

2 months ago[Project] Start implementation of the rules maps
Vsevolod Stakhov [Thu, 26 Jun 2025 08:01:01 +0000 (09:01 +0100)] 
[Project] Start implementation of the rules maps

2 months ago[WebUI] Add fuzzy flag selectors to Scan/Learn tab 5524/head
Alexander Moisseev [Wed, 25 Jun 2025 14:20:19 +0000 (17:20 +0300)] 
[WebUI] Add fuzzy flag selectors to Scan/Learn tab

- Populate selectors with symbols and flags from writable fuzzy storages.
- Cache each server’s config_id to avoid redundant `/plugins/fuzzy/storages` calls.
- Dynamically show, hide, and disable controls based on fuzzy_check module availability and storage writability.

2 months ago[Feature] Allow lua callback maps to be filled line by line
Vsevolod Stakhov [Wed, 25 Jun 2025 14:03:25 +0000 (15:03 +0100)] 
[Feature] Allow lua callback maps to be filled line by line

2 months ago[Minor] Improve JSDoc for ui.query()
Alexander Moisseev [Mon, 23 Jun 2025 15:14:22 +0000 (18:14 +0300)] 
[Minor] Improve JSDoc for ui.query()

2 months ago[Minor] Add /plugins/fuzzy/storages endpoint
Alexander Moisseev [Sat, 21 Jun 2025 14:12:03 +0000 (17:12 +0300)] 
[Minor] Add /plugins/fuzzy/storages endpoint

Allows retrieving the list of configured fuzzy storages via HTTP API,
including associated flags, read_only status, and server addresses.

2 months ago[Fix] GPT: fix processing of messages with no subject
Vsevolod Stakhov [Wed, 25 Jun 2025 08:39:44 +0000 (09:39 +0100)] 
[Fix] GPT: fix processing of messages with no subject

2 months agoMerge pull request #5520 from moisseev/webui
Vsevolod Stakhov [Mon, 23 Jun 2025 07:56:04 +0000 (13:56 +0600)] 
Merge pull request #5520 from moisseev/webui

[WebUI] Add Bayes classifier selector to Learn card

2 months agoMerge pull request #5523 from rspamd/vstakhov-more-utils
Vsevolod Stakhov [Mon, 23 Jun 2025 07:55:53 +0000 (13:55 +0600)] 
Merge pull request #5523 from rspamd/vstakhov-more-utils

[Feature] Add some more OS utility functions

2 months agoMerge pull request #5510 from rspamd/vstakhov-flexible-tokenizer
Vsevolod Stakhov [Mon, 23 Jun 2025 07:55:40 +0000 (13:55 +0600)] 
Merge pull request #5510 from rspamd/vstakhov-flexible-tokenizer

[Project] Add ability to create custom tokenizers for languages

2 months ago[Minor] Make fundamental structures less GLib polluted 5510/head
Vsevolod Stakhov [Sun, 22 Jun 2025 21:34:20 +0000 (22:34 +0100)] 
[Minor] Make fundamental structures less GLib polluted

2 months ago[Project] Rework tokenizers initialisation
Vsevolod Stakhov [Sun, 22 Jun 2025 21:00:16 +0000 (22:00 +0100)] 
[Project] Rework tokenizers initialisation