Jason Ish [Tue, 1 Apr 2025 15:31:07 +0000 (09:31 -0600)]
threads/lib: fix coverity check for unchecked return code
In thread startup, return error of TmThreadsWaitForUnpause() fails.
Fixed coverity check as in all other places the return value of this
function is checked and acted on.
Jeff Lucovsky [Tue, 1 Apr 2025 12:14:31 +0000 (08:14 -0400)]
doc/entropy: Add documentation for the entropy keyword
This commits adds documentation for the entropy keyword.
The entropy keyword calculates the Shannon entropy value for content
with the calculated value used to determine whether an alert occurs.
Jeff Lucovsky [Wed, 5 Feb 2025 20:53:30 +0000 (15:53 -0500)]
detect/entropy: Add entropy keyword
This commit adds keyword/build support for the entropy keyword. The
entropy keyword compares an entropy value with a value calculated
according to the Shannon entropy on the available content.
Jeff Lucovsky [Wed, 5 Feb 2025 20:46:10 +0000 (15:46 -0500)]
rust/detect: Add entropy support
This commit adds
- Parser for the entropy keyword
- Calculation of content the Shannon entropy value
Issue: 4162
The entropy keyword syntax is the keyword entropy followed by options
and the entropy value for comparison.
The minimum entropy keyword specification is:
entropy: value <entropy-spec>
This results in the calculated entropy value being compared with
<entropy-spec> with the equality operator.
Calculated entropy values are between 0.0 and 8.0, inclusive.
A match occurs when the values and operator agree. This example matches
if the calculated and entropy value are the same.
When entropy keyword options are specified, all options and "value" must
be comma-separated. Options and value may be specified in any order.
Options have default values:
- bytes is equal to the current content length
- offset is 0
- comparison with value is equality
entropy: [bytes <byteval>] [offset <offsetval>] value <entropy-spec>
Using default values:
entropy: bytes 0, offset 0, value =<entropy-spec>
<entropy-spec> is: <operator> (see below) and a value, e.g., "< 4.1"
The following operators are available from the float crate:
- = (default): Match when calculated entropy value equals specified entropy value
- < Match when calculated entropy value is strictly less than specified entropy value
- <= Match when calculated entropy value is less than or equal to specified entropy value
- > Match when calculated entropy value is strictly greater than specified entropy value
- >= Match when calculated entropy value is greater than or equal to specified entropy value
- != Match when calculated entropy value is not equal to specified entropy value
- x-y Match when calculated entropy value is in the range, exclusive
- !x-y Match when calculated entropy value is not in the range, exclusive
Shivani Bhardwaj [Wed, 26 Mar 2025 06:04:11 +0000 (11:34 +0530)]
util/lua: change arg name to reflect correctness
lua fns do not care about the argument count, they work with the index of
the item in the stack. Before library, there was just one item on the stack
so it worked out, however, with the library, the first item in the stack is
the library userdata, so, the fn would fail with the existing hard coded
setting of 1 for argc which can easily be confused with the number of
arguments passed to the fn.
Jason Ish [Mon, 26 Aug 2024 20:18:22 +0000 (14:18 -0600)]
runmodes: typedef runmode enum and use as type
Also remove function to set the library mode. This is easy enough to
do with SCRunmodeSet, and we don't want to add a specific setter for
each and every runmode.
Jason Ish [Thu, 18 Apr 2024 22:53:47 +0000 (16:53 -0600)]
examples/lib: better command line handling
Use the more conventional "--" command line handling to separate the
arguments. The first set will be passed to Suricata, and the args
after "--" will be handled by the example. Currently this is a single
PCAP filename, but will be extended to a list of PCAP filenames.
Jason Ish [Thu, 18 Apr 2024 06:33:09 +0000 (00:33 -0600)]
threads: don't attempt to join threads with an id of 0
Worker threads not created by Suricata, but instead a library user
should not be joined, as Suricata does not have access to their thread
handle, and it may in-fact be an unjoinable thread, such as the main
process.
When the thread ID is 0, assume the thread is "externally" managed,
but still mark is as dead to satisfy Suricata's view of the thread.
Add library source and runmode modules. Reorganized
library example to create a worker thread and replay a pcap
file using the library mode.
No API layer is added at this stage.
Edits by Jason Ish:
- fix guard
- add copyright/license headers
Lukas Sismis [Sat, 29 Mar 2025 12:31:15 +0000 (13:31 +0100)]
dpdk: allow zero TX queues when running in IDS mode
When running in non-forwarding (IDS) mode, it is not required
to create TX queues for the interface.
This can be acheived by setting tx-descriptors configuration
field to 0.
Lukas Sismis [Tue, 4 Feb 2025 10:13:42 +0000 (11:13 +0100)]
dpdk: check for link up before full startup
ICE card (Intel E810) was not receiving packets immediatelly
after startup, Suricata workers would act as processing while
it was not. This eliminates the problem by only continuing
in the initialization if the link is already up.
The setting can be turned off manually from the configuraiton
file.
It turned out that having global (interface-specific) mempool
that is shared by the threads of the interface is slower than
having individual mempools per queue for each interface.
The commit brings this change and should be user-invisible,
the config setting remains still as a number of objects of
all mempools summed (of that interface).
Victor Julien [Mon, 24 Mar 2025 10:20:39 +0000 (11:20 +0100)]
detect: per tx detect flags to a u8 progress value
Reduce per tx space for tracking detection/prefilter progress. Instead
of a per direction u64 of flags, where each bit reflected a progress
value, use a simple u8 to track the linear progression through the
progress values. Use an offset to allow 0 to mean no value.
Add flags field as well to track "skip detect" and "inspect complete".
Lukas Sismis [Mon, 28 Oct 2024 14:46:17 +0000 (15:46 +0100)]
hyperscan: add caching mechanism for hyperscan contexts
Cache Hyperscan serialized databases to disk to prevent compilation
of the same databases when Suricata is run again with the same
ruleset.
Hyperscan binary files are stored per rulegroup in the designated
folder, by default in the cached library folder.
Since caching is per signature group heads,
some chunk of the ruleset can change and it still can reuse part of
the unchanged signature groups.
Loading *fresh* ET Open ruleset: 19 seconds
Loading *cached* ET Open ruleset: 07 seconds
hashlittle: add a safe variant of hashlittle2 function
This variant of hashlittle2() ensures that it avoids
accesses beyond the last byte of the string, which will
cause warnings from tools like Valgrind or Address
Sanitizer.
Jeff Lucovsky [Fri, 28 Feb 2025 16:02:29 +0000 (11:02 -0500)]
output/buffer: Change buffer-size default value
The buffer-size value that controls file output buffering defaults to
8k. To be consistent with previous logic, the default is being changed
to 0 (e.g., needed if there are old config files that don't specifically
enable the new value).
We do not want to see hard-coded protocol constants such as
ALPROTO_QUIC directly used in generic code in detect-parse.c
How ?
From the keyword point of view, this commit adds the function
DetectSignatureSetMultiAppProto which is similar to
DetectSignatureSetAppProto but takes multiple alprotos.
It restricts the signature alprotos to a set of possible alprotos
and errors out if the interstion gets empty.
The data structure SignatureInitData gets extended with
a fixed-length array, as the use case is a sparse number of protocols