Shivani Bhardwaj [Mon, 25 Mar 2024 13:38:31 +0000 (19:08 +0530)]
detect/port: handle range and upper boundary ports
So far, if a port was found to be single which was earlier a part of the
range, port + 1 was added to the list to honor the range that it was a
part of. But, this is incorrect in case the port is 65535 or if the port
was found to be of range when it was earlier a single port.
Jason Ish [Mon, 4 Mar 2024 23:08:08 +0000 (17:08 -0600)]
suricata: expose and break out configuration loading
Expose LoadYamlConfig as SCLoadYamlConfig and remove it from
SuricataInit. This is required to allow the library user the ability
customize the loading of the configuration, for example doing some
programmatic configuration then loading a configuration file.
Jason Ish [Mon, 26 Feb 2024 22:45:07 +0000 (16:45 -0600)]
suricata: move SuricataMain into main()
Move the contents of SuricataMain into the `main()` function found in
main.c. This forces the Suricata application to bootstrap and run
Suricata through the same interfaces as a library user might do.
Required exposing StartInternalRunMode as SCStartInternalRunmode. Its
arguable whether those "actions" belong in the library or just the
application, but I think that is separation we can look at later.
For now the lib example and Suricata's own main are the same, however
the example will probably extend more into programmatically
configuring Suricata or dynamically registering a runmode, which
doesn't really belong the main Suricata application.
Jason Ish [Mon, 26 Feb 2024 22:29:25 +0000 (16:29 -0600)]
suricata: expose FinalizeRunMode and ParseCommandLine
Expose the functions FinalizeRunMode and ParseCommandLine to library
users, renaming with the `SC` prefix in the process.
This involves moving "application" level details from SuricataInit
into SuricataMain, as parsing command line options should be opt-in
for a library user.
Jason Ish [Mon, 26 Feb 2024 21:55:04 +0000 (15:55 -0600)]
suricata: remove instance from ParseCommandLine
We want to be able to call ParseCommandLine from library users, but
currently library users don't have access to the `suricata` instance
type. Since this var is used other places as a global, use the global
one in ParseCommandLine as well.
Not ideal, but isolating SCInstance to a non-global will be another
challenge on its own.
Jason Ish [Mon, 26 Feb 2024 21:39:56 +0000 (15:39 -0600)]
suricata: move WindowsInitService to SuricataMain
Move WindowsInitService from SuricataInit() to SuricataMain(), as
initializing Suricata as a service is very specific to the application
and not something you'd want to happen in a library, and SuricataInit
is more common initialization for application and library usage.
Jason Ish [Thu, 22 Feb 2024 20:25:33 +0000 (14:25 -0600)]
suricata: expose SuricataMainLoop and GlobalsDestroy
Expose SuricataMainLoop and GlobalsDestroy so that SuricataMain can be
replicated by a library user of Suricata.
These removes the `suricata` instance as a function argument to some
of these functions, as the way we use it now, it serves no
purpose. However, it is a reminder that it should probably be
refactored to not be a global, as at some point it might be desirable
for to have multiple instances active without data sharing.
Philippe Antoine [Thu, 21 Mar 2024 15:15:53 +0000 (16:15 +0100)]
rust/smb: fix clippy nightly warning
error: unnecessary use of `to_vec`
--> src/smb/smb.rs:1048:62
|
1048 | let (name, is_dcerpc) = match self.guid2name_map.get(&guid.to_vec()) {
| ^^^^^^^^^^^^^^ help: replace it with: `guid`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_to_owned
= note: `#[deny(clippy::unnecessary_to_owned)]` implied by `#[deny(warnings)]`
Philippe Antoine [Thu, 21 Mar 2024 15:02:23 +0000 (16:02 +0100)]
rust: fix clippy 1.77 warning
Ticket: 6883
error: field `0` is never read
--> src/asn1/mod.rs:36:14
|
36 | BerError(Err<der_parser::error::BerError>),
| -------- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| |
| field in this variant
|
Jeff Lucovsky [Sat, 16 Mar 2024 12:58:11 +0000 (08:58 -0400)]
profiling/rules: Improve dynamic rule handling
Issue: 6861
Without this commit, disabling rule profiling via suricatasc's command
'ruleset-profile-stop' may crash because profiling_rules_entered becomes
negative.
This can happen because
- There can be multiple rules evaluated for a single packet
- Each rule is profiled individually.
- Starting profiling is gated by a configuration setting and rule
profiling being active
- Ending profiling is gated by the same configuration setting and
whether the packet was marked as profiling.
The crash can occur when a rule is being profiled and rule profiling
is then disabled after one at least one rule was profiled for the packet
(which marks the packet as being profiled).
In this scenario, the value of profiling_rules_entered was
not incremented so the BUG_ON in the end profiling macro trips
because it is 0.
The changes to fix the problem are:
- In the profiling end macro, gate the actions taken there by the same
configuration setting and use the profiling_rues_entered (instead of
the per-packet profiling flag). Since the start and end macros are
tightly coupled, this will permit profiling to "finish" if started.
- Modify SCProfileRuleStart to only check the sampling values if the
packet hasn't been marked for profiling already. This change makes all
rules for a packet (once selected) to be profiled (without this change
sampling is applied to each *rule* that applies to the packet.
Philippe Antoine [Thu, 22 Feb 2024 09:14:36 +0000 (10:14 +0100)]
ssh: avoid quadratic complexity from long banner
Ticket: 6799
When we find an overlong banner, we get into the state just
waiting for end of line, and we just want to skip the bytes
until then.
Returning AppLayerResult::incomplete made TCP engine retain
the bytes and grow the buffer that we parsed again and again...
When running FlowWorkerStreamTCPUpdate, one of the dequeued packet
may set the flow action to drop, without updating the not-pseudo
packet action, as is done usually with a previous call to
FlowHandlePacketUpdate
Jeff Lucovsky [Mon, 11 Mar 2024 18:57:16 +0000 (14:57 -0400)]
flow/inject: Ensure initialized thread value used
Issue: 6835
When injecting a flow, ensure that the selected thread_id has been
initialized. When a flow is picked up midstream, the initialized thread
can be the second thread element.
Lukas Sismis [Sat, 2 Mar 2024 17:15:16 +0000 (18:15 +0100)]
dpdk: only close the port when workers are synchronized
When Suricata was running in IPS mode and received a signal to stop,
the first worker of every interface/port stopped the port and
proactively stopped the peered interface as well.
This was done to be as accurate with port stats as possible.
However, in a highly active scenarios (lots of packets moving around)
the peered workers might still be in the process of a packet
release operation. These workers would then attempt to transmit
on a stopped interface - resulting in an errorneous operation.
Instead, this patch proposes a worker synchronization of the given
port. After these workers are synchronized, it is known that no packets
will be sent of the peered interface, therefore the first worker can
stop it. This however cannot be assumed about "its own" port as the
peered workers can still try to send the packets. Therefore, ports
are only stopped by the peered workers.
Jason Ish [Fri, 8 Mar 2024 06:23:25 +0000 (00:23 -0600)]
eve/filetypes: common init for threaded and non-threaded
In 7.0 if EVE was non-threaded, the ThreadInit for the filetype was
not called meaning that the filetype author had to handle the threaded
and non-threaded cases.
To simplify this, if non-threaded, still call ThreadInit (and
ThreadDeinit) once with a thread_id of 0. This should simplify
authoring EVE filetype plugins.
Jason Ish [Thu, 7 Mar 2024 21:40:03 +0000 (15:40 -0600)]
plugins: remove conf.h from suricata-plugin.h
Remove "conf.h" from suricata-plugin.h as its not needed by that
header. However, some other files became transitively dependent on
through other includes, so fix those up.
Jason Ish [Thu, 7 Mar 2024 21:33:28 +0000 (15:33 -0600)]
eve/filetypes: remove from plugin context
Remove EVE filetypes from plugin context as they are not only used
from plugins. Plugins allow user code to register filetypes, but we
also have internal file types that use this api including the null
output and syslog. Additionally library users can use this API to
register filetypes, and they are not plugins.
Ideally this code would go in "output-json.[ch]" as the "primary" eve
API, however there are currently some include circular include issues
there, so start new cleaned up EVE API in "output-eve.[ch]" which is
"clean" with respect to includes, and as we cleanup existing EVE API for
"public" use, it can be moved here.
Victor Julien [Mon, 4 Dec 2023 05:49:40 +0000 (06:49 +0100)]
nfq: stricter thread sync
No longer update `Packet::flags` for tracking packet modifications,
as thread safety was not guaranteed.
Clearly separate between various kinds of `Packet::nfq_v` accesses for:
- mark
- mark_modified
- verdicted
These are either done under lock (Packet::persistent.tunnel_lock) or,
if the Packet is not part of a tunnel, not under lock.
This is safe as in all the related logic the Packet's tunnel state
is fixed and can no longer change.
Jason Ish [Sat, 9 Mar 2024 18:12:43 +0000 (12:12 -0600)]
src: make include guards more library friendly
Include guards for libraries should use a prefix that is meaningful for
the library to avoid conflicts with other user code. For Suricata, use
SURICATA.
Additionally, remove the pattern of leading and trailing underscores as
these are reserved for the language implementation per the C and C++
standards.
If a port point is single but later on also a part of a range, it ends
up only creating the port groups for single points and not the range.
Fix it by adding the port next to current single one to unique points
and marking it a range port.
dns.rcode matches the rcode header field in DNS messages
It's an unsigned integer
valid ranges = [0-15]
Does not support prefilter
Supports matches in both flow directions
Arne Welzel [Sat, 17 Feb 2024 17:19:27 +0000 (18:19 +0100)]
stats: Fix non-worker stats missing
Commit b8b8aa69b49ac0dd222446c28d00a50f9fd7d716 used tm_name of the
first StatsRecord of a thread block as key for the "threads" object.
However, depending on the type of thread, tm_name can be NULL and would
result in no entry being included for that thread at all. This caused
non-worker metrics to vanish from the "threads" object in the
dump-counters output.
This patch fixes this by remembering the first occurrence of a valid
tm_name within the per-thread block and adds another unittest to
cover this scenario.
Victor Julien [Thu, 29 Feb 2024 10:02:40 +0000 (11:02 +0100)]
rust: update parser dependencies
Time locked to 0.3.20 to guarantee MSRV of 1.63.
Update snmp-parser to 0.10.0.
Update asn1-rs to 0.6.1.
Update kerberos-parser to 0.8.0.
Update x509-parser 0.16.0.
Update der-parser to 9.0.0.
Remove specific use of der-parser 6.
Shivani Bhardwaj [Wed, 28 Feb 2024 14:29:04 +0000 (19:59 +0530)]
detect/port: remove SigGroupHead* ops
The functions in detect-engine-port.c are only being used at the time of
parsing the ports from rules initially. Since there are no SGHs at that
point, remove the ops related to them too.
Victor Julien [Mon, 26 Feb 2024 16:08:21 +0000 (21:38 +0530)]
detect/port: use qsort instead of insert sort
Instead of using in place insertion sort on linked list based on two
keys, convert the linked list to an array, perform sorting on it using
qsort and convert it back to a linked list. This turns out to be much
faster.
Shivani Bhardwaj [Wed, 21 Feb 2024 06:42:30 +0000 (12:12 +0530)]
detect/port: merge port ranges for same signatures
To avoid getting multiple entries in the final port list and to also
make the next step more efficient by reducing the size of the items to
traverse over.
Shivani Bhardwaj [Tue, 20 Feb 2024 16:22:38 +0000 (21:52 +0530)]
detect/port: create list of small port ranges
Using the unique port points, create a list of small port ranges which
contain the DetectPort objects and the designated SGHs found by finding
the overlaps with the existing ports and copying the SGHs accordingly.
Shivani Bhardwaj [Fri, 16 Feb 2024 09:18:46 +0000 (14:48 +0530)]
detect/port: create a tree of given ports
After all the SGHs have been appropriately copied to the designated
ports, create an interval tree out of it for a faster lookup when later
a search for overlaps is made.
Shivani Bhardwaj [Fri, 16 Feb 2024 08:57:52 +0000 (14:27 +0530)]
detect/port: find unique port points
In order to create the smallest possible port ranges, it is convenient
to first have a list of unique ports. Then, the work becomes simple. See
below:
Given, a port range P1 = [1, 8]; SGH1
and another, P2 = [3, 94]; SGH2
right now, the code will follow a logic of recursively cutting port
ranges until we create the small ranges. But, with the help of unique
port points, we get, unique_port_points = [1, 3, 8, 94]
So, now, in a later stage, we can create the ranges as
[1, 2], [3, 7], [8, 8], [9, 94] and copy the designated SGHs where they
belong. Note that the intervals are closed which means that the range
is inclusive of both the points.
The final result becomes:
1. [1, 2]; SGH1
2. [3, 7]; SGH1 + SGH2
3. [8, 8]; SGH1 + SGH2
4. [9, 94]; SGH2
There would be 3 unique rule groups made for the case above.
Group 1: [1, 2]
Group 2: [3, 7], [8, 8]
Group 3: [9, 94]
Warning was:
src/util-port-interval-tree.c:50:1: warning: Either the condition 'tmp!=NULL' is redundant or there is possible null pointer dereference: tmp. [nullPointerRedundantCheck]
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Assuming that condition 'tmp!=NULL' is not redundant
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Null pointer dereference
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: warning: Either the condition 'oleft!=NULL' is redundant or there is possible null pointer dereference: oleft. [nullPointerRedundantCheck]
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Assuming that condition 'oleft!=NULL' is not redundant
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Null pointer dereference
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: warning: Either the condition 'oright!=NULL' is redundant or there is possible null pointer dereference: oright. [nullPointerRedundantCheck]
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Assuming that condition 'oright!=NULL' is not redundant
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Null pointer dereference
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: warning: Either the condition 'left!=NULL' is redundant or there is possible null pointer dereference: left. [nullPointerRedundantCheck]
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Assuming that condition 'left!=NULL' is not redundant
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Null pointer dereference
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
Shivani Bhardwaj [Fri, 16 Feb 2024 08:07:23 +0000 (13:37 +0530)]
util/interval-tree: add utility fns
Add new utility files to deal with the interval trees. These cover the
basic ops:
1. Creation/Destruction of the tree
2. Creation/Destruction of the nodes
It also adds the support for finding overlaps for a given set of ports.
This function is used by the detection engine is the Stage 2 of
signature preparation.
Shivani Bhardwaj [Mon, 29 Jan 2024 06:08:51 +0000 (11:38 +0530)]
interval-tree: add augmentation fns to the tree
An interval tree uses red-black tree as its base data structure and
follows all the properties of a usual red-black tree. The additional
params are:
1. An interval such as [low, high] per node.
2. A max attribute per node. This attribute stores the maximum high
value of any subtree rooted at this node.
At any point in time, an inorder traversal of an interval tree should
give the port ranges sorted by the low key in ascending order.
This commit modifies the IRB_AUGMENT macro and it's call sites to make
sure that on every insertion, the max attribute of the tree is properly
updated.