]> git.ipfire.org Git - thirdparty/suricata.git/log
thirdparty/suricata.git
16 months agosrc: make include guards more library friendly 10610/head
Jason Ish [Sat, 9 Mar 2024 18:12:43 +0000 (12:12 -0600)] 
src: make include guards more library friendly

Include guards for libraries should use a prefix that is meaningful for
the library to avoid conflicts with other user code. For Suricata, use
SURICATA.

Additionally, remove the pattern of leading and trailing underscores as
these are reserved for the language implementation per the C and C++
standards.

16 months agodetect/port: handle single port that is range too 10608/head
Shivani Bhardwaj [Sat, 9 Mar 2024 04:21:57 +0000 (09:51 +0530)] 
detect/port: handle single port that is range too

If a port point is single but later on also a part of a range, it ends
up only creating the port groups for single points and not the range.
Fix it by adding the port next to current single one to unique points
and marking it a range port.

Bug 6843

16 months agodns: add dns.rcode keyword 10603/head
Hadiqa Alamdar Bukhari [Wed, 24 Jan 2024 12:10:49 +0000 (17:10 +0500)] 
dns: add dns.rcode keyword

dns.rcode matches the rcode header field in DNS messages
It's an unsigned integer
valid ranges = [0-15]
Does not support prefilter
Supports matches in both flow directions

Task #6621

16 months agodevguide: add chapter and short intro to libsuricata
Juliana Fajardini [Mon, 4 Mar 2024 16:12:08 +0000 (13:12 -0300)] 
devguide: add chapter and short intro to libsuricata

With this, we intend to make more users aware of this use case, and that
we are working towards this.

Related to
Task #2693

16 months agoutil/interval-tree: fix coverity warning 10600/head
Shivani Bhardwaj [Fri, 8 Mar 2024 08:36:31 +0000 (14:06 +0530)] 
util/interval-tree: fix coverity warning

Fix Coverity warning

** CID 1592992:  Incorrect expression  (COPY_PASTE_ERROR)
/src/util-port-interval-tree.c: 255 in SCPortIntervalFindOverlaps()

________________________________________________________________________________________________________
*** CID 1592992:  Incorrect expression  (COPY_PASTE_ERROR)
/src/util-port-interval-tree.c: 255 in SCPortIntervalFindOverlaps()
249                      * will be sorted, insert any new ports to the end of the list
250                      * and avoid walking the entire list */
251                     if (*list == NULL) {
252                         *list = new_port;
253                         (*list)->last = new_port;
254                     } else if (((*list)->last->port != new_port->port) &&
>>>     CID 1592992:  Incorrect expression  (COPY_PASTE_ERROR)
>>>     "port" in "(*list)->last->port2 != new_port->port" looks like a copy-paste error.
255                                ((*list)->last->port2 != new_port->port)) {
256                         DEBUG_VALIDATE_BUG_ON(new_port->port < (*list)->last->port);
257                         (*list)->last->next = new_port;
258                         new_port->prev = (*list)->last;
259                         (*list)->last = new_port;
260                     } else {

The code does not generate two port ranges that are same other than the
cases where port == port2 which is why it worked so far. Fix it.

Bug 6839

16 months agonetmap: Avoid HW errors when using pipes
Jeff Lucovsky [Thu, 7 Mar 2024 20:24:36 +0000 (15:24 -0500)] 
netmap: Avoid HW errors when using pipes

Issue: 6837

When using netmap pipes (with lb, for example), avoid direct hardware
related IOCTLs that will fail (not supported with pipes).

16 months agoschema: Add stats.capture and in_iface properties 10590/head
Arne Welzel [Tue, 20 Feb 2024 11:50:40 +0000 (12:50 +0100)] 
schema: Add stats.capture and in_iface properties

New suricata-verify test listens on loopback interface, resulting
in the capture and in_iface fields in the stats and event objects.

16 months agostats: Fix non-worker stats missing
Arne Welzel [Sat, 17 Feb 2024 17:19:27 +0000 (18:19 +0100)] 
stats: Fix non-worker stats missing

Commit b8b8aa69b49ac0dd222446c28d00a50f9fd7d716 used tm_name of the
first StatsRecord of a thread block as key for the "threads" object.
However, depending on the type of thread, tm_name can be NULL and would
result in no entry being included for that thread at all. This caused
non-worker metrics to vanish from the "threads" object in the
dump-counters output.

This patch fixes this by remembering the first occurrence of a valid
tm_name within the per-thread block and adds another unittest to
cover this scenario.

16 months agorust: update parser dependencies
Victor Julien [Thu, 29 Feb 2024 10:02:40 +0000 (11:02 +0100)] 
rust: update parser dependencies

Time locked to 0.3.20 to guarantee MSRV of 1.63.
Update snmp-parser to 0.10.0.
Update asn1-rs to 0.6.1.
Update kerberos-parser to 0.8.0.
Update x509-parser 0.16.0.
Update der-parser to 9.0.0.
Remove specific use of der-parser 6.

Ticket: #6817.
Ticket: #6818.

17 months agodetect/port: cleanup address artifacts 10569/head
Shivani Bhardwaj [Wed, 28 Feb 2024 15:14:04 +0000 (20:44 +0530)] 
detect/port: cleanup address artifacts

A lot of code uses variable names and comments derived from the code
about addresses, make them about port.

17 months agodetect/port: remove SigGroupHead* ops
Shivani Bhardwaj [Wed, 28 Feb 2024 14:29:04 +0000 (19:59 +0530)] 
detect/port: remove SigGroupHead* ops

The functions in detect-engine-port.c are only being used at the time of
parsing the ports from rules initially. Since there are no SGHs at that
point, remove the ops related to them too.

17 months agodetect: optimize sig_cnt setting
Victor Julien [Mon, 26 Feb 2024 11:17:15 +0000 (12:17 +0100)] 
detect: optimize sig_cnt setting

Utilize _popcnt64 where available.

17 months agodetect: optimize group head bitarray handling
Victor Julien [Mon, 26 Feb 2024 10:28:53 +0000 (11:28 +0100)] 
detect: optimize group head bitarray handling

During startup large rulesets use a lot of large bitarrays, that
are frequently merged (OR'd).

Optimize this using SSE2 _mm_or_si128.

17 months agodetect: prepare for SIMD optimizations
Victor Julien [Mon, 26 Feb 2024 09:52:09 +0000 (10:52 +0100)] 
detect: prepare for SIMD optimizations

Make rule group head bitarray 16 bytes aligned and padded to 16 bytes
boundaries to assist SIMD operations in follow up commits.

17 months agodetect/port: use qsort instead of insert sort
Victor Julien [Mon, 26 Feb 2024 16:08:21 +0000 (21:38 +0530)] 
detect/port: use qsort instead of insert sort

Instead of using in place insertion sort on linked list based on two
keys, convert the linked list to an array, perform sorting on it using
qsort and convert it back to a linked list. This turns out to be much
faster.

Ticket #6795

17 months agodetect/port: merge port ranges for same signatures
Shivani Bhardwaj [Wed, 21 Feb 2024 06:42:30 +0000 (12:12 +0530)] 
detect/port: merge port ranges for same signatures

To avoid getting multiple entries in the final port list and to also
make the next step more efficient by reducing the size of the items to
traverse over.

Ticket 6792
Bug 6414

17 months agodetect/port: remove the port cut/insertion stage
Shivani Bhardwaj [Tue, 20 Feb 2024 16:23:14 +0000 (21:53 +0530)] 
detect/port: remove the port cut/insertion stage

As this is already taken care of and a list of ports is available for
use by the next stage.

Ticket 6792
Bug 6414

17 months agodetect/port: create list of small port ranges
Shivani Bhardwaj [Tue, 20 Feb 2024 16:22:38 +0000 (21:52 +0530)] 
detect/port: create list of small port ranges

Using the unique port points, create a list of small port ranges which
contain the DetectPort objects and the designated SGHs found by finding
the overlaps with the existing ports and copying the SGHs accordingly.

Ticket 6792
Bug 6414

17 months agodetect/port: create a tree of given ports
Shivani Bhardwaj [Fri, 16 Feb 2024 09:18:46 +0000 (14:48 +0530)] 
detect/port: create a tree of given ports

After all the SGHs have been appropriately copied to the designated
ports, create an interval tree out of it for a faster lookup when later
a search for overlaps is made.

Ticket 6792
Bug 6414

17 months agodetect/port: find unique port points
Shivani Bhardwaj [Fri, 16 Feb 2024 08:57:52 +0000 (14:27 +0530)] 
detect/port: find unique port points

In order to create the smallest possible port ranges, it is convenient
to first have a list of unique ports. Then, the work becomes simple. See
below:

Given, a port range P1 = [1, 8]; SGH1
and another, P2 = [3, 94]; SGH2

right now, the code will follow a logic of recursively cutting port
ranges until we create the small ranges. But, with the help of unique
port points, we get, unique_port_points = [1, 3, 8, 94]

So, now, in a later stage, we can create the ranges as
[1, 2], [3, 7], [8, 8], [9, 94] and copy the designated SGHs where they
belong. Note that the intervals are closed which means that the range
is inclusive of both the points.

The final result becomes:
1. [1, 2]; SGH1
2. [3, 7]; SGH1 + SGH2
3. [8, 8]; SGH1 + SGH2
4. [9, 94]; SGH2

There would be 3 unique rule groups made for the case above.
Group 1: [1, 2]
Group 2: [3, 7], [8, 8]
Group 3: [9, 94]

Ticket 6792
Bug 6414

17 months agoutil/interval-tree: suppress cppcheck warnings
Shivani Bhardwaj [Mon, 4 Mar 2024 08:21:59 +0000 (13:51 +0530)] 
util/interval-tree: suppress cppcheck warnings

Warning was:
src/util-port-interval-tree.c:50:1: warning: Either the condition 'tmp!=NULL' is redundant or there is possible null pointer dereference: tmp. [nullPointerRedundantCheck]
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Assuming that condition 'tmp!=NULL' is not redundant
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Null pointer dereference
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: warning: Either the condition 'oleft!=NULL' is redundant or there is possible null pointer dereference: oleft. [nullPointerRedundantCheck]
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Assuming that condition 'oleft!=NULL' is not redundant
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Null pointer dereference
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: warning: Either the condition 'oright!=NULL' is redundant or there is possible null pointer dereference: oright. [nullPointerRedundantCheck]
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Assuming that condition 'oright!=NULL' is not redundant
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Null pointer dereference
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: warning: Either the condition 'left!=NULL' is redundant or there is possible null pointer dereference: left. [nullPointerRedundantCheck]
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Assuming that condition 'left!=NULL' is not redundant
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Null pointer dereference
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^

17 months agoutil/interval-tree: add utility fns
Shivani Bhardwaj [Fri, 16 Feb 2024 08:07:23 +0000 (13:37 +0530)] 
util/interval-tree: add utility fns

Add new utility files to deal with the interval trees. These cover the
basic ops:
1. Creation/Destruction of the tree
2. Creation/Destruction of the nodes

It also adds the support for finding overlaps for a given set of ports.
This function is used by the detection engine is the Stage 2 of
signature preparation.

Ticket 6792
Bug 6414

Co-authored-by: Victor Julien <vjulien@oisf.net>
17 months agodetect/port: make DetectPortInit non static
Shivani Bhardwaj [Wed, 14 Feb 2024 11:09:02 +0000 (16:39 +0530)] 
detect/port: make DetectPortInit non static

as this fn will be called upon and further used by other files later on.

Ticket 6792
Bug 6414

17 months agointerval-tree: add augmentation fns to the tree
Shivani Bhardwaj [Mon, 29 Jan 2024 06:08:51 +0000 (11:38 +0530)] 
interval-tree: add augmentation fns to the tree

An interval tree uses red-black tree as its base data structure and
follows all the properties of a usual red-black tree. The additional
params are:
1. An interval such as [low, high] per node.
2. A max attribute per node. This attribute stores the maximum high
   value of any subtree rooted at this node.

At any point in time, an inorder traversal of an interval tree should
give the port ranges sorted by the low key in ascending order.

This commit modifies the IRB_AUGMENT macro and it's call sites to make
sure that on every insertion, the max attribute of the tree is properly
updated.

Ticket 6792
Bug 6414

17 months agointerval-tree: remove splay tree implementation
Shivani Bhardwaj [Fri, 16 Feb 2024 07:17:49 +0000 (12:47 +0530)] 
interval-tree: remove splay tree implementation

Ticket 6792
Bug 6414

17 months agointerval-tree: add base data structure
Shivani Bhardwaj [Fri, 16 Feb 2024 07:14:10 +0000 (12:44 +0530)] 
interval-tree: add base data structure

Ticket 6792
Bug 6414

17 months agodetect/engine: fix whitelisting check
Victor Julien [Fri, 12 Jan 2024 07:03:06 +0000 (12:33 +0530)] 
detect/engine: fix whitelisting check

In the commit 4a00ae607, the whitelisting check was updated in a quest
to make use of the conditional better but it made things worse as every
range would be whitelisted as long as it had any of the default
whitelisted port which is very common.

17 months agodpdk: output unknown NIC's NUMA node message only on multi-node systems
Lukas Sismis [Sat, 2 Mar 2024 14:57:43 +0000 (15:57 +0100)] 
dpdk: output unknown NIC's NUMA node message only on multi-node systems

Ticket: #6715

17 months agogithub-actions: bump actions/download-artifact from 4.1.3 to 4.1.4 10552/head
dependabot[bot] [Fri, 1 Mar 2024 19:56:33 +0000 (19:56 +0000)] 
github-actions: bump actions/download-artifact from 4.1.3 to 4.1.4

Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 4.1.3 to 4.1.4.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/87c55149d96e628cc2ef7e6fc2aab372015aec85...c850b930e6ba138125429b7e5c93fc707a7f8427)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
17 months agogithub-actions: bump github/codeql-action from 3.24.5 to 3.24.6
dependabot[bot] [Thu, 29 Feb 2024 19:49:17 +0000 (19:49 +0000)] 
github-actions: bump github/codeql-action from 3.24.5 to 3.24.6

Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.24.5 to 3.24.6.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Commits](https://github.com/github/codeql-action/compare/v3.24.5...v3.24.6)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
17 months agoexamples: minimal example capture plugin for ci
Jason Ish [Wed, 28 Feb 2024 21:21:57 +0000 (15:21 -0600)] 
examples: minimal example capture plugin for ci

Create a mininal capture plugin that injects one packet. While it can
also be a template, we should be able to run this in CI to test the
loading and registration of the capture plugin mechanisms.

17 months agothreads: tm-modules.h depends on Packet so include decode.h
Jason Ish [Wed, 28 Feb 2024 22:07:17 +0000 (16:07 -0600)] 
threads: tm-modules.h depends on Packet so include decode.h

17 months agoplugins: initialize plugins earlier
Jason Ish [Tue, 27 Feb 2024 22:16:08 +0000 (16:16 -0600)] 
plugins: initialize plugins earlier

Capture plugins need to be registered before LiveDeviceFinalize,
otherwise Suricata errors before the plugin gets a chance to load.

Bug: #6811

17 months agothread modules: separate initialization from registration
Jason Ish [Tue, 27 Feb 2024 22:07:33 +0000 (16:07 -0600)] 
thread modules: separate initialization from registration

Move the zero'ing to the thread module array InitGlobal in an effort
to fix capture modules.

At some point device validation moved to a point in startup before
plugins are loaded meaning that capture plugins could not be
used. Moving plugin registration early enough caused some of their
registration to be wiped out as clearing the array was done after.

Bug: #6811

17 months agoschema: add flow.wrong_thread
Jason Ish [Thu, 29 Feb 2024 14:19:58 +0000 (08:19 -0600)] 
schema: add flow.wrong_thread

17 months agodependabot: reduce to monthly update 10536/head
Victor Julien [Wed, 28 Feb 2024 17:33:37 +0000 (18:33 +0100)] 
dependabot: reduce to monthly update

17 months agodns: add dns.rrtype keyword
Hadiqa Alamdar Bukhari [Wed, 31 Jan 2024 12:56:33 +0000 (17:56 +0500)] 
dns: add dns.rrtype keyword

It matches the rrtype field in DNS
It's an unsigned integer match
valid ranges = [0-65535]
Does not support prefilter
Supports flow in both directions
Feature #6666

17 months agopfring: fix leak of configuration data and in single mode 10525/head
Jason Ish [Tue, 27 Feb 2024 22:55:59 +0000 (16:55 -0600)] 
pfring: fix leak of configuration data and in single mode

Fix leak of configuration data on exit. Also, in single mode set
thread count to one instead of the CPU count.

Bug: #4734

17 months agopfring: fix leak of packet on exit
Jason Ish [Tue, 27 Feb 2024 23:19:54 +0000 (17:19 -0600)] 
pfring: fix leak of packet on exit

Bug: #4734

17 months agooutput/filestore: delay snprintf until needed
Philippe Antoine [Tue, 27 Feb 2024 20:45:03 +0000 (21:45 +0100)] 
output/filestore: delay snprintf until needed

Perf optimization so that we do not call snprintf in the
common code path.

Ticket: 6796

17 months agooutput/filestore: remove duplicate snprintf
Philippe Antoine [Mon, 26 Feb 2024 16:28:25 +0000 (17:28 +0100)] 
output/filestore: remove duplicate snprintf

Ticket: 6796

17 months agooutput: log tx only when there is a fresh app update
Philippe Antoine [Thu, 22 Feb 2024 08:42:41 +0000 (09:42 +0100)] 
output: log tx only when there is a fresh app update

Ticket: 6796

Similar to commit for detection
9240ae250cc369306803740279df2ab3eca6b54a

We only have more logging to do if the app update was fresh,
ie if p->app_update_direction != 0

If we have data acknowledged in one direction,
and then many packets in the other direction,
the APP_UPDATED flow flags did not get reset because we did not
run detection yet in this direction,
but there is nothing more to do after the first packet in the
other direction.

17 months agogithub-actions: bump codecov/codecov-action from 4.0.1 to 4.1.0
dependabot[bot] [Tue, 27 Feb 2024 19:59:34 +0000 (19:59 +0000)] 
github-actions: bump codecov/codecov-action from 4.0.1 to 4.1.0

Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 4.0.1 to 4.1.0.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/e0b68c6749509c5f83f984dd99a76a1c1a231044...54bcd8715eee62d40e33596ef5e8f0f48dbbccab)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
17 months agogithub-actions: bump actions/download-artifact from 4.1.2 to 4.1.3
dependabot[bot] [Tue, 27 Feb 2024 19:59:31 +0000 (19:59 +0000)] 
github-actions: bump actions/download-artifact from 4.1.2 to 4.1.3

Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 4.1.2 to 4.1.3.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/eaceaf801fd36c7dee90939fad912460b18a1ffe...87c55149d96e628cc2ef7e6fc2aab372015aec85)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
17 months agodoc: add upgrade section for 8 10522/head
Giuseppe Longo [Fri, 24 Nov 2023 08:54:53 +0000 (09:54 +0100)] 
doc: add upgrade section for 8

17 months agorust/sip: register pattern matching
Giuseppe Longo [Fri, 24 Nov 2023 22:18:49 +0000 (23:18 +0100)] 
rust/sip: register pattern matching

This permits to detect the SIP protocol using pattern matching instead of
probing parser.

Since it is no longer used, the respective probing functions have been removed.

17 months agosuricata.yaml: define SIP_PORTS
Giuseppe Longo [Tue, 1 Aug 2023 19:24:28 +0000 (21:24 +0200)] 
suricata.yaml: define SIP_PORTS

17 months agorust/sip: add direction to transaction
Giuseppe Longo [Tue, 1 Aug 2023 19:20:58 +0000 (21:20 +0200)] 
rust/sip: add direction to transaction

This patch permits to set a direction when a new transaction is created in order
to avoid 'signature shadowing' as reported by Eric Leblond in commit
5aaf50760f546e9047da508f725f43a7ad9b8a35

17 months agorust/sip: register parser for tcp
Giuseppe Longo [Tue, 1 Aug 2023 19:08:50 +0000 (21:08 +0200)] 
rust/sip: register parser for tcp

This patch lets the parser to work over tcp protocol, taking care of handling
data before calling the request/response parsers.

Ticket #3351.

17 months agosip/parser: enforce valid chars for sip version
Giuseppe Longo [Sat, 25 Nov 2023 08:39:54 +0000 (09:39 +0100)] 
sip/parser: enforce valid chars for sip version

The `is_version_char` function incorrectly allowed characters that are not
part of the valid SIP version "SIP/2.0".

For instance, 'HTTP/1.1' was mistakenly accepted as a valid SIP version,
although it's not.

This commit fixes the issue by updating the condition to strictly
check for the correct version string.

17 months agosip/parser: accept valid chars
Giuseppe Longo [Tue, 1 Aug 2023 18:50:17 +0000 (20:50 +0200)] 
sip/parser: accept valid chars

Accepts valid characters as defined in RFC3261.

17 months agorust/sip: rustfmt sip module
Giuseppe Longo [Mon, 14 Aug 2023 18:10:36 +0000 (20:10 +0200)] 
rust/sip: rustfmt sip module

17 months agodetect: do not run tx detection on non established packets
Philippe Antoine [Fri, 2 Feb 2024 10:11:10 +0000 (11:11 +0100)] 
detect: do not run tx detection on non established packets

Follows commit 2fb5059

Ticket: 6775

17 months agodoc: there is no right shift for integer bitmasks 10501/head
Philippe Antoine [Tue, 30 Jan 2024 14:45:17 +0000 (15:45 +0100)] 
doc: there is no right shift for integer bitmasks

Ticket: 6628

17 months agogithub-actions: bump github/codeql-action from 3.24.3 to 3.24.5 10496/head
dependabot[bot] [Fri, 23 Feb 2024 19:36:10 +0000 (19:36 +0000)] 
github-actions: bump github/codeql-action from 3.24.3 to 3.24.5

Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.24.3 to 3.24.5.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Commits](https://github.com/github/codeql-action/compare/v3.24.3...v3.24.5)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
17 months agobuild: don't build the lib example by default
Jason Ish [Thu, 22 Feb 2024 17:50:11 +0000 (11:50 -0600)] 
build: don't build the lib example by default

Don't build the library example by default as it is not needed by most
users.

17 months agoexamples/lib: work with bundled libhtp
Jason Ish [Thu, 22 Feb 2024 17:24:52 +0000 (11:24 -0600)] 
examples/lib: work with bundled libhtp

The simple example Makefile.am was unconditionally including
$(HTP_LDADD) which might be empty resulting in "../.." ending up in the
Makefile causing the build to fail.

Instead, also make HTP_LDADD a conditional, so we can only include it
when actually set, and its only set when libhtp is bundled.

The reason this Makefile needs to include the path components "../.."
is because the HTP_LDADD value is relative to the top level "src/"
directory.

17 months agoaf-xdp: suppress cppcheck false positive 10486/head
Victor Julien [Tue, 20 Feb 2024 15:07:37 +0000 (16:07 +0100)] 
af-xdp: suppress cppcheck false positive

As the cppcheck test suite does as well:
https://github.com/danmar/cppcheck/blob/main/test/cfg/posix.c#L311

Requires cppcheck --inline-suppr option to be used.

17 months agodetect: fix memory leak in error handling
Victor Julien [Tue, 20 Feb 2024 15:06:57 +0000 (16:06 +0100)] 
detect: fix memory leak in error handling

Pointed out by cppcheck:

src/detect-parse.c:1544:9: error: Memory leak: sig.init_data [memleak]
        SCFree(sig);
        ^

Bug: #6527.

17 months agodetect/address: avoid cppcheck false positive
Victor Julien [Tue, 20 Feb 2024 15:06:08 +0000 (16:06 +0100)] 
detect/address: avoid cppcheck false positive

src/detect-engine-address.c:1386:5: error: Memory leak: map.string [memleak]
    return true;
    ^

Seems cppcheck looses track of the pointer after the unnecessary cast to
void.

Bug: #6527.

17 months agodns: add missing dns keywords to schema.json
Hadiqa Alamdar Bukhari [Tue, 13 Feb 2024 12:55:38 +0000 (17:55 +0500)] 
dns: add missing dns keywords to schema.json

Found and added missing dns fields in schema.json after manual code review.
Added description to these newly added dns fields.
Feature #5642

17 months agotcp: do not assign TCP flags to pseudopackets 10475/head
Lukas Sismis [Tue, 6 Feb 2024 10:38:19 +0000 (11:38 +0100)] 
tcp: do not assign TCP flags to pseudopackets

Previously pseudopackets were assigned with ACK flag which falsely turned
"SYN" flows to "SYN/ACK" flows when Suricata ran with raw content-matching
rules. The problem occured during the flow timeout or Suricata shutdown,
essentially, when the flow was being kicked out (with a pseudopacket).

When Suricata ran without raw content-matching rules (the ruleset did not
contain content matching keywords or it only contained keywords that are
app-layer content-matching) then raw stream reassembly tracking is turned off
(SignatureHasStreamContent()).
This in turn disabled a check in StreamNeedsReassembly() and the right edge
was not checked with the raw stream progress. In turn, it did not generate
a pseudopacket that would go through the detection engine. Suricata with
raw content-matching keywords would therefore on a flow with SYN packet only
return STREAM_HAS_UNPROCESSED_SEGMENTS_NEED_ONLY_DETECTION which would generate
the pseudopacket.

In Suricata versions <= 6.0.x, the flow output was correct because
only the commit 1bb6f44ff01363fa29488f1ae83b9368e33c2770 started to
differentiate the right edge calculation between the raw and application
layer streams. The older Suricata versions used only the application layer
right edge equation and therefore did not generate a pseudopacket.

Ticket: #6733

17 months agodpdk: remove redundant port checks and set copy iface socket id correctly
Lukas Sismis [Fri, 16 Feb 2024 20:40:59 +0000 (21:40 +0100)] 
dpdk: remove redundant port checks and set copy iface socket id correctly

The function to retrieve port ID from the port name was used multiple times.
This commit removes the redundant usage of the function.

Additionally, in the DeviceConfigureIPS(), the socket ID was wrongly retrieved
for the original interface and not for the out port interface.

17 months agodpdk: add a user warning on the change of RX/TX descriptors
Lukas Sismis [Sun, 11 Feb 2024 16:25:30 +0000 (17:25 +0100)] 
dpdk: add a user warning on the change of RX/TX descriptors

Ticket: #6739

17 months agodpdk: refactor log messages to have the same error format
Lukas Sismis [Sun, 11 Feb 2024 16:21:14 +0000 (17:21 +0100)] 
dpdk: refactor log messages to have the same error format

17 months agodpdk: split out the configure function into smaller ones
Lukas Sismis [Sun, 11 Feb 2024 16:18:13 +0000 (17:18 +0100)] 
dpdk: split out the configure function into smaller ones

17 months agodpdk: add missing SCEnter macros to functions with SCReturns
Lukas Sismis [Sun, 11 Feb 2024 16:16:23 +0000 (17:16 +0100)] 
dpdk: add missing SCEnter macros to functions with SCReturns

17 months agohugepages: run hugepage check only on DPDK runmode and on Linux
Lukas Sismis [Sat, 10 Feb 2024 19:04:55 +0000 (20:04 +0100)] 
hugepages: run hugepage check only on DPDK runmode and on Linux

Previous implementation allowed FreeBSD to enter into the hugepage
analysis. It then failed with an error message because hugepage/
NUMA node paths that are used in the codebase to retrieve info about
the system are not the same with the structure in Linux.

Additionally, the messages were logged on error level. It has been
demoted to info level because the whole hugepage analysis checkup is
only for informational purposes and does not affect Suricata operation.

The hugepage analysis and the hugepage snapshots are now limited to
only run in the DPDK runmode.

Ticket: #6760
Ticket: #6762

17 months agoerrors: add SC_ENOENT error
Lukas Sismis [Sat, 10 Feb 2024 19:04:24 +0000 (20:04 +0100)] 
errors: add SC_ENOENT error

17 months agodetect/mpm: don't test for impossible condition
Victor Julien [Fri, 5 Jan 2024 07:29:16 +0000 (08:29 +0100)] 
detect/mpm: don't test for impossible condition

Functions can't get called with NULL signature or NULL
Signature::init_data.

17 months agodetect/iponly: code cleanups
Victor Julien [Fri, 5 Jan 2024 07:25:46 +0000 (08:25 +0100)] 
detect/iponly: code cleanups

Constify and flatten branches.

17 months agodetect/iponly: move parsing only fields to init_data
Victor Julien [Fri, 5 Jan 2024 11:56:24 +0000 (12:56 +0100)] 
detect/iponly: move parsing only fields to init_data

IP-only parse results were not used at runtime.

17 months agorunmode/unix-socket: fix cppcheck warnings 10469/head
Victor Julien [Sat, 17 Feb 2024 09:52:59 +0000 (10:52 +0100)] 
runmode/unix-socket: fix cppcheck warnings

src/runmode-unix-socket.c:547:9: warning: %d in format string (no. 1) requires 'int' but the argument type is 'unsigned int'. [invalidPrintfArgType_sint]
        snprintf(tstr, sizeof(tstr), "%d", cfile->tenant_id);
        ^
src/runmode-unix-socket.c:1040:5: warning: %d in format string (no. 1) requires 'int' but the argument type is 'unsigned int'. [invalidPrintfArgType_sint]
    snprintf(prefix, sizeof(prefix), "multi-detect.%d", tenant_id);
    ^
src/runmode-unix-socket.c:1189:5: warning: %d in format string (no. 1) requires 'int' but the argument type is 'unsigned int'. [invalidPrintfArgType_sint]
    snprintf(prefix, sizeof(prefix), "multi-detect.%d", tenant_id);
    ^

17 months agogithub-actions: bump github/codeql-action from 3.24.1 to 3.24.3
dependabot[bot] [Thu, 15 Feb 2024 19:04:35 +0000 (19:04 +0000)] 
github-actions: bump github/codeql-action from 3.24.1 to 3.24.3

Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.24.1 to 3.24.3.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Commits](https://github.com/github/codeql-action/compare/v3.24.1...v3.24.3)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
17 months agobuild: remove autoconf.h from dist
Jason Ish [Thu, 15 Feb 2024 15:19:14 +0000 (09:19 -0600)] 
build: remove autoconf.h from dist

We were including "autoconf.h" in our dist releases, however this file
is generated by ./configure. This can be problematic in out of tree
builds as the one included in the dist, not the generated one may be
used.

Also using "autoconf.h" instead of <autoconf.h> so it is searched for
relative to the file including it which is necessary if a library/plugin
does something like "#include "suricata/foo.h".

17 months agoexamples: program linking against library
Jason Ish [Mon, 5 Feb 2024 22:49:45 +0000 (16:49 -0600)] 
examples: program linking against library

Provide an example of an extremely simple application that links
against Suricata. This provides a Makefile integrated with the
Suricata build system for in-tree building, as well as an example
Makefile for building out of tree.

Currently this application just wraps SuricataMain and does nothing
else.

17 months agoexamples/plugin: simplify Makefile
Jason Ish [Wed, 14 Feb 2024 21:23:44 +0000 (15:23 -0600)] 
examples/plugin: simplify Makefile

Simplify the Makefile by avoiding automake and providing our own
Makefile.in that is suitable for in-tree builds of the plugin and can
also serve as an example for standalone plugins.

But the bigger benefit of this is to allow building the example plugin
even with --disable-shared provided to configure, as this is just a
phony limitation imposed by automake/libtool.

17 months agonapatech: Close streams on hard errors
Jeff Lucovsky [Mon, 29 Jan 2024 18:39:40 +0000 (13:39 -0500)] 
napatech: Close streams on hard errors

Issue: 5172

This commit forces the current stream to be closed when a hard error
occurs.

Without this commit, a hard error will force a 2nd connection
the stream that raised the error.

With this commit, the stream with the hard error is closed before
opening another connection to the stream.

Verified with the Python snippet in the redmine issue.

17 months agococci/run-check: log if parallel command is not found 10445/head
Jason Ish [Tue, 13 Feb 2024 19:38:57 +0000 (13:38 -0600)] 
cocci/run-check: log if parallel command is not found

If CONCURRENCY_LEVEL was set, the script would log a concurrency level
even if the parallel command was not available. Not log if parallel is
not available and set concurrency to 1.

17 months agogithub-ci: use all cpus for coccinelle checks
Jason Ish [Tue, 13 Feb 2024 19:21:26 +0000 (13:21 -0600)] 
github-ci: use all cpus for coccinelle checks

Also put "cocci" in the job name and install parallel so the script can
actually run with concurrency.

17 months agoeve/stats: add description for applayer flows
Shivani Bhardwaj [Fri, 9 Feb 2024 10:12:42 +0000 (15:42 +0530)] 
eve/stats: add description for applayer flows

Ticket 6434

17 months agoeve/stats: add description for expectations
Shivani Bhardwaj [Mon, 5 Feb 2024 10:52:18 +0000 (16:22 +0530)] 
eve/stats: add description for expectations

Ticket 6434

17 months agoeve/stats: add description for applayer errors
Shivani Bhardwaj [Mon, 5 Feb 2024 10:51:34 +0000 (16:21 +0530)] 
eve/stats: add description for applayer errors

Ticket 6434

17 months agoeve/stats: add description for common fields
Shivani Bhardwaj [Mon, 5 Feb 2024 10:49:24 +0000 (16:19 +0530)] 
eve/stats: add description for common fields

Ticket 6434

17 months agomulti-tenant: remove futile mutex lock 10436/head
Shivani Bhardwaj [Thu, 15 Feb 2024 04:47:38 +0000 (10:17 +0530)] 
multi-tenant: remove futile mutex lock

No shared resource is being changed when the lock is held, it is
immediately unlocked. So, remove it.

17 months agostats: Add unittest for basic stats serialization 10426/head
Arne Welzel [Thu, 8 Feb 2024 17:23:38 +0000 (18:23 +0100)] 
stats: Add unittest for basic stats serialization

Main purpose is to validate that the 30 of bond0.30 isn't expanded into
a nested object during serialization.

17 months agostats: Do not expand dots of tm_name
Arne Welzel [Mon, 5 Feb 2024 16:45:30 +0000 (17:45 +0100)] 
stats: Do not expand dots of tm_name

When an interface with dots is used, per worker stats are nested by the
dot-separated-components of the interface due to the usage of
OutputStats2Json().

Prevent this by using OutputStats2Json() on a per-thread specific object
and setting this object into the threads object using the
json_object_set_new() which won't do the dot expansion.

This was tested by creating an interface with dots in the name
and checking the stats.

    ip link add name a.b.c type dummy

With Suricata 7.0.2, sniffing on the a.b.c interface results in the
following worker stats format:

    "threads": {
      "W#01-a": {
        "b": {
          "c": {
            "capture": {
              "kernel_packets": 0,

After this fix, the output looks as follows:

    "threads": {
      "W#01-a.b.c": {
        "capture": {
          "kernel_packets": 0,

Ticket: #6732

17 months agodetect-engine-iponly: improve ip list performance
Simon Dugas [Fri, 29 Dec 2023 16:58:50 +0000 (11:58 -0500)] 
detect-engine-iponly: improve ip list performance

The runtime complexity of insertion sort is approx. O(h*n)^2 where
h is the size of the HOME_NET and n is the number of ip only rules
that use the HOME_NET.

Replacing this with qsort significantly improves rule load time when
a large HOME_NET is used in combination with a moderate amount of ip
only rules.

17 months agodetect: respect directionality for filestore
Philippe Antoine [Thu, 25 Jan 2024 13:26:09 +0000 (14:26 +0100)] 
detect: respect directionality for filestore

Ticket: 6617

So that rules with keyword like `filestore:to_server,flow`
only store the files to server and not the ones to client...

Directionality only worked with the default scope, ie the
current file, and not the scope tx or scope flow.
For non-default scope, tx or flow, both directions were stored
whatever the directionality specified.

For these non-default scopes, this commit keeps a default
of both directions, but use only one direction if specified.

Need to split flag FLOWFILE_STORE per direction, so that Suricata
can retain this (optional) directional info from the filestore
keyword.

Fixes: 79499e476979 ("app-layer: move files into transactions")
17 months agoci: right sha for authors check 10419/head
Philippe Antoine [Tue, 13 Feb 2024 20:11:32 +0000 (21:11 +0100)] 
ci: right sha for authors check

17 months agomulti-tenant: fix coverity warning
Victor Julien [Tue, 13 Feb 2024 08:51:15 +0000 (09:51 +0100)] 
multi-tenant: fix coverity warning

Rework locking logic to avoid the following coverity warning.

** CID 1591966:  Concurrent data access violations  (MISSING_LOCK)
/src/detect-engine-loader.c: 475 in DetectLoadersSync()

    474                     SCCtrlMutexLock(loader->tv->ctrl_mutex);
    >>>     CID 1591966:  Concurrent data access violations  (MISSING_LOCK)
    >>>     Accessing "loader->tv" without holding lock "DetectLoaderControl_.m". Elsewhere, "DetectLoaderControl_.tv" is written to with "DetectLoaderControl_.m" held 1 out of 1 times (1 of these accesses strongly imply that it is necessary).
    475                     pthread_cond_broadcast(loader->tv->ctrl_cond);
    476                     SCCtrlMutexUnlock(loader->tv->ctrl_mutex);

The warning itself is harmless.

17 months agomqtt: Improve frame parsing w/mult. PDUs
Jeff Lucovsky [Wed, 3 Jan 2024 14:28:41 +0000 (09:28 -0500)] 
mqtt: Improve frame parsing w/mult. PDUs

This commit improves the mqtt parsing of frames to handle multiple PDUs.

Issue: 6592

17 months agogithub-actions: bump github/codeql-action from 2.24.0 to 3.24.1 10413/head
dependabot[bot] [Tue, 13 Feb 2024 19:27:24 +0000 (19:27 +0000)] 
github-actions: bump github/codeql-action from 2.24.0 to 3.24.1

Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2.24.0 to 3.24.1.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Commits](https://github.com/github/codeql-action/compare/v2.24.0...v3.24.1)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
17 months agodetect/tls.certs: fix direction handling
Victor Julien [Sat, 27 Jan 2024 08:59:55 +0000 (09:59 +0100)] 
detect/tls.certs: fix direction handling

Direction flag was checked against wrong field, leading to undefined behavior.

Bug: #6778.

17 months agogithub-ci: fix authors check with special characters
Jason Ish [Tue, 13 Feb 2024 17:57:02 +0000 (11:57 -0600)] 
github-ci: fix authors check with special characters

Dependabot is always getting flagged as a new author even tho it uses
a consistent author of:

dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

But this doesn't work with plain grep. Fix by telling grep to treat
the value as a fixed string instead of a regular expression.

17 months agodependabot: disable rust checks
Jason Ish [Tue, 13 Feb 2024 16:08:37 +0000 (10:08 -0600)] 
dependabot: disable rust checks

As we don't have a Cargo.toml and a Cargo.lock, dependabot for Rust
hasn't been working correctly. Disable, as we now have our own cargo
audit and update workflows.

17 months agodependabot: ignore actions/{cache,checkout} v3
Jason Ish [Tue, 13 Feb 2024 16:05:02 +0000 (10:05 -0600)] 
dependabot: ignore actions/{cache,checkout} v3

The CentOS 7 build requires older GitHub actions, try to make
dependabot ignore these older versions.

17 months agogithub-ci: apply read-only permissions to more workflows
Jason Ish [Tue, 13 Feb 2024 15:42:55 +0000 (09:42 -0600)] 
github-ci: apply read-only permissions to more workflows

- authors.yml
- codeql.yml
- scan-build.yml

17 months agosecurity: update policy wrt CVE ID's
Victor Julien [Sat, 10 Feb 2024 09:41:05 +0000 (10:41 +0100)] 
security: update policy wrt CVE ID's

To match that we'll now request CVE ID's ourselves as well,
and we can do it for reported issues as well.

See also:
https://forum.suricata.io/t/security-new-cve-policy/4473

17 months agodoc: mention the limited number of RX/TX descriptors on Intel NICs
Lukas Sismis [Sun, 11 Feb 2024 20:14:08 +0000 (21:14 +0100)] 
doc: mention the limited number of RX/TX descriptors on Intel NICs

Ticket: 6748