]> git.ipfire.org Git - thirdparty/suricata.git/log
thirdparty/suricata.git
15 months agojsonbuilder: fix serialization of nan and inf 10774/head
Jason Ish [Fri, 5 Apr 2024 14:35:36 +0000 (08:35 -0600)] 
jsonbuilder: fix serialization of nan and inf

When outputting a float, check if its infinity, or not a number and
output a null instead.

Using a null was chosen as this is what serde_yaml, Firefox, Chrome,
Node, etc. do.

Ticket: #6921

16 months agohost-info: remove pcre2_substring_list_free use 10768/head
Victor Julien [Thu, 4 Apr 2024 15:51:48 +0000 (17:51 +0200)] 
host-info: remove pcre2_substring_list_free use

Function prototype has changed in a recent release. Rather than dealing
with detecting that, fall back to our regular pattern of using
pcre2_substring_copy_bynumber().

Bug: #6918.

16 months agogithub-actions: bump github/codeql-action from 3.24.6 to 3.24.9
dependabot[bot] [Mon, 1 Apr 2024 19:39:26 +0000 (19:39 +0000)] 
github-actions: bump github/codeql-action from 3.24.6 to 3.24.9

Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.24.6 to 3.24.9.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Commits](https://github.com/github/codeql-action/compare/v3.24.6...v3.24.9)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
16 months agogithub-ci: set checkout directory as safe before running git commands 10767/head
Jason Ish [Tue, 2 Apr 2024 17:04:08 +0000 (11:04 -0600)] 
github-ci: set checkout directory as safe before running git commands

While the checkout job appears to do this, it is done with a different
version of git which seems to be the cause for it not having an effect
when doing manual git operations from within a job.

Also removes duplicate checkout statements in Windows builds.

16 months agoconfigure: export release date for documentation
Jason Ish [Mon, 1 Apr 2024 16:37:49 +0000 (10:37 -0600)] 
configure: export release date for documentation

Sphinx embeds a date in the generated man pages, and to provide
reproducible builds this date needs to be provided to Sphinx,
otherwise it will use the current date.

If building from Git, extract the date from the most recent commit. In
a release, this commit would be the commit that sets the version so is
accurate.

If .git does not exist, use the most recent data found in the
ChangeLog.

The ChangeLog is not used when building from git, as the main/master
branch may not have recent enough timestamps.

This should provide a consistent date when re-building the
distribution from the same non-git archive, or from the same git
commit.

Ticket: #6911

16 months agodocs/userguide: use a consistent date for reproducible builds
Jason Ish [Mon, 1 Apr 2024 16:35:39 +0000 (10:35 -0600)] 
docs/userguide: use a consistent date for reproducible builds

By default, when Sphinx generates the man pages, the current date will
be embedded in them. This can be set to a specific date with the
"today" variable. Typically the date embedded in manpages in the
release date.

To achieve this, attempt to use the environment variable, RELEASE_DATE
to set the "today" variable, reverting back to the empty string if not
set. It is up to our build system to properly set this date.

Ticket: #6911

16 months agoconfigure: don't check ./revision, it never exists
Jason Ish [Tue, 2 Apr 2024 15:41:08 +0000 (09:41 -0600)] 
configure: don't check ./revision, it never exists

Stop checking the ./revision file for the git revision info, its never
created.

16 months agodocs/conf.py: fix python escape warning
Jason Ish [Mon, 1 Apr 2024 16:34:45 +0000 (10:34 -0600)] 
docs/conf.py: fix python escape warning

/home/jason/oisf/dev/suricata/master/doc/userguide/conf.py:74: SyntaxWarning: invalid escape sequence '\('
  "AC_INIT\(\[suricata\],\s*\[(.*)?\]\)",

16 months agogithub-actions: bump codecov/codecov-action from 4.1.0 to 4.1.1 10745/head
dependabot[bot] [Mon, 1 Apr 2024 19:38:51 +0000 (19:38 +0000)] 
github-actions: bump codecov/codecov-action from 4.1.0 to 4.1.1

Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 4.1.0 to 4.1.1.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/54bcd8715eee62d40e33596ef5e8f0f48dbbccab...c16abc29c95fcf9174b58eb7e1abf4c866893bc8)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
16 months agosip: convert transaction list to vecdeque 10720/head
Philippe Antoine [Sun, 24 Mar 2024 20:31:22 +0000 (21:31 +0100)] 
sip: convert transaction list to vecdeque

Ticket: 6891

So as to avoid quadratic complexity on tx cleanup with SIP/TCP
that can create many transactions in one go.

16 months agodetect/port: handle range and upper boundary ports
Shivani Bhardwaj [Mon, 25 Mar 2024 13:38:31 +0000 (19:08 +0530)] 
detect/port: handle range and upper boundary ports

So far, if a port was found to be single which was earlier a part of the
range, port + 1 was added to the list to honor the range that it was a
part of. But, this is incorrect in case the port is 65535 or if the port
was found to be of range when it was earlier a single port.

Bug 6896

16 months agoexamples/capture: fix the slot
Jason Ish [Fri, 22 Mar 2024 19:47:11 +0000 (13:47 -0600)] 
examples/capture: fix the slot

Use slot->slot_next, not the slot as passed in.

16 months agosuricata: expose and break out configuration loading
Jason Ish [Mon, 4 Mar 2024 23:08:08 +0000 (17:08 -0600)] 
suricata: expose and break out configuration loading

Expose LoadYamlConfig as SCLoadYamlConfig and remove it from
SuricataInit. This is required to allow the library user the ability
customize the loading of the configuration, for example doing some
programmatic configuration then loading a configuration file.

16 months agosmtp: fix configuration node creation
Jason Ish [Mon, 4 Mar 2024 21:59:09 +0000 (15:59 -0600)] 
smtp: fix configuration node creation

A configuration sequence node needs to have name, which is just its
index in the sequence.

Discovered by calling ConfDump() after Suricata was fully initialized.

16 months agosuricata: move SuricataMain into main()
Jason Ish [Mon, 26 Feb 2024 22:45:07 +0000 (16:45 -0600)] 
suricata: move SuricataMain into main()

Move the contents of SuricataMain into the `main()` function found in
main.c. This forces the Suricata application to bootstrap and run
Suricata through the same interfaces as a library user might do.

Required exposing StartInternalRunMode as SCStartInternalRunmode. Its
arguable whether those "actions" belong in the library or just the
application, but I think that is separation we can look at later.

For now the lib example and Suricata's own main are the same, however
the example will probably extend more into programmatically
configuring Suricata or dynamically registering a runmode, which
doesn't really belong the main Suricata application.

16 months agosuricata: expose FinalizeRunMode and ParseCommandLine
Jason Ish [Mon, 26 Feb 2024 22:29:25 +0000 (16:29 -0600)] 
suricata: expose FinalizeRunMode and ParseCommandLine

Expose the functions FinalizeRunMode and ParseCommandLine to library
users, renaming with the `SC` prefix in the process.

This involves moving "application" level details from SuricataInit
into SuricataMain, as parsing command line options should be opt-in
for a library user.

16 months agowindows: remove argc, argv from SCServiceRemove
Jason Ish [Mon, 26 Feb 2024 22:04:32 +0000 (16:04 -0600)] 
windows: remove argc, argv from SCServiceRemove

Not used.

16 months agosuricata: remove argv from FinalizeRunMode
Jason Ish [Mon, 26 Feb 2024 21:59:59 +0000 (15:59 -0600)] 
suricata: remove argv from FinalizeRunMode

FinalizeRunMode can get the program name from suri->progname.

16 months agosuricata: remove instance from ParseCommandLine
Jason Ish [Mon, 26 Feb 2024 21:55:04 +0000 (15:55 -0600)] 
suricata: remove instance from ParseCommandLine

We want to be able to call ParseCommandLine from library users, but
currently library users don't have access to the `suricata` instance
type. Since this var is used other places as a global, use the global
one in ParseCommandLine as well.

Not ideal, but isolating SCInstance to a non-global will be another
challenge on its own.

16 months agosuricata: move WindowsInitService to SuricataMain
Jason Ish [Mon, 26 Feb 2024 21:39:56 +0000 (15:39 -0600)] 
suricata: move WindowsInitService to SuricataMain

Move WindowsInitService from SuricataInit() to SuricataMain(), as
initializing Suricata as a service is very specific to the application
and not something you'd want to happen in a library, and SuricataInit
is more common initialization for application and library usage.

16 months agoexamples/lib: replicate Suricata using the library
Jason Ish [Thu, 22 Feb 2024 20:27:50 +0000 (14:27 -0600)] 
examples/lib: replicate Suricata using the library

With more functions exposed via the library, a library user can now
replicate the Suricata "main" function.

16 months agosuricata: expose SuricataMainLoop and GlobalsDestroy
Jason Ish [Thu, 22 Feb 2024 20:25:33 +0000 (14:25 -0600)] 
suricata: expose SuricataMainLoop and GlobalsDestroy

Expose SuricataMainLoop and GlobalsDestroy so that SuricataMain can be
replicated by a library user of Suricata.

These removes the `suricata` instance as a function argument to some
of these functions, as the way we use it now, it serves no
purpose. However, it is a reminder that it should probably be
refactored to not be a global, as at some point it might be desirable
for to have multiple instances active without data sharing.

16 months agolibsuricata: reorganize SuricataMain code
Angelo Mirabella [Wed, 21 Feb 2024 10:05:58 +0000 (02:05 -0800)] 
libsuricata: reorganize SuricataMain code

Split SuricataMain code in smaller functions. This is a first step
towards running as a library.

16 months agodetect/port: fix grouping of ports w gaps 10697/head
Shivani Bhardwaj [Thu, 21 Mar 2024 08:45:39 +0000 (14:15 +0530)] 
detect/port: fix grouping of ports w gaps

If a single port happens before a range port, the port groups created
were incorrect. Fix it to use smarter range check.

For example, given,
80:80 - SGH1
100:120 - SGH2

Range created should be
80:80 - SGH1
100:120 - SGH2

Bug 6881

16 months agoeve/alert: fix validation check
Victor Julien [Wed, 20 Mar 2024 06:18:44 +0000 (07:18 +0100)] 
eve/alert: fix validation check

Bug: #6875.

16 months agorust/smb: fix clippy nightly warning 10691/head 10693/head
Philippe Antoine [Thu, 21 Mar 2024 15:15:53 +0000 (16:15 +0100)] 
rust/smb: fix clippy nightly warning

error: unnecessary use of `to_vec`
    --> src/smb/smb.rs:1048:62
     |
1048 |         let (name, is_dcerpc) = match self.guid2name_map.get(&guid.to_vec()) {
     |                                                              ^^^^^^^^^^^^^^ help: replace it with: `guid`
     |
     = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_to_owned
     = note: `#[deny(clippy::unnecessary_to_owned)]` implied by `#[deny(warnings)]`

And also other uses of to_vec() on already Vec

16 months agorust: fix clippy 1.77 warning 10690/head
Philippe Antoine [Thu, 21 Mar 2024 15:02:23 +0000 (16:02 +0100)] 
rust: fix clippy 1.77 warning

Ticket: 6883

error: field `0` is never read
  --> src/asn1/mod.rs:36:14
   |
36 |     BerError(Err<der_parser::error::BerError>),
   |     -------- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |     |
   |     field in this variant
   |

16 months agorust: add MSRV as rust-version
Victor Julien [Wed, 20 Mar 2024 10:02:44 +0000 (11:02 +0100)] 
rust: add MSRV as rust-version

Update github-actions to use it for the MSRV check.

16 months agoprofiling/rules: Improve dynamic rule handling 10667/head
Jeff Lucovsky [Sat, 16 Mar 2024 12:58:11 +0000 (08:58 -0400)] 
profiling/rules: Improve dynamic rule handling

Issue: 6861

Without this commit, disabling rule profiling via suricatasc's command
'ruleset-profile-stop' may crash because profiling_rules_entered becomes
negative.

This can happen because
- There can be multiple rules evaluated for a single packet
- Each rule is profiled individually.
- Starting profiling is gated by a configuration setting and rule
  profiling being active
- Ending profiling is gated by the same configuration setting and
  whether the packet was marked as profiling.

The crash can occur when a rule is being profiled and rule profiling
is then disabled after one at least one rule was profiled for the packet
(which marks the packet as being profiled).

In this scenario, the value of profiling_rules_entered was
not incremented so the BUG_ON in the end profiling macro trips
because it is 0.

The changes to fix the problem are:
- In the profiling end macro, gate the actions taken there by the same
  configuration setting and use the profiling_rues_entered (instead of
  the per-packet profiling flag). Since the start and end macros are
  tightly coupled, this will permit profiling to "finish" if started.
- Modify SCProfileRuleStart to only check the sampling values if the
  packet hasn't been marked for profiling already. This change makes all
  rules for a packet (once selected) to be profiled (without this change
  sampling is applied to each *rule* that applies to the packet.

16 months agossh: limit length for banner logs 10666/head
Philippe Antoine [Mon, 19 Feb 2024 15:57:55 +0000 (16:57 +0100)] 
ssh: limit length for banner logs

Ticket: 6770

16 months agossh: avoid quadratic complexity from long banner
Philippe Antoine [Thu, 22 Feb 2024 09:14:36 +0000 (10:14 +0100)] 
ssh: avoid quadratic complexity from long banner

Ticket: 6799

When we find an overlong banner, we get into the state just
waiting for end of line, and we just want to skip the bytes
until then.
Returning AppLayerResult::incomplete made TCP engine retain
the bytes and grow the buffer that we parsed again and again...

16 months agodetect: flush when setting no_inspection 10657/head
Philippe Antoine [Mon, 27 Nov 2023 16:07:21 +0000 (17:07 +0100)] 
detect: flush when setting no_inspection

Ticket: 6578

When a protocol such as SSH sets no_inspection, we still have to
flush the current streams and packets that contain clear-text
for detection.

16 months agodetect: update packet action on protocol change
Philippe Antoine [Tue, 3 Oct 2023 10:39:13 +0000 (12:39 +0200)] 
detect: update packet action on protocol change

Ticket: #6305

When running FlowWorkerStreamTCPUpdate, one of the dequeued packet
may set the flow action to drop, without updating the not-pseudo
packet action, as is done usually with a previous call to
FlowHandlePacketUpdate

16 months agomembuffer: annotate printf style function 10654/head
Victor Julien [Fri, 24 Nov 2023 16:06:20 +0000 (17:06 +0100)] 
membuffer: annotate printf style function

16 months agoeve/alert: break out of payload logging callback if buffer is full
Victor Julien [Fri, 24 Nov 2023 15:02:14 +0000 (16:02 +0100)] 
eve/alert: break out of payload logging callback if buffer is full

16 months agoeve/frame: break out of logging callback if buffer is full
Victor Julien [Fri, 24 Nov 2023 14:53:23 +0000 (15:53 +0100)] 
eve/frame: break out of logging callback if buffer is full

16 months agomembuffer: return bytes written
Victor Julien [Fri, 24 Nov 2023 15:10:16 +0000 (16:10 +0100)] 
membuffer: return bytes written

16 months agomembuffer: use buffer pointer as flexible array member
Victor Julien [Fri, 24 Nov 2023 15:12:47 +0000 (16:12 +0100)] 
membuffer: use buffer pointer as flexible array member

16 months agomembuffer: turn complex macros into functions
Victor Julien [Fri, 24 Nov 2023 12:58:43 +0000 (13:58 +0100)] 
membuffer: turn complex macros into functions

For better readability and type checking.

16 months agounix-manager: add \n string to buffer using correct API call
Victor Julien [Fri, 24 Nov 2023 12:58:12 +0000 (13:58 +0100)] 
unix-manager: add \n string to buffer using correct API call

16 months agoeve/frame: improve frame payload logging
Victor Julien [Tue, 21 Nov 2023 15:27:16 +0000 (16:27 +0100)] 
eve/frame: improve frame payload logging

Log using stream callback API, meaning that data will also
be logged if there are GAPs.

Also implement GAP indicators: '[123 bytes missing]'.

16 months agoeve/frames: pass membuffer to API
Victor Julien [Tue, 21 Nov 2023 13:24:12 +0000 (14:24 +0100)] 
eve/frames: pass membuffer to API

In preparation of stream logging changes.

16 months agoeve/alert: init membuffer size on missing config
Victor Julien [Thu, 23 Nov 2023 05:49:12 +0000 (06:49 +0100)] 
eve/alert: init membuffer size on missing config

Don't init buffer to 0 size but use the desired default of 4k.

16 months agoeve/alert: log payload directly from stream buffer
Victor Julien [Mon, 20 Nov 2023 09:57:38 +0000 (10:57 +0100)] 
eve/alert: log payload directly from stream buffer

This avoids looping over partly duplicate segments that cause
output data corruption by logging parts of the stream data multiple
times.

For data with GAPs now add a indicator '[4 bytes missing]' similar
to how Wireshark does it.

Bug: #6553.

16 months agoeve/frame: implement payload-buffer-size option
Victor Julien [Thu, 23 Nov 2023 05:49:41 +0000 (06:49 +0100)] 
eve/frame: implement payload-buffer-size option

Modeled after the same option in eve/alert. Defaults to 4k.

16 months agostream: const args for StreamReassembleLog
Victor Julien [Fri, 24 Nov 2023 10:10:35 +0000 (11:10 +0100)] 
stream: const args for StreamReassembleLog

Needed a workaround cast for RBTREE use.

16 months agoyaml: remove newline from error message
Victor Julien [Mon, 20 Nov 2023 08:31:40 +0000 (09:31 +0100)] 
yaml: remove newline from error message

16 months agogen/typo: Correct comment typo 10652/head
Jeff Lucovsky [Mon, 11 Mar 2024 18:59:38 +0000 (14:59 -0400)] 
gen/typo: Correct comment typo

16 months agoflow: Swap thread_ids
Jeff Lucovsky [Mon, 11 Mar 2024 18:58:07 +0000 (14:58 -0400)] 
flow: Swap thread_ids

Issue: 6835

When swapping the flow's direction, also swap the thread_ids.

This should help with the issues identified in
https://redmine.openinfosecfoundation.org/issues/2725

16 months agoflow/inject: Ensure initialized thread value used
Jeff Lucovsky [Mon, 11 Mar 2024 18:57:16 +0000 (14:57 -0400)] 
flow/inject: Ensure initialized thread value used

Issue: 6835

When injecting a flow, ensure that the selected thread_id has been
initialized. When a flow is picked up midstream, the initialized thread
can be the second thread element.

16 months agoci: update ubuntu22.04 builds with clang14+asan
Philippe Antoine [Thu, 14 Mar 2024 08:00:15 +0000 (09:00 +0100)] 
ci: update ubuntu22.04 builds with clang14+asan

using a workround about ASLR

16 months agodpdk: fix typo in the struct member name
Lukas Sismis [Thu, 14 Mar 2024 11:49:14 +0000 (12:49 +0100)] 
dpdk: fix typo in the struct member name

16 months agodpdk: refactor the main packet loop into smaller functions
Lukas Sismis [Tue, 12 Mar 2024 22:24:07 +0000 (23:24 +0100)] 
dpdk: refactor the main packet loop into smaller functions

16 months agodpdk: only close the port when workers are synchronized
Lukas Sismis [Sat, 2 Mar 2024 17:15:16 +0000 (18:15 +0100)] 
dpdk: only close the port when workers are synchronized

When Suricata was running in IPS mode and received a signal to stop,
the first worker of every interface/port stopped the port and
proactively stopped the peered interface as well.
This was done to be as accurate with port stats as possible.
However, in a highly active scenarios (lots of packets moving around)
the peered workers might still be in the process of a packet
release operation. These workers would then attempt to transmit
on a stopped interface - resulting in an errorneous operation.

Instead, this patch proposes a worker synchronization of the given
port. After these workers are synchronized, it is known that no packets
will be sent of the peered interface, therefore the first worker can
stop it. This however cannot be assumed about "its own" port as the
peered workers can still try to send the packets. Therefore, ports
are only stopped by the peered workers.

Ticket: #6790

16 months agohttp: event on request line missing protocol
Philippe Antoine [Mon, 11 Mar 2024 10:18:34 +0000 (11:18 +0100)] 
http: event on request line missing protocol

Ticket: 6856

16 months agodevguide: add an upgrade section
Jason Ish [Mon, 11 Mar 2024 23:23:33 +0000 (17:23 -0600)] 
devguide: add an upgrade section

Add an upgrade section to the devguide. This should cover any changes
to APIs that users might be using from plugins or as a library user.

16 months agoeve/filetype: reorder fields to match lifecycle
Jason Ish [Mon, 11 Mar 2024 23:14:30 +0000 (17:14 -0600)] 
eve/filetype: reorder fields to match lifecycle

Enhances readability.

16 months agoeve/filetypes: use more const
Jason Ish [Mon, 11 Mar 2024 23:13:25 +0000 (17:13 -0600)] 
eve/filetypes: use more const

16 months agoeve/filetype: ThreadDeinit can return void
Jason Ish [Mon, 11 Mar 2024 23:06:50 +0000 (17:06 -0600)] 
eve/filetype: ThreadDeinit can return void

Change ThreadDeinit to return void instead of an int, there is nothing
to be done on success or failure.

16 months agooutput-eve: doxygen docs for SCEveFileType
Jason Ish [Mon, 11 Mar 2024 22:59:25 +0000 (16:59 -0600)] 
output-eve: doxygen docs for SCEveFileType

Add documentation for the SCEveFileType in Doxygen format.

16 months agodoxygen: document the examples directory
Jason Ish [Mon, 11 Mar 2024 22:59:14 +0000 (16:59 -0600)] 
doxygen: document the examples directory

16 months agoeve/filetypes: common init for threaded and non-threaded
Jason Ish [Fri, 8 Mar 2024 06:23:25 +0000 (00:23 -0600)] 
eve/filetypes: common init for threaded and non-threaded

In 7.0 if EVE was non-threaded, the ThreadInit for the filetype was
not called meaning that the filetype author had to handle the threaded
and non-threaded cases.

To simplify this, if non-threaded, still call ThreadInit (and
ThreadDeinit) once with a thread_id of 0. This should simplify
authoring EVE filetype plugins.

16 months agoeve: rename plugin to filetypes
Jason Ish [Thu, 7 Mar 2024 22:01:48 +0000 (16:01 -0600)] 
eve: rename plugin to filetypes

EVE filetypes are not always plugins, for example, null and syslog
that are built-in filetypes.

16 months agoplugins: remove conf.h from suricata-plugin.h
Jason Ish [Thu, 7 Mar 2024 21:40:03 +0000 (15:40 -0600)] 
plugins: remove conf.h from suricata-plugin.h

Remove "conf.h" from suricata-plugin.h as its not needed by that
header. However, some other files became transitively dependent on
through other includes, so fix those up.

16 months agoeve/filetypes: remove from plugin context
Jason Ish [Thu, 7 Mar 2024 21:33:28 +0000 (15:33 -0600)] 
eve/filetypes: remove from plugin context

Remove EVE filetypes from plugin context as they are not only used
from plugins. Plugins allow user code to register filetypes, but we
also have internal file types that use this api including the null
output and syslog.  Additionally library users can use this API to
register filetypes, and they are not plugins.

Ideally this code would go in "output-json.[ch]" as the "primary" eve
API, however there are currently some include circular include issues
there, so start new cleaned up EVE API in "output-eve.[ch]" which is
"clean" with respect to includes, and as we cleanup existing EVE API for
"public" use, it can be moved here.

Ticket: #6838

16 months agoexample/plugin: Use ThreadId
Jeff Lucovsky [Sun, 7 Jan 2024 14:34:45 +0000 (09:34 -0500)] 
example/plugin: Use ThreadId

16 months agooutput/plugin: Use Suri thread-id for plugins
Jeff Lucovsky [Mon, 16 Oct 2023 14:43:27 +0000 (10:43 -0400)] 
output/plugin: Use Suri thread-id for plugins

Issue: 6408

Use the Suricata thread id for plugin thread initialization to give the
plugin a better correlating factor to the actual Suricata threads.

16 months agompm/ac: minor test cleanups
Victor Julien [Tue, 16 Jan 2024 15:33:53 +0000 (16:33 +0100)] 
mpm/ac: minor test cleanups

16 months agompm/ac: add endswith test
Victor Julien [Tue, 16 Jan 2024 15:40:25 +0000 (16:40 +0100)] 
mpm/ac: add endswith test

16 months agompm/ac: implement endswith
Victor Julien [Mon, 15 Jan 2024 19:42:28 +0000 (20:42 +0100)] 
mpm/ac: implement endswith

When a pattern is using endswith, only consider it a match when it
is the end of the data.

Ticket: #6852.

16 months agompm: register algo features
Victor Julien [Tue, 19 Dec 2023 10:42:42 +0000 (11:42 +0100)] 
mpm: register algo features

This is so patterns can reply on mpm match meaning a full match.

Not yet used.

16 months agompm/ac: tidy up main search loop
Victor Julien [Sat, 13 Jan 2024 18:41:40 +0000 (19:41 +0100)] 
mpm/ac: tidy up main search loop

16 months agodetect: minor cleanup
Victor Julien [Tue, 9 Jan 2024 13:44:32 +0000 (14:44 +0100)] 
detect: minor cleanup

16 months agorequirements.txt: use suricata-update master 10629/head
Jason Ish [Mon, 11 Mar 2024 17:09:13 +0000 (11:09 -0600)] 
requirements.txt: use suricata-update master

16 months agonfq: stricter thread sync
Victor Julien [Mon, 4 Dec 2023 05:49:40 +0000 (06:49 +0100)] 
nfq: stricter thread sync

No longer update `Packet::flags` for tracking packet modifications,
as thread safety was not guaranteed.

Clearly separate between various kinds of `Packet::nfq_v` accesses for:
- mark
- mark_modified
- verdicted
These are either done under lock (Packet::persistent.tunnel_lock) or,
if the Packet is not part of a tunnel, not under lock.

This is safe as in all the related logic the Packet's tunnel state
is fixed and can no longer change.

16 months agodecode/tunnel: split verdict logic
Victor Julien [Mon, 4 Dec 2023 09:46:34 +0000 (10:46 +0100)] 
decode/tunnel: split verdict logic

Allows caller to take their own lock.

16 months agodecode/tunnel: improve tunnel handling
Victor Julien [Sun, 3 Dec 2023 18:37:31 +0000 (19:37 +0100)] 
decode/tunnel: improve tunnel handling

Give each packet explicit tunnel type `ttype`: none, root, child.

Assigning happens when a (tunnel) packet is set up and is thread
safe.

16 months agodecode/tunnel: move tunnel verdicted logic
Victor Julien [Mon, 4 Dec 2023 16:59:22 +0000 (17:59 +0100)] 
decode/tunnel: move tunnel verdicted logic

In preparation of cleaning up thread safety, move "verdicted"
logic out of Packet::flags. Unsafe writes to "flags" can potentially
have side effects.

16 months agonfq: remove obsolete comment
Victor Julien [Mon, 4 Dec 2023 13:53:00 +0000 (14:53 +0100)] 
nfq: remove obsolete comment

16 months agonfq: minor code cleanup
Victor Julien [Mon, 4 Dec 2023 08:18:00 +0000 (09:18 +0100)] 
nfq: minor code cleanup

16 months agonfq: use bool for verdicted packet var
Victor Julien [Sun, 3 Dec 2023 20:48:44 +0000 (21:48 +0100)] 
nfq: use bool for verdicted packet var

16 months agodefrag: match up v4 and v6 packet setup
Victor Julien [Sun, 3 Dec 2023 20:15:36 +0000 (21:15 +0100)] 
defrag: match up v4 and v6 packet setup

v4 was doing redundant recursion level setup.

v6 was missing PKT_REBUILT_FRAGMENT flag.

16 months agodecode/pppoe: skip useless loop unless debug 10615/head 10618/head
Philippe Antoine [Mon, 11 Mar 2024 14:58:24 +0000 (15:58 +0100)] 
decode/pppoe: skip useless loop unless debug

Looping over tags has no effects, except debug logging

16 months agodecode/pppoe: pointer cast consistency
Philippe Antoine [Mon, 19 Feb 2024 09:53:02 +0000 (10:53 +0100)] 
decode/pppoe: pointer cast consistency

Ticket: 6787

To do pointer arithmetic, we need to use uint8_t* pointer :
Pointer arithmetic in C is automatically scaled according
to the size of the data type.

Also simplifies the loop condition

16 months agosrc: make include guards more library friendly 10610/head
Jason Ish [Sat, 9 Mar 2024 18:12:43 +0000 (12:12 -0600)] 
src: make include guards more library friendly

Include guards for libraries should use a prefix that is meaningful for
the library to avoid conflicts with other user code. For Suricata, use
SURICATA.

Additionally, remove the pattern of leading and trailing underscores as
these are reserved for the language implementation per the C and C++
standards.

16 months agodetect/port: handle single port that is range too 10608/head
Shivani Bhardwaj [Sat, 9 Mar 2024 04:21:57 +0000 (09:51 +0530)] 
detect/port: handle single port that is range too

If a port point is single but later on also a part of a range, it ends
up only creating the port groups for single points and not the range.
Fix it by adding the port next to current single one to unique points
and marking it a range port.

Bug 6843

16 months agodns: add dns.rcode keyword 10603/head
Hadiqa Alamdar Bukhari [Wed, 24 Jan 2024 12:10:49 +0000 (17:10 +0500)] 
dns: add dns.rcode keyword

dns.rcode matches the rcode header field in DNS messages
It's an unsigned integer
valid ranges = [0-15]
Does not support prefilter
Supports matches in both flow directions

Task #6621

16 months agodevguide: add chapter and short intro to libsuricata
Juliana Fajardini [Mon, 4 Mar 2024 16:12:08 +0000 (13:12 -0300)] 
devguide: add chapter and short intro to libsuricata

With this, we intend to make more users aware of this use case, and that
we are working towards this.

Related to
Task #2693

16 months agoutil/interval-tree: fix coverity warning 10600/head
Shivani Bhardwaj [Fri, 8 Mar 2024 08:36:31 +0000 (14:06 +0530)] 
util/interval-tree: fix coverity warning

Fix Coverity warning

** CID 1592992:  Incorrect expression  (COPY_PASTE_ERROR)
/src/util-port-interval-tree.c: 255 in SCPortIntervalFindOverlaps()

________________________________________________________________________________________________________
*** CID 1592992:  Incorrect expression  (COPY_PASTE_ERROR)
/src/util-port-interval-tree.c: 255 in SCPortIntervalFindOverlaps()
249                      * will be sorted, insert any new ports to the end of the list
250                      * and avoid walking the entire list */
251                     if (*list == NULL) {
252                         *list = new_port;
253                         (*list)->last = new_port;
254                     } else if (((*list)->last->port != new_port->port) &&
>>>     CID 1592992:  Incorrect expression  (COPY_PASTE_ERROR)
>>>     "port" in "(*list)->last->port2 != new_port->port" looks like a copy-paste error.
255                                ((*list)->last->port2 != new_port->port)) {
256                         DEBUG_VALIDATE_BUG_ON(new_port->port < (*list)->last->port);
257                         (*list)->last->next = new_port;
258                         new_port->prev = (*list)->last;
259                         (*list)->last = new_port;
260                     } else {

The code does not generate two port ranges that are same other than the
cases where port == port2 which is why it worked so far. Fix it.

Bug 6839

16 months agonetmap: Avoid HW errors when using pipes
Jeff Lucovsky [Thu, 7 Mar 2024 20:24:36 +0000 (15:24 -0500)] 
netmap: Avoid HW errors when using pipes

Issue: 6837

When using netmap pipes (with lb, for example), avoid direct hardware
related IOCTLs that will fail (not supported with pipes).

16 months agoschema: Add stats.capture and in_iface properties 10590/head
Arne Welzel [Tue, 20 Feb 2024 11:50:40 +0000 (12:50 +0100)] 
schema: Add stats.capture and in_iface properties

New suricata-verify test listens on loopback interface, resulting
in the capture and in_iface fields in the stats and event objects.

16 months agostats: Fix non-worker stats missing
Arne Welzel [Sat, 17 Feb 2024 17:19:27 +0000 (18:19 +0100)] 
stats: Fix non-worker stats missing

Commit b8b8aa69b49ac0dd222446c28d00a50f9fd7d716 used tm_name of the
first StatsRecord of a thread block as key for the "threads" object.
However, depending on the type of thread, tm_name can be NULL and would
result in no entry being included for that thread at all. This caused
non-worker metrics to vanish from the "threads" object in the
dump-counters output.

This patch fixes this by remembering the first occurrence of a valid
tm_name within the per-thread block and adds another unittest to
cover this scenario.

16 months agorust: update parser dependencies
Victor Julien [Thu, 29 Feb 2024 10:02:40 +0000 (11:02 +0100)] 
rust: update parser dependencies

Time locked to 0.3.20 to guarantee MSRV of 1.63.
Update snmp-parser to 0.10.0.
Update asn1-rs to 0.6.1.
Update kerberos-parser to 0.8.0.
Update x509-parser 0.16.0.
Update der-parser to 9.0.0.
Remove specific use of der-parser 6.

Ticket: #6817.
Ticket: #6818.

17 months agodetect/port: cleanup address artifacts 10569/head
Shivani Bhardwaj [Wed, 28 Feb 2024 15:14:04 +0000 (20:44 +0530)] 
detect/port: cleanup address artifacts

A lot of code uses variable names and comments derived from the code
about addresses, make them about port.

17 months agodetect/port: remove SigGroupHead* ops
Shivani Bhardwaj [Wed, 28 Feb 2024 14:29:04 +0000 (19:59 +0530)] 
detect/port: remove SigGroupHead* ops

The functions in detect-engine-port.c are only being used at the time of
parsing the ports from rules initially. Since there are no SGHs at that
point, remove the ops related to them too.

17 months agodetect: optimize sig_cnt setting
Victor Julien [Mon, 26 Feb 2024 11:17:15 +0000 (12:17 +0100)] 
detect: optimize sig_cnt setting

Utilize _popcnt64 where available.

17 months agodetect: optimize group head bitarray handling
Victor Julien [Mon, 26 Feb 2024 10:28:53 +0000 (11:28 +0100)] 
detect: optimize group head bitarray handling

During startup large rulesets use a lot of large bitarrays, that
are frequently merged (OR'd).

Optimize this using SSE2 _mm_or_si128.

17 months agodetect: prepare for SIMD optimizations
Victor Julien [Mon, 26 Feb 2024 09:52:09 +0000 (10:52 +0100)] 
detect: prepare for SIMD optimizations

Make rule group head bitarray 16 bytes aligned and padded to 16 bytes
boundaries to assist SIMD operations in follow up commits.

17 months agodetect/port: use qsort instead of insert sort
Victor Julien [Mon, 26 Feb 2024 16:08:21 +0000 (21:38 +0530)] 
detect/port: use qsort instead of insert sort

Instead of using in place insertion sort on linked list based on two
keys, convert the linked list to an array, perform sorting on it using
qsort and convert it back to a linked list. This turns out to be much
faster.

Ticket #6795

17 months agodetect/port: merge port ranges for same signatures
Shivani Bhardwaj [Wed, 21 Feb 2024 06:42:30 +0000 (12:12 +0530)] 
detect/port: merge port ranges for same signatures

To avoid getting multiple entries in the final port list and to also
make the next step more efficient by reducing the size of the items to
traverse over.

Ticket 6792
Bug 6414