]> git.ipfire.org Git - thirdparty/suricata.git/log
thirdparty/suricata.git
2 years agodetect: use explicit rule types
Victor Julien [Wed, 24 May 2023 08:44:45 +0000 (10:44 +0200)] 
detect: use explicit rule types

Instead of using flags to indicate a rule type, use an explicit `type`
field.

This will make it more clean in code paths what paths a rule is taking,
and will allow easier debugging as well as analyzer output.

Define the following fields:

- SIG_TYPE_IPONLY: sig meets IP-only criteria and is handled by the IP-only
  engine.
- SIG_TYPE_PDONLY: sig inspects protocol detection results only.
- SIG_TYPE_DEONLY: sig inspects decoder events only.
- SIG_TYPE_PKT:    sig is inspected per packet.
- SIG_TYPE_PKT_STREAM: sig is inspected against either packet payload or
  stream payload.
- SIG_TYPE_STREAM: sig is inspected against the reassembled stream
- SIG_TYPE_APPLAYER: sig is inspected against an app-layer property, but not
  against a tx engine.
- SIG_TYPE_APP_TX: sig is inspected the tx aware inspection engine(s).

Ticket: #6085.

2 years agodetect/pcre: remove redundant applayer flag set
Victor Julien [Tue, 2 Aug 2022 08:56:30 +0000 (10:56 +0200)] 
detect/pcre: remove redundant applayer flag set

2 years agodetect: minor cleanup
Victor Julien [Fri, 29 Jul 2022 09:15:03 +0000 (11:15 +0200)] 
detect: minor cleanup

2 years agohttp: fix leak of normailzed uri 8918/head
Philippe Antoine [Tue, 23 May 2023 12:15:51 +0000 (14:15 +0200)] 
http: fix leak of normailzed uri

if tx_ud == NULL, still need to free alloced normailzed uri

2 years agorust: use 2021 edition 8907/head
Jason Ish [Mon, 22 May 2023 04:01:10 +0000 (22:01 -0600)] 
rust: use 2021 edition

With the MSRV being bumped to 1.62 for 7.0, we can move the edition up
to 2021.

2 years agostreaming: minor guards cleanup 8904/head
Victor Julien [Sat, 20 May 2023 19:51:50 +0000 (21:51 +0200)] 
streaming: minor guards cleanup

2 years agostreaming: fix region buf_offset update
Victor Julien [Sat, 20 May 2023 19:43:57 +0000 (21:43 +0200)] 
streaming: fix region buf_offset update

If region wasn't the first region, the buf_offset could get out of
sync.

Bug: #6041.

2 years agostreaming: stricter validation check
Victor Julien [Sat, 20 May 2023 19:16:33 +0000 (21:16 +0200)] 
streaming: stricter validation check

2 years agostreaming: fix intersect detection
Victor Julien [Sat, 20 May 2023 05:05:16 +0000 (07:05 +0200)] 
streaming: fix intersect detection

Update logic to always use data right edge.

Bug: #5834.

2 years agogithub: change codeowners back to team
Philippe Antoine [Thu, 18 May 2023 09:51:24 +0000 (11:51 +0200)] 
github: change codeowners back to team

as in master-6.0.x

2 years agofuzz: make targets more resitant to allocation failures 8891/head
Philippe Antoine [Wed, 17 May 2023 12:33:16 +0000 (14:33 +0200)] 
fuzz: make targets more resitant to allocation failures

2 years agoflowworker: simplify pseudo packet use 8888/head
Victor Julien [Mon, 15 May 2023 08:02:26 +0000 (10:02 +0200)] 
flowworker: simplify pseudo packet use

Pseudo packets originating in the flow worker do not need to leave the
flow worker. Putting those in the ThreadVars::decode_pq will make them
be evaluated by the next steps in the pipeline, but those will all
ignore pseudo packets.

Instead, this patch returns them to the packet pool, while still honoring
the IPS verdict logic.

2 years agoall: remove unused literals
Philippe Antoine [Fri, 7 Apr 2023 16:02:02 +0000 (18:02 +0200)] 
all: remove unused literals

2 years agorust/doc: restore comment with code example, but ignore 8879/head
Jason Ish [Fri, 12 May 2023 14:59:54 +0000 (08:59 -0600)] 
rust/doc: restore comment with code example, but ignore

Use backticks for proper markdown processing. As Rust code in
backticks is compiled, and this is a non-complete example, tag the
code sample to be ignored.

2 years agogithub-ci: check for new authors in pull-request
Jason Ish [Tue, 9 May 2023 20:42:30 +0000 (14:42 -0600)] 
github-ci: check for new authors in pull-request

On pull request, get a list of commit authors for the pull request and
compare to the list of authors in git master. If any differ, save to
new-authors.txt and upload this as an artifact.

As a workflow-run, download this artifact and if non-empty, add a
comment to the pull-request that new authors may be part of the pull
request.

This 2 step approach is because GitHub actions running in pull-request
context are not allowed to comment on the pull request, instead a
post-workflow workflow has been added that runs in the context of the
repo which can then comment on the pull request.

2 years agoprofiling/rules: minor fatal error cleanup
Victor Julien [Fri, 12 May 2023 14:05:20 +0000 (16:05 +0200)] 
profiling/rules: minor fatal error cleanup

2 years agoprofiling/rules: add BUG_ON to profile tracking
Victor Julien [Fri, 12 May 2023 12:21:37 +0000 (14:21 +0200)] 
profiling/rules: add BUG_ON to profile tracking

2 years agoprofiling/rules: simplify return code handling
Victor Julien [Fri, 12 May 2023 12:12:19 +0000 (14:12 +0200)] 
profiling/rules: simplify return code handling

2 years agoprofiling/rules: minor code cleanup
Victor Julien [Fri, 12 May 2023 12:09:10 +0000 (14:09 +0200)] 
profiling/rules: minor code cleanup

2 years agoprofiling/rules: use atomics for rule flag
Victor Julien [Fri, 12 May 2023 12:08:44 +0000 (14:08 +0200)] 
profiling/rules: use atomics for rule flag

2 years agoprofiling/rules: doc updates
Victor Julien [Fri, 12 May 2023 11:59:10 +0000 (13:59 +0200)] 
profiling/rules: doc updates

2 years agoprofiling/rules: reduce sync logic scope
Victor Julien [Fri, 12 May 2023 11:56:51 +0000 (13:56 +0200)] 
profiling/rules: reduce sync logic scope

Use a simple once a second scheme.

2 years agodoc: add rule profiling information
Eric Leblond [Fri, 11 Nov 2022 13:37:12 +0000 (14:37 +0100)] 
doc: add rule profiling information

2 years agoprofiling: socket command to control rules profiling
Eric Leblond [Mon, 15 Nov 2021 06:40:32 +0000 (07:40 +0100)] 
profiling: socket command to control rules profiling

This patch adds unix socket command to start and stop the collection
of stats when running in rules profiling mode.

2 years agoprofiling: set sample rate to power of 2
Eric Leblond [Thu, 26 Aug 2021 16:11:42 +0000 (18:11 +0200)] 
profiling: set sample rate to power of 2

For the rules profiling, we really want to limit the performance
impact to the maximum. So let's use an hash size that is a power
of 2. This will allow to not use the modulo operation that is
costly and simply use a single binary operator.

This code is only active for rules profiling so we are backward
compatible.

2 years agoprofiling: let 'ruleset-profile' send message
Eric Leblond [Mon, 23 Aug 2021 19:29:24 +0000 (21:29 +0200)] 
profiling: let 'ruleset-profile' send message

Let's send the profile output as an answer on the Unix socket.

2 years agoprofiling: add 'ruleset-profile' unix command
Eric Leblond [Fri, 11 Nov 2022 13:40:27 +0000 (14:40 +0100)] 
profiling: add 'ruleset-profile' unix command

This patch adds a new unix command that allows the user to trigger
a dump of the ruleset profile to the file without having to stop
Suricata.

This will be really useful to debug performance issue related to
signatures in production environment.

2 years agoprofiling: introduce rules profiling
Eric Leblond [Sun, 22 Aug 2021 21:55:42 +0000 (23:55 +0200)] 
profiling: introduce rules profiling

Performance measurement of rules is important on live Suricata
as bad rules can cause severe performance regression. This patch
introduces the --enable-profiling-rules that activate profiling
for the rules. This reduces the performance impact of full
profiling  and provide visiblity on the rules performance at
the same time.

2 years agodetect: http_client_body for HTTP2
Philippe Antoine [Mon, 3 Apr 2023 08:09:25 +0000 (10:09 +0200)] 
detect: http_client_body for HTTP2

By using the file.data logic

Ticket: #4067

2 years agodetect: http.response_body for HTTP2
Philippe Antoine [Mon, 3 Apr 2023 08:10:43 +0000 (10:10 +0200)] 
detect: http.response_body for HTTP2

Already using the file.data logic

Ticket: #4067

2 years agodetect: http_response_line for HTTP2
Philippe Antoine [Mon, 3 Apr 2023 08:36:25 +0000 (10:36 +0200)] 
detect: http_response_line for HTTP2

Ticket: #4067

Synthetized as HTTP/2 <STAT>\r\n

2 years agodetect: http_request_line support for HTTP2
Philippe Antoine [Mon, 3 Apr 2023 08:26:36 +0000 (10:26 +0200)] 
detect: http_request_line support for HTTP2

Ticket: #4067

Synthetized as <METHOD> <URI> HTTP/2\r\n

2 years agodetect: http_stat_msg supported for HTTP2
Philippe Antoine [Mon, 3 Apr 2023 07:59:18 +0000 (09:59 +0200)] 
detect: http_stat_msg supported for HTTP2

By providing a constant empty buffer

Ticket: #4067

2 years agodetect: support http.protocol for HTTP2
Philippe Antoine [Mon, 3 Apr 2023 07:55:35 +0000 (09:55 +0200)] 
detect: support http.protocol for HTTP2

Ticket: #4067

By having a synthetic constant HTTP/2 buffer

2 years agodpdk: support capture timeout support 8877/head
Victor Julien [Fri, 12 May 2023 14:53:58 +0000 (16:53 +0200)] 
dpdk: support capture timeout support

To not inject too many packets, simulate a 100ms timeout based on
af-packet's 100ms poll() timeout.

Bug: #6064.

2 years agorust/doc: wrap some code examples in backticks 8867/head
Jason Ish [Thu, 11 May 2023 21:49:59 +0000 (15:49 -0600)] 
rust/doc: wrap some code examples in backticks

2 years agorust/doc: fix doc compile issues
Victor Julien [Fri, 12 May 2023 05:04:17 +0000 (07:04 +0200)] 
rust/doc: fix doc compile issues

2 years agogithub-ci: fix rust check by fixing git directory permissions
Jason Ish [Thu, 11 May 2023 16:06:41 +0000 (10:06 -0600)] 
github-ci: fix rust check by fixing git directory permissions

Due to the update to AlmaLinux 9.2, a new version of git with stricter
permissions was introduced.  Mark the directory as safe as we have
done in other builds with newer versions of it.

2 years agodataset: Address compile-time error
Jeff Lucovsky [Thu, 11 May 2023 13:54:57 +0000 (09:54 -0400)] 
dataset: Address compile-time error

This commit fixes an issue with using a `in_addr` when an IPv6 structure
should be used.

2 years agocounters: make tcp stats independent of flow, ssn
Victor Julien [Tue, 25 Apr 2023 08:09:27 +0000 (10:09 +0200)] 
counters: make tcp stats independent of flow, ssn

Counters depended on availability of flow and tcp session, meaning
that 2 memcaps could affect the counters.

Bug: #5017.

2 years agostream: update no-flow checks
Victor Julien [Tue, 25 Apr 2023 08:22:11 +0000 (10:22 +0200)] 
stream: update no-flow checks

2 years agobuild: harden rust.yml permissions
Alex [Thu, 8 Dec 2022 18:19:17 +0000 (20:19 +0200)] 
build: harden rust.yml permissions

Signed-off-by: Alex <aleksandrosansan@gmail.com>
2 years agodoc/userguide: provide more RPM doc
Jason Ish [Mon, 13 Mar 2023 22:47:14 +0000 (16:47 -0600)] 
doc/userguide: provide more RPM doc

- Address the various RPM distributions
- User info
- Systemd info

Related issue: #5884

2 years agokrb5: update krb5_msg_type keyword docs
Eloy Pérez González [Thu, 28 Apr 2022 10:48:28 +0000 (12:48 +0200)] 
krb5: update krb5_msg_type keyword docs

2 years agokrb5: use req_type instead of msg_type to get request type
Eloy Pérez González [Thu, 28 Apr 2022 10:47:43 +0000 (12:47 +0200)] 
krb5: use req_type instead of msg_type to get request type

2 years agokrb5: set msg_type for KRB-ERROR messages to MessageType::KRB_ERROR
Eloy Pérez González [Thu, 28 Apr 2022 10:47:06 +0000 (12:47 +0200)] 
krb5: set msg_type for KRB-ERROR messages to MessageType::KRB_ERROR

2 years agokrb5: add AS-REQ and TGS-REQ transactions
Eloy Pérez González [Thu, 28 Apr 2022 10:40:56 +0000 (12:40 +0200)] 
krb5: add AS-REQ and TGS-REQ transactions

Fix bug in ticket #4529

2 years agodpdk: warn about processing segmented DPDK mbufs 8861/head
Lukas Sismis [Mon, 24 Apr 2023 16:04:42 +0000 (18:04 +0200)] 
dpdk: warn about processing segmented DPDK mbufs

Segmented mbufs should never happen in Suricata.
Mbuf segmentation divides the received packet into multiple
mbufs. This can happen when MTU of the NIC is larger than
the allocated mbufs in the mbuf mempool. As Suricata sets the size
of the mbuf to be slightly higher than the configured MTU, mbuf
segmentation should never happen in Suricata. This is especially
true, if Suricata runs as a primary process and configures the
packet source (NIC).

Processing segmented mbufs can lead to missed/false
(pattern-matching) detections as Suricata only inspects the first
segment of the packet. It can also lead to segfault if Suricata
moves the detection window out of the segment boundaries.

2 years agodpdk: query device stats only with one worker
Lukas Sismis [Tue, 18 Apr 2023 11:50:28 +0000 (13:50 +0200)] 
dpdk: query device stats only with one worker

Function rte_eth_stats_get is not thread-safe and
the result is only used by one thread.
Running with multiple workers led to very high values in rx_missed
counters (buffer-overflow-like behavior).

Ticket: #6006

2 years agodocs: refactor DPDK docs and add performance tuning section
Lukas Sismis [Thu, 4 May 2023 14:57:29 +0000 (16:57 +0200)] 
docs: refactor DPDK docs and add performance tuning section

Ticket: #5857
Ticket: #5858

2 years agodocs: wrap DPDK doc section at 80 chars
Lukas Sismis [Thu, 4 May 2023 14:05:50 +0000 (16:05 +0200)] 
docs: wrap DPDK doc section at 80 chars

2 years agodpdk: add configure option
Lukas Sismis [Mon, 27 Mar 2023 13:05:01 +0000 (15:05 +0200)] 
dpdk: add configure option

Ticket: #5859

2 years agodpdk: update version checking to DPDK-intended version checking API
Lukas Sismis [Thu, 23 Mar 2023 14:47:18 +0000 (15:47 +0100)] 
dpdk: update version checking to DPDK-intended version checking API

Ticket: #5937

2 years agodpdk: release mempool after the device is closed
Lukas Sismis [Thu, 23 Mar 2023 12:48:03 +0000 (13:48 +0100)] 
dpdk: release mempool after the device is closed

Ticket: #5936

2 years agodpdk: fix multibit comparison
Lukas Sismis [Wed, 22 Mar 2023 17:54:35 +0000 (18:54 +0100)] 
dpdk: fix multibit comparison

Ticket: #5925

2 years agodpdk: fix DPDK API change on NUMA/socket retrieval
Lukas Sismis [Tue, 21 Mar 2023 09:22:33 +0000 (10:22 +0100)] 
dpdk: fix DPDK API change on NUMA/socket retrieval

Ticket: #5923

2 years agothreshold: fixed example rules to match description 8858/head
Thomas Norheim [Fri, 3 Sep 2021 09:21:45 +0000 (11:21 +0200)] 
threshold: fixed example rules to match description

2 years agothreading: wait for flow housekeeping at shutdown
Victor Julien [Tue, 9 May 2023 13:54:02 +0000 (15:54 +0200)] 
threading: wait for flow housekeeping at shutdown

Flow house keeping can accumulate work that wasn't taken into account
during shutdown. This could lead to flows still in the flowworker
thread context when being it was freed, leading to missed work and
memory leaks.

This patch adds a new way of checking if a thread module is still
busy.

Bug: #6062.

2 years agoflowworker: flush local work queue on capture timeout
Victor Julien [Wed, 10 May 2023 05:02:27 +0000 (07:02 +0200)] 
flowworker: flush local work queue on capture timeout

Capture timeout means no packets have been received for some
time, so this is a good time to flush out all work.

2 years agoflowworker: rate limit flow queue processing
Victor Julien [Tue, 9 May 2023 12:56:33 +0000 (14:56 +0200)] 
flowworker: rate limit flow queue processing

Until this patch the logic the flow worker flow house keeping used was:
- at most 2 flows are handled per packet
- pseudo packets could flush the entire queue

This patch changes that. Pseudo packets are fairly common, and can lead
to packet stalls / latency spikes if the number of flows in the queue
is large.

It does that by adding a new packet type only used at shutdown, which
flushes out the queues completely. All other packets will now stick
to the 2 flow rate limit.

2 years agodatasets: bugfix to load ip types from yaml 8852/head
Scott Jordan [Tue, 9 May 2023 20:38:55 +0000 (16:38 -0400)] 
datasets: bugfix to load ip types from yaml

2 years agosrc: various comment spelling fixes 8840/head
Victor Julien [Mon, 8 May 2023 09:56:38 +0000 (11:56 +0200)] 
src: various comment spelling fixes

Thanks to Josh Soref.

2 years agodpdk: minor output fixup
Victor Julien [Mon, 8 May 2023 09:56:11 +0000 (11:56 +0200)] 
dpdk: minor output fixup

Thanks to Josh Soref.

2 years agodetect/parse: fix test spelling
Victor Julien [Mon, 8 May 2023 09:48:03 +0000 (11:48 +0200)] 
detect/parse: fix test spelling

Thanks to Josh Soref.

2 years agostream: fix test code spelling
Victor Julien [Mon, 8 May 2023 09:47:09 +0000 (11:47 +0200)] 
stream: fix test code spelling

Thanks to Josh Soref.

2 years agodpdk: fix variable spelling
Victor Julien [Mon, 8 May 2023 09:45:30 +0000 (11:45 +0200)] 
dpdk: fix variable spelling

Thanks to Josh Soref.

2 years agoscripts: fix bundle script spelling
Victor Julien [Mon, 8 May 2023 09:44:10 +0000 (11:44 +0200)] 
scripts: fix bundle script spelling

Thanks to Josh Soref.

2 years agorust: spelling fixes
Victor Julien [Mon, 8 May 2023 09:42:53 +0000 (11:42 +0200)] 
rust: spelling fixes

Thanks to Josh Soref.

2 years agocommandline: fix help output spelling
Victor Julien [Mon, 8 May 2023 09:41:47 +0000 (11:41 +0200)] 
commandline: fix help output spelling

Thanks to Josh Soref.

2 years agosnmp: fix spelling
Victor Julien [Mon, 8 May 2023 09:40:53 +0000 (11:40 +0200)] 
snmp: fix spelling

Thanks to Josh Soref.

2 years agolicense: fixup file headers
Victor Julien [Mon, 8 May 2023 09:38:05 +0000 (11:38 +0200)] 
license: fixup file headers

Thanks to Josh Soref.

2 years agodoc: spelling
Victor Julien [Mon, 8 May 2023 09:36:21 +0000 (11:36 +0200)] 
doc: spelling

Thanks to Josh Soref.

2 years agochangelog: spelling
Victor Julien [Mon, 8 May 2023 09:35:56 +0000 (11:35 +0200)] 
changelog: spelling

Thanks to Josh Soref.

2 years agodetect/csum: test cleanups
Victor Julien [Mon, 8 May 2023 09:16:24 +0000 (11:16 +0200)] 
detect/csum: test cleanups

2 years agodetect/content: test cleanups
Victor Julien [Mon, 8 May 2023 09:09:34 +0000 (11:09 +0200)] 
detect/content: test cleanups

2 years agowindivert: compile fixes 8834/head
Victor Julien [Sun, 7 May 2023 06:17:31 +0000 (08:17 +0200)] 
windivert: compile fixes

2 years agogithub-ci: add windivert build
Victor Julien [Sun, 7 May 2023 05:42:13 +0000 (07:42 +0200)] 
github-ci: add windivert build

2 years agooutput: target keys have port 8829/head
Eric Leblond [Tue, 30 Aug 2022 12:10:53 +0000 (14:10 +0200)] 
output: target keys have port

Update JSON schema to support signature with target keyword

2 years agodoc: explain where tls.store stores certificates
Philippe Antoine [Fri, 5 May 2023 15:00:11 +0000 (17:00 +0200)] 
doc: explain where tls.store stores certificates

By adding a reference/link to the doc about the suricata.yaml
config section pecifying the directory where the certificates
are stored

2 years agoutil/rule-vars: spelling
Victor Julien [Sat, 6 May 2023 05:22:48 +0000 (07:22 +0200)] 
util/rule-vars: spelling

2 years agocommon: spelling
Victor Julien [Sat, 6 May 2023 05:22:21 +0000 (07:22 +0200)] 
common: spelling

2 years agothreshold: spelling
Victor Julien [Sat, 6 May 2023 05:21:48 +0000 (07:21 +0200)] 
threshold: spelling

2 years agodebug: spelling
Victor Julien [Sat, 6 May 2023 05:21:33 +0000 (07:21 +0200)] 
debug: spelling

2 years agoredis: spelling
Victor Julien [Sat, 6 May 2023 05:20:12 +0000 (07:20 +0200)] 
redis: spelling

2 years agomime: spelling
Victor Julien [Sat, 6 May 2023 05:19:43 +0000 (07:19 +0200)] 
mime: spelling

2 years agounix-socket: spelling
Victor Julien [Sat, 6 May 2023 05:19:28 +0000 (07:19 +0200)] 
unix-socket: spelling

2 years agoreputation: spelling
Victor Julien [Sat, 6 May 2023 05:19:08 +0000 (07:19 +0200)] 
reputation: spelling

2 years agoippair: spelling
Victor Julien [Sat, 6 May 2023 05:18:51 +0000 (07:18 +0200)] 
ippair: spelling

2 years agohost: spelling
Victor Julien [Sat, 6 May 2023 05:18:40 +0000 (07:18 +0200)] 
host: spelling

2 years agostats: spelling
Victor Julien [Sat, 6 May 2023 05:18:27 +0000 (07:18 +0200)] 
stats: spelling

2 years agoradix: spelling
Victor Julien [Sat, 6 May 2023 05:18:08 +0000 (07:18 +0200)] 
radix: spelling

2 years agohash: spelling
Victor Julien [Sat, 6 May 2023 05:17:51 +0000 (07:17 +0200)] 
hash: spelling

2 years agofile: spelling
Victor Julien [Sat, 6 May 2023 05:17:21 +0000 (07:17 +0200)] 
file: spelling

2 years agoprofiling: spelling
Victor Julien [Sat, 6 May 2023 05:16:46 +0000 (07:16 +0200)] 
profiling: spelling

2 years agounittests: spelling
Victor Julien [Sat, 6 May 2023 05:15:52 +0000 (07:15 +0200)] 
unittests: spelling

2 years agoreference: spelling
Victor Julien [Sat, 6 May 2023 05:14:30 +0000 (07:14 +0200)] 
reference: spelling

2 years agoclassification: spelling
Victor Julien [Sat, 6 May 2023 05:13:23 +0000 (07:13 +0200)] 
classification: spelling

2 years agoutil/lua: spelling
Victor Julien [Sat, 6 May 2023 05:12:52 +0000 (07:12 +0200)] 
util/lua: spelling

2 years agohash/lookup3: spelling
Victor Julien [Sat, 6 May 2023 05:08:25 +0000 (07:08 +0200)] 
hash/lookup3: spelling

2 years agoutil/misc: spelling
Victor Julien [Sat, 6 May 2023 05:07:37 +0000 (07:07 +0200)] 
util/misc: spelling