Lukas Sismis [Mon, 9 Dec 2024 15:07:57 +0000 (16:07 +0100)]
dpdk: set ice PMD RSS key length to 52 bytes for all DPDK versions
ICE driver (Intel E810 NIC) requires/supports 52-byte long RSS key.
The 52 byte key length was mandatory from DPDK 23.11 when Suricata
was starting with independently configured ice PMD.
However, Suricata failed to start when ice PMD was part of
net_bonding PMD, requiring 52 byte RSS key even in DPDK versions
lower than 23.11. Since the support for the longer key is present
since DPDK 19.11 the key is set to 52 bytes for all versions.
Similar keywords use `isnotset`, while `flowint` only accepted `notset`
Opted to change the code, not only the regex, to keep the underlying
code also following the same patterns.
Victor Julien [Tue, 3 Dec 2024 15:55:38 +0000 (16:55 +0100)]
eve/alert: enrich decoder event
Default decoder event alert was very sparse, not even logging packet
type and pcap_cnt. Expand support for this record type. It will be more
useful with the ethernet headers and packet field, but these are still
disabled by default.
Jason Ish [Wed, 4 Dec 2024 17:28:17 +0000 (11:28 -0600)]
requires: add option to ignore unknown requirements
The new behavior in 8, and backported is to treat unknown requirements
as unsatisfied requirements.
For 7.0.8, add a configuration option, "ignore-unknown-requirements"
to completely ignore unknown requirements, effectively treating them
as available.
Jason Ish [Wed, 20 Nov 2024 16:46:38 +0000 (10:46 -0600)]
requires: treat unknown requires keywords as unmet requirements
For example, "requires: foo bar" is an unknown requirement, however
its not tracked, nor an error as it follows the syntax. Instead,
record these unknown keywords, and fail the requirements check if any
are present.
A future version of Suricata may have new requires keywords, for
example a check for keywords.
Jason Ish [Wed, 4 Dec 2024 18:22:28 +0000 (12:22 -0600)]
rust: update num-derive to 0.4.2
Includes Cargo.lock.in generated for just this single crate update
(minimal atomic update to keep Cargo.lock in sync with Cargo.toml).
This prevents the clippy warning:
508 | #[derive(FromPrimitive, Debug)]
| ^------------
| |
| `FromPrimitive` is not local
| move the `impl` block outside of this constant `_IMPL_NUM_FromPrimitive_FOR_IsakmpPayloadType`
509 | pub enum IsakmpPayloadType {
| ----------------- `IsakmpPayloadType` is not local
|
= note: the derive macro `FromPrimitive` defines the non-local `impl`, and may need to be changed
= note: the derive macro `FromPrimitive` may come from an old version of the `num_derive` crate, try updating your dependency with `cargo update -p num_derive`
= note: an `impl` is never scoped, even when it is nested inside an item, as it may impact type checking outside of that item, which can be the case if neither the trait or the self type are at the same nesting level as the `impl`
= note: items in an anonymous const item (`const _: () = { ... }`) are treated as in the same scope as the anonymous const's declaration for the purpose of this lint
= note: this warning originates in the derive macro `FromPrimitive` (in Nightly builds, run with -Z macro-backtrace for more info)
Jason Ish [Fri, 22 Nov 2024 21:26:49 +0000 (15:26 -0600)]
output-json: drop eve records that are too long
In the situation where the mem buffer cannot be expanded to the
requested size, drop the log message.
For each JSON log context, a warning will be emitted once with a partial
bit of the log record being dropped to identify what event types may be
leading to large log records.
This also fixes the call to MemBufferExpand which is supposed be
passed the amount to expand by, not the new size required.
Philippe Antoine [Tue, 15 Oct 2024 19:05:19 +0000 (21:05 +0200)]
detect/http: fix progress for headers keywords
Ticket: 7326
Having a lower progress than one where we actually can get
occurences of the multibuffer made prefilter
bail out too early, not having found a buffer in the multi-buffer
that matiched the prefilter.
For example, we registered http_request_header with progress 0
instad of progress HTP_REQUEST_HEADERS==2, and if the first
packet had only the request line, we would consider
that signatures with http_request_header as prefilter/fast_pattern
could not match for this transaction, even if they in fact
could have a later packet with matching headers.
Hence, we got false negatives, if http.request_header or
http.response_header was used as fast pattern, and if the request
or response came in multiple packets, and the first of these packets
did not have enough data (like only http request line),
and the next packets did have the matching data.
Noah Liu [Mon, 23 Sep 2024 03:07:47 +0000 (11:07 +0800)]
stream/reassembly: optimize GetBlock
Current GetBlock degrees the sbb search from rb tree to
line, which costs much cpu time, and could be replaced by
SBB_RB_FIND_INCLUSIVE. It reduces time complexity from
O(nlogn) to O(logn).
Jeff Lucovsky [Wed, 16 Oct 2024 14:37:20 +0000 (10:37 -0400)]
profiling: Correct profiling data array size
The profiling arrays are incorrectly sized by the number of thread
modules. Since they contain app-layer protocol data, they should be
sized by ALPROTO_MAX.
Commit changes are made to avoid possible memory leaks. If the parser
is initialized before configuration file checking, there was no deinit
call before function return. Do check config file existance and type
before YAML parser initialization, so we don't need to deinit parser
before exiting the function.
detect: add new_de_ctx release in case of errors in initialization
Detect engine tenant reloading function hasn't got engine release call
under error label, so it is possible memory leak in case of errors in
further new detect engine initialization.
Ilya Bakhtin [Sat, 31 Aug 2024 11:44:25 +0000 (13:44 +0200)]
detect: pseudo-packets inherit inspect flags from parent packet
Instead of inheriting from flow, because encrypted protocols like TLS
and SSH may have just set the flow flags to indicate rest of stream is
encrypted and does not need to run stream inspection. But inspection
still needs to be run detection on this last flushing packet.
So as to avoid quadratic complexity in libhtp.
Make the limit configurable from suricata.yaml,
and have an event when network traffic goes over the limit.
Victor Julien [Fri, 20 Sep 2024 07:54:57 +0000 (09:54 +0200)]
stream: improve 3whs completed by ACK with data
If the ACK packet completing the 3whs is received, the stream engine will
transition to "established". However, the packet itself will not be tagged
as "established". This will only happen for the next packet after the 3whs,
so that `flow:established` only matches after the 3whs.
It is possible that the ACK packet completing the 3whs was lost. Since the
ACK packets themselves are not acknowledged, there will be no retransmission
of them. Instead, the next packet can have the expected ACK flag as well as
data.
This case was mishandled in a subtle way. The stream engine state transition
was done correctly, as well as the data handling and app-layer updates.
However, the packet itself was not tagged as "established", which meant
that `flow:established` would not yet match.
This patch detects this case and tags the packet as established if ACK
with data is received that completes the 3whs.
pgsql: trigger raw stream reassembly at tx completion
Once we are tracking tx progress per-direction for PGSQL, we can trigger
the raw stream reassembly, for detection purposes, as soon as the
transactions are completed in the given direction.
PGSQL's current implementation tracks the transaction progress without
taking into consideration flow direction, and also has indirections
that make it harder to understand how the progress is tracked, as well
as when a request or response is actually complete.
This patch introduces tracking such progress per direction and adds
completion status per direction, too. This will help when triggering
raw stream reassembly or for unidirectional transactions, and may be
useful when we implement sub-protocols that can have multiple requests
per transaction, as well.
CancelRequests and TerminationRequests are examples of unidirectional
transactions. There won't be any responses to those requests, so we can
also mark the response side as done, and set their transactions as
completed.
Victor Julien [Wed, 31 Jul 2024 11:58:29 +0000 (13:58 +0200)]
dcerpc: don't reuse completed tx
In the DCERPC over TCP pcap, logging and rule matching is disrupted by adding a simple rule:
alert tcp any any -> any any (flow:to_server,established; \
dce_iface:5d2b62aa-ee0a-4a95-91ae-b064fdb471fc; dce_opnum:1; \
dce_stub_data; content:"|42 77 4E 6F 64 65 49 50 2E 65 78 65 20|"; \
content:!"|00|"; within:100; distance:97; sid:1; rev:1; )
Works: alert + 3 dcerpc records.
But when adding a trivial rule:
alert tcp any any -> any any (flow:to_server,established; \
dce_iface:5d2b62aa-ee0a-4a95-91ae-b064fdb471fc; dce_opnum:1; \
dce_stub_data; content:"|42 77 4E 6F 64 65 49 50 2E 65 78 65 20|"; \
content:!"|00|"; within:100; distance:97; sid:1; rev:1; )
alert tcp any any -> any any (dsize:3; sid:2; rev:1; )
The alert for sid:1 disappears and also there is one dcerpc event less.
In the single rule case we can aggressively free the transactions, as there
is only an sgh in the toserver direction.
This means that when we encounter the 2nd REQUEST, the first 2 transactions
have already been processed and freed. So for the 2nd REQUEST we open a new
TX and run inspection and logging on it.
When the 2nd rule is added, it adds toclient sgh as well. This means that we
will now slightly delay the freeing of the transactions.
As a consequence we still have the TX for the first REQUEST when the 2nd REQUEST
is parsed. This leads to the 2nd REQUEST re-using the TX. Since the TX is
already marked as inspected, it means the toserver rule now no longer matches.
Also we're not logging this TX correctly now.
This commit fixes the issue by not "finding" a TX that as already been
marked complete in the search direction.
Philippe Antoine [Thu, 12 Sep 2024 11:07:48 +0000 (13:07 +0200)]
frames: do not only rely on FRAME_STREAM_ID
As stream frame is not always created,
hence the first frame is not always a stream frame :
If stream frame is not enabled, it does not get created,
and other enabled frames may be created first.
See use of FrameConfigTypeIsEnabled
This resulted that this other frame got its length updated
on stream end, which led to false positives.
Also checking FRAME_STREAM_TYPE is more consistent.
Not a clean cherry-pick as AppLayerFrameGetLastOpenByType
does not exist in main7
warning: first doc comment paragraph is too long
--> src/detect/iprep.rs:57:1
|
57 | / /// value matching is done use `DetectUintData` logic.
58 | | /// isset matching is done using special `DetectUintData` value ">= 0"
59 | | /// isnotset matching bypasses `DetectUintData` and is handled directly
60 | | /// in the match function (in C).
| |_
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#too_long_first_doc_paragraph
= note: `#[warn(clippy::too_long_first_doc_paragraph)]` on by default
help: add an empty line
warning: you seem to be trying to use `match` for destructuring a single pattern. Consider using `if let`
--> src/dcerpc/log.rs:36:33
|
36 | DCERPC_TYPE_BIND => match &state.bind {
| _________________________________^
37 | | Some(bind) => {
38 | | jsb.open_array("interfaces")?;
39 | | for uuid in &bind.uuid_list {
... |
51 | | None => {}
52 | | },
| |_____________^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#single_match
= note: `#[warn(clippy::single_match)]` on by default
Philippe Antoine [Wed, 31 Jul 2024 12:15:14 +0000 (14:15 +0200)]
rust/ike: fix collapsible_match clippy warning
warning: this `match` can be collapsed into the outer `match`
help: the outer pattern can be modified to include the inner pattern
(cherry picked from commit 42e5e556e59fcd10efa89fcc75ad9f081ee25e93)
Philippe Antoine [Wed, 31 Jul 2024 12:10:17 +0000 (14:10 +0200)]
rust: fix byte_char_slices clippy warnings
warning: can be more succinctly written as a byte str
--> src/mime/smtp.rs:762:37
|
762 | mime_smtp_find_url_strings(ctx, &[b'\n']);
| ^^^^^^^^ help: try: `b"\n"`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#byte_char_slices
= note: `#[warn(clippy::byte_char_slices)]` on by default
Sascha Steinbiss [Wed, 14 Aug 2024 08:11:48 +0000 (10:11 +0200)]
userguide: fix spelling of `security_result` EVE field
This ensures that the correct spelling of the `security_result` EVE
field for RFB (as opposed to `security-result`) is also reflected in the
documentation.
rust/rfb: use consistent key name for security_result
A typo caused a slightly different key (`security-result`) to be used
for the case in which the result was `FAIL`. This commit addresses this
by ensuring the same string is used for all cases.
Eric Leblond [Fri, 19 Apr 2024 15:07:48 +0000 (17:07 +0200)]
datasets: fix parsing of ip4 in ip6
The lookup function was not taking into account that we can have
an IPv4 or an IPv6 address as parameters and that this addresses
need to be converted to Suricata internal storage.
By using the already defined dedicated parsing function, we are
fixing the issue.
It was brought to my attention by GLongo that Pgsql parser handled eof
diffrently for requests and responses, and apparently there isn't a good
reason for such a difference therefore, apply same logic used for
rs_pgsql_parse_request for checking for eof when parsing a response.
pgsql/logger: open json object from logger function
Before, the JsonBuilder object for the pgsql event was being created
from the C-side function that actually called the Rust logger.
This resulted that if another module - such as the Json Alert called the
PGSQL logger, we wouldn't have the `pgsql` key present in the log output
- only its inner fields.
projects / thirdparty / suricata.git / log
