Philippe Antoine [Tue, 15 Oct 2024 19:05:19 +0000 (21:05 +0200)]
detect/http: fix progress for headers keywords
Ticket: 7326
Having a lower progress than one where we actually can get
occurences of the multibuffer made prefilter
bail out too early, not having found a buffer in the multi-buffer
that matiched the prefilter.
For example, we registered http_request_header with progress 0
instad of progress HTP_REQUEST_HEADERS==2, and if the first
packet had only the request line, we would consider
that signatures with http_request_header as prefilter/fast_pattern
could not match for this transaction, even if they in fact
could have a later packet with matching headers.
Hence, we got false negatives, if http.request_header or
http.response_header was used as fast pattern, and if the request
or response came in multiple packets, and the first of these packets
did not have enough data (like only http request line),
and the next packets did have the matching data.
Noah Liu [Mon, 23 Sep 2024 03:07:47 +0000 (11:07 +0800)]
stream/reassembly: optimize GetBlock
Current GetBlock degrees the sbb search from rb tree to
line, which costs much cpu time, and could be replaced by
SBB_RB_FIND_INCLUSIVE. It reduces time complexity from
O(nlogn) to O(logn).
Jeff Lucovsky [Wed, 16 Oct 2024 14:37:20 +0000 (10:37 -0400)]
profiling: Correct profiling data array size
The profiling arrays are incorrectly sized by the number of thread
modules. Since they contain app-layer protocol data, they should be
sized by ALPROTO_MAX.
Commit changes are made to avoid possible memory leaks. If the parser
is initialized before configuration file checking, there was no deinit
call before function return. Do check config file existance and type
before YAML parser initialization, so we don't need to deinit parser
before exiting the function.
detect: add new_de_ctx release in case of errors in initialization
Detect engine tenant reloading function hasn't got engine release call
under error label, so it is possible memory leak in case of errors in
further new detect engine initialization.
Ilya Bakhtin [Sat, 31 Aug 2024 11:44:25 +0000 (13:44 +0200)]
detect: pseudo-packets inherit inspect flags from parent packet
Instead of inheriting from flow, because encrypted protocols like TLS
and SSH may have just set the flow flags to indicate rest of stream is
encrypted and does not need to run stream inspection. But inspection
still needs to be run detection on this last flushing packet.
So as to avoid quadratic complexity in libhtp.
Make the limit configurable from suricata.yaml,
and have an event when network traffic goes over the limit.
Victor Julien [Fri, 20 Sep 2024 07:54:57 +0000 (09:54 +0200)]
stream: improve 3whs completed by ACK with data
If the ACK packet completing the 3whs is received, the stream engine will
transition to "established". However, the packet itself will not be tagged
as "established". This will only happen for the next packet after the 3whs,
so that `flow:established` only matches after the 3whs.
It is possible that the ACK packet completing the 3whs was lost. Since the
ACK packets themselves are not acknowledged, there will be no retransmission
of them. Instead, the next packet can have the expected ACK flag as well as
data.
This case was mishandled in a subtle way. The stream engine state transition
was done correctly, as well as the data handling and app-layer updates.
However, the packet itself was not tagged as "established", which meant
that `flow:established` would not yet match.
This patch detects this case and tags the packet as established if ACK
with data is received that completes the 3whs.
pgsql: trigger raw stream reassembly at tx completion
Once we are tracking tx progress per-direction for PGSQL, we can trigger
the raw stream reassembly, for detection purposes, as soon as the
transactions are completed in the given direction.
PGSQL's current implementation tracks the transaction progress without
taking into consideration flow direction, and also has indirections
that make it harder to understand how the progress is tracked, as well
as when a request or response is actually complete.
This patch introduces tracking such progress per direction and adds
completion status per direction, too. This will help when triggering
raw stream reassembly or for unidirectional transactions, and may be
useful when we implement sub-protocols that can have multiple requests
per transaction, as well.
CancelRequests and TerminationRequests are examples of unidirectional
transactions. There won't be any responses to those requests, so we can
also mark the response side as done, and set their transactions as
completed.
Victor Julien [Wed, 31 Jul 2024 11:58:29 +0000 (13:58 +0200)]
dcerpc: don't reuse completed tx
In the DCERPC over TCP pcap, logging and rule matching is disrupted by adding a simple rule:
alert tcp any any -> any any (flow:to_server,established; \
dce_iface:5d2b62aa-ee0a-4a95-91ae-b064fdb471fc; dce_opnum:1; \
dce_stub_data; content:"|42 77 4E 6F 64 65 49 50 2E 65 78 65 20|"; \
content:!"|00|"; within:100; distance:97; sid:1; rev:1; )
Works: alert + 3 dcerpc records.
But when adding a trivial rule:
alert tcp any any -> any any (flow:to_server,established; \
dce_iface:5d2b62aa-ee0a-4a95-91ae-b064fdb471fc; dce_opnum:1; \
dce_stub_data; content:"|42 77 4E 6F 64 65 49 50 2E 65 78 65 20|"; \
content:!"|00|"; within:100; distance:97; sid:1; rev:1; )
alert tcp any any -> any any (dsize:3; sid:2; rev:1; )
The alert for sid:1 disappears and also there is one dcerpc event less.
In the single rule case we can aggressively free the transactions, as there
is only an sgh in the toserver direction.
This means that when we encounter the 2nd REQUEST, the first 2 transactions
have already been processed and freed. So for the 2nd REQUEST we open a new
TX and run inspection and logging on it.
When the 2nd rule is added, it adds toclient sgh as well. This means that we
will now slightly delay the freeing of the transactions.
As a consequence we still have the TX for the first REQUEST when the 2nd REQUEST
is parsed. This leads to the 2nd REQUEST re-using the TX. Since the TX is
already marked as inspected, it means the toserver rule now no longer matches.
Also we're not logging this TX correctly now.
This commit fixes the issue by not "finding" a TX that as already been
marked complete in the search direction.
Philippe Antoine [Thu, 12 Sep 2024 11:07:48 +0000 (13:07 +0200)]
frames: do not only rely on FRAME_STREAM_ID
As stream frame is not always created,
hence the first frame is not always a stream frame :
If stream frame is not enabled, it does not get created,
and other enabled frames may be created first.
See use of FrameConfigTypeIsEnabled
This resulted that this other frame got its length updated
on stream end, which led to false positives.
Also checking FRAME_STREAM_TYPE is more consistent.
Not a clean cherry-pick as AppLayerFrameGetLastOpenByType
does not exist in main7
warning: first doc comment paragraph is too long
--> src/detect/iprep.rs:57:1
|
57 | / /// value matching is done use `DetectUintData` logic.
58 | | /// isset matching is done using special `DetectUintData` value ">= 0"
59 | | /// isnotset matching bypasses `DetectUintData` and is handled directly
60 | | /// in the match function (in C).
| |_
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#too_long_first_doc_paragraph
= note: `#[warn(clippy::too_long_first_doc_paragraph)]` on by default
help: add an empty line
warning: you seem to be trying to use `match` for destructuring a single pattern. Consider using `if let`
--> src/dcerpc/log.rs:36:33
|
36 | DCERPC_TYPE_BIND => match &state.bind {
| _________________________________^
37 | | Some(bind) => {
38 | | jsb.open_array("interfaces")?;
39 | | for uuid in &bind.uuid_list {
... |
51 | | None => {}
52 | | },
| |_____________^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#single_match
= note: `#[warn(clippy::single_match)]` on by default
Philippe Antoine [Wed, 31 Jul 2024 12:15:14 +0000 (14:15 +0200)]
rust/ike: fix collapsible_match clippy warning
warning: this `match` can be collapsed into the outer `match`
help: the outer pattern can be modified to include the inner pattern
(cherry picked from commit 42e5e556e59fcd10efa89fcc75ad9f081ee25e93)
Philippe Antoine [Wed, 31 Jul 2024 12:10:17 +0000 (14:10 +0200)]
rust: fix byte_char_slices clippy warnings
warning: can be more succinctly written as a byte str
--> src/mime/smtp.rs:762:37
|
762 | mime_smtp_find_url_strings(ctx, &[b'\n']);
| ^^^^^^^^ help: try: `b"\n"`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#byte_char_slices
= note: `#[warn(clippy::byte_char_slices)]` on by default
Sascha Steinbiss [Wed, 14 Aug 2024 08:11:48 +0000 (10:11 +0200)]
userguide: fix spelling of `security_result` EVE field
This ensures that the correct spelling of the `security_result` EVE
field for RFB (as opposed to `security-result`) is also reflected in the
documentation.
rust/rfb: use consistent key name for security_result
A typo caused a slightly different key (`security-result`) to be used
for the case in which the result was `FAIL`. This commit addresses this
by ensuring the same string is used for all cases.
Eric Leblond [Fri, 19 Apr 2024 15:07:48 +0000 (17:07 +0200)]
datasets: fix parsing of ip4 in ip6
The lookup function was not taking into account that we can have
an IPv4 or an IPv6 address as parameters and that this addresses
need to be converted to Suricata internal storage.
By using the already defined dedicated parsing function, we are
fixing the issue.
It was brought to my attention by GLongo that Pgsql parser handled eof
diffrently for requests and responses, and apparently there isn't a good
reason for such a difference therefore, apply same logic used for
rs_pgsql_parse_request for checking for eof when parsing a response.
pgsql/logger: open json object from logger function
Before, the JsonBuilder object for the pgsql event was being created
from the C-side function that actually called the Rust logger.
This resulted that if another module - such as the Json Alert called the
PGSQL logger, we wouldn't have the `pgsql` key present in the log output
- only its inner fields.
Jason Ish [Tue, 6 Aug 2024 16:45:05 +0000 (10:45 -0600)]
rust/dcerpc: fix rustdoc indentation
Fixes clippy lint:
error: doc list item missing indentation
--> src/dcerpc/dcerpc.rs:511:9
|
511 | /// description: direction of the flow
| ^
|
= help: if this is supposed to be its own paragraph, add a blank line
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#doc_lazy_continuation
Jason Ish [Tue, 6 Aug 2024 16:43:19 +0000 (10:43 -0600)]
rust/conf: collapse match pattern into if
Fixes clippy lint for collapsible_match.
error: this `match` can be collapsed into the outer `if let`
--> src/conf.rs:85:9
|
85 | / match val {
86 | | "1" | "yes" | "true" | "on" => {
87 | | return true;
88 | | },
89 | | _ => {},
90 | | }
| |_________^
|
help: the outer pattern can be modified to include the inner pattern
--> src/conf.rs:84:17
|
84 | if let Some(val) = conf_get(key) {
| ^^^ replace this binding
85 | match val {
86 | "1" | "yes" | "true" | "on" => {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^ with this pattern
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#collapsible_match
Jason Ish [Tue, 6 Aug 2024 16:30:13 +0000 (10:30 -0600)]
rust/dcerpc: clippy fix for match
error: this `match` can be collapsed into the outer `match`
--> src/dcerpc/detect.rs:215:20
|
215 | Some(x) => match x {
| ____________________^
216 | | DCERPC_TYPE_REQUEST | DCERPC_TYPE_RESPONSE => {}
217 | | _ => {
218 | | return 0;
219 | | }
220 | | },
| |_________^
|
help: the outer pattern can be modified to include the inner pattern
--> src/dcerpc/detect.rs:215:14
|
215 | Some(x) => match x {
| ^ replace this binding
216 | DCERPC_TYPE_REQUEST | DCERPC_TYPE_RESPONSE => {}
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ with this pattern
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#collapsible_match
dpdk: replace TSC clock with GetTime (gettimeofday) function
Getting clock through Time Stamp Counter (TSC) can be precise and fast,
however only for a short duration of time.
The implementation across CPUs seems to vary. The original idea is to
increment the counter with every tick. Then dividing the delta of CPU ticks
by the CPU frequency can return the time that passed.
However, the CPU clock/frequency can change over time, resulting in uneven
incrementation of TSC. On some CPUs this is handled by extra logic.
As a result, obtaining time through this method might drift from the real
time.
This commit therefore substitues TSC time retrieval by the standard system
call wrapped in GetTime function - on Linux it is gettimeofday.
Victor Julien [Tue, 25 Jun 2024 08:35:35 +0000 (10:35 +0200)]
smb/ntlmssp: improve version check
Don't assume the ntlmssp version field is always present if the flag is
set. Instead keep track of the offsets of the data of the various blobs
and see if there is space for the version.
projects / thirdparty / suricata.git / log
