The differences on how the `need` key works, depending on script
usage (output or detection) confuses users, sometimes (cf doc#4725).
While we don't fix that, just explain this behavior.
Fix this by using the new `Default` method for registering parsers.
However, the buffer still needs an explicit configuration check.
Also convert Notice debug messages to Debug to reduce output when in
unittest mode. If we feel stronly this should still be Notice in the
template, that is a conversion we can make in the generation script when
generating a new parser.
If a HTTP2 transaction gets freed before the end of the range
request, we need to have the files container which is in
the state, to transfer owernship of this file to the files
container.
rust/nfs: add a maximum number of operations per compound
The `count` combinator preallocates a number of bytes. Since the value
is untrusted, this can result in an Out Of Memory allocation.
Use a maximum value, large enough to cover all current implementations.
Jason Ish [Wed, 10 Nov 2021 22:38:35 +0000 (16:38 -0600)]
smtp: log transaction even if no email present
The SMTP transaction logger was not writing the log if the email
portion of the logger failed, such as in the case of STARTTLS
where this is no email decoded.
Jason Ish [Wed, 17 Nov 2021 01:34:11 +0000 (19:34 -0600)]
dhcp: fix url in comment
rustdoc was complaining about the format of the URL in a comment
while trying to generate documentation. Convert the comment to a
non-rustdoc comment for now to satisfy rustdoc.
Jason Ish [Thu, 14 Oct 2021 18:37:03 +0000 (12:37 -0600)]
rust/app-layer: provide generic implementation of iterator
Create traits for app-layer State and Transaction that allow
a generic implementation of a transaction iterator that parser
can use when the follow the common pattern for iterating
transactions.
Also convert DNS to use the generic for testing purposes.
Eric Leblond [Wed, 17 Nov 2021 15:43:23 +0000 (16:43 +0100)]
profiling: fix profiling with sample rate
Rules profiling was returning invalid results when used with sample
rate. The problem was that the sample condition was run twice in the
packet flow. As a result, the second pass was not initializing the
variable storing the initial CPU ticks and the resulting performance
counters were reporting invalid values.
Philippe Antoine [Thu, 16 Sep 2021 14:54:37 +0000 (16:54 +0200)]
tcp: rejects FIN+SYN packets as invalid
Ticket: #4569
If a FIN+SYN packet is sent, the destination may keep the
connection alive instead of starting to close it.
In this case, a later SYN packet will be ignored by the
destination.
Previously, Suricata considered this a session reuse, and thus
used the sequence number of the last SYN packet, instead of
using the one of the live connection, leading to evasion.
This commit errors on FIN+SYN so that they do not get
processed as regular FIN packets.
Victor Julien [Tue, 5 Oct 2021 12:48:27 +0000 (14:48 +0200)]
stream/tcp: handle RST with MD5 or AO header
Special handling for RST packets if they have an TCP MD5 or AO header option.
The options hash can't be validated. The end host might be able to validate
it, as it can have a key/password that was communicated out of band.
The sender could use this to move the TCP state to 'CLOSED', leading to
a desync of the TCP session.
This patch builds on top of 843d0b7a10bb ("stream: support RST getting lost/ignored")
It flags the receiver as having received an RST and moves the TCP state
into the CLOSED state. It then reverts this if the sender continues to
send traffic. In this case it sets the following event:
Philippe Antoine [Mon, 13 Sep 2021 10:18:34 +0000 (12:18 +0200)]
detect: fixes InspectionBuffer id with transforms
When InspectionBufferGet gets called with base_id
Later InspectionBufferSetup must also be called with base_id
In case there were transforms, we had base_id != list_id
Not calling InspectionBufferSetup with the right id
resulted in leaving a dangling pointer,
because it was not added to det_ctx->inspect.to_clear_queue
Victor Julien [Sun, 7 Nov 2021 05:25:31 +0000 (06:25 +0100)]
flow/manager: fix flows not evicted & freed in time
Flows have been shown to linger for a long time w/o giving up their
resources. This would lead to higher memory use and memcaps getting
reached.
Three main causes have been identified:
Slow passes hash passes. By default the flow manager will scan the
flow hash slowly. It is based on the flow timeout settings, and with
the default config it will take 4 minutes for a full scan to be
complete. This leaves a window for flows that are timed out to linger
for minutes longer than expected.
Flow Manager yields under pressure. The per row TryLock causes work
to be delayed more. The Flow manager will use trylock on a hash row
and will yield immediately if the row is busy. This means that it will
take a full pass before the row is revisited again. If the row holds
busy flows, this could happen many times in a row.
Flow Manager favors evicted flows over active flows. The Flow Manager
will only process the evicted flows if they are present. These flows
have been evicted by workers. The active flows on that hash row will
have to wait until the next hash pass. Of course by then there could
be more evicted flows.
Combined these factors could lead to flows not being considered for
freeing and logging for a very long time, potentially even indefinitly.
The patch addresses the latter two flow manager issues by no longer
using TryLock. It will now simply wait for the lock to be released and
then do its work on it. Additionally for each row both the evicted list
and the active flow list will be processed.