Clarify the transform validation step. When a transform indicates that
the content/byte-array is not compatible, validation will stop.
Content is incompatible is some cases -- e.g., following the
to_lowercase transform with content containing uppercase characters.
An alert is not possible since the content contains uppercase and the
transform has converted the buffer into all lowercase.
Add a more visible explanation of that requests, responses, frontend and
and backend are, in Pgsql context, to avoid having to repeat that over
different portions of the docs.
A CanceldRequest can occur after any query request, and is sent over a
new connection, leading to a new flow. It won't take any reply, but, if
processed by the backend, will lead to an ErrorResponse.
With the changes in the probing_ts function, this other one could become
obsolete. Remove it, and directly call `parser::parse_request` when
checking for gaps, instead.
Some non-pgsql traffic seen by Suricata is mistankenly identified as
pgsql, as the probing function is too generic. Now, if the parser sees
an unknown message type, even if it looks like pgsql, it will fail.
We had unkonwn message type for the backend, but not the frontend
messages. It's important to better identify those to improve pgsql
probing functions.
Jason Ish [Wed, 1 Nov 2023 22:57:39 +0000 (16:57 -0600)]
outputs: call plugin ThreadDeinit, not Deinit
With the change to the hash table for tracking threaded loggers, this
call is now called once per thread, so should be changed to the
ThreadDeinit, as that is not longer being called.
Then call Deinit for the primary logger. In threaded mode this would be
the parent, its just the logger in non-threaded mode.
Victor Julien [Thu, 14 Sep 2023 04:49:31 +0000 (06:49 +0200)]
packetpool: signal waiter within lock
Needed for predictable scheduling. From pthread_cond_signal man page:
"The pthread_cond_signal() or pthread_cond_broadcast() functions may
be called by a thread whether or not it currently owns the mutex that
threads calling pthread_cond_wait() or pthread_cond_timedwait() have
associated with the condition variable during their waits; however, if
predictable scheduling behaviour is required, then that mutex is locked
by the thread calling pthread_cond_signal() or pthread_cond_broadcast()."
Jason Ish [Fri, 27 Oct 2023 16:19:31 +0000 (10:19 -0600)]
dns/eve: use default formats if formats is empty
If the configuration field "formats" is empty, DNS response records do
not have any relevant information other than that there was a
response, but not much about the response.
I'm pretty sure the intention here was to log the response details if
no formats were provided, which is what happens when the field is
commented out.
So if no formats are specified, use the default of all.
Make sure Suricata is in the running state before
you attempt to execute commands on the Unix sockets.
UnixMain is being called in an infinite loop where
TmThreadsCheckFlag(th_v, THV_KILL) is checked for the
deinit phase. However, it may take some time between
the start of Suricata's deinitialization and
the receipt of THV_KILL flag in the Unix thread.
In between this time period, the Unix manager can still
perform select() operation on the Unix socket while
the socket being already deinitialized.
Likely with a longer time span between the initial shutdown
command and actual closing of Unix sockets resulted in
an error of invalid file descriptors.
Victor Julien [Fri, 13 Oct 2023 11:47:05 +0000 (13:47 +0200)]
detect: inspect all packets in multi-layer tunneling
When the decoders encounter multiple layers of tunneling, multiple tunnel
packets are created. These are then stored in ThreadVars::decode_pq, where
they are processed after the current thread "slot" is done. However, due
to a logic error, the tunnel packets after the first, where not called
for the correct position in the packet pipeline. This would lead to these
packets not going through the FlowWorker module, so skipping everything
from flow tracking, detection and logging.
This would only happen for single and workers, due to how the pipelines
are constructed.
The "slot" holding the decoder, would contain 2 packets in
ThreadVars::decode_pq. Then it would call the pipeline on the first
packet with the next slot of the pipeline through a indirect call to
TmThreadsSlotVarRun(), so it would be called for the FlowWorker.
However when that first (the most inner) packet was done, the call
to TmThreadsSlotVarRun() would again service the ThreadVars::decode_pq
and process it, again moving the slot pointer forward, so past the
FlowWorker.
This patch addresses the issue by making sure only a "decode" thread
slot will service the ThreadVars::decode_pq, thus never moving the
slot past the FlowWorker.
Lukas Sismis [Wed, 16 Aug 2023 21:51:10 +0000 (23:51 +0200)]
dpdk: add hugepage hint to lower the amount of reserved hugepages
If a user allocates too many hugepages and those are largely not used
then Suricata suggests that the user can lower the amount of hugepages
and therefore save memory for other purposes.
Our documentation was failing to build, seems connected to the new way
of indicating build options (cf
https://readthedocs.org/projects/suricata/builds/22112658/,
https://docs.readthedocs.io/en/stable/config-file/v2.html#build,
and https://docs.readthedocs.io/en/stable/config-file/v2.html#build-os).
Added the build.os required new field, and adjusted the way python
version is passed.
For the new configuration style for read the docs, one of the ways to
pass extra configuration for python is having a requirements file.
email_ctx->fields only gets populated when smtp.custom setting is on.
The fn EveEmailLogJSONCustom is called when either
1. smtp.extended setting is on or,
2. email_ctx->fields is populated which means smtp.custom setting is on
In case neither of these are set in suricata.yaml, no call should
ideally be made to the fn EveEmailLogJSONCustom.
However, it turns out that email_ctx->fields is unset and then set only
after the smtp config was found. This leads to email_ctx->fields
sometimes contain value even when no config was given to the smtp
section and can lead to unexpected output.
Fix this by using SCCalloc while initializing OutputJsonEmailCtx struct
instead of SCMalloc.
Although we have an updated version of instructions for installation
from git, our install guide was only referring to RedMine, which is less
up-to-date.
Kept that reference, since it might still be useful for non-Ubuntu
cases.
Philippe Antoine [Wed, 27 Sep 2023 12:15:18 +0000 (14:15 +0200)]
detect: check if signature uses too many buffers
Ticket: #6104
The approach in master branch is to change the prototype of
SigMatchAppendSMToList so that it allocates itself the new SigMatch
This approach requires to change all the 100-ish calls to
SigMatchAppendSMToList and is thus quite a big change.
For branch 7, we still wanted to avoid the buffer overflow, but
did not want such an intrusive change, and still wanted to make
the signature invalid. Instead of changing the prototype of the
function, we make it return early, and set a flag in the signature
which can be later checked by SigValidate
Jason Ish [Thu, 28 Sep 2023 22:19:51 +0000 (16:19 -0600)]
readme: add a resources section
Consolidate a few items into a resources section, and add few more
items, most importantly the bug tracker as it can't currently be found
from our GitHub presence.
dpdk: reset stats just before the start of packet receive loop
While Suricata initializes, the device must be started to e.g.
apply rte_flow rules on some devices. But in the meantime, the
NIC started receiving packets but accounted those as missed.
Stats reset was added to better represent true packet drop.
dpdk: stop devices immediately after Suricata stop command
To better represent port stats of Suricata stop the device
right after Suricata is deemed to shut down.
While Suricata deinitialization happened, the device kept
receiving packets. But because Suricata was no longer
interested in the packets, the device accounted these packets
as missed and it could have alter true stats of Suricata.
Shivani Bhardwaj [Thu, 10 Aug 2023 15:35:45 +0000 (21:05 +0530)]
af-packet: fetch mtu info once
With the current layout and fn calls, it was seen that once in the
beginning after the MTU was found and displayed to the user, when the
threads spawned, each thread displayed MTU info as a part of
AFPPeersListAdd fn. This happened in AF_PACKET IPS mode and led to
excessive MTU logs.
Save this info in the LiveDevice struct and avoid calling the unneeded
fns later on.
Shivani Bhardwaj [Thu, 10 Aug 2023 15:34:56 +0000 (21:04 +0530)]
util/ioctl: use LiveDevice to retrieve name
The fn GetIfaceMaxPacketSize now uses LiveDevice object as a param
instead of a string. This was done to keep the logic of checking for the
device to this function itself instead of having callers first determine
whether the device exists or not.
This also falls in line with the changes made to avoid excessive MTU
logs in the following commit.
projects / thirdparty / suricata.git / log
