Victor Julien [Fri, 22 Feb 2019 19:41:41 +0000 (20:41 +0100)]
ips/stream: handle low mem(cap) crash
In low memory or memcap reached conditions a crash could happen in
inline stream detection.
The crash had the following path:
A packet would come in and it's data was added to the stream. Due
to earlier packet loss, the stream buffer uses a stream buffer block
tree to track the data blocks. When trying to add the current packets
block to the tree, the memory limit was reached and the add fails.
A bit later in the pipeline for the same packet, the inline stream
mpm inspection function gets the data to inspect. For inline mode
this is the current packet + stream data before and after the packet,
if available.
The code looking up the packets data in the stream would not
consider the possibility that the stream block returned wasn't
the right one. The tree search returns either the correct or the
next block. In adjusting the returned block to add the extra stream
data it would miscalculate offsets leading to a corrupt pointer to the
data.
This patch more carefully checks the result of the lookup, and
falls back to simply inspecting the packet payload if the lookup
didn't produce the expected result.
Mats Klepsland [Sat, 16 Feb 2019 20:55:19 +0000 (21:55 +0100)]
app-layer-ssl: check that cipher suites length is divisible by two
Cipher suites length should always be divisible by two. If it is a
odd number, which should not happen with normal traffic, it ends up
reading one byte too much.
No resizing is done in Ja3BufferResizeIfFull() when the buffer is
empty. This leads to a potential overflow when this happens, since
a ',' is appended even when the buffer is empty.
Jason Ish [Thu, 7 Feb 2019 19:53:23 +0000 (13:53 -0600)]
issue 2795: python 3 fix in Rust C header gen
The C header generation script was failing with a unicode error
in Python 3 on FreeBSD. Fix the reading of files to properly
handle unicode in all Python 3 environments.
Fabrice Fontaine [Thu, 31 Jan 2019 07:56:15 +0000 (08:56 +0100)]
configure.ac: fix --{disable,enable}-xxx options
Currently, if the user provides --enable-libmagic or
--disable-libmagic, libmagic will be disabled because $enableval is not
used to know if the user provided --enable or --disable
Most of the options have this issue so fix them all by using $enableval
Victor Julien [Fri, 18 Jan 2019 14:03:39 +0000 (15:03 +0100)]
stream: fix false negative on bad RST
If a bad RST was received the stream inspection would not happen
for that packet, but it would still move the 'raw progress' tracker
forward. Following good packets would then fail to detect anything
before the 'raw progress' position.
Victor Julien [Wed, 23 Jan 2019 20:18:59 +0000 (21:18 +0100)]
eve: fix missing decoder-events in stats
In the eve log the decoder events are added as optional counters. This
behaviour is enabled by default. However, lots of the counters are
missing, as the names colide with other counters.
E.g.
decoder.ipv6 counts ipv6 packets
decoder.ipv6.unknown_next_header counts how often an unknown next
header is encountered.
In this example 'ipv6' would be both a json integer and a json object.
It appears that jansson favours the first that is generated, so the
event counters are mostly missing.
This patch registers them as 'decoder.events.<event>' instead. As
these names are generated on the fly, a hash table to contain the
allocated strings was added as well.
Victor Julien [Tue, 22 Jan 2019 20:00:57 +0000 (21:00 +0100)]
af-packet: fix sync start for tpacket v3
The tpacket-v3 implementation of the synchonize start logic would
not correctly consider the timestamp parameter, leading to threads
starting before synchronization between threads was complete.
Victor Julien [Wed, 19 Dec 2018 10:49:42 +0000 (11:49 +0100)]
proto/detect: workaround dns misdetected as dcerpc
The DCERPC UDP detection would misfire on DNS with transaction
ID 0x0400. This would happen as the protocol detection engine
gives preference to pattern based detection over probing parsers for
performance reasons.
This hack/workaround fixes this specific case by still running the
probing parser if DCERPC has been detected on UDP. The probing
parser result will take precedence.
Victor Julien [Wed, 19 Dec 2018 08:45:35 +0000 (09:45 +0100)]
teredo: be stricter on what to consider valid teredo
Invalid Teredo can lead to valid DNS traffic (or other UDP traffic)
being misdetected as Teredo. This leads to false negatives in the
UDP payload inspection.
Make the teredo code only consider a packet teredo if the encapsulated
data was decoded without any 'invalid' events being set.
Victor Julien [Sat, 8 Dec 2018 17:51:23 +0000 (18:51 +0100)]
stats: more accurate interval handling
In the stats loop sleep for a time period more closely matching
the stats.interval setting. Fix an off by one that would make
the loop wake up ~1 second early.
Victor Julien [Thu, 6 Dec 2018 14:35:40 +0000 (15:35 +0100)]
app-layer: improve transaction cleanup handling
The app layers with a custom iterator would skip a tx if during
the ..Cleanup() pass a transaction was removed.
Address this by storing the current index instead of the next
index. Also pass in the next "min_tx_id" to be incremented from
the last TX. Update loops to do this increment.
Also make sure that the min_id is properly updated if the last
TX is removed when out of order.
Eric Leblond [Mon, 8 Oct 2018 21:51:37 +0000 (23:51 +0200)]
af-packet: improve error handling
Stress condition in Suricata could lead to interface to disconnect
when it is not necessary. This patch updates the error handling
code to try to continue reading when such a case occurs.
Jason Ish [Sun, 25 Nov 2018 15:03:22 +0000 (09:03 -0600)]
configure: allow for --disable-suricata-update
This is to prevent suricata-update from being installed if it
would otherwise be installed based on in being bundled, and
its dependencies being available.
Warn the user that Suricata-Update will not be installed if it
is bundled, but python-yaml is missing (this will also cover
the case where Python is missing).
Add "Install suricata-update" to the build summary. For consistency,
relable "Suricatasc install" as "Install suricatasc".
Victor Julien [Wed, 18 Apr 2018 13:25:40 +0000 (15:25 +0200)]
detect: improve inspect buffer handling
Fix and Optimize cleanup. For the simple single inspect buffer optimize
the cleanup by keeping track of the actually used buffers. This avoid
looping over unused buffers.
Fix the case of cleaning not being done after a tx if the next tx is
also inspected in the context of the same packet.
Fix cleanup of the multi-inspect buffers. Optimize in 2 ways. First
like with single keep track of which multi-inspect buffers have been
used. Second, keep a max of the buffers within a multi-inspect buffer.
Use this max to limit (nested) looping.