Duarte Silva [Tue, 12 Aug 2014 17:13:52 +0000 (18:13 +0100)]
Simple code fixes
- Removed unnecessary assignment of the data field
- Removed else condition (same function called for IPv4 and IPV6)
- Fixed constants to be a power of two (used in bitwise operations)
Eric Leblond [Wed, 13 Aug 2014 08:48:26 +0000 (10:48 +0200)]
packet pool: fix ext_pkt cleaning
The field ext_pkt was cleaned before calling the release function.
The result was that IPS mode such as the one of AF_PACKET were not
working anymore because they were not able to send the data which
were initially pointed by ext_pkt.
This patch moves the ext_pkt cleaning to the cleaning macro. This
ensures that the cleaning is done for allocated and pool packets.
Victor Julien [Wed, 9 Jul 2014 06:51:29 +0000 (08:51 +0200)]
packet recycle: do most clean up on packet reuse
Call PACKET_RELEASE_REFS from PacketPoolGetPacket() so that
we only access the large packet structure just before actually
using it. Should give better cache behaviour.
Ken Steele [Wed, 5 Feb 2014 23:00:19 +0000 (18:00 -0500)]
Fix GRE Source Routing Header definition
The Source Routing Header had routing defined as a char* for a field
of variable size. Since that field was not being used in the code, I
removed the pointer and added a comment.
Ken Steele [Fri, 20 Dec 2013 16:52:12 +0000 (11:52 -0500)]
Add Packed attribute on Header structures
Structures that are used to cast packet data into fields need to be packed
so that the compiler doesn't add any padding to these fields. This also helps
Tile-Gx to avoid unaligned loads because the compiler will insert code to
handle the possible unaligned load.
Victor Julien [Mon, 11 Aug 2014 12:14:59 +0000 (14:14 +0200)]
lua: improve configure checks
The base 'lua' library has different names on different OS' and even
Linux distro's. Instead of selecting the proper one, we now just try
all. This way no OS/distro specific knowledge about the name is needed.
Victor Julien [Wed, 16 Jul 2014 22:23:50 +0000 (00:23 +0200)]
stream: detect and filter out bad window updates
Reported in bug 1238 is an issue where stream reassembly can be
disrupted.
A packet that was in-window, but otherwise unexpected set the
window to a really low value, causing the next *expected* packet
to be considered out of window. This lead to missing data in the
stream reassembly.
The packet was unexpected in various ways:
- it would ack unseen traffic
- it's sequence number would not match the expected next_seq
- set a really low window, while not being a proper window update
Detection however, it greatly hampered by the fact that in case of
packet loss, quite similar packets come in. Alerting in this case
is unwanted. Ignoring/skipping packets in this case as well.
The logic used in this patch is as follows. If:
- the packet is not a window update AND
- packet seq > next_seq AND
- packet acq > next_seq (packet acks unseen data) AND
- packet shrinks window more than it's own data size
THEN set event and skip the packet in the stream engine.
So in case of a segment with no data, any window shrinking is rejected.
Victor Julien [Tue, 5 Aug 2014 15:28:17 +0000 (17:28 +0200)]
defrag: use 'struct timeval' for timeout tracking
Until now the time out handling in defrag was done using a single
uint32_t that tracked seconds. This lead to corner cases, where
defrag trackers could be timed out a little too early.
Victor Julien [Thu, 24 Jul 2014 11:39:10 +0000 (13:39 +0200)]
ipv6 defrag: fix unfragmentable exthdr handling
Fix or rather implement handling of unfragmentable exthdrs in ipv6.
The exthdr(s) appearing before the frag header were copied into the
reassembled packet correctly, however the stripping of the frag header
did not work correctly.
Example:
The common case is a frag header directly after the ipv6 header:
The result would be:
[ipv6 header]->[hop header]->[icmpv6]
However, here too the ipv6 header would have been updated to point
to what the frag header pointed at. So it would consider the hop header
as if it was an ICMPv6 header, or whatever the frag header pointed at.
The result is that packets would not be correctly parsed, and thus this
issue can lead to evasion.
This patch implements handling of the unfragmentable part. In the first
segment that is stored in the list for reassembly, this patch detects
unfragmentable headers and updates it to have the last unfragmentable
header point to the layer after the frag header.
Also, the ipv6 headers 'next hdr' is only updated if no unfragmentable
headers are used. If they are used, the original value is correct.
Reported-By: Rafael Schaefer <rschaefer@ernw.de>
Bug #1244.
Eric Leblond [Tue, 29 Jul 2014 08:05:23 +0000 (10:05 +0200)]
unittests: don't register app layer test
Some tests are already registered via the function
AppLayerParserRegisterProtocolUnittests. So we don't need to
egister them during runmode initialization.
Ken Steele [Thu, 3 Jul 2014 16:42:12 +0000 (12:42 -0400)]
Reduce reallocation in AC Tile MPM creation.
Exponentially increase the memory allocated for new states when adding new
states, then at the end resize down to the actually final size so that no space is wasted.
Ken Steele [Tue, 29 Jul 2014 13:31:49 +0000 (09:31 -0400)]
Fix Packet Stacks for non-TLS Operating Systems
On non-TLS systems, check each time the Thread Local Storage
is requested and if it has not been initialized for this thread, initialize it.
The prevents not initializing the worker threads in autofp run mode.
Handle misc tasks only in instance 1: Handle defrag hash timeout
handing, host hash timeout handling and flow spare queue updating
only from the first instance.
Victor Julien [Wed, 16 Jul 2014 07:59:48 +0000 (09:59 +0200)]
threads: add management API
Currently management threads do their own thread setup and handling. This
patch introduces a new way of handling management threads.
Functionality that needs to run as a management thread can now register
itself as a regular 'thread module' (TmModule), where the 'Management'
callback is registered.
Victor Julien [Fri, 23 May 2014 12:54:05 +0000 (14:54 +0200)]
flow: add flow_end_flags field, add logging
The flow end flags field is filled by the flow manager or the flow
hash (in case of forced timeout of a flow) to record the timeout
conditions in the flow:
- emergency mode
- state
- reason (timed out or forced)
Victor Julien [Thu, 22 May 2014 10:53:51 +0000 (12:53 +0200)]
flow-recycler: speed up flow-recycler shutdown
Thread was killed by the generic TmThreadKillThreads instead of
the FlowKillFlowRecyclerThread. The latter wakes the thread up, so
that shutdown is quite a bit faster.
Victor Julien [Wed, 21 May 2014 12:29:15 +0000 (14:29 +0200)]
stream: track TCP flags per stream direction
For netflow logging track TCP flags per stream direction. As the struct
had no more space left without expanding it, the flags and wscale
fields are now compressed.
Victor Julien [Fri, 9 May 2014 12:37:07 +0000 (14:37 +0200)]
flow: prepare flow forced reuse logging
Most flows are marked for clean up by the flow manager, which then
passes them to the recycler. The recycler logs and cleans up. However,
under resource stress conditions, the packet threads can recycle
existing flow directly. So here the recycler has no role to play, as
the flow is immediately used.
For this reason, the packet threads need to be able to invoke the
flow logger directly.
The flow logging thread ctx will stored in the DecodeThreadVars
stucture. Therefore, this patch makes the DecodeThreadVars an argument
to FlowHandlePacket.