Victor Julien [Sat, 19 Sep 2015 20:59:03 +0000 (22:59 +0200)]
mpm: introduce ac-ks
Introduce 'ac-ks' or the Kenneth Steele AC implementation. It's
actually 'ac-tile' written by Ken for the Tilera platform. This
patch adds support for it on other architectures as well.
Enable ac-tile for other archs as 'ac-ks'.
Fix a bunch of OOB reads in the loops that triggered ASAN.
*** CID 1358023: Null pointer dereferences (REVERSE_INULL)
/src/util-mpm-hs.c: 860 in SCHSDestroyThreadCtx()
854 if (thr_ctx->scratch != NULL) {
855 hs_free_scratch(thr_ctx->scratch);
856 mpm_thread_ctx->memory_cnt--;
857 mpm_thread_ctx->memory_size -= thr_ctx->scratch_size;
858 }
859
>>> CID 1358023: Null pointer dereferences (REVERSE_INULL)
>>> Null-checking "mpm_thread_ctx->ctx" suggests that it may be null, but it has already been dereferenced on all paths leading to the check.
860 if (mpm_thread_ctx->ctx != NULL) {
861 SCFree(mpm_thread_ctx->ctx);
862 mpm_thread_ctx->ctx = NULL;
863 mpm_thread_ctx->memory_cnt--;
864 mpm_thread_ctx->memory_size -= sizeof(SCHSThreadCtx);
865 }
Victor Julien [Wed, 30 Mar 2016 07:39:46 +0000 (09:39 +0200)]
detect: fix small mem leak on duplicate sigs
Direct leak of 80 byte(s) in 5 object(s) allocated from:
#0 0x4c673b in __interceptor_malloc (/home/victor/dev/suricata/src/suricata+0x4c673b)
#1 0xb7a425 in DetectEngineSignatureIsDuplicate /home/victor/dev/suricata/src/detect-parse.c:1715:10
#2 0xb79390 in DetectEngineAppendSig /home/victor/dev/suricata/src/detect-parse.c:1836:19
#3 0x86fe56 in DetectLoadSigFile /home/victor/dev/suricata/src/detect.c:357:15
#4 0x815fee in ProcessSigFiles /home/victor/dev/suricata/src/detect.c:419:13
#5 0x8139a8 in SigLoadSignatures /home/victor/dev/suricata/src/detect.c:499:15
#6 0xfe435d in LoadSignatures /home/victor/dev/suricata/src/suricata.c:1979:9
#7 0xfcd87e in main /home/victor/dev/suricata/src/suricata.c:2345:17
#8 0x7fb66bf7cec4 in __libc_start_main /build/eglibc-3GlaMS/eglibc-2.19/csu/libc-start.c:287
Justin Viiret [Mon, 28 Mar 2016 22:32:26 +0000 (09:32 +1100)]
util-hash-lookup3: Add hashlittle_safe() variant
By default, hashlittle() will read off the end of the key, up to the
next four-byte boundary, although the data beyond the end of the key
doesn't affect the hash. This read causes uninitialized read warnings
from Valgrind and Address Sanitizer.
Here we add hashlittle_safe(), which avoids reading off the end of the
buffer (using the code inside the VALGRIND-guarded block in the original
hashlittle() implementation).
Victor Julien [Wed, 23 Mar 2016 16:05:14 +0000 (17:05 +0100)]
detect reload: generic packet injection for capture
Capture methods that are non blocking will still not generate packets
that go through the system if there is no traffic. Some maintenance
tasks, like rule reloads rely on packets to complete.
This patch introduces a new thread flag, THV_CAPTURE_INJECT_PKT, that
instructs the capture thread to create a fake packet.
The capture implementations can call the TmThreadsCaptureInjectPacket
utility function either with the packet they already got from the pool
or without a packet. In this case the util func will get it's own
packet.
Victor Julien [Wed, 23 Mar 2016 15:16:41 +0000 (16:16 +0100)]
detect reload: call 'breakloop' on capture method
Split wait loop into three steps:
- first insert pseudo packets
- 2nd nudge all capture threads to break out of their loop
- third, wait for the detection thread contexts to be used
Victor Julien [Thu, 24 Mar 2016 10:51:49 +0000 (11:51 +0100)]
signals: cleanup signal handling
Simplify handling of USR2 signal. The SCLogInfo usage could lead to
dead locks as the SCLog API can do many complicated things including
memory allocations, syslog calls, libjansson message construction.
If an existing malloc call was interupted, it could lead to the
following dead lock:
0 __lll_lock_wait_private () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:97
1 0x0000003140c7d2df in _L_lock_10176 () from /lib64/libc.so.6
2 0x0000003140c7ab83 in __libc_malloc (bytes=211543457408) at malloc.c:3655
3 0x0000003140c80ec2 in __strdup (s=0x259ca40 "[%i] %t - (%f:%l) <%d> (%n) -- ") at strdup.c:43
4 0x000000000059dd4a in SCLogMessageGetBuffer (tval=0x7fff52b47360, color=1, type=SC_LOG_OP_TYPE_REGULAR, buffer=0x7fff52b47370 "", buffer_size=2048,
log_format=0x259ca40 "[%i] %t - (%f:%l) <%d> (%n) -- ", log_level=SC_LOG_INFO, file=0x63dd00 "suricata.c", line=287, function=0x640f50 "SignalHandlerSigusr2StartingUp", error_code=SC_OK,
message=0x7fff52b47bb0 "Live rule reload only possible after engine completely started.") at util-debug.c:307
5 0x000000000059e940 in SCLogMessage (log_level=SC_LOG_INFO, file=0x63dd00 "suricata.c", line=287, function=0x640f50 "SignalHandlerSigusr2StartingUp", error_code=SC_OK,
message=0x7fff52b47bb0 "Live rule reload only possible after engine completely started.") at util-debug.c:549
6 0x000000000057e374 in SignalHandlerSigusr2StartingUp (sig=12) at suricata.c:287
7 <signal handler called>
8 _int_malloc (av=0x3140f8fe80, bytes=<value optimized out>) at malloc.c:4751
9 0x0000003140c7ab1c in __libc_malloc (bytes=296) at malloc.c:3657
10 0x0000000000504d55 in FlowAlloc () at flow-util.c:60
11 0x00000000004fd909 in FlowInitConfig (quiet=0 '\000') at flow.c:454
12 0x0000000000584c8e in main (argc=6, argv=0x7fff52b4a3b8) at suricata.c:2300
This patch simply sets a variable and lets the main loop act on that.
Victor Julien [Sat, 26 Mar 2016 11:05:50 +0000 (12:05 +0100)]
tcp: fix unlikely NULL-ptr dereference
If a TCP packet could not get a flow (flow engine out of flows/memory)
and there were *only* TCP inspecting rules with the direction
explicitly set to 'to_server', a NULL pointer deref could happen.
PacketPatternSearchWithStreamCtx would fall through to the 'to_client'
case which was not initialized.
cdwakelin [Wed, 23 Mar 2016 17:13:55 +0000 (17:13 +0000)]
autofp: add "ippair" scheduler
Add "ippair" autofp scheduler to split traffic based on source and
destination IP only (not ports).
- This is useful when using the "xbits" feature to track events
that occur between the same hosts but not necessarily the same
flow (such as exploit kit landings/expoits/payloads)
- The disadvantage is that traffic may be balanced very unevenly
between threads if some host pairs are much more frequently seen
than others, so it may be only practicable for sandbox or pcap
analysis
- not tested for IPv6
See https://redmine.openinfosecfoundation.org/issues/1661
Eric Leblond [Fri, 18 Mar 2016 08:53:21 +0000 (09:53 +0100)]
output-json-email: fix memleak
This possibly fix:
ndirect leak of 64 byte(s) in 1 object(s) allocated from:
#0 0x4c264b in malloc (/home/victor/qa/buildbot/donkey/z600fuzz/Private/src/.libs/lt-suricata+0x4c264b)
#1 0x7fb09c1e8aaa in json_array (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x6aaa)
#2 0xd67553 in JsonEmailLogJsonData /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/output-json-email-common.c:290:27
#3 0xd6a272 in JsonEmailLogJson /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/output-json-email-common.c:370:19
#4 0xd956b9 in JsonSmtpLogger /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/output-json-smtp.c:103:9
#5 0xdcedac in OutputTxLog /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/output-tx.c:165:17
#6 0xff6669 in TmThreadsSlotVarRun /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/tm-threads.c:132:17
#7 0xffecc1 in TmThreadsSlotVar /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/tm-threads.c:474:17
#8 0x7fb09bfcc181 in start_thread /build/eglibc-3GlaMS/eglibc-2.19/nptl/pthread_create.c:312
Eric Leblond [Fri, 18 Mar 2016 08:28:58 +0000 (09:28 +0100)]
output-json-email: fix memleak in error path
In JsonEmailLogJsonData function, an invalid state was leading to
early exit without a proper freeing of resources.
This should fix:
Indirect leak of 72 byte(s) in 1 object(s) allocated from:
#0 0x4c264b in malloc (/home/victor/qa/buildbot/donkey/z600fuzz/Private/src/.libs/lt-suricata+0x4c264b)
#1 0x7fb09c1e886a in json_object (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x686a)
#2 0xd6a272 in JsonEmailLogJson /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/output-json-email-common.c:370:19
#3 0xd956b9 in JsonSmtpLogger /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/output-json-smtp.c:103:9
#4 0xdcedac in OutputTxLog /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/output-tx.c:165:17
#5 0xff6669 in TmThreadsSlotVarRun /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/tm-threads.c:132:17
#6 0xffecc1 in TmThreadsSlotVar /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/tm-threads.c:474:17
#7 0x7fb09bfcc181 in start_thread /build/eglibc-3GlaMS/eglibc-2.19/nptl/pthread_create.c:312
Victor Julien [Wed, 16 Mar 2016 11:37:25 +0000 (12:37 +0100)]
ip parsing: fix cppcheck negative shift warning
[src/util-ip.c:104]: (error) Shifting a negative value is undefined behaviour
[src/util-radix-tree.c:1160]: (error) Shifting a negative value is undefined behaviour
[src/util-radix-tree.c:1357]: (error) Shifting a negative value is undefined behaviour
[src/util-radix-tree.c:1380]: (error) Shifting a negative value is undefined behaviour
[src/util-radix-tree.c:1438]: (error) Shifting a negative value is undefined behaviour
Victor Julien [Wed, 16 Mar 2016 11:20:17 +0000 (12:20 +0100)]
stats: fix unix socket crash
Reset counters_global_id at ctx destruction. In the unix socket
runmode the lack of this reset would cause the id's to increase with
each pcap, leading to an ever larger stats array.
Jason Ish [Wed, 27 Jan 2016 05:22:27 +0000 (23:22 -0600)]
detect: denote the max detection list; fix issue 1674.
Denotes the max detection list so that rule validation can
allow post-detection lists to come after base64_data, but
disallow detection lists to come after it.
Victor Julien [Wed, 9 Mar 2016 19:43:54 +0000 (20:43 +0100)]
file: optionally use detect tracking in pruning
When the file API is used to do content inspection (currently only
smtp does this), the detection should be considered while pruning
the file chunks.
This patch introduces a new flag for the file API: FILE_USE_DETECT
When it is used, 'FilePrune' will not remove chunks that are (partly)
beyond the File::content_inspected tracker.
When using this flag, it's important to realize that when the detect
engine is disabled or rules are not matching, content_inspected
might not get updated.
While this appears to be a FP, in most cases the initialization to ""
was unnecessary as the snprintf statement immediately follows the
variable declaration.
Maurizio Abba [Thu, 10 Mar 2016 13:58:21 +0000 (13:58 +0000)]
decode-events: counters for decode events errors
We want to add counters in order to track the number of times we hit a
decode event. A decode event is related to an error in the protocol
decoding over a certain packet.
This patch fist modifies the decode-event list, reordering it in order
to separate single packet events from stream-related events and adding
the prefix "decoder" to decode events.
The counters are created during the decode setup and the relative event
counter is increased every time a packet with the flag PKT_IS_INVALID is
finalized in the decode phase
Victor Julien [Tue, 8 Mar 2016 15:15:45 +0000 (16:15 +0100)]
defrag: fix bad packet error handling
When defrag creates a new reassembled IP packet, it then passes this
packet to the IP decoder. If this decoder returns an error the packet
is returned back to the packet pool with a call to TmqhOutputPacketpool
This lead to the first problem. The returned packet had it's p->root
pointer set, and it's PKT_TUNNEL flag set. This could cause problems
in TmqhOutputPacketpool, as this may reference the packet referenced
in p->root.
The second and more glaring problem is that the packet that was
returned to the packetpool, was still returned by the Defrag function
and processed further. It would then at the end of it's processing
be returned to the packet pool, which at this point already had a
reference to this packet.
This patch fixes both issues by unsetting the tunnel references and
returning NULL from Defrag in this case.
Andreas Herz [Mon, 7 Mar 2016 20:33:14 +0000 (21:33 +0100)]
configure: bypass libpcre 8.35 check
When --with-libpcre-libraries is used we skip the libpcre 8.35 check
since pkg-config might still point to the 8.35 version installed
although newer version was passed with --with-libpcre-libraries.
Eric Leblond [Mon, 7 Mar 2016 08:16:41 +0000 (09:16 +0100)]
util-radix-tree: fix memleak
Logic used when adding a new prefix to a node was not correct
as we were allocating a prefix that could be at the end unused.
This patch is updating the code to have a complete creation to
be done if and only if we are needing the complete object. In
the other cases, it was enough to use the function input values.
This fixes:
104 (48 direct, 56 indirect) bytes in 2 blocks are definitely lost in loss record 184 of 327
at 0x4C29C0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
by 0x9C2DAD: SCRadixCreatePrefix (util-radix-tree.c:144)
by 0x9AFA5B: SCRadixAddKey (util-radix-tree.c:522)
by 0x9B1A4D: SCRadixAddKeyIPV4Netblock (util-radix-tree.c:897)
by 0x67C824: IPOnlyPrepare (detect-engine-iponly.c:1197)
by 0x55172B: SigAddressPrepareStage2 (detect.c:3534)
by 0x5486F4: SigGroupBuild (detect.c:4671)
by 0x547C87: SigLoadSignatures (detect.c:538)
by 0x8FB5D0: LoadSignatures (suricata.c:1976)
by 0x8F3B32: main (suricata.c:2342)
projects / thirdparty / suricata.git / log
