]> git.ipfire.org Git - thirdparty/suricata.git/log
thirdparty/suricata.git
9 years agodetect: rename groupings vars
Victor Julien [Fri, 2 Oct 2015 18:23:50 +0000 (20:23 +0200)] 
detect: rename groupings vars

9 years agodetect: remove unused grouping settings
Victor Julien [Fri, 2 Oct 2015 18:16:25 +0000 (20:16 +0200)] 
detect: remove unused grouping settings

9 years agodetect: make port grouping use config limits
Victor Julien [Wed, 30 Sep 2015 10:41:42 +0000 (12:41 +0200)] 
detect: make port grouping use config limits

9 years agodetect: change port grouping
Victor Julien [Wed, 30 Sep 2015 09:56:42 +0000 (11:56 +0200)] 
detect: change port grouping

Update port grouping logic. Previously it would create one consistent
list w/o overlap. It largely still does this, except for the 'catch
all' port group at the end of the list. This port group contains all
the sigs that didn't fit into the other groups.

9 years agodetect: sort/group port sigs
Victor Julien [Tue, 29 Sep 2015 16:42:16 +0000 (18:42 +0200)] 
detect: sort/group port sigs

9 years agodetect: display unique sgh count
Victor Julien [Tue, 29 Sep 2015 15:40:59 +0000 (17:40 +0200)] 
detect: display unique sgh count

9 years agodetect: group proto sghs
Victor Julien [Tue, 29 Sep 2015 14:46:21 +0000 (16:46 +0200)] 
detect: group proto sghs

9 years agodetect-mpm: make sgh setup proto aware
Victor Julien [Mon, 28 Sep 2015 14:03:48 +0000 (16:03 +0200)] 
detect-mpm: make sgh setup proto aware

Allow multi-proto, multi-direction sgh's.

9 years agodetect: remove obsolete grouping code
Victor Julien [Tue, 29 Sep 2015 11:20:20 +0000 (13:20 +0200)] 
detect: remove obsolete grouping code

9 years agodetect: debug output
Victor Julien [Mon, 28 Sep 2015 21:20:03 +0000 (23:20 +0200)] 
detect: debug output

9 years agodetect: per port and proto rule grouping
Victor Julien [Fri, 7 Nov 2014 22:14:26 +0000 (23:14 +0100)] 
detect: per port and proto rule grouping

Replace tree based approach for rule grouping with a per port (tcp/udp)
and per protocol approach.

Grouping now looks like:

               +----+
               |icmp+--->
               +----+
               |gre +--->
               +----+
               |esp +--->
               +----+
          other|... |
        +----->-----+
        |      |N   +--->
        |      +----+
        |
        | tcp  +----+   +----+
        +----->+ 80 +-->+ 139+-->
        |      +----+   +----+
        |
        | udp  +----+   +----+
    +---+----->+ 53 +-->+ 135+-->
    |          +----+   +----+
    |toserver
+--->
    |toclient
    |
    +--->

So the first 'split' in the rules is the direction: toserver or toclient.
Rules that don't have a direction, are in both branches.

Then the split is between tcp/udp and the other protocols. For tcp and
udp port lists are used. For the other protocols, grouping is simply per
protocol.

The ports used are the destination ports for toserver sigs and source
ports for toclient sigs.

9 years agodetect: track direction and ipproto of sgh
Victor Julien [Mon, 28 Sep 2015 08:10:58 +0000 (10:10 +0200)] 
detect: track direction and ipproto of sgh

Each SGH has a unique ipproto and direction.

9 years agodetect: pass ipproto to rule grouping funcs
Victor Julien [Mon, 28 Sep 2015 08:00:36 +0000 (10:00 +0200)] 
detect: pass ipproto to rule grouping funcs

9 years agodetect: delay sgh cleanup
Victor Julien [Mon, 28 Sep 2015 14:02:15 +0000 (16:02 +0200)] 
detect: delay sgh cleanup

9 years agodetect: delay sgh mpm setup
Victor Julien [Mon, 28 Sep 2015 08:17:34 +0000 (10:17 +0200)] 
detect: delay sgh mpm setup

9 years agoStart rule inspect with mask check
Victor Julien [Fri, 7 Nov 2014 22:43:45 +0000 (23:43 +0100)] 
Start rule inspect with mask check

9 years agorule analyzer: add no/both direction warning
Victor Julien [Sun, 27 Sep 2015 08:33:48 +0000 (10:33 +0200)] 
rule analyzer: add no/both direction warning

9 years agodetect: SYN flags
Victor Julien [Wed, 7 Oct 2015 13:38:58 +0000 (15:38 +0200)] 
detect: SYN flags

Add funcs to see if a rule needs a SYN flag in the packet.

9 years agodetect: validate http_method pattern
Victor Julien [Fri, 9 Oct 2015 09:18:36 +0000 (11:18 +0200)] 
detect: validate http_method pattern

Leading and trailing spaces and tabs are invalid as these are not part
of the buffer as returned by libhtp.

9 years agodetect: remove dead code
Victor Julien [Mon, 5 Oct 2015 17:00:24 +0000 (19:00 +0200)] 
detect: remove dead code

9 years agodetect mpm: remove dead code
Victor Julien [Mon, 5 Oct 2015 10:40:57 +0000 (12:40 +0200)] 
detect mpm: remove dead code

9 years agodetect/mpm: remove unused max_id param from API
Victor Julien [Sat, 3 Oct 2015 15:57:27 +0000 (17:57 +0200)] 
detect/mpm: remove unused max_id param from API

9 years agodetect: constify mpm/detect funcs
Victor Julien [Mon, 28 Sep 2015 09:15:09 +0000 (11:15 +0200)] 
detect: constify mpm/detect funcs

9 years agompm: remove obsolete mpm algos
Victor Julien [Tue, 20 Oct 2015 08:19:40 +0000 (10:19 +0200)] 
mpm: remove obsolete mpm algos

Remove: ac-gfbs, wumanber, b2g, b3g.

9 years agompm: fix ac-ks compilation on cygwin
Victor Julien [Mon, 12 Oct 2015 10:49:06 +0000 (12:49 +0200)] 
mpm: fix ac-ks compilation on cygwin

9 years agodetect mpm: ac-tile/ac-ks default to single
Victor Julien [Sun, 20 Sep 2015 08:17:17 +0000 (10:17 +0200)] 
detect mpm: ac-tile/ac-ks default to single

Use sgh-mpm-context single is it is set to 'auto' when ac-ks is used.

9 years agoac-ks: 32bit fixes
Victor Julien [Mon, 12 Oct 2015 06:47:37 +0000 (08:47 +0200)] 
ac-ks: 32bit fixes

9 years agoac-ks: fix mem leaks
Victor Julien [Fri, 9 Oct 2015 14:21:17 +0000 (16:21 +0200)] 
ac-ks: fix mem leaks

9 years agompm: introduce ac-ks
Victor Julien [Sat, 19 Sep 2015 20:59:03 +0000 (22:59 +0200)] 
mpm: introduce ac-ks

Introduce 'ac-ks' or the Kenneth Steele AC implementation. It's
actually 'ac-tile' written by Ken for the Tilera platform. This
patch adds support for it on other architectures as well.

Enable ac-tile for other archs as 'ac-ks'.

Fix a bunch of OOB reads in the loops that triggered ASAN.

9 years agoOpen Suricata 3.1 development branch
Victor Julien [Mon, 4 Apr 2016 16:04:32 +0000 (18:04 +0200)] 
Open Suricata 3.1 development branch

9 years agoUpdate Changelog for 3.0.1 2093/head suricata-3.0.1
Victor Julien [Mon, 4 Apr 2016 10:15:12 +0000 (12:15 +0200)] 
Update Changelog for 3.0.1

9 years agoutil-decode-der: fix NULL dereference bug 1975/head
Mats Klepsland [Thu, 31 Mar 2016 14:15:26 +0000 (16:15 +0200)] 
util-decode-der: fix NULL dereference bug

Make sure that the length is not longer than the size of the buffer
provided.

9 years agoutil-decode-der: fix heap-buffer-overflow
Mats Klepsland [Thu, 31 Mar 2016 12:28:24 +0000 (14:28 +0200)] 
util-decode-der: fix heap-buffer-overflow

9 years agoapp-layer-tls-handshake: fix heap-buffer overflow
Mats Klepsland [Thu, 31 Mar 2016 12:21:21 +0000 (14:21 +0200)] 
app-layer-tls-handshake: fix heap-buffer overflow

Fix heap-buffer overflow that occurs when we are given repeatedly
certificates with the length of zero.

9 years agohyperscan: fix minor coverity warning 1358023 1970/head
Victor Julien [Thu, 31 Mar 2016 08:07:01 +0000 (10:07 +0200)] 
hyperscan: fix minor coverity warning 1358023

*** CID 1358023:  Null pointer dereferences  (REVERSE_INULL)
/src/util-mpm-hs.c: 860 in SCHSDestroyThreadCtx()
854         if (thr_ctx->scratch != NULL) {
855             hs_free_scratch(thr_ctx->scratch);
856             mpm_thread_ctx->memory_cnt--;
857             mpm_thread_ctx->memory_size -= thr_ctx->scratch_size;
858         }
859
>>>     CID 1358023:  Null pointer dereferences  (REVERSE_INULL)
>>>     Null-checking "mpm_thread_ctx->ctx" suggests that it may be null, but it has already been dereferenced on all paths leading to the check.
860         if (mpm_thread_ctx->ctx != NULL) {
861             SCFree(mpm_thread_ctx->ctx);
862             mpm_thread_ctx->ctx = NULL;
863             mpm_thread_ctx->memory_cnt--;
864             mpm_thread_ctx->memory_size -= sizeof(SCHSThreadCtx);
865         }

9 years agohyperscan: fix minor coverity warning 1358024
Victor Julien [Thu, 31 Mar 2016 08:04:44 +0000 (10:04 +0200)] 
hyperscan: fix minor coverity warning 1358024

*** CID 1358024:  Null pointer dereferences  (REVERSE_INULL)
/src/util-mpm-hs.c: 1043 in SCHSPrintInfo()
1037         printf("  SCHSPattern    %" PRIuMAX "\n", (uintmax_t)sizeof(SCHSPattern));
1038         printf("Unique Patterns: %" PRIu32 "\n", mpm_ctx->pattern_cnt);
1039         printf("Smallest:        %" PRIu32 "\n", mpm_ctx->minlen);
1040         printf("Largest:         %" PRIu32 "\n", mpm_ctx->maxlen);
1041         printf("\n");
1042
>>>     CID 1358024:  Null pointer dereferences  (REVERSE_INULL)
>>>     Null-checking "ctx" suggests that it may be null, but it has already been dereferenced on all paths leading to the check.
1043         if (ctx) {
1044             char *db_info = NULL;
1045             if (hs_database_info(pd->hs_db, &db_info) == HS_SUCCESS) {
1046                 printf("HS Database Info: %s\n", db_info);
1047                 SCFree(db_info);
1048             }

9 years agosource-pcap-file: fix stats not being reset
browner87 [Thu, 24 Mar 2016 16:55:34 +0000 (12:55 -0400)] 
source-pcap-file: fix stats not being reset

Fix for redmine #1737. Clears stats between each scanned PCAP instead of just
on load in Unix socket mode.

9 years agoFix for redmine bug 1737
browner87 [Wed, 23 Mar 2016 15:05:05 +0000 (11:05 -0400)] 
Fix for redmine bug 1737

PCAP stats not reset between files in Unix socket mode. Added a memset to the Global Init function to clear these stats.

9 years agobyte-extract: for string, default to base "dec" if not provided
Jason Ish [Wed, 30 Mar 2016 14:46:30 +0000 (08:46 -0600)] 
byte-extract: for string, default to base "dec" if not provided

For Snort compatibility, as Snort defaults to "dec" if not provided.

Fixed issue 1697:
https://redmine.openinfosecfoundation.org/issues/1697

9 years agodetect: fix potential deadlock during reload
Victor Julien [Wed, 30 Mar 2016 17:53:04 +0000 (19:53 +0200)] 
detect: fix potential deadlock during reload

If interrupted during the BreakLoop stage during reload, a deadlock
could happen.

9 years agodetect: fix small mem leak on duplicate sigs 1968/head
Victor Julien [Wed, 30 Mar 2016 07:39:46 +0000 (09:39 +0200)] 
detect: fix small mem leak on duplicate sigs

Direct leak of 80 byte(s) in 5 object(s) allocated from:
    #0 0x4c673b in __interceptor_malloc (/home/victor/dev/suricata/src/suricata+0x4c673b)
    #1 0xb7a425 in DetectEngineSignatureIsDuplicate /home/victor/dev/suricata/src/detect-parse.c:1715:10
    #2 0xb79390 in DetectEngineAppendSig /home/victor/dev/suricata/src/detect-parse.c:1836:19
    #3 0x86fe56 in DetectLoadSigFile /home/victor/dev/suricata/src/detect.c:357:15
    #4 0x815fee in ProcessSigFiles /home/victor/dev/suricata/src/detect.c:419:13
    #5 0x8139a8 in SigLoadSignatures /home/victor/dev/suricata/src/detect.c:499:15
    #6 0xfe435d in LoadSignatures /home/victor/dev/suricata/src/suricata.c:1979:9
    #7 0xfcd87e in main /home/victor/dev/suricata/src/suricata.c:2345:17
    #8 0x7fb66bf7cec4 in __libc_start_main /build/eglibc-3GlaMS/eglibc-2.19/csu/libc-start.c:287

9 years agohyperscan: add DrMemory suppressions 1966/head
Victor Julien [Tue, 29 Mar 2016 09:44:00 +0000 (11:44 +0200)] 
hyperscan: add DrMemory suppressions

9 years agompm: add Hyperscan integration
Justin Viiret [Tue, 15 Mar 2016 01:40:24 +0000 (12:40 +1100)] 
mpm: add Hyperscan integration

This adds an MPM implementation that uses the Hyperscan regex engine
library from Intel, accessible as the "hs" mpm-algo.

9 years agompm: pass offset, depth args to add functions
Justin Viiret [Tue, 15 Mar 2016 01:38:23 +0000 (12:38 +1100)] 
mpm: pass offset, depth args to add functions

MpmAddPatternCI and MpmAddPatternCS had arguments for offset and depth,
but these were not being passed in by the caller.

9 years agoutil-hash-lookup3: Add hashlittle_safe() variant
Justin Viiret [Mon, 28 Mar 2016 22:32:26 +0000 (09:32 +1100)] 
util-hash-lookup3: Add hashlittle_safe() variant

By default, hashlittle() will read off the end of the key, up to the
next four-byte boundary, although the data beyond the end of the key
doesn't affect the hash. This read causes uninitialized read warnings
from Valgrind and Address Sanitizer.

Here we add hashlittle_safe(), which avoids reading off the end of the
buffer (using the code inside the VALGRIND-guarded block in the original
hashlittle() implementation).

9 years agocapture: warn -i user if faster options are available
Victor Julien [Wed, 25 Jun 2014 14:16:30 +0000 (16:16 +0200)] 
capture: warn -i user if faster options are available

If af-packet, netmap or pfring are available, users should use those
for best performance.

9 years agonetmap: implement capture inject packet flag
Victor Julien [Sat, 26 Mar 2016 18:56:00 +0000 (19:56 +0100)] 
netmap: implement capture inject packet flag

9 years agodetect reload: generic packet injection for capture
Victor Julien [Wed, 23 Mar 2016 16:05:14 +0000 (17:05 +0100)] 
detect reload: generic packet injection for capture

Capture methods that are non blocking will still not generate packets
that go through the system if there is no traffic. Some maintenance
tasks, like rule reloads rely on packets to complete.

This patch introduces a new thread flag, THV_CAPTURE_INJECT_PKT, that
instructs the capture thread to create a fake packet.

The capture implementations can call the TmThreadsCaptureInjectPacket
utility function either with the packet they already got from the pool
or without a packet. In this case the util func will get it's own
packet.

Implementations for pcap, AF_PACKET and PF_RING.

9 years agodetect reload: call 'breakloop' on capture method
Victor Julien [Wed, 23 Mar 2016 15:16:41 +0000 (16:16 +0100)] 
detect reload: call 'breakloop' on capture method

Split wait loop into three steps:
- first insert pseudo packets
- 2nd nudge all capture threads to break out of their loop
- third, wait for the detection thread contexts to be used

Interupt capture more than once if needed

Move packet injection into util func

9 years agosignals: cleanup signal handling
Victor Julien [Thu, 24 Mar 2016 10:51:49 +0000 (11:51 +0100)] 
signals: cleanup signal handling

Simplify handling of USR2 signal. The SCLogInfo usage could lead to
dead locks as the SCLog API can do many complicated things including
memory allocations, syslog calls, libjansson message construction.

If an existing malloc call was interupted, it could lead to the
following dead lock:

 0  __lll_lock_wait_private () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:97
 1  0x0000003140c7d2df in _L_lock_10176 () from /lib64/libc.so.6
 2  0x0000003140c7ab83 in __libc_malloc (bytes=211543457408) at malloc.c:3655
 3  0x0000003140c80ec2 in __strdup (s=0x259ca40 "[%i] %t - (%f:%l) <%d> (%n) -- ") at strdup.c:43
 4  0x000000000059dd4a in SCLogMessageGetBuffer (tval=0x7fff52b47360, color=1, type=SC_LOG_OP_TYPE_REGULAR, buffer=0x7fff52b47370 "", buffer_size=2048,
    log_format=0x259ca40 "[%i] %t - (%f:%l) <%d> (%n) -- ", log_level=SC_LOG_INFO, file=0x63dd00 "suricata.c", line=287, function=0x640f50 "SignalHandlerSigusr2StartingUp", error_code=SC_OK,
    message=0x7fff52b47bb0 "Live rule reload only possible after engine completely started.") at util-debug.c:307
 5  0x000000000059e940 in SCLogMessage (log_level=SC_LOG_INFO, file=0x63dd00 "suricata.c", line=287, function=0x640f50 "SignalHandlerSigusr2StartingUp", error_code=SC_OK,
    message=0x7fff52b47bb0 "Live rule reload only possible after engine completely started.") at util-debug.c:549
 6  0x000000000057e374 in SignalHandlerSigusr2StartingUp (sig=12) at suricata.c:287
 7  <signal handler called>
 8  _int_malloc (av=0x3140f8fe80, bytes=<value optimized out>) at malloc.c:4751
 9  0x0000003140c7ab1c in __libc_malloc (bytes=296) at malloc.c:3657
 10 0x0000000000504d55 in FlowAlloc () at flow-util.c:60
 11 0x00000000004fd909 in FlowInitConfig (quiet=0 '\000') at flow.c:454
 12 0x0000000000584c8e in main (argc=6, argv=0x7fff52b4a3b8) at suricata.c:2300

This patch simply sets a variable and lets the main loop act on that.

9 years agosignals: handle INT/TERM signals in the main loop
Victor Julien [Thu, 24 Mar 2016 09:21:11 +0000 (10:21 +0100)] 
signals: handle INT/TERM signals in the main loop

9 years agopfring: use likely for fast path
Victor Julien [Thu, 24 Mar 2016 08:31:19 +0000 (09:31 +0100)] 
pfring: use likely for fast path

9 years agotcp: fix unlikely NULL-ptr dereference
Victor Julien [Sat, 26 Mar 2016 11:05:50 +0000 (12:05 +0100)] 
tcp: fix unlikely NULL-ptr dereference

If a TCP packet could not get a flow (flow engine out of flows/memory)
and there were *only* TCP inspecting rules with the direction
explicitly set to 'to_server', a NULL pointer deref could happen.

PacketPatternSearchWithStreamCtx would fall through to the 'to_client'
case which was not initialized.

9 years agodetect: add corner case mpm test
Victor Julien [Mon, 28 Sep 2015 10:12:37 +0000 (12:12 +0200)] 
detect: add corner case mpm test

9 years agotx logging: fix potential missed logging issue
Victor Julien [Thu, 17 Dec 2015 09:34:17 +0000 (10:34 +0100)] 
tx logging: fix potential missed logging issue

Wrong scope of proto_logged variable could potentially lead to
incrementing logged tx id w/o actually being logged.

Reported-By: Jason Ish
9 years agostats-log: fix layout issue due to decoder stats
Victor Julien [Sat, 26 Mar 2016 10:28:30 +0000 (11:28 +0100)] 
stats-log: fix layout issue due to decoder stats

9 years agoautofp: print packet scheduler info only on autofp
Victor Julien [Sat, 26 Mar 2016 09:59:52 +0000 (10:59 +0100)] 
autofp: print packet scheduler info only on autofp

To avoid confusion about what runmode is active, only print autofp
related scheduler information if autofp is the actual runmode.

9 years agoautofp: add "ippair" scheduler
cdwakelin [Wed, 23 Mar 2016 17:13:55 +0000 (17:13 +0000)] 
autofp: add "ippair" scheduler

Add "ippair" autofp scheduler to split traffic based on source and
destination IP only (not ports).

- This is useful when using the "xbits" feature to track events
  that occur between the same hosts but not necessarily the same
  flow (such as exploit kit landings/expoits/payloads)
- The disadvantage is that traffic may be balanced very unevenly
  between threads if some host pairs are much more frequently seen
  than others, so it may be only practicable for sandbox or pcap
  analysis
- not tested for IPv6

See https://redmine.openinfosecfoundation.org/issues/1661

9 years agochangelog: update for 3.0.1RC1 suricata-3.0.1RC1
Victor Julien [Wed, 23 Mar 2016 10:42:05 +0000 (11:42 +0100)] 
changelog: update for 3.0.1RC1

9 years agoflow: record the flow hash for use as the output flow id 1954/head
Jason Ish [Thu, 10 Mar 2016 20:32:06 +0000 (14:32 -0600)] 
flow: record the flow hash for use as the output flow id

Provides a consistent hash for a flow, as well as a better
distribution than using a memory address.

9 years agooutput-json-email: fix memleak
Eric Leblond [Mon, 21 Mar 2016 14:27:55 +0000 (15:27 +0100)] 
output-json-email: fix memleak

This patch fixes some error handling in code generating JSON output for
email event.

This fixes:

Indirect leak of 128 byte(s) in 1 object(s) allocated from:
    #0 0x50c142 in malloc (/home/eric/git/oisf/src/.libs/lt-suricata+0x50c142)
    #1 0x7ff92394771c  (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x371c)
    #2 0x1bd3309 in JsonEmailLogJson /home/eric/git/oisf/src/output-json-email-common.c:376:19
    #3 0x1bfe774 in JsonSmtpLogger /home/eric/git/oisf/src/output-json-smtp.c:103:9
    #4 0x1c378ff in OutputTxLog /home/eric/git/oisf/src/output-tx.c:165:17
    #5 0x1f94ef3 in TmThreadsSlotVarRun /home/eric/git/oisf/src/tm-threads.c:134:17
    #6 0x1d33478 in TmThreadsSlotProcessPkt /home/eric/git/oisf/src/./tm-threads.h:150:9
    #7 0x1d32dd4 in PcapFileCallbackLoop /home/eric/git/oisf/src/source-pcap-file.c:184:9
    #8 0x7ff924199013  (/usr/lib/x86_64-linux-gnu/libpcap.so.0.8+0x1f013)

Indirect leak of 96 byte(s) in 3 object(s) allocated from:
    #0 0x50c142 in malloc (/home/eric/git/oisf/src/.libs/lt-suricata+0x50c142)
    #1 0x7ff92394bc7b  (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x7c7b)

Indirect leak of 82 byte(s) in 3 object(s) allocated from:
    #0 0x50c142 in malloc (/home/eric/git/oisf/src/.libs/lt-suricata+0x50c142)
    #1 0x7ff923949924  (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x5924)

Indirect leak of 72 byte(s) in 1 object(s) allocated from:
    #0 0x50c142 in malloc (/home/eric/git/oisf/src/.libs/lt-suricata+0x50c142)
    #1 0x7ff92394bcda in json_object (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x7cda)
    #2 0x1bd3309 in JsonEmailLogJson /home/eric/git/oisf/src/output-json-email-common.c:376:19
    #3 0x1bfe774 in JsonSmtpLogger /home/eric/git/oisf/src/output-json-smtp.c:103:9
    #4 0x1c378ff in OutputTxLog /home/eric/git/oisf/src/output-tx.c:165:17
    #5 0x1f94ef3 in TmThreadsSlotVarRun /home/eric/git/oisf/src/tm-threads.c:134:17
    #6 0x1d33478 in TmThreadsSlotProcessPkt /home/eric/git/oisf/src/./tm-threads.h:150:9
    #7 0x1d32dd4 in PcapFileCallbackLoop /home/eric/git/oisf/src/source-pcap-file.c:184:9
    #8 0x7ff924199013  (/usr/lib/x86_64-linux-gnu/libpcap.so.0.8+0x1f013)

Indirect leak of 64 byte(s) in 1 object(s) allocated from:
    #0 0x50c142 in malloc (/home/eric/git/oisf/src/.libs/lt-suricata+0x50c142)
    #1 0x7ff92394bf5a in json_array (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x7f5a)
    #2 0x1bd04b5 in JsonEmailLogJsonData /home/eric/git/oisf/src/output-json-email-common.c:296:27
    #3 0x1bd3309 in JsonEmailLogJson /home/eric/git/oisf/src/output-json-email-common.c:376:19
    #4 0x1bfe774 in JsonSmtpLogger /home/eric/git/oisf/src/output-json-smtp.c:103:9
    #5 0x1c378ff in OutputTxLog /home/eric/git/oisf/src/output-tx.c:165:17
    #6 0x1f94ef3 in TmThreadsSlotVarRun /home/eric/git/oisf/src/tm-threads.c:134:17
    #7 0x1d33478 in TmThreadsSlotProcessPkt /home/eric/git/oisf/src/./tm-threads.h:150:9
    #8 0x1d32dd4 in PcapFileCallbackLoop /home/eric/git/oisf/src/source-pcap-file.c:184:9
    #9 0x7ff924199013  (/usr/lib/x86_64-linux-gnu/libpcap.so.0.8+0x1f013)

Indirect leak of 48 byte(s) in 1 object(s) allocated from:
    #0 0x50c142 in malloc (/home/eric/git/oisf/src/.libs/lt-suricata+0x50c142)
    #1 0x7ff92394bf2a in json_array (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x7f2a)
    #2 0x1bd04b5 in JsonEmailLogJsonData /home/eric/git/oisf/src/output-json-email-common.c:296:27
    #3 0x1bd3309 in JsonEmailLogJson /home/eric/git/oisf/src/output-json-email-common.c:376:19
    #4 0x1bfe774 in JsonSmtpLogger /home/eric/git/oisf/src/output-json-smtp.c:103:9
    #5 0x1c378ff in OutputTxLog /home/eric/git/oisf/src/output-tx.c:165:17
    #6 0x1f94ef3 in TmThreadsSlotVarRun /home/eric/git/oisf/src/tm-threads.c:134:17
    #7 0x1d33478 in TmThreadsSlotProcessPkt /home/eric/git/oisf/src/./tm-threads.h:150:9
    #8 0x1d32dd4 in PcapFileCallbackLoop /home/eric/git/oisf/src/source-pcap-file.c:184:9
    #9 0x7ff924199013  (/usr/lib/x86_64-linux-gnu/libpcap.so.0.8+0x1f013)

9 years agooutput-json-email: fix memleak
Eric Leblond [Fri, 18 Mar 2016 08:53:21 +0000 (09:53 +0100)] 
output-json-email: fix memleak

This possibly fix:

ndirect leak of 64 byte(s) in 1 object(s) allocated from:
    #0 0x4c264b in malloc (/home/victor/qa/buildbot/donkey/z600fuzz/Private/src/.libs/lt-suricata+0x4c264b)
    #1 0x7fb09c1e8aaa in json_array (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x6aaa)
    #2 0xd67553 in JsonEmailLogJsonData /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/output-json-email-common.c:290:27
    #3 0xd6a272 in JsonEmailLogJson /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/output-json-email-common.c:370:19
    #4 0xd956b9 in JsonSmtpLogger /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/output-json-smtp.c:103:9
    #5 0xdcedac in OutputTxLog /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/output-tx.c:165:17
    #6 0xff6669 in TmThreadsSlotVarRun /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/tm-threads.c:132:17
    #7 0xffecc1 in TmThreadsSlotVar /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/tm-threads.c:474:17
    #8 0x7fb09bfcc181 in start_thread /build/eglibc-3GlaMS/eglibc-2.19/nptl/pthread_create.c:312

9 years agooutput-json-email: fix memleak in error path
Eric Leblond [Fri, 18 Mar 2016 08:28:58 +0000 (09:28 +0100)] 
output-json-email: fix memleak in error path

In JsonEmailLogJsonData function, an invalid state was leading to
early exit without a proper freeing of resources.

This should fix:

Indirect leak of 72 byte(s) in 1 object(s) allocated from:
    #0 0x4c264b in malloc (/home/victor/qa/buildbot/donkey/z600fuzz/Private/src/.libs/lt-suricata+0x4c264b)
    #1 0x7fb09c1e886a in json_object (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x686a)
    #2 0xd6a272 in JsonEmailLogJson /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/output-json-email-common.c:370:19
    #3 0xd956b9 in JsonSmtpLogger /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/output-json-smtp.c:103:9
    #4 0xdcedac in OutputTxLog /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/output-tx.c:165:17
    #5 0xff6669 in TmThreadsSlotVarRun /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/tm-threads.c:132:17
    #6 0xffecc1 in TmThreadsSlotVar /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/tm-threads.c:474:17
    #7 0x7fb09bfcc181 in start_thread /build/eglibc-3GlaMS/eglibc-2.19/nptl/pthread_create.c:312

9 years agooutput-json-email: fix error code
Eric Leblond [Fri, 18 Mar 2016 08:23:31 +0000 (09:23 +0100)] 
output-json-email: fix error code

Returning early if first function call fail could avoid some issue.

9 years agoFix typos in an error message
JonZeolla [Wed, 16 Mar 2016 13:28:02 +0000 (09:28 -0400)] 
Fix typos in an error message

Fix two minor typos in a SC_ERR_INVALID_ARGUMENT error message.

9 years agonetmap: fix issue 1717 1950/head
gureedo [Mon, 21 Mar 2016 10:14:34 +0000 (15:14 +0500)] 
netmap: fix issue 1717
Use packet pool only without packet allocation.
Wait for N packets available in packet pool before netmap ring fetching.

9 years agonetmap: fix log typo
gureedo [Mon, 21 Mar 2016 09:41:42 +0000 (14:41 +0500)] 
netmap: fix log typo

9 years agonetmap: remove workers zero copy mode restriction between interfaces
gureedo [Mon, 21 Mar 2016 09:38:53 +0000 (14:38 +0500)] 
netmap: remove workers zero copy mode restriction between interfaces
Rationale: https://github.com/luigirizzo/netmap/issues/126

9 years agoapp-layer-ssl: fix out of bounds memory read 1946/head
Mats Klepsland [Thu, 17 Mar 2016 10:56:52 +0000 (11:56 +0100)] 
app-layer-ssl: fix out of bounds memory read

Added several checks to avoid out of bounds memory read caused
by malformed traffic.

9 years agoip parsing: fix cppcheck negative shift warning 1943/head
Victor Julien [Wed, 16 Mar 2016 11:37:25 +0000 (12:37 +0100)] 
ip parsing: fix cppcheck negative shift warning

[src/util-ip.c:104]: (error) Shifting a negative value is undefined behaviour
[src/util-radix-tree.c:1160]: (error) Shifting a negative value is undefined behaviour
[src/util-radix-tree.c:1357]: (error) Shifting a negative value is undefined behaviour
[src/util-radix-tree.c:1380]: (error) Shifting a negative value is undefined behaviour
[src/util-radix-tree.c:1438]: (error) Shifting a negative value is undefined behaviour

9 years agostats: fix unix socket crash
Victor Julien [Wed, 16 Mar 2016 11:20:17 +0000 (12:20 +0100)] 
stats: fix unix socket crash

Reset counters_global_id at ctx destruction. In the unix socket
runmode the lack of this reset would cause the id's to increase with
each pcap, leading to an ever larger stats array.

9 years agodetect: denote the max detection list; fix issue 1674.
Jason Ish [Wed, 27 Jan 2016 05:22:27 +0000 (23:22 -0600)] 
detect: denote the max detection list; fix issue 1674.

Denotes the max detection list so that rule validation can
allow post-detection lists to come after base64_data, but
disallow detection lists to come after it.

9 years agosmtp: use detection file tracking
Victor Julien [Wed, 9 Mar 2016 19:44:26 +0000 (20:44 +0100)] 
smtp: use detection file tracking

As SMTP file_data detection uses the file API, the file's inspect
tracker should be considered when pruning files.

This patch sets the FILE_USE_DETECT flag on files tracked by smtp.

It also adds logic to move inspected tracker ahead if detection
doesn't do it, like when no rules are matching or detection engine
is disabled.

9 years agofile: optionally use detect tracking in pruning
Victor Julien [Wed, 9 Mar 2016 19:43:54 +0000 (20:43 +0100)] 
file: optionally use detect tracking in pruning

When the file API is used to do content inspection (currently only
smtp does this), the detection should be considered while pruning
the file chunks.

This patch introduces a new flag for the file API: FILE_USE_DETECT

When it is used, 'FilePrune' will not remove chunks that are (partly)
beyond the File::content_inspected tracker.

When using this flag, it's important to realize that when the detect
engine is disabled or rules are not matching, content_inspected
might not get updated.

9 years agosmtp: add file_data debug code
Victor Julien [Wed, 9 Mar 2016 16:24:51 +0000 (17:24 +0100)] 
smtp: add file_data debug code

9 years agofile logging: fix bad end of file check
Victor Julien [Thu, 10 Mar 2016 08:56:15 +0000 (09:56 +0100)] 
file logging: fix bad end of file check

File storage could end file storing when the logger was invoked
while no new data chunks were available.

9 years agosmtp: fix config parsing and config defaults
Victor Julien [Tue, 15 Dec 2015 09:13:48 +0000 (10:13 +0100)] 
smtp: fix config parsing and config defaults

9 years agosmtp: consider file state in file_data inspection limits
Victor Julien [Mon, 14 Dec 2015 17:34:38 +0000 (18:34 +0100)] 
smtp: consider file state in file_data inspection limits

9 years agopfring: strip prio from vlan id 1938/head
Victor Julien [Tue, 15 Mar 2016 17:01:06 +0000 (18:01 +0100)] 
pfring: strip prio from vlan id

9 years agoafpacket: strip prio from vlan id
Victor Julien [Tue, 15 Mar 2016 17:00:54 +0000 (18:00 +0100)] 
afpacket: strip prio from vlan id

9 years agocppcheck: work around snprintf warning 1937/head
Victor Julien [Mon, 7 Mar 2016 15:55:09 +0000 (16:55 +0100)] 
cppcheck: work around snprintf warning

Cppcheck 1.72 gives a warning on the following code pattern:

    char blah[32] = "";
    snprintf(blah, sizeof(blah), "something");

The warning is:

    (error) Buffer is accessed out of bounds.

While this appears to be a FP, in most cases the initialization to ""
was unnecessary as the snprintf statement immediately follows the
variable declaration.

9 years agorules: add rules for TLS SNI app layer events
Mats Klepsland [Tue, 15 Mar 2016 07:08:07 +0000 (08:08 +0100)] 
rules: add rules for TLS SNI app layer events

9 years agoapp-layer-ssl: fix memleak
Mats Klepsland [Fri, 11 Mar 2016 11:55:38 +0000 (12:55 +0100)] 
app-layer-ssl: fix memleak

Avoid that the SNI extension code is executed twice sometimes, causing
memory leaks.

9 years agodaemon: suppress coverity resource leak warning (CID 400725) 1927/head
Victor Julien [Fri, 11 Mar 2016 08:49:57 +0000 (09:49 +0100)] 
daemon: suppress coverity resource leak warning (CID 400725)

9 years agocoverity: suppress CID 1038112
Victor Julien [Fri, 11 Mar 2016 08:46:38 +0000 (09:46 +0100)] 
coverity: suppress CID 1038112

9 years agohtp: optimize multipart parsing
Victor Julien [Fri, 11 Mar 2016 08:39:20 +0000 (09:39 +0100)] 
htp: optimize multipart parsing

Store boundary in a single stack buffer instead of 2 heap allocated
buffers.

9 years agohtp: fix boundry error handling (CID 1153934)
Victor Julien [Fri, 11 Mar 2016 08:29:20 +0000 (09:29 +0100)] 
htp: fix boundry error handling (CID 1153934)

9 years agohtp: fix dead code warning (CID 1153933)
Victor Julien [Fri, 11 Mar 2016 08:25:23 +0000 (09:25 +0100)] 
htp: fix dead code warning (CID 1153933)

9 years agounified2: fix unused value warning (CID 1247404)
Victor Julien [Fri, 11 Mar 2016 08:23:07 +0000 (09:23 +0100)] 
unified2: fix unused value warning (CID 1247404)

9 years agolua: fix unlikely null-deref during setup (CID 1351209)
Victor Julien [Fri, 11 Mar 2016 06:14:39 +0000 (07:14 +0100)] 
lua: fix unlikely null-deref during setup (CID 1351209)

9 years agobase64_decode: fix potential keyword parsing issue (CID 1340064)
Victor Julien [Fri, 11 Mar 2016 06:07:07 +0000 (07:07 +0100)] 
base64_decode: fix potential keyword parsing issue (CID 1340064)

9 years agoioctl: fix NULL check after dereference (CID 1204326)
Victor Julien [Fri, 11 Mar 2016 06:01:06 +0000 (07:01 +0100)] 
ioctl: fix NULL check after dereference (CID 1204326)

9 years agolua: fix minor coverity issues
Victor Julien [Fri, 11 Mar 2016 05:56:35 +0000 (06:56 +0100)] 
lua: fix minor coverity issues

Remove checks that can never be false. CID 12320761312012

9 years agoafpacket: suppress harmless Coverity warning (1192960)
Victor Julien [Fri, 11 Mar 2016 05:54:03 +0000 (06:54 +0100)] 
afpacket: suppress harmless Coverity warning (1192960)

9 years agoFix compilation on systems with Musl 1926/head
Victor Julien [Sat, 12 Mar 2016 09:32:54 +0000 (10:32 +0100)] 
Fix compilation on systems with Musl

9 years agosmtp: clean up thread local memory 1923/head
Victor Julien [Wed, 9 Mar 2016 16:02:06 +0000 (17:02 +0100)] 
smtp: clean up thread local memory

9 years agostream-tcp: improve function doc
Victor Julien [Tue, 22 Dec 2015 09:09:49 +0000 (10:09 +0100)] 
stream-tcp: improve function doc

9 years agoAdd global MAX macro.
Victor Julien [Wed, 7 Oct 2015 13:56:01 +0000 (15:56 +0200)] 
Add global MAX macro.

9 years agodecode-events: counters for decode events errors 1921/head
Maurizio Abba [Thu, 10 Mar 2016 13:58:21 +0000 (13:58 +0000)] 
decode-events: counters for decode events errors

We want to add counters in order to track the number of times we hit a
decode event. A decode event is related to an error in the protocol
decoding over a certain packet.

This patch fist modifies the decode-event list, reordering it in order
to separate single packet events from stream-related events and adding
the prefix "decoder" to decode events.
The counters are created during the decode setup and the relative event
counter is increased every time a packet with the flag PKT_IS_INVALID is
finalized in the decode phase

9 years agoaf-packet: don't check GRO LRO on non ethernet 1917/head
Eric Leblond [Wed, 9 Mar 2016 13:29:19 +0000 (14:29 +0100)] 
af-packet: don't check GRO LRO on non ethernet

This way we avoid an error message when sniffing on a non Ethernet
device.