]> git.ipfire.org Git - thirdparty/suricata.git/log
thirdparty/suricata.git
3 years agologopenfile: fix minor format string warning 7322/head
Victor Julien [Wed, 27 Apr 2022 09:39:27 +0000 (11:39 +0200)] 
logopenfile: fix minor format string warning

cppcheck:

src/util-logopenfile.c:743:13: warning: %d in format string (no. 2) requires 'int' but the argument type is 'unsigned int'. [invalidPrintfArgType_sint]
            snprintf(threaded_name, len, "%s.%d.%s", tname, unique_id, ext);
            ^
src/util-logopenfile.c:752:9: warning: %d in format string (no. 2) requires 'int' but the argument type is 'unsigned int'. [invalidPrintfArgType_sint]
        snprintf(threaded_name, len, "%s.%d", original_name, unique_id);
        ^

Bug: #5291.

3 years agoja3: fix minor format string warning
Victor Julien [Wed, 27 Apr 2022 09:38:37 +0000 (11:38 +0200)] 
ja3: fix minor format string warning

cppcheck:

src/util-ja3.c:197:28: warning: %d in format string (no. 1) requires 'int' but the argument type is 'unsigned int'. [invalidPrintfArgType_sint]
        (*buffer)->used += snprintf((*buffer)->data, (*buffer)->size, "%d",
                           ^
src/util-ja3.c:201:28: warning: %d in format string (no. 1) requires 'int' but the argument type is 'unsigned int'. [invalidPrintfArgType_sint]
        (*buffer)->used += snprintf((*buffer)->data + (*buffer)->used,
                           ^

Bug: #5291.

3 years agolog-pcap: remove redundant check
Victor Julien [Wed, 27 Apr 2022 09:36:21 +0000 (11:36 +0200)] 
log-pcap: remove redundant check

Check is always true but confuses cppcheck:

src/log-pcap.c:1224:32: warning: Either the condition 'filename' is redundant or there is possible null pointer dereference: filename. [nullPointerRedundantCheck]
    if ((pl->prefix = SCStrdup(filename)) == NULL) {
                               ^
src/log-pcap.c:1421:9: note: Assuming that condition 'filename' is not redundant
    if (filename) {
        ^
src/log-pcap.c:1224:32: note: Null pointer dereference
    if ((pl->prefix = SCStrdup(filename)) == NULL) {
                               ^

Bug: #5291.

3 years agoaf-packet/v2: use proper type for ring
Victor Julien [Wed, 27 Apr 2022 09:32:22 +0000 (11:32 +0200)] 
af-packet/v2: use proper type for ring

cppcheck:

src/source-af-packet.c:1762:19: warning: Size of pointer 'v2' used instead of size of its data. This is likely to lead to a buffer overflow. You probably intend to write 'sizeof(*v2)'. [pointerSize]
        ptv->ring.v2 = SCMalloc(ptv->req.v2.tp_frame_nr * sizeof (union thdr *));
                  ^
src/source-af-packet.c:1767:26: warning: Size of pointer 'v2' used instead of size of its data. This is likely to lead to a buffer overflow. You probably intend to write 'sizeof(*v2)'. [pointerSize]
        memset(ptv->ring.v2, 0, ptv->req.v2.tp_frame_nr * sizeof (union thdr *));
                         ^

scan-build:

CC       source-af-packet.o
source-af-packet.c:1762:24: warning: Result of 'malloc' is converted to a pointer of type 'char', which is incompatible with sizeof operand type 'union thdr *' [unix.MallocSizeof]
        ptv->ring.v2 = SCMalloc(ptv->req.v2.tp_frame_nr * sizeof (union thdr *));
                       ^~~~~~~~                           ~~~~~~~~~~~~~~~~~~~~~
./util-mem.h:35:18: note: expanded from macro 'SCMalloc'
                 ^~~~~~
1 warning generated.

Bug: #5291.

3 years agodetect/pcre: assist code analyzer around pointer logic
Victor Julien [Tue, 26 Apr 2022 19:47:37 +0000 (21:47 +0200)] 
detect/pcre: assist code analyzer around pointer logic

cppcheck:

src/detect-pcre.c:381:27: warning: Either the condition 'pcap' is redundant or there is overflow in pointer subtraction. [nullPointerArithmeticRedundantCheck]
            cut_capture = MIN((pcap - regexstr), (fcap - regexstr));
                          ^
src/detect-pcre.c:378:18: note: Assuming that condition 'pcap' is not redundant
        else if (pcap && !fcap)
                 ^
src/detect-pcre.c:381:27: note: Null pointer subtraction
            cut_capture = MIN((pcap - regexstr), (fcap - regexstr));
                          ^

Bug: #5291.

3 years agodevice: avoid uninit var warning
Victor Julien [Tue, 26 Apr 2022 19:35:29 +0000 (21:35 +0200)] 
device: avoid uninit var warning

cppcheck:

src/util-device.c:455:17: error: Uninitialized variables: *ndev.dev, *ndev.tenant_id_set, *ndev.id, *ndev.next, *ndev.tenant_id, *ndev.offload_orig [uninitvar]
        *ldev = *ndev;
                ^
src/util-device.c:618:36: note: Calling function 'LiveDeviceForEach', 2nd argument '&ndev' value is <Uninit>
    while(LiveDeviceForEach(&ldev, &ndev)) {
                                   ^
src/util-device.c:455:17: note: Uninitialized variables: *ndev.dev, *ndev.tenant_id_set, *ndev.id, *ndev.next, *ndev.tenant_id, *ndev.offload_orig
        *ldev = *ndev;
                ^

Bug: #5291.

3 years agodetect: fix bad BUG_ON pattern
Victor Julien [Tue, 26 Apr 2022 19:33:52 +0000 (21:33 +0200)] 
detect: fix bad BUG_ON pattern

cppcheck:

src/detect-engine-uint.c:73:13: warning: Conversion of string literal "unknown mode" to bool always evaluates to true. [incorrectStringBooleanError]
            BUG_ON("unknown mode");
            ^
src/detect-engine-uint.c:328:13: warning: Conversion of string literal "unknown mode" to bool always evaluates to true. [incorrectStringBooleanError]
            BUG_ON("unknown mode");
            ^
src/detect-pcre.c:291:25: warning: Conversion of string literal "Impossible captype" to bool always evaluates to true. [incorrectStringBooleanError]
                        BUG_ON("Impossible captype");
                        ^

Bug: #5291.

3 years agotime: fix warning in timestring creation
Victor Julien [Tue, 26 Apr 2022 19:03:42 +0000 (21:03 +0200)] 
time: fix warning in timestring creation

cppcheck:

src/util-time.c:255:18: warning: Either the condition 'str!=NULL' is redundant or there is possible null pointer dereference: str. [nullPointerRedundantCheck]
        snprintf(str, size, "ts-error");
                 ^
src/util-time.c:252:48: note: Assuming that condition 'str!=NULL' is not redundant
    if (likely(t != NULL && fmt != NULL && str != NULL)) {
                                               ^
src/util-time.c:255:18: note: Null pointer dereference
        snprintf(str, size, "ts-error");
                 ^

Only `t` could possibly be NULL if `localtime_r` fails elsewhere.

Bug: #5291.

3 years agodetect/multi-tentancy: minor format string fixes
Victor Julien [Tue, 26 Apr 2022 18:36:36 +0000 (20:36 +0200)] 
detect/multi-tentancy: minor format string fixes

cppcheck:

src/detect-engine.c:3643:5: warning: %d in format string (no. 1) requires 'int' but the argument type is 'unsigned int'. [invalidPrintfArgType_sint]
    snprintf(prefix, sizeof(prefix), "multi-detect.%d", tenant_id);
    ^
src/detect-engine.c:3707:5: warning: %d in format string (no. 1) requires 'int' but the argument type is 'unsigned int'. [invalidPrintfArgType_sint]
    snprintf(prefix, sizeof(prefix), "multi-detect.%d.reload.%d", tenant_id, reload_cnt);
    ^
src/detect-engine.c:4086:17: warning: %d in format string (no. 1) requires 'int' but the argument type is 'unsigned int'. [invalidPrintfArgType_sint]
                snprintf(prefix, sizeof(prefix), "multi-detect.%d", tenant_id);
                ^

Bug: #5291.

3 years agoreference: remove useless var reset
Victor Julien [Tue, 26 Apr 2022 18:18:28 +0000 (20:18 +0200)] 
reference: remove useless var reset

cppcheck:

src/util-reference-config.c:179:9: warning: Assignment of function parameter has no effect outside the function. Did you forget dereferencing it? [uselessAssignmentPtrArg]
        fd = NULL;
        ^

Bug: #5291.

3 years agorunmodes: minor format string fixes
Victor Julien [Tue, 26 Apr 2022 18:17:27 +0000 (20:17 +0200)] 
runmodes: minor format string fixes

cppcheck:

src/util-runmodes.c:210:9: warning: %u in format string (no. 2) requires 'unsigned int' but the argument type is 'signed int'. [invalidPrintfArgType_uint]
        snprintf(tname, sizeof(tname), "%s#%02u", thread_name_workers, thread+1);
        ^
src/util-runmodes.c:211:9: warning: %u in format string (no. 1) requires 'unsigned int' but the argument type is 'signed int'. [invalidPrintfArgType_uint]
        snprintf(qname, sizeof(qname), "pickup%u", thread+1);
        ^
src/util-runmodes.c:455:9: warning: %u in format string (no. 2) requires 'unsigned int' but the argument type is 'signed int'. [invalidPrintfArgType_uint]
        snprintf(tname, sizeof(tname), "%s#%02u", thread_name_workers, thread+1);
        ^
src/util-runmodes.c:457:9: warning: %u in format string (no. 1) requires 'unsigned int' but the argument type is 'signed int'. [invalidPrintfArgType_uint]
        snprintf(qname, sizeof(qname), "pickup%u", thread+1);
        ^

src/runmode-erf-file.c:188:9: warning: %u in format string (no. 2) requires 'unsigned int' but the argument type is 'signed int'. [invalidPrintfArgType_uint]
        snprintf(tname, sizeof(tname), "%s#%02u", thread_name_workers, thread+1);
        ^
src/runmode-erf-file.c:189:9: warning: %u in format string (no. 1) requires 'unsigned int' but the argument type is 'signed int'. [invalidPrintfArgType_uint]
        snprintf(qname, sizeof(qname), "pickup%u", thread+1);
        ^
src/runmode-pcap-file.c:201:9: warning: %u in format string (no. 2) requires 'unsigned int' but the argument type is 'signed int'. [invalidPrintfArgType_uint]
        snprintf(tname, sizeof(tname), "%s#%02u", thread_name_workers, thread+1);
        ^
src/runmode-pcap-file.c:202:9: warning: %u in format string (no. 1) requires 'unsigned int' but the argument type is 'signed int'. [invalidPrintfArgType_uint]
        snprintf(qname, sizeof(qname), "pickup%u", thread+1);
        ^

Bug: #5291.

3 years agompm/ac-ks: address int handling issues
Victor Julien [Tue, 26 Apr 2022 18:14:39 +0000 (20:14 +0200)] 
mpm/ac-ks: address int handling issues

cppcheck:

src/util-mpm-ac-ks.c:1452:5: warning: %d in format string (no. 1) requires 'int' but the argument type is 'unsigned int'. [invalidPrintfArgType_sint]
    printf("Total states in the state table:    %d\n", ctx->state_count);
    ^
src/util-mpm-ac-ks.c:606:34: error: Signed integer overflow for expression '1<<31'. [integerOverflow]
        encoded_next_state |= (1 << 31);
                                 ^

Bug: #5291.

3 years agoclassification: remove useless clear
Victor Julien [Tue, 26 Apr 2022 18:12:20 +0000 (20:12 +0200)] 
classification: remove useless clear

cppcheck:

src/util-classification-config.c:189:9: warning: Assignment of function parameter has no effect outside the function. Did you forget dereferencing it? [uselessAssignmentPtrArg]
        fd = NULL;
        ^

Bug: #5291.

3 years agodetect/content-inspect: code cleanup
Victor Julien [Tue, 26 Apr 2022 18:06:43 +0000 (20:06 +0200)] 
detect/content-inspect: code cleanup

Rearrange code slightly to make it more clear that `found` cannot
be NULL further down the loop.

cppcheck:

src/detect-engine-content-inspection.c:316:50: warning: Either the condition 'found!=NULL' is redundant or there is overflow in pointer subtraction. [nullPointerArithmeticRedundantCheck]
                match_offset = (uint32_t)((found - buffer) + cd->content_len);
                                                 ^
src/detect-engine-content-inspection.c:308:30: note: Assuming that condition 'found!=NULL' is not redundant
            } else if (found != NULL && (cd->flags & DETECT_CONTENT_NEGATED)) {
                             ^
src/detect-engine-content-inspection.c:316:50: note: Null pointer subtraction
                match_offset = (uint32_t)((found - buffer) + cd->content_len);
                                                 ^

Bug: #5291.

3 years agodetect/analyzer: minor format string fixes
Victor Julien [Tue, 26 Apr 2022 18:05:51 +0000 (20:05 +0200)] 
detect/analyzer: minor format string fixes

cppcheck flagged this as:

src/detect-engine-analyzer.c:1359:13: warning: %d in format string (no. 1) requires 'int' but the argument type is 'unsigned int'. [invalidPrintfArgType_sint]
            fprintf(rule_engine_analysis_FD, "    Rule contains %d content options, %d http content options, %d pcre options, and %d pcre options with http modifiers.\n", rule_content, rule_content_http, rule_pcre, rule_pcre_http);
            ^
src/detect-engine-analyzer.c:1359:13: warning: %d in format string (no. 2) requires 'int' but the argument type is 'unsigned int'. [invalidPrintfArgType_sint]
            fprintf(rule_engine_analysis_FD, "    Rule contains %d content options, %d http content options, %d pcre options, and %d pcre options with http modifiers.\n", rule_content, rule_content_http, rule_pcre, rule_pcre_http);
            ^
src/detect-engine-analyzer.c:1359:13: warning: %d in format string (no. 3) requires 'int' but the argument type is 'unsigned int'. [invalidPrintfArgType_sint]
            fprintf(rule_engine_analysis_FD, "    Rule contains %d content options, %d http content options, %d pcre options, and %d pcre options with http modifiers.\n", rule_content, rule_content_http, rule_pcre, rule_pcre_http);
            ^
src/detect-engine-analyzer.c:1359:13: warning: %d in format string (no. 4) requires 'int' but the argument type is 'unsigned int'. [invalidPrintfArgType_sint]
            fprintf(rule_engine_analysis_FD, "    Rule contains %d content options, %d http content options, %d pcre options, and %d pcre options with http modifiers.\n", rule_content, rule_content_http, rule_pcre, rule_pcre_http);
            ^

Bug: #5291.

3 years agodetect/address: remove useless checks
Victor Julien [Tue, 26 Apr 2022 18:04:28 +0000 (20:04 +0200)] 
detect/address: remove useless checks

Cppcheck flagged this:

src/detect-engine-address.c:1035:48: warning: Either the condition 'ghn!=NULL' is redundant or there is possible null pointer dereference: gh. [nullPointerRedundantCheck]
    int r = DetectAddressIsCompleteIPSpaceIPv4(gh->ipv4_head);
                                               ^
src/detect-engine-address.c:1297:17: note: Assuming that condition 'ghn!=NULL' is not redundant
        if (ghn != NULL) {
                ^
src/detect-engine-address.c:1283:44: note: Calling function 'DetectAddressIsCompleteIPSpace', 1st argument 'ghn' value is 0
        if (DetectAddressIsCompleteIPSpace(ghn)) {
                                           ^
src/detect-engine-address.c:1035:48: note: Null pointer dereference
    int r = DetectAddressIsCompleteIPSpaceIPv4(gh->ipv4_head);
                                               ^

Cleanup code could only be reached with non-NULL pointers, so simplify checks.

Bug: #5291.

3 years agodetect/ipv6: remove useless code
Victor Julien [Tue, 26 Apr 2022 18:02:19 +0000 (20:02 +0200)] 
detect/ipv6: remove useless code

Remove useless allocation and free.

Found by cppcheck as a potential issue:

src/detect-engine-address-ipv6.c:385:12: warning: Either the condition 'tmp!=NULL' is redundant or there is possible null pointer dereference: tmp. [nullPointerRedundantCheck]
    memset(tmp,0,sizeof(DetectAddress));
           ^
src/detect-engine-address-ipv6.c:525:13: note: Assuming that condition 'tmp!=NULL' is not redundant
    if (tmp != NULL)
            ^
src/detect-engine-address-ipv6.c:385:12: note: Null pointer dereference
    memset(tmp,0,sizeof(DetectAddress));
           ^

But code turned out not to do anything, so removed.

Bug: #5291.

3 years agodatasets: fix cppcheck warning
Victor Julien [Tue, 26 Apr 2022 18:01:19 +0000 (20:01 +0200)] 
datasets: fix cppcheck warning

src/datasets.c:107:17: error: Uninitialized variable: hash [uninitvar]
    memcpy(out, hash, outs);
                ^
src/datasets.c:93:26: note: Assuming condition is false
    for (x = 0, i = 0; i < ins; i+=2, x++) {
                         ^
src/datasets.c:107:17: note: Uninitialized variable: hash
    memcpy(out, hash, outs);
                ^

Bug: #5291.

3 years agodetect: fix rule inspection order 7308/head
Victor Julien [Mon, 25 Apr 2022 16:00:24 +0000 (18:00 +0200)] 
detect: fix rule inspection order

Fix rules from the 'match' list getting added to the tx candidates list
unsorted. In some cases this could lead to the same sid getting inspected
twice leading to a DEBUG_VALIDATION_BUG_ON trigger.

Bug: #5144.

3 years agostream: improve flow end payload logging
Victor Julien [Sat, 23 Apr 2022 11:59:34 +0000 (13:59 +0200)] 
stream: improve flow end payload logging

Use all available data, including un-ACK'd, when in flow timeout
mode.

Bug: #5276.

3 years agoeve/alert: add pkt_src/pcap_cnt to tunnel
Victor Julien [Sat, 23 Apr 2022 08:00:32 +0000 (10:00 +0200)] 
eve/alert: add pkt_src/pcap_cnt to tunnel

Makes it easier to correlate an alert to the original packet.

3 years agoeve: add pkt_src
Victor Julien [Sat, 23 Apr 2022 07:59:21 +0000 (09:59 +0200)] 
eve: add pkt_src

This will tell the user if a record was generated based on a real packet,
a flow timeout packet or others.

3 years agogithub-actions: bump actions/checkout from 3.0.1 to 3.0.2 7293/head
dependabot[bot] [Thu, 21 Apr 2022 19:38:07 +0000 (19:38 +0000)] 
github-actions: bump actions/checkout from 3.0.1 to 3.0.2

Bumps [actions/checkout](https://github.com/actions/checkout) from 3.0.1 to 3.0.2.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/dcd71f646680f2efd8db4afa5ad64fdcba30e748...2541b1294d2704b0964813337f33b291d3f8596b)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
3 years agogithub-actions: bump codecov/codecov-action from 3.0.0 to 3.1.0
dependabot[bot] [Thu, 21 Apr 2022 19:38:03 +0000 (19:38 +0000)] 
github-actions: bump codecov/codecov-action from 3.0.0 to 3.1.0

Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 3.0.0 to 3.1.0.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/e3c560433a6cc60aec8812599b7844a7b4fa0d71...81cd2dc8148241f03f5839d295e000b8f761e378)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
3 years agostream/unittests: fix failures after last_ack fix
Victor Julien [Fri, 22 Apr 2022 17:33:13 +0000 (19:33 +0200)] 
stream/unittests: fix failures after last_ack fix

Work around many tests not setting up stream completely or correctly.

3 years agostream: improve last_ack validation check
Victor Julien [Fri, 22 Apr 2022 16:27:15 +0000 (18:27 +0200)] 
stream: improve last_ack validation check

If a packet after the initialization would come with ACK flag set
but a ACK value of 0, the last_ack tracking could get confused. Fix
this by not checking for 0 but instead checking if the ACK flag
has been seen.

Bug: #4549.

3 years agolibhtp: require 0.5.40 7292/head
Victor Julien [Fri, 22 Apr 2022 12:30:26 +0000 (14:30 +0200)] 
libhtp: require 0.5.40

Ticket: #4970.

3 years agocbindgen: handle version to stderr change 7291/head
Victor Julien [Fri, 22 Apr 2022 06:10:58 +0000 (08:10 +0200)] 
cbindgen: handle version to stderr change

3 years agosmb: protocol detection on pattern without midstream 7282/head
Philippe Antoine [Mon, 22 Nov 2021 10:30:08 +0000 (11:30 +0100)] 
smb: protocol detection on pattern without midstream

To recognize a protocol, Suricata first looks for
patterns, which can be confirmed by a probing parser.
If this does not work, Suricata can try to run
some probing parsers on some ports.

This is the case for SMB.

This commit makes handling the confirming and the probing
paser differently even if they share much code.

The confirmation parser knows that a pattern has been found.
So, it must not do the midstream case of looking for this
pattern in the whole buffer, but only check it at the beginning.
But it must reverse direction if needed.

3 years agosmb: fix event types for limit exceeded rules
Victor Julien [Wed, 20 Apr 2022 19:39:01 +0000 (21:39 +0200)] 
smb: fix event types for limit exceeded rules

3 years agosmtp: don't pass partial boundary on to mime parser
Victor Julien [Fri, 15 Apr 2022 13:51:10 +0000 (15:51 +0200)] 
smtp: don't pass partial boundary on to mime parser

If the start of a line looks like it might be a mime boundary we
yield to the get line logic if we don't have enough data to be
conclusive.

3 years agomime: allow partial lines as input
Victor Julien [Fri, 15 Apr 2022 13:49:09 +0000 (15:49 +0200)] 
mime: allow partial lines as input

If we get a zero length delim we assume its a partial line and we
won't append CRLF just yet.

3 years agosmtp: pre process DATA and BDAT commands
Shivani Bhardwaj [Thu, 14 Apr 2022 15:59:32 +0000 (21:29 +0530)] 
smtp: pre process DATA and BDAT commands

The input data received in DATA and BDAT command modes can be huge and
could have important data, like a legit huge email. Therefore, exempt
these from the line buffering limits which were introduced to regulate
the size of lines that we buffer at any point in time.

As a part of this patch, anything that comes under DATA or BDAT is
processed early without buffering as and when it arrives. The ways of
processing remain the same as before.

3 years agosmtp: fix indefinite buffering if no LF in line
Shivani Bhardwaj [Mon, 14 Feb 2022 11:23:52 +0000 (16:53 +0530)] 
smtp: fix indefinite buffering if no LF in line

Issue
-----
So far, with the SMTP parser, we would buffer data up until an LF char
was found indicating the end of one line. This would happen in case of
fragmented data where a line might come broken into multiple chunks.
This was problematic if there was a really long line without any LF
character. It would mean that we'd keep buffering data up until we
encounter one such LF char which may be many many bytes of data later.

Fix
---
Fix this issue by setting an upper limit of 4KB on the buffering of
lines. If the limit is reached then we save the data into current line
and process it as if it were a regular request/response up until 4KB
only. Any data after 4KB is discarded up until there is a new LF char in
the received input.

Cases
-----
1. Fragmentation
The limit is enforced for any cases where a line of >= 4KB comes as diff
fragments that are each/some < 4KB.
2. Single too long line
The limit is also enforced for any cases where a single line exceeds the
limit of buffer.

Reported by Victor Julien.
Ticket 5023

3 years agosmtp: add truncated line event
Shivani Bhardwaj [Wed, 20 Apr 2022 07:25:54 +0000 (12:55 +0530)] 
smtp: add truncated line event

3 years agodoc/userguide: document ftp max-line-length
Jason Ish [Thu, 7 Apr 2022 21:58:58 +0000 (15:58 -0600)] 
doc/userguide: document ftp max-line-length

3 years agoftp: truncate command data that is too long
Jason Ish [Wed, 6 Apr 2022 21:38:35 +0000 (15:38 -0600)] 
ftp: truncate command data that is too long

FTP control commands will be buffered forever until a new line is seen,
this can lead to memory exhaustion in Suricata.

To fix, set an upper bound, 4096 bytes on the size of the command that
is saved in the transaction. The input continues to be parsed to find
the end of the command so the parser can continue to move onto the next
command.

The result is that the command data in the transaction is truncated,
which also shows up in the ftp transaction logs.

This value is configurable with the max-line-length field in the ftp
app-layer.protocols section.

As FTP doesn't have events at this time, add a new fields to eve-log
that specificy if the request, or the response has been truncated.

Ticket #5024

3 years agoprotocol: forbids concurrent protocol upgrades
Philippe Antoine [Fri, 8 Apr 2022 12:40:02 +0000 (14:40 +0200)] 
protocol: forbids concurrent protocol upgrades

Ticket: 5243

When switching from SMTP to TLS, and getting HTTP1 instead of
expected TLS, and HTTP1 requesting upgrade to HTTP2, we do not
overwrite the alproto_orig value so as not to have type confusion
in AppLayerParserStateProtoCleanup

3 years agodns: better error handling when parsing names
Jason Ish [Tue, 1 Feb 2022 21:44:43 +0000 (15:44 -0600)] 
dns: better error handling when parsing names

The DNS name parser will error out with an error even if the
error is incomplete. Instead of manually generating errors,
use '?' to let the nom error ripple up the error handling chain.

The reason this wasn't done in the first place is this code
predates the ? operator, or we were not aware of it at the time.

This prevents the case where probing fails when there is enough data to
parse the header, but not enough to complete name parser. In such a case
a parse error is returned (instead of incomplete) resulting in the
payload not being detected as DNS.

Ticket #5034

3 years agodns: don't parse a full request during probe if not enough data
Jason Ish [Mon, 28 Feb 2022 22:48:34 +0000 (16:48 -0600)] 
dns: don't parse a full request during probe if not enough data

If there is more data than a header, but not enough for a complete DNS
message, the hostname parser could return an error causing the probe to
fail on valid DNS messages.

So only parse the complete message if we have enough input data. This is
reliable for TCP as DNS messages are prefixed, but for UDP its just
going to be the size of the input buffer presented to the parser, so
incomplete could still happen.

Ticket #5034

3 years agogithub-actions: bump actions/upload-artifact from 1 to 3 7273/head
dependabot[bot] [Tue, 19 Apr 2022 19:37:14 +0000 (19:37 +0000)] 
github-actions: bump actions/upload-artifact from 1 to 3

Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 1 to 3.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v1...6673cd052c4cd6fcf4b4e6e60ea986c889389535)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
3 years agodetect/frame: fix frame detect registration 7269/head
Victor Julien [Tue, 19 Apr 2022 16:54:08 +0000 (18:54 +0200)] 
detect/frame: fix frame detect registration

Rewrite keyword parser.

Duplicate short names could lead to buffer confusion and memory leaks.

Bug: #5238.

3 years agosmb/rules: add rules for new events 7262/head
Victor Julien [Tue, 19 Apr 2022 10:35:52 +0000 (12:35 +0200)] 
smb/rules: add rules for new events

3 years agodoc/smb: add resource limits section
Victor Julien [Tue, 19 Apr 2022 10:17:31 +0000 (12:17 +0200)] 
doc/smb: add resource limits section

3 years agosmb2: validate negotiate read/write max sizes
Victor Julien [Tue, 19 Apr 2022 06:13:48 +0000 (08:13 +0200)] 
smb2: validate negotiate read/write max sizes

Raise event if they exceed the configured limit.

3 years agosmb2: allow limiting in-flight data size/cnt
Victor Julien [Sat, 16 Apr 2022 04:58:20 +0000 (06:58 +0200)] 
smb2: allow limiting in-flight data size/cnt

Allow limiting in-flight out or order data chunks per size or count.

Implemented for read and writes separately:

app-layer.protocols.smb.max-write-queue-size
app-layer.protocols.smb.max-write-queue-cnt
app-layer.protocols.smb.max-read-queue-size
app-layer.protocols.smb.max-read-queue-cnt

3 years agofiletracker: track total queued data (in_flight)
Victor Julien [Sat, 16 Apr 2022 04:57:56 +0000 (06:57 +0200)] 
filetracker: track total queued data (in_flight)

As well as expose number of chunks.

3 years agosmb: log max read/write sizes
Victor Julien [Mon, 18 Apr 2022 20:14:36 +0000 (22:14 +0200)] 
smb: log max read/write sizes

3 years agosmb2: add options for max read/write size
Victor Julien [Mon, 18 Apr 2022 19:47:39 +0000 (21:47 +0200)] 
smb2: add options for max read/write size

Add options for the max read/write size accepted by the parser.

3 years agosmb2: track max read/write size and enforce its values
Victor Julien [Mon, 18 Apr 2022 15:49:58 +0000 (17:49 +0200)] 
smb2: track max read/write size and enforce its values

3 years agosmb: minor function cleanup
Victor Julien [Fri, 15 Apr 2022 18:52:48 +0000 (20:52 +0200)] 
smb: minor function cleanup

Remove used argument from `filetracker_newchunk()`. We're not
using fill_bytes with smb.

3 years agofiletracker: make FileChunk private
Victor Julien [Fri, 15 Apr 2022 18:00:55 +0000 (20:00 +0200)] 
filetracker: make FileChunk private

3 years agogithub-actions: bump codecov/codecov-action from 2.1.0 to 3 7261/head
dependabot[bot] [Tue, 19 Apr 2022 06:14:45 +0000 (06:14 +0000)] 
github-actions: bump codecov/codecov-action from 2.1.0 to 3

Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 2.1.0 to 3.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/f32b3a3741e1053eb607407145bc9619351dc93b...e3c560433a6cc60aec8812599b7844a7b4fa0d71)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
3 years agogithub-actions: bump github/codeql-action from 1.0.26 to 2.1.8
dependabot[bot] [Tue, 19 Apr 2022 06:14:42 +0000 (06:14 +0000)] 
github-actions: bump github/codeql-action from 1.0.26 to 2.1.8

Bumps [github/codeql-action](https://github.com/github/codeql-action) from 1.0.26 to 2.1.8.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/5f532563584d71fdef14ee64d17bafb34f751ce5...1ed1437484560351c5be56cf73a48a279d116b78)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
3 years agogithub-actions: bump actions/cache from 2.1.7 to 3.0.2
dependabot[bot] [Tue, 19 Apr 2022 06:14:37 +0000 (06:14 +0000)] 
github-actions: bump actions/cache from 2.1.7 to 3.0.2

Bumps [actions/cache](https://github.com/actions/cache) from 2.1.7 to 3.0.2.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/937d24475381cd9c75ae6db12cb4e79714b926ed...48af2dc4a9e8278b89d7fa154b955c30c6aaab09)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
3 years agogithub-actions: bump actions/download-artifact from 2 to 3
dependabot[bot] [Tue, 19 Apr 2022 06:14:33 +0000 (06:14 +0000)] 
github-actions: bump actions/download-artifact from 2 to 3

Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 2 to 3.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v2...fb598a63ae348fa914e94cd0ff38f362e927b741)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
3 years agogithub-actions: bump ossf/scorecard-action from 1.0.1 to 1.0.4 7256/head
dependabot[bot] [Tue, 19 Apr 2022 06:14:30 +0000 (06:14 +0000)] 
github-actions: bump ossf/scorecard-action from 1.0.1 to 1.0.4

Bumps [ossf/scorecard-action](https://github.com/ossf/scorecard-action) from 1.0.1 to 1.0.4.
- [Release notes](https://github.com/ossf/scorecard-action/releases)
- [Commits](https://github.com/ossf/scorecard-action/compare/e3e75cf2ffbf9364bbff86cdbdf52b23176fe492...c1aec4ac820532bab364f02a81873c555a0ba3a1)

---
updated-dependencies:
- dependency-name: ossf/scorecard-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
3 years agodetect/frames: reduce severity of validation check 7255/head
Victor Julien [Tue, 19 Apr 2022 05:33:39 +0000 (07:33 +0200)] 
detect/frames: reduce severity of validation check

3 years agodependabot: monitor github actions
Jason Ish [Thu, 14 Apr 2022 18:35:57 +0000 (12:35 -0600)] 
dependabot: monitor github actions

3 years agogithub-ci: set safe directory before reset
Jason Ish [Thu, 14 Apr 2022 19:16:55 +0000 (13:16 -0600)] 
github-ci: set safe directory before reset

While the latest checkout action does set the "safe.directory"
parameter, it doesn't appear to stick for the following "git fetch", so
call this command again.

3 years agogithub-ci: pin checkout action to latest release
Jason Ish [Thu, 14 Apr 2022 18:34:47 +0000 (12:34 -0600)] 
github-ci: pin checkout action to latest release

3 years agomqtt: fix consumed bytes computation for truncated msg 7253/head
Philippe Antoine [Mon, 11 Apr 2022 19:29:33 +0000 (21:29 +0200)] 
mqtt: fix consumed bytes computation for truncated msg

Ticket: 5268

3 years agodetect/frame: get data using stream callback 7244/head
Victor Julien [Wed, 13 Apr 2022 05:47:42 +0000 (07:47 +0200)] 
detect/frame: get data using stream callback

Inspect only data that has already been consumed by the
app-layer parser. This allows for simpler progress tracking.

3 years agoframe: introduce entry for getting stream data for frame
Victor Julien [Wed, 13 Apr 2022 05:42:56 +0000 (07:42 +0200)] 
frame: introduce entry for getting stream data for frame

3 years agostream: make raw data handling more generally usable
Victor Julien [Wed, 13 Apr 2022 05:42:09 +0000 (07:42 +0200)] 
stream: make raw data handling more generally usable

Move raw detection logic out of main StreamReassembleRawDo() so that
it can be reused for other parts of the engine.

The caller now has to specify a right edge of the data.

3 years agostream: add offset to raw stream callback
Victor Julien [Tue, 12 Apr 2022 13:22:23 +0000 (15:22 +0200)] 
stream: add offset to raw stream callback

This gives the called function to understand where it is in the
stream.

3 years agoapp-layer: disable stream app tracking on no parser
Victor Julien [Wed, 13 Apr 2022 12:00:37 +0000 (14:00 +0200)] 
app-layer: disable stream app tracking on no parser

If protocol has no parser enabled or implemented, disable the app
progress tracking in the stream engine to reduce the workload in
the stream engine.

3 years agosource: pcap timestamp microsecond consistency
Philippe Antoine [Fri, 1 Apr 2022 15:55:33 +0000 (17:55 +0200)] 
source: pcap timestamp microsecond consistency

That is it should be less than 1 000 000.
Have the same for fuzz targets where the bug came from.

https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44177

3 years agodcerpc: store consumed_bytes as i32
Philippe Antoine [Sat, 2 Apr 2022 19:16:53 +0000 (21:16 +0200)] 
dcerpc: store consumed_bytes as i32

As it can grow bigger than u16

3 years agoike: fix integer underflow in parse_proposal
Philippe Antoine [Fri, 8 Apr 2022 06:49:18 +0000 (08:49 +0200)] 
ike: fix integer underflow in parse_proposal

By not restricting a usize to i16

3 years agodetect: config checks alstate before getting tx 7242/head
Philippe Antoine [Fri, 8 Apr 2022 13:15:23 +0000 (15:15 +0200)] 
detect: config checks alstate before getting tx

Ticket: 4972

As is done in detect-lua-extensions.
We can have a flow with alproto unknown, no state, and therefore
cannot run AppLayerParserGetTx which could try to run a NULL
function

3 years agodetect: faster linked list copy
Philippe Antoine [Mon, 5 Jul 2021 15:05:10 +0000 (17:05 +0200)] 
detect: faster linked list copy

In DetectAppLayerInspectEngineCopyListToDetectCtx
Avoid quadratic complexity by remembering last element
of the linked list we are inserting into

3 years agoflow: fix integer warnings
Philippe Antoine [Thu, 14 Apr 2022 12:44:20 +0000 (14:44 +0200)] 
flow: fix integer warnings

Ticket: 4516

3 years agohost/ippair: fix integer warnings
Philippe Antoine [Thu, 14 Apr 2022 12:41:45 +0000 (14:41 +0200)] 
host/ippair: fix integer warnings

Ticket: 4516

3 years agoutil: using size_t len for byte utils
Philippe Antoine [Thu, 14 Apr 2022 12:36:57 +0000 (14:36 +0200)] 
util: using size_t len for byte utils

Ticket: 4516

Like ByteExtractStringUint64, because most of their inputs come
from strlen which returns a size_t

3 years agoapp-layer: fix integer warnings
Philippe Antoine [Tue, 18 Jan 2022 10:19:21 +0000 (11:19 +0100)] 
app-layer: fix integer warnings

Ticket: 4516

3 years agodebug: support %m output format again
Victor Julien [Fri, 25 Feb 2022 14:40:41 +0000 (15:40 +0100)] 
debug: support %m output format again

Use thread local storage to avoid the previous dead lock issues.

3 years agothreading: simplify thread name logic
Victor Julien [Fri, 4 Mar 2022 13:31:24 +0000 (14:31 +0100)] 
threading: simplify thread name logic

3 years agorust: update regex & memchr dependencies 7230/head
Victor Julien [Mon, 11 Apr 2022 15:25:47 +0000 (17:25 +0200)] 
rust: update regex & memchr dependencies

Bug: #5260.

3 years agosmb/ntlmssp: add stricter len/offset validation
Victor Julien [Mon, 11 Apr 2022 10:33:43 +0000 (12:33 +0200)] 
smb/ntlmssp: add stricter len/offset validation

3 years agosmb: prevents integer underflow
Philippe Antoine [Fri, 8 Apr 2022 09:23:09 +0000 (11:23 +0200)] 
smb: prevents integer underflow

Ticket: 5246

If msg_id is 0, we cannot find the previous request

3 years agosmb: ntlmssp domain_blob_offset underflow check
Philippe Antoine [Mon, 4 Apr 2022 20:51:01 +0000 (22:51 +0200)] 
smb: ntlmssp domain_blob_offset underflow check

Ticket: 5246

3 years agosmb: check on param parsing
Philippe Antoine [Mon, 4 Apr 2022 20:45:56 +0000 (22:45 +0200)] 
smb: check on param parsing

Ticket: 5246

so as not to overflow u16

3 years agoframes: remove dead condition in eof check 7226/head
Victor Julien [Sun, 10 Apr 2022 18:22:00 +0000 (20:22 +0200)] 
frames: remove dead condition in eof check

3 years agoapp-layer: don't switch dir if proto already known
Victor Julien [Sun, 10 Apr 2022 18:21:18 +0000 (20:21 +0200)] 
app-layer: don't switch dir if proto already known

3 years agofuzz/sigpcap_aware: set pkt_src to wire
Victor Julien [Sun, 10 Apr 2022 13:46:38 +0000 (15:46 +0200)] 
fuzz/sigpcap_aware: set pkt_src to wire

Avoids an assert if DEBUG is compiled in:

fuzz_sigpcap_aware: source-pcap-file.c:420: TmEcode DecodePcapFile(ThreadVars *, Packet *, void *): Assertion `!(p->pkt_src != PKT_SRC_WIRE && p->pkt_src != PKT_SRC_FFR)' failed.

3 years agodetect/frame: improve assert accuracy
Victor Julien [Sun, 10 Apr 2022 13:45:36 +0000 (15:45 +0200)] 
detect/frame: improve assert accuracy

Handle frames of unknown size correctly.

Bug: #5226.

3 years agoeve: allow /dev/null in threaded mode 7225/head
Victor Julien [Sat, 9 Apr 2022 15:24:33 +0000 (17:24 +0200)] 
eve: allow /dev/null in threaded mode

Avoids creation of actual files called /dev/null.N which take
up space in /dev/ which lives in memory.

3 years agoflow: cleanup locking debug leftovers
Victor Julien [Sat, 9 Apr 2022 08:56:04 +0000 (10:56 +0200)] 
flow: cleanup locking debug leftovers

3 years agoflow: fix and simplify locking
Victor Julien [Fri, 8 Apr 2022 20:06:09 +0000 (22:06 +0200)] 
flow: fix and simplify locking

Since:

9551cd053579 ("threading: don't pass locked flow between threads")

`MoveToWorkQueue()` unconditionally unlocks the flow. This allows simpler
locking handling, including of tcp reuse flows.

The simpler logic also fixes a scenario where TCP reuse flows got "unlocked"
twice, once in `FlowGetFlowFromHash()` and once in `MoveToWorkQueue()`.

Bug: #5248.
Coverity: 1494354.

3 years agomqtt: remove redundant "where" keyword 7223/head
Sascha Steinbiss [Fri, 11 Mar 2022 18:05:03 +0000 (19:05 +0100)] 
mqtt: remove redundant "where" keyword

3 years agomqtt: make some functions non-public
Sascha Steinbiss [Tue, 8 Mar 2022 22:23:47 +0000 (23:23 +0100)] 
mqtt: make some functions non-public

3 years agomqtt: rustfmt
Sascha Steinbiss [Tue, 8 Mar 2022 22:19:22 +0000 (23:19 +0100)] 
mqtt: rustfmt

3 years agomqtt: raise event on parse error
Sascha Steinbiss [Tue, 8 Mar 2022 22:18:36 +0000 (23:18 +0100)] 
mqtt: raise event on parse error

3 years agomqtt: ensure we do not request extra data after buffering
Sascha Steinbiss [Tue, 8 Mar 2022 22:15:05 +0000 (23:15 +0100)] 
mqtt: ensure we do not request extra data after buffering

This addresses Redmine bug #5018 by ensuring that the parser
never requests additional data via the Incomplete error, but to
raise an actual parse error, since it is supposed to have all
the data as specified by the message length in the header already.

3 years agooutput: fix integer warnings 7219/head
Philippe Antoine [Tue, 18 Jan 2022 09:56:48 +0000 (10:56 +0100)] 
output: fix integer warnings

Ticket: 4516

3 years agossh: install app-layer events rules 7210/head
Philippe Antoine [Fri, 25 Mar 2022 14:03:12 +0000 (15:03 +0100)] 
ssh: install app-layer events rules

3 years agodetect: not an iponly signature if it needs app-layer
Philippe Antoine [Wed, 30 Mar 2022 13:24:32 +0000 (15:24 +0200)] 
detect: not an iponly signature if it needs app-layer

Ticket: 4972

This may happen with `config` keyword which is postmatch,
but may require a transaction

3 years agodoc/userguide: sphinx syntax correction
William Harding [Tue, 29 Mar 2022 18:53:46 +0000 (14:53 -0400)] 
doc/userguide: sphinx syntax correction

3 years agounittests: alloc Packet with PacketGetFromAlloc
Juliana Fajardini [Fri, 28 Jan 2022 21:20:31 +0000 (21:20 +0000)] 
unittests: alloc Packet with PacketGetFromAlloc

Some unittests used SCMalloc for allocating new Packet the unittests.
While this is valid, it leads to segmentation faults when we move to
dynamic allocation of the maximum alerts allowed to be triggered by a
single packet.

This massive patch uses PacketGetFromAlloc, which initializes a Packet
in such a way that any dynamic allocated structures within will also be
initialized.

Related to
Task #4207