tests: add tests for unseen http midstream traffic

In a pcap where just `http` midstream traffic is seen, Suri is
unable to see the packtes as `http` traffic (Wireshark tags them
correctly).

This also seems to result in Suri sometimes not adding the packet
payload to the associated alert event in the eve-log.

`bug-5437-01` has the pcap where http packets are not seen `bug-5437-02`
has a more complete pcap, and the same packets are properly
identified by Suri.

Related to
Bug #5437

diff --git a/tests/bug-5437-01/README.md b/tests/bug-5437-01/README.md
new file mode 100644 (file)
index 0000000..787099d
--- /dev/null
+++ b/tests/bug-5437-01/README.md
@@ -0,0 +1,22 @@
+Test
+====
+
+This is a test for the bug 5437 about unseen http midstream packets/flow.
+
+Behavior
+========
+
+Suri seems unable to properly identify `http` traffic in this payload,
+despite having `-k none` and `midstream=true` set.
+
+Here we only have two `http` GET request packets in the pcap file: the `http`,
+and the stream is not seen (Wireshark tags those correctly).
+
+Compare with `bug-5437-02`, from which the 2 packets in the pcap from
+the present test come from: the `http` stream is seen and logged there.
+
+PCAP
+====
+
+Pcap was shared on Suricata's Discord server by the users who observed this
+behavior.

diff --git a/tests/bug-5437-01/input.pcap b/tests/bug-5437-01/input.pcap
new file mode 100644 (file)
index 0000000..99abaa0
Binary files /dev/null and b/tests/bug-5437-01/input.pcap differ

diff --git a/tests/bug-5437-01/test.rules b/tests/bug-5437-01/test.rules
new file mode 100644 (file)
index 0000000..19d01d5
--- /dev/null
+++ b/tests/bug-5437-01/test.rules
@@ -0,0 +1,3 @@
+alert ip any any -> any any (msg:"SURICATA IPv4 invalid checksum"; ipv4-csum:invalid; classtype:protocol-command-decode; sid:2200073; rev:2;)
+alert tcp any any -> any $HTTP_PORTS (msg:"MALWARE-CNC User-Agent known malicious user-agent string - Mirai"; flow:to_server,established; content:"User-Agent|3A| Hello, world"; fast_pattern:only; http_header; metadata:impact_flag red, policy balanced-ips drop, policy max-detect-ips drop, policy security-ips drop, service http; reference:url,www.virustotal.com/en/file/3908cc1d8001f926031fbe55ce104448dbc20c9795b7c3cfbd9abe7b789f899d/analysis/; classtype:trojan-activity; sid:58992; rev:1;)
+alert http any any -> any any (msg:"ET MALWARE JAWS Webserver Unauthenticated Shell Command Execution"; flow:established,to_server; http.method; content:"GET"; http.uri.raw; content:"/shell?cd+/tmp|3b|rm+-rf+*|3b|wget+"; depth:29; fast_pattern; reference:md5,fea9e4132fc9d30bda5eb6b1d9d0b9b9; classtype:web-application-attack; sid:2030092; rev:2; metadata:affected_product Linux, attack_target Web_Server, created_at 2020_05_04, deployment Perimeter, signature_severity Major, updated_at 2020_05_04;)

diff --git a/tests/bug-5437-01/test.yaml b/tests/bug-5437-01/test.yaml
new file mode 100644 (file)
index 0000000..7c387e0
--- /dev/null
+++ b/tests/bug-5437-01/test.yaml
@@ -0,0 +1,13 @@
+requires:
+  min-version: 7
+
+args:
+- -k none
+- --set stream.midstream=true
+
+checks:
+- filter:
+    count: 2
+    match:
+      event_type: flow
+      app_proto: http

diff --git a/tests/bug-5437-02/README.md b/tests/bug-5437-02/README.md
new file mode 100644 (file)
index 0000000..027a80a
--- /dev/null
+++ b/tests/bug-5437-02/README.md
@@ -0,0 +1,19 @@
+Test
+====
+
+This is a test for the bug 5437 about unseen http midstream packets.
+
+
+Behavior
+========
+
+This test shows the desired behavior. This is the pcap from which the traffic
+seen in the extracted pcap for test `bug-5437-01` comes.
+
+In this one Suri is able to identify the `http` packets.
+
+PCAP
+====
+
+Pcap was shared on Suricata's Discord server by the users who observed this
+behavior.

diff --git a/tests/bug-5437-02/input.pcap b/tests/bug-5437-02/input.pcap
new file mode 100644 (file)
index 0000000..543da02
Binary files /dev/null and b/tests/bug-5437-02/input.pcap differ

diff --git a/tests/bug-5437-02/test.rules b/tests/bug-5437-02/test.rules
new file mode 100644 (file)
index 0000000..19d01d5
--- /dev/null
+++ b/tests/bug-5437-02/test.rules
@@ -0,0 +1,3 @@
+alert ip any any -> any any (msg:"SURICATA IPv4 invalid checksum"; ipv4-csum:invalid; classtype:protocol-command-decode; sid:2200073; rev:2;)
+alert tcp any any -> any $HTTP_PORTS (msg:"MALWARE-CNC User-Agent known malicious user-agent string - Mirai"; flow:to_server,established; content:"User-Agent|3A| Hello, world"; fast_pattern:only; http_header; metadata:impact_flag red, policy balanced-ips drop, policy max-detect-ips drop, policy security-ips drop, service http; reference:url,www.virustotal.com/en/file/3908cc1d8001f926031fbe55ce104448dbc20c9795b7c3cfbd9abe7b789f899d/analysis/; classtype:trojan-activity; sid:58992; rev:1;)
+alert http any any -> any any (msg:"ET MALWARE JAWS Webserver Unauthenticated Shell Command Execution"; flow:established,to_server; http.method; content:"GET"; http.uri.raw; content:"/shell?cd+/tmp|3b|rm+-rf+*|3b|wget+"; depth:29; fast_pattern; reference:md5,fea9e4132fc9d30bda5eb6b1d9d0b9b9; classtype:web-application-attack; sid:2030092; rev:2; metadata:affected_product Linux, attack_target Web_Server, created_at 2020_05_04, deployment Perimeter, signature_severity Major, updated_at 2020_05_04;)

diff --git a/tests/bug-5437-02/test.yaml b/tests/bug-5437-02/test.yaml
new file mode 100644 (file)
index 0000000..7c387e0
--- /dev/null
+++ b/tests/bug-5437-02/test.yaml
@@ -0,0 +1,13 @@
+requires:
+  min-version: 7
+
+args:
+- -k none
+- --set stream.midstream=true
+
+checks:
+- filter:
+    count: 2
+    match:
+      event_type: flow
+      app_proto: http