]> git.ipfire.org Git - thirdparty/suricata-verify.git/log
thirdparty/suricata-verify.git
7 years agoremove support for "skip" file test skips
Jason Ish [Thu, 21 Dec 2017 00:20:49 +0000 (18:20 -0600)] 
remove support for "skip" file test skips

wasn't being used anymore as there are other ways to
skip a test

7 years agoremove run.sh; use run.py instead
Jason Ish [Thu, 21 Dec 2017 00:16:56 +0000 (18:16 -0600)] 
remove run.sh; use run.py instead

7 years agoremove leftovers vars.sh files: not used anymore
Jason Ish [Thu, 21 Dec 2017 00:16:26 +0000 (18:16 -0600)] 
remove leftovers vars.sh files: not used anymore

7 years agotls test: based on tls tests in @regit suripcap branch
Jason Ish [Thu, 21 Dec 2017 00:09:04 +0000 (18:09 -0600)] 
tls test: based on tls tests in @regit suripcap branch

combines "TLS 1" and "TLS alert" into a single test

7 years agosome check.sh to test.yaml verifications.
Jason Ish [Wed, 20 Dec 2017 23:04:59 +0000 (17:04 -0600)] 
some check.sh to test.yaml verifications.

7 years agosmtp: basic smtp test showing filter and stats check
Jason Ish [Wed, 20 Dec 2017 23:03:53 +0000 (17:03 -0600)] 
smtp: basic smtp test showing filter and stats check

7 years agoadd new checks to test.yaml
Jason Ish [Wed, 20 Dec 2017 23:02:16 +0000 (17:02 -0600)] 
add new checks to test.yaml

shell - for checking shell command output
filter - for counting number of events matching a filter
stats: for matching values in stats

7 years agonew test for dns lua detect
Jason Ish [Wed, 20 Dec 2017 20:26:31 +0000 (14:26 -0600)] 
new test for dns lua detect

Tests:
- dns.rrname
- dns.request
- dns.response

7 years agoadd suricata 4.0.3 default config
Jason Ish [Wed, 20 Dec 2017 20:26:12 +0000 (14:26 -0600)] 
add suricata 4.0.3 default config

7 years agoadd signature-id validation to test.yaml
Jason Ish [Wed, 20 Dec 2017 19:58:27 +0000 (13:58 -0600)] 
add signature-id validation to test.yaml

With the following in a test.yaml:
  checks:
  - signature-id: 1
  - signature-id: 2
  - signature-id: 3
the eve.json will be checked to make sure it alerts for each
signature id provided.

7 years agoallow test to require an env var to be set
Jason Ish [Wed, 20 Dec 2017 17:25:13 +0000 (11:25 -0600)] 
allow test to require an env var to be set

Useful for only enabling tests when an environment
variable is set.

7 years agoallow command to be set in test.yaml
Jason Ish [Mon, 4 Dec 2017 18:02:36 +0000 (12:02 -0600)] 
allow command to be set in test.yaml

Removes the need to support a test specific run.sh.

Also, don't require a check.sh. In some tests, just a
successful exit code is being tested for.

7 years agoMark 2 tests with min-version 4.0.
Jason Ish [Mon, 4 Dec 2017 13:39:22 +0000 (07:39 -0600)] 
Mark 2 tests with min-version 4.0.

7 years agoadd feature requires; remove skip.sh scripts
Jason Ish [Fri, 1 Dec 2017 23:06:10 +0000 (17:06 -0600)] 
add feature requires; remove skip.sh scripts

requires.features for required features

requires.not-features for when a test should be skipped if a
feature is present.

7 years agodon't run linktype 228 test on suricata < 4.1.0
Jason Ish [Fri, 1 Dec 2017 22:43:34 +0000 (16:43 -0600)] 
don't run linktype 228 test on suricata < 4.1.0

Introduces a test.yaml file that can be placed in a test
directory that provides a test configuration.

For now just a min-version can be configured.

7 years agobasic test for linktype 228
Jason Ish [Fri, 24 Nov 2017 16:55:55 +0000 (10:55 -0600)] 
basic test for linktype 228

7 years agoreadme: document that a pcap is all that is needed.
Jason Ish [Fri, 24 Nov 2017 16:58:11 +0000 (10:58 -0600)] 
readme: document that a pcap is all that is needed.

7 years agorun tests from test directory
Jason Ish [Fri, 24 Nov 2017 16:03:24 +0000 (10:03 -0600)] 
run tests from test directory

Fixes the case where we had a hardcoded relative path
to deal with Lua scripts.

7 years agoput all tests in a tests/ directory
Jason Ish [Fri, 24 Nov 2017 15:14:12 +0000 (09:14 -0600)] 
put all tests in a tests/ directory

7 years agolua-output-http: make ports an int instead of a float
Jason Ish [Mon, 25 Sep 2017 08:40:02 +0000 (09:40 +0100)] 
lua-output-http: make ports an int instead of a float

7 years agobsd fix: pipe wc output to xargs
Jason Ish [Sun, 24 Sep 2017 10:36:51 +0000 (11:36 +0100)] 
bsd fix: pipe wc output to xargs

On BSD and Mac, wc -l outputs right justified strings, causing
the check to fail. Passing through xargs is a hack to strip
the white space.

7 years agolua script tests: fix path to rule directory
Jason Ish [Sun, 24 Sep 2017 10:36:37 +0000 (11:36 +0100)] 
lua script tests: fix path to rule directory

7 years agorename: output-lua-smtp -> lua-output-smtp
Jason Ish [Wed, 20 Sep 2017 09:09:14 +0000 (10:09 +0100)] 
rename: output-lua-smtp -> lua-output-smtp

7 years agooutput-lua: rename to lua-output-http
Jason Ish [Wed, 20 Sep 2017 09:04:26 +0000 (10:04 +0100)] 
output-lua: rename to lua-output-http

As it tests lua output of http metadata.

7 years agooutput-lua: fix lua script
Jason Ish [Wed, 20 Sep 2017 09:02:38 +0000 (10:02 +0100)] 
output-lua: fix lua script

On some distros it was correct, others it failed. I think this was
due to a type in string.gsub, using %g instead of %c.

7 years agoDisable ASAN leak detection.
Jason Ish [Mon, 11 Sep 2017 12:16:04 +0000 (13:16 +0100)] 
Disable ASAN leak detection.

8 years agoreadme: update
Jason Ish [Thu, 3 Aug 2017 21:55:14 +0000 (15:55 -0600)] 
readme: update

8 years agotest: eve/dns aaaa only logging
Jason Ish [Thu, 13 Jul 2017 16:33:11 +0000 (10:33 -0600)] 
test: eve/dns aaaa only logging

8 years agoadd a dns test - has pcap for some unit tests
Jason Ish [Wed, 14 Jun 2017 16:33:26 +0000 (10:33 -0600)] 
add a dns test - has pcap for some unit tests

8 years agoskip some tests when rust is enabled
Jason Ish [Fri, 9 Jun 2017 16:11:02 +0000 (10:11 -0600)] 
skip some tests when rust is enabled

8 years agodns-tcp-ts-gap: test gap in tcp dns traffic
Jason Ish [Fri, 9 Jun 2017 16:10:22 +0000 (10:10 -0600)] 
dns-tcp-ts-gap: test gap in tcp dns traffic

8 years agodns txt test
Jason Ish [Thu, 8 Jun 2017 19:01:09 +0000 (13:01 -0600)] 
dns txt test

8 years agoadd tests for dns log filtering
Jason Ish [Tue, 9 May 2017 21:20:56 +0000 (15:20 -0600)] 
add tests for dns log filtering

8 years agouse python test runner
Jason Ish [Thu, 4 May 2017 19:38:19 +0000 (13:38 -0600)] 
use python test runner

its a little more robust on multiple platforms...

8 years agoremove more direct file comparisons
Jason Ish [Thu, 4 May 2017 15:31:26 +0000 (09:31 -0600)] 
remove more direct file comparisons

8 years agoreadme: expected directory is no longer
Jason Ish [Thu, 4 May 2017 15:25:42 +0000 (09:25 -0600)] 
readme: expected directory is no longer

8 years agoupdate to not do any exact file matches...
Jason Ish [Thu, 4 May 2017 15:22:52 +0000 (09:22 -0600)] 
update to not do any exact file matches...

changes in flow id can break the tests

8 years agotest for lua dns output
Jason Ish [Tue, 2 May 2017 21:28:40 +0000 (15:28 -0600)] 
test for lua dns output

8 years agoadd test for issue 856
Jason Ish [Mon, 1 May 2017 21:36:21 +0000 (15:36 -0600)] 
add test for issue 856
https://redmine.openinfosecfoundation.org/issues/856

8 years agomac fixups
Jason Ish [Fri, 7 Apr 2017 21:48:46 +0000 (15:48 -0600)] 
mac fixups

8 years agodns tests: check for results instead of a file match
Jason Ish [Mon, 1 May 2017 16:31:58 +0000 (10:31 -0600)] 
dns tests: check for results instead of a file match

so we don't fail when something like the flow id is changed
due to other internal suricata changes

8 years agoadd help/usage
Jason Ish [Fri, 7 Apr 2017 17:06:30 +0000 (11:06 -0600)] 
add help/usage

8 years agolog cmdline used to output/cmdline
Jason Ish [Fri, 7 Apr 2017 16:56:49 +0000 (10:56 -0600)] 
log cmdline used to output/cmdline

8 years agoruntime fixups
Jason Ish [Fri, 7 Apr 2017 21:39:03 +0000 (15:39 -0600)] 
runtime fixups

8 years agoempty rule file test
Jason Ish [Fri, 7 Apr 2017 21:38:56 +0000 (15:38 -0600)] 
empty rule file test

8 years agodns-udp-double-request-response: fix check
Jason Ish [Wed, 1 Mar 2017 17:01:10 +0000 (11:01 -0600)] 
dns-udp-double-request-response: fix check

8 years agoskip lua tests if lua support not compiled in
Jason Ish [Sat, 25 Feb 2017 13:59:37 +0000 (07:59 -0600)] 
skip lua tests if lua support not compiled in

8 years agoreplace from exact match test with event count tests
Jason Ish [Tue, 21 Feb 2017 20:57:31 +0000 (14:57 -0600)] 
replace from exact match test with event count tests

8 years agotest dns.log for unanswered requests suricata-3.2.1
Jason Ish [Tue, 31 Jan 2017 18:58:23 +0000 (12:58 -0600)] 
test dns.log for unanswered requests

8 years agoSome new DNS tests.
Jason Ish [Tue, 31 Jan 2017 18:51:49 +0000 (12:51 -0600)] 
Some new DNS tests.

8 years agoreplace pcap with one that has distinct timestamps
Jason Ish [Wed, 25 Jan 2017 22:32:56 +0000 (16:32 -0600)] 
replace pcap with one that has distinct timestamps

8 years agodnp3: update test for more responses being logged
Jason Ish [Wed, 18 Jan 2017 18:55:04 +0000 (12:55 -0600)] 
dnp3: update test for more responses being logged

8 years agorunner: --continue to continue running on failures
Jason Ish [Mon, 16 Jan 2017 17:16:01 +0000 (11:16 -0600)] 
runner: --continue to continue running on failures

8 years agorename skip script
Jason Ish [Tue, 10 Jan 2017 16:51:26 +0000 (10:51 -0600)] 
rename skip script

8 years agonew test: dns-udp-unsolicited-response
Jason Ish [Tue, 10 Jan 2017 16:48:49 +0000 (10:48 -0600)] 
new test: dns-udp-unsolicited-response

Test for unsolicited DNS response on existing flow.

8 years agoproto-mismatch-http-ssh: test app-layer mismatch
Jason Ish [Wed, 4 Jan 2017 17:59:38 +0000 (11:59 -0600)] 
proto-mismatch-http-ssh: test app-layer mismatch

Tests that an app-layer mismatch is detected.

8 years agorename verify.sh to check.sh
Jason Ish [Wed, 4 Jan 2017 17:50:03 +0000 (11:50 -0600)] 
rename verify.sh to check.sh

8 years agorm unused rules file
Jason Ish [Wed, 4 Jan 2017 17:49:30 +0000 (11:49 -0600)] 
rm unused rules file

8 years agooutput-pcap-log: update expected output for snaplen change
Jason Ish [Tue, 20 Dec 2016 17:12:42 +0000 (11:12 -0600)] 
output-pcap-log: update expected output for snaplen change

8 years agoconvert expected output to UTC.
Jason Ish [Tue, 15 Nov 2016 19:48:46 +0000 (13:48 -0600)] 
convert expected output to UTC.

8 years agorun suricata with TZ=UTC
Jason Ish [Tue, 15 Nov 2016 19:48:33 +0000 (13:48 -0600)] 
run suricata with TZ=UTC

8 years agoenable dnp3 for dnp3 tests
Jason Ish [Sun, 13 Nov 2016 14:53:01 +0000 (08:53 -0600)] 
enable dnp3 for dnp3 tests

8 years agodon't skip flow:not_established test
Jason Ish [Sat, 29 Oct 2016 06:56:06 +0000 (00:56 -0600)] 
don't skip flow:not_established test

8 years agofix dnp3 tests; don't skip
Jason Ish [Sat, 29 Oct 2016 06:55:53 +0000 (00:55 -0600)] 
fix dnp3 tests; don't skip

8 years agooutput-lua-smtp: document pcap source
Jason Ish [Wed, 26 Oct 2016 21:14:14 +0000 (15:14 -0600)] 
output-lua-smtp: document pcap source

8 years agoAdd test for multi-dns request without response in-between.
Jason Ish [Wed, 26 Oct 2016 21:09:15 +0000 (15:09 -0600)] 
Add test for multi-dns request without response in-between.

8 years agoignore ~ files
Jason Ish [Wed, 26 Oct 2016 21:09:11 +0000 (15:09 -0600)] 
ignore ~ files

8 years agoallow for private tests in private/
Jason Ish [Wed, 26 Oct 2016 20:56:26 +0000 (14:56 -0600)] 
allow for private tests in private/

8 years agoUpdate README.
Jason Ish [Wed, 26 Oct 2016 20:37:52 +0000 (14:37 -0600)] 
Update README.

8 years agorework the tests a little
Jason Ish [Wed, 26 Oct 2016 20:34:38 +0000 (14:34 -0600)] 
rework the tests a little

8 years agonew test: dns tcp with a buffer containing multiple requests
Jason Ish [Wed, 26 Oct 2016 20:31:39 +0000 (14:31 -0600)] 
new test: dns tcp with a buffer containing multiple requests

8 years agonew test: dnp3_data; skipped for now
Jason Ish [Wed, 26 Oct 2016 20:31:00 +0000 (14:31 -0600)] 
new test: dnp3_data; skipped for now

8 years agonew test: not_established keyword
Jason Ish [Wed, 26 Oct 2016 20:30:25 +0000 (14:30 -0600)] 
new test: not_established keyword

also includes an example of how to programmaticly skip a test

8 years agonew test: single-dns-request
Jason Ish [Wed, 26 Oct 2016 20:30:02 +0000 (14:30 -0600)] 
new test: single-dns-request

8 years agotest tls fingerprint alert direction
Jason Ish [Tue, 27 Sep 2016 15:37:43 +0000 (09:37 -0600)] 
test tls fingerprint alert direction

8 years agoTests for TLS events in IDS and IPS (--simulate-ips) modes.
Jason Ish [Tue, 27 Sep 2016 15:22:21 +0000 (09:22 -0600)] 
Tests for TLS events in IDS and IPS (--simulate-ips) modes.

8 years agoclean.sh: script to cleanup after tests
Jason Ish [Mon, 26 Sep 2016 17:17:09 +0000 (11:17 -0600)] 
clean.sh: script to cleanup after tests

8 years agodnp3-dnp3_func-alert: test dnp3_func keyword
Jason Ish [Mon, 26 Sep 2016 17:16:37 +0000 (11:16 -0600)] 
dnp3-dnp3_func-alert: test dnp3_func keyword

8 years agoif a file skip exists, skip test
Jason Ish [Mon, 26 Sep 2016 16:53:16 +0000 (10:53 -0600)] 
if a file skip exists, skip test

8 years agoadd dnp3 logging test
Jason Ish [Mon, 26 Sep 2016 16:52:58 +0000 (10:52 -0600)] 
add dnp3 logging test

8 years agoadd test.rules
Jason Ish [Mon, 26 Sep 2016 16:45:47 +0000 (10:45 -0600)] 
add test.rules

8 years agorename rules.rules to test.rules
Jason Ish [Mon, 26 Sep 2016 16:45:15 +0000 (10:45 -0600)] 
rename rules.rules to test.rules

8 years agoREADME
Jason Ish [Mon, 26 Sep 2016 16:35:50 +0000 (10:35 -0600)] 
README

8 years agomake quiet
Jason Ish [Mon, 26 Sep 2016 15:55:58 +0000 (09:55 -0600)] 
make quiet

8 years agotest alert for www.testmyids.com
Jason Ish [Mon, 26 Sep 2016 15:49:37 +0000 (09:49 -0600)] 
test alert for www.testmyids.com

8 years agorun with reference.config and classification.config
Jason Ish [Mon, 26 Sep 2016 15:48:39 +0000 (09:48 -0600)] 
run with reference.config and classification.config

8 years agofix expected flow_id
Jason Ish [Mon, 26 Sep 2016 15:47:07 +0000 (09:47 -0600)] 
fix expected flow_id

9 years agofix dns output: directional logging changes timestamps
Jason Ish [Wed, 13 Jul 2016 22:39:59 +0000 (16:39 -0600)] 
fix dns output: directional logging changes timestamps

9 years agoadd smtp lua output test
Jason Ish [Wed, 13 Jul 2016 22:39:39 +0000 (16:39 -0600)] 
add smtp lua output test

9 years agoCleanup runner.
Jason Ish [Wed, 8 Jun 2016 23:02:09 +0000 (17:02 -0600)] 
Cleanup runner.

9 years agoAdd tcp data output test.
Jason Ish [Wed, 8 Jun 2016 23:01:54 +0000 (17:01 -0600)] 
Add tcp data output test.

9 years agoAdd eve fileinfo output test.
Jason Ish [Wed, 8 Jun 2016 23:01:22 +0000 (17:01 -0600)] 
Add eve fileinfo output test.

9 years agoInitial commit of Suricata verification tests.
Jason Ish [Wed, 8 Jun 2016 19:21:52 +0000 (13:21 -0600)] 
Initial commit of Suricata verification tests.