rule analyzer: make analyzer aware of http_user_agent pcre flag /V.

[people/ms/suricata.git] / ChangeLog

1.3 -- 2012-07-06

- make live rule reloads optional and disabled by default
- fix a shutdown bug
- fix several memory leaks (#492)
- warn user if global and rule thresholding conflict (#455)
- set thread names on FreeBSD (Nikolay Denev)
- Fix PF_RING building on Ubuntu 12.04
- rule analyzer updates
- file inspection improvements when dealing with limits (#493)

1.3rc1 -- 2012-06-29

- experimental live rule reload by sending a USR2 signal (#279)
- AF_PACKET BPF support (#449)
- AF_PACKET live packet loss counters (#441)
- Rule analyzer (#349)
- add pcap workers runmode for use with libpcap wrappers that support load balancing, such as  Napatech's or Myricom's
- negated filemd5 matching, allowing for md5 whitelisting
- signatures with depth and/or offset are now checked against packets in addition to the stream (#404)
- http_cookie keyword now also inspects "Set-Cookie" header (#479)
- filemd5 keyword no longer depends on log-file output module (#447)
- http_raw_header keyword inspects original header line terminators (#475)
- deal with double encoded URI (#464)
- improved SMB/SMB2/DCERPC robustness
- ICMPv6 parsing fixes
- improve HTTP body inspection
- stream.inline accuracy issues fixed (#339)
- general stability fixes (#482, #486)
- missing unittests added (#471)
- "threshold.conf not found" error made more clear (#446)
- IPS mode segment logging for Unified2 improved

1.3beta2 -- 2012-06-08

- experimental support for matching on large lists of known file MD5 checksums
- Improved performance for file_data, http_server_body and http_client_body keywords
- Improvements to HTTP handling: multipart parsing, gzip decompression
- Byte_extract can support negative offsets now (#445)
- Support for PF_RING 5.4 added. Many thanks to Chris Wakelin (#459)
- HOME_NET and EXTERNAL_NET and the other vars are now checked for common errors (#454)
- Improved error reporting when using too long address strings (#451)
- MD5 calculation improvements for daemon mode and other cases (#449)
- File inspection scripts: Added Syslog action for logging to local syslog. Thanks to Martin Holste.
- Rule parser is made more strict.
- Unified2 output overhaul, logging individual segments in more cases.
- detection_filter keyword accuracy problem was fixed (#453)
- Don't inspect cookie header with http header (#461)
- Crash with a rule with two byte_extract keywords (#456)
- SSL parser fixes. Thanks to Chris Wakelin for testing the patches! (#476)
- Accuracy issues in HTTP inspection fixed. Thanks to Rmkml (#452)
- Improve escaping of some characters in logs (#418)
- Checksum calculation bugs fixed
- IPv6 parsing issues fixed. Thanks to Michel Saborde.
- Endace DAG issues fixed. Thanks to Jason Ish from Endace.
- Various OpenBSD related fixes.
- Fixes for bugs found by Coverity source code analyzer.

1.3beta1 -- 2012-04-04

- TLS/SSL handshake parser, tls.subjectdn and tls.issuerdn keywords (#296, contributed by Pierre Chifflier)
- Napatech capture card support (contributed by Randy Caldejon -- nPulse)
- Scripts for looking up files / file md5's at Virus Total and others (contributed by Martin Holste)
- Test mode: -T option to test the config (#271)
- Ringbuffer and zero copy support for AF_PACKET
- Commandline options to list supported app layer protocols and keywords (#344, #414)
- File extraction for HTTP POST request that do not use multipart bodies
- On the fly md5 checksum calculation of extracted files
- Line based file log, in json format
- Basic support for including other yaml files into the main yaml
- New multi pattern engine: ac-bs
- Profiling improvements, added lock profiling code
- Improved HTTP CONNECT support in libhtp (#427, Brian Rectanus -- Qualys)
- Unified yaml naming convention, including fallback support (by Nikolay Denev)
- Improved Endace DAG support (#431, Jason Ish -- Endace)
- New default runmode: "autofp" (#433)
- Major rewrite of flow engine, improving scalability.
- Improved http_stat_msg and http_stat_code keywords (#394)
- Improved scalability for Tag and Threshold subsystems
- Made the rule keyword parser much stricter in detecting syntax errors
- Split "file" output into "file-store" and "file-log" outputs
- Much improved file extraction
- CUDA build fixes (#421)
- Various FP's reported by Rmkml (#403, #405, #411)
- IPv6 decoding and detection issues (reported by Michel Sarborde)
- PCAP logging crash (#422)
- Fixed many (potential) issues with the help of the Coverity source code analyzer
- Fixed several (potential) issues with the help of the cppcheck and clang/scan-build source code analyzers

1.2.1 -- 2012-01-20

- fix malformed unified2 records when writing alerts trigger by stream inspection (#402)
- only force a pseudo packet inspection cycle for TCP streams in a state >= established

1.2 -- 2012-01-19

- improved Windows/CYGWIN path handling (#387)
- fixed some issues with passing an interface or ip address with -i
- make live worker runmode threads adhere to the 'detect' cpu affinity settings

1.2rc1 -- 2012-01-11

- app-layer-events keyword: similar to the decoder-events and stream-events, this will allow matching on HTTP and SMTP events
- auto detection of checksum offloading per interface (#311)
- urilen options to match on raw or normalized URI (#341)
- flow keyword option "only_stream" and "no_stream"
- unixsock output options for all outputs except unified2 (PoC python script in the qa/ dir) (#250)
- in IPS mode, reject rules now also drop (#399)
- http_header now also inspects response headers (#389)
- "worker" runmodes for NFQ and IPFW
- performance improvement for "ac" pattern matcher
- allow empty/non-initialized flowints to be incremented
- PCRE-JIT is now enabled by default if available (#356)
- many file inspection and extraction improvements
- flowbits and flowints are now modified in a post-match action list
- general performance increasements
- fixed parsing really high sid numbers >2 Billion (#393)
- fixed ICMPv6 not matching in IP-only sigs (#363)

1.2beta1 -- 2011-12-19

- File name, type inspection and extraction for HTTP
- filename, fileext, filemagic and filestore keywords added
- "file" output for storing extracted files to disk
- file_data keyword support, inspecting normalized, dechunked, decompressed HTTP response body (feature #241
- new keyword http_server_body, pcre regex /S option
- Option to enable/disable core dumping from the suricata.yaml (enabled by default)
- Human readable size limit settings in suricata.yaml
- PF_RING bpf support (required PF_RING >= 5.1) (feature #334)
- tos keyword support (feature #364)
- IPFW IPS mode does now support multiple divert sockets
- New IPS running modes, Linux and FreeBSD do now support "worker" and "autofp"
- Improved alert accuracy in autofp and single runmodes
- major performance optimizations for the ac-gfbs pattern matcher implementation
- unified2 output fixes
- PF_RING supports privilege dropping now (bug #367)
- Improved detection of duplicate signatures

1.1.1 -- 2011-12-07

- Fix for a error in the smtp parser that could crash Suricata.
- Fix for AF_PACKET not compiling on modern linux systems like Fedora 16.

1.1 -- 2011-11-10

- CUDA build fixed
- minor pcap, AF_PACKET and PF_RING fixes (#368)
- bpf handling fix
- Windows CYGWIN build
- more cleanups

1.1rc1 -- 2011-11-03

- extended HTTP request logging for use with (among other things) http_agent for Sguil (#38)
- AF_PACKET report drop stats on shutdown (#325)
- new counters in stats.log for flow and stream engines (#348)
- SMTP parsing code support for BDAT command (#347)
- HTTP URI normalization no longer converts to lowercase (#362)
- AF_PACKET works with privileges dropping now (#361)
- Prelude output for state matches (#264, #355)
- update of the pattern matching code that should improve accuracy
- rule parser was made more strict (#295, #312)
- multiple event suppressions for the same SID was fixed (#366)
- several accuracy fixes
- removal of the unified1 output plugins (#353)

1.1beta3 -- 2011-10-25

- af-packet support for high speed packet capture
- "replace" keyword support (#303)
- new "workers" runmode for multi-dev and/or clustered PF_RING, AF_PACKET, pcap
- added "stream-event" keyword to match on TCP session anomalies
- support for suppress keyword was added (#274)
- byte_extract keyword support was added
- improved handling of timed out TCP sessions in the detection engine
- unified2 payload logging if detection was in the HTTP state (#264)
- improved accuracy of the HTTP transaction logging
- support for larger (64 bit) Flow/Stream memcaps (#332)
- major speed improvements for PCRE, including support for PCRE JIT
- support setting flowbits in ip-only rules (#292)
- performance increases on SSE3+ CPU's
- overhaul of the packet acquisition subsystem
- packet based performance profiling subsystem was added
- TCP SACK support was added to the stream engine
- updated included libhtp to 0.2.6 which fixes several issues

1.1beta2 -- 2011-04-13

- New keyword support: http_raw_uri (including /I for pcre), ssl_state, ssl_version (#258, #259, #260, #262).
- Inline mode for the stream engine (#230, #248).
- New keyword support: nfq_set_mark
- Included an example decoder-events.rules file
- api for adding and selecting runmodes was added
- pcap logging / recording output was added
- basic SCTP protocol parsing was added
- more fine grained CPU affinity setting support was added
- stream engine inspects stream in larger chunks
- fast_pattern support for http_method content modifier (#255)
- negation support for isdataat keyword (#257)
- configurable interval for stats.log updates (#247)
- new pf_ring runmode was added that scales better
- pcap live mode now handles the monitor interface going up and down
- several QA additions to "make check"
- NFQ (linux inline) mode was improved
- Alerts classification fix (#275)
- compiles and runs on big-endian systems (#63)
- unified2 output works around barnyard2 issues with DLT_RAW + IPv6

1.1beta1 -- 2010-12-21

- New keyword support: http_raw_header, http_stat_msg, http_stat_code.
- A new default pattern matcher, Aho-Corasick based, that uses much less memory.
- reference.config support as supplied by ET/ETpro and VRT.
- Much improved fast_pattern support, including for http_uri, http_client_body, http_header, http_raw_header.
- Improved parsers, especially the DCERPC parser.
- Much improved performance & accuracy.

1.0.5 -- 2011-07-25

- Fix stream reassembly bug #300. Thanks to Rmkml for the report.
- Fix several (potential) issues fixed after a source code scan with Coverity generously contributed by RedHat.

1.0.4 -- 2011-06-24

- LibHTP updated to 0.2.6
- Large number of (potential) issues fixed after a source code scan with Coverity generously contributed by RedHat.
- Large number of (potential) issues fixed after source code scans with the Clang static analizer.

1.0.3 -- 2011-04-13

- Fix broken checksum calculation for TCP/UDP in some cases
- Fix errors in the byte_test, byte_jump, http_method and http_header keywords
- Fix a ASN1 parsing issue
- Improve LibHTP memory handling
- Fix a defrag issue
- Fix several stream engine issues