dev reference documentation for stream compression

diff --git a/doc/dev-reference/runtime.rst b/doc/dev-reference/runtime.rst
index 665395a83ac82c853eb53bb2f6ccbdd94b4712d4..dbfe7633f022de44fa3ad7abf73375c7b7c19e31 100644 (file)
--- a/doc/dev-reference/runtime.rst
+++ b/doc/dev-reference/runtime.rst
@@ -80,6 +80,42 @@ functions for the management of streams:
   another, resetting the destination stream first. This call avoids the
   allocation done by :c:func:`hs_copy_stream`.
 
+==================
+Stream Compression
+==================
+
+A stream object is allocated as a fixed size region of memory which has been
+sized to ensure that no memory allocations are required during scan
+operations. When the system is under memory pressure, it may be useful to reduce
+the memory consumed by streams that are not expected to be used soon. The
+Hyperscan API provides calls for translating a stream to and from a compressed
+representation for this purpose. The compressed representation differs from the
+full stream object as it does not reserve space for components which are not
+required given the current stream state. The Hyperscan API functions for this
+functionality are:
+
+* :c:func:`hs_compress_stream`: fills the provided buffer with a compressed
+  representation of the stream and returns the number of bytes consumed by the
+  compressed representation. If the buffer is not large enough to hold the
+  compressed representation, :c:member:`HS_INSUFFICIENT_SPACE` is returned along
+  with the required size. This call does not modify the original stream in any
+  way: it may still be written to with :c:func:`hs_scan_stream`, used as part of
+  the various reset calls to reinitialise its state, or
+  :c:func:`hs_close_stream` may be called to free its resources.
+
+* :c:func:`hs_expand_stream`: creates a new stream based on a buffer containing
+  a compressed representation.
+
+* :c:func:`hs_reset_and_expand_stream`: constructs a stream based on a buffer
+  containing a compressed representation on top of an existing stream, resetting
+  the existing stream first. This call avoids the allocation done by
+  :c:func:`hs_expand_stream`.
+
+Note: it is not recommended to use stream compression between every call to scan
+for performance reasons as it takes time to convert between the compressed
+representation and a standard stream.
+
+
 **********
 Block Mode
 **********