From 53c8a0f8f182824727a0e7f4b36246ecd034e081 Mon Sep 17 00:00:00 2001 From: Jeff Lucovsky Date: Wed, 18 Sep 2024 08:18:07 -0400 Subject: [PATCH] doc: Document luaxform transform Issue: 2290 --- doc/userguide/configuration/suricata-yaml.rst | 5 +- doc/userguide/lua/lua-functions.rst | 3 +- doc/userguide/lua/lua-usage.rst | 19 +++- doc/userguide/rules/lua-detection.rst | 27 ++++- doc/userguide/rules/transforms.rst | 106 ++++++++++++++++++ 5 files changed, 147 insertions(+), 13 deletions(-) diff --git a/doc/userguide/configuration/suricata-yaml.rst b/doc/userguide/configuration/suricata-yaml.rst index 5881a5401d..a772181d28 100644 --- a/doc/userguide/configuration/suricata-yaml.rst +++ b/doc/userguide/configuration/suricata-yaml.rst @@ -2993,8 +2993,9 @@ Lua ~~~ Suricata 8.0 sandboxes Lua rules by default. The restrictions on the sandbox for Lua rules can be -modified in the ``security.lua`` section of the configuration file. Additionally, Lua rules -can be completely disabled the same as the Suricata 7.0 default: +modified in the ``security.lua`` section of the configuration file. This section also applies to +Lua transforms. Additionally, Lua rules can be completely disabled in the same way as for as the +Suricata 7.0 default: :: diff --git a/doc/userguide/lua/lua-functions.rst b/doc/userguide/lua/lua-functions.rst index dadbb534a1..546edfa38f 100644 --- a/doc/userguide/lua/lua-functions.rst +++ b/doc/userguide/lua/lua-functions.rst @@ -7,6 +7,7 @@ Differences between `output` and `detect`: ------------------------------------------ Currently, the ``needs`` key initialization varies, depending on what is the goal of the script: output or detection. +The Lua script for the ``luaxform`` transform **does not use ``needs``**. If the script is for detection, the ``needs`` initialization should be as seen in the example below (see :ref:`lua-detection` for a complete example of a detection script): @@ -422,7 +423,7 @@ index so in our case we need to use 0. SCFlowintSet(0, a + 1) else SCFlowintSet(0, 1) - end + end SCFlowintGet ~~~~~~~~~~~~ diff --git a/doc/userguide/lua/lua-usage.rst b/doc/userguide/lua/lua-usage.rst index 19946db5e5..8b478b21af 100644 --- a/doc/userguide/lua/lua-usage.rst +++ b/doc/userguide/lua/lua-usage.rst @@ -1,20 +1,29 @@ Lua usage in Suricata ===================== -Lua scripting can be used in two components of Suricata. The first is in -output and the second one in rules in the detection engine. +Lua scripting can be used in two components of Suricata: + + * Output + * Detection: ``lua`` keyword and ``luaxform`` transform Both features are using a list of functions to access the data extracted by Suricata. You can get the list of functions in the :ref:`lua-functions` page. -.. note:: Currently, there is a difference in the ``needs`` key in the ``init`` function, depending on what is the usage: ``output`` or ``detection``. The list of available functions may also differ. +.. note:: Currently, there is a difference in the ``needs`` key in the ``init`` function, + depending on what is the usage: ``output`` or ``detection``. The list of available + functions may also differ. The ``luaxform`` doesn't use the ``needs`` key. Lua output ---------- -Lua can be used to write arbitrary output. See :ref:`lua-output` for more information. +Lua scripts can be used to write arbitrary output. See :ref:`lua-output` for more information. Lua detection ------------- -Lua script can be used as a filter condition in signatures. See :ref:`lua-detection` for more information. +Lua scripts can be used as a filter condition in signatures. See :ref:`lua-detection` for more information. + +Lua transform +------------- + +The ``luaxform`` transform can be used in signatures. See :ref:`lua-transform` for more information. diff --git a/doc/userguide/rules/lua-detection.rst b/doc/userguide/rules/lua-detection.rst index e70d1f07c3..14f4f2ffb5 100644 --- a/doc/userguide/rules/lua-detection.rst +++ b/doc/userguide/rules/lua-detection.rst @@ -3,6 +3,18 @@ Lua Scripting for Detection =========================== +There are 2 ways that Lua can be used with detection. These are + +* ``lua`` rule keyword. +* ``luaxform`` transform. + +.. note:: Lua is disabled by default for use in rules, it must be + enabled in the configuration file. See the ``security.lua`` + section of ``suricata.yaml`` and enable ``allow-rules``. + +Lua Rule Keyword +---------------- + Syntax: :: @@ -17,7 +29,7 @@ A Lua rule script has 2 required functions, an ``init`` function and Additionally, the script will run in a limited sandbox by default. Init function -------------- +^^^^^^^^^^^^^ .. code-block:: lua @@ -52,7 +64,7 @@ All the HTTP buffers have a limitation: only one can be inspected by a script at a time. Match function --------------- +^^^^^^^^^^^^^^ .. code-block:: lua @@ -93,8 +105,13 @@ Entire script: return 0 -Sandbox and Available functions -------------------------------- +Lua Transform: ``luaxform`` +--------------------------- + +More details in :ref:`lua-transform`. + +Lua Sandbox and Available functions +----------------------------------- Lua rule scripts are run in a sandbox environment the applies the following restrictions: @@ -130,7 +147,7 @@ Of note, the following standard libraries are not available: This behavior can be modified via the ``security.lua`` section of :ref:`suricata-yaml-lua-config` -.. note:: Suricata 8.0 has moved to Lua 5.4 and has builtin support for bitwise and utf8 operations now. +.. note:: Suricata 8.0 has moved to Lua 5.4 and now has builtin support for bitwise and utf8 operations. A comprehensive list of existing lua functions - with examples - can be found at :ref:`lua-functions` (some of them, however, work only for diff --git a/doc/userguide/rules/transforms.rst b/doc/userguide/rules/transforms.rst index 0047211eba..5c0527aa50 100644 --- a/doc/userguide/rules/transforms.rst +++ b/doc/userguide/rules/transforms.rst @@ -279,3 +279,109 @@ This example transforms `"Zm 9v Ym Fy"` to `"foobar"`:: content:"/?arg=Zm 9v Ym Fy"; from_base64: offset 6, mode rfc2045; \ content:"foobar"; + +.. _lua-transform: + +luaxform +-------- + +This transform allows a Lua script to apply a transformation +to a buffer. + +Lua scripts that are used for transformations *must* contain a function +named ``transform``. + +Lua transforms can be passed optional arguments -- see the examples below -- but they +are not required to do so. Arguments are comma-separated. + +A Lua transform function is not invoked if the buffer is empty or the Lua framework is +not accessible (rare). + +Lua transform functions must return two values (see below) or the buffer is not modified. + +Note that the arguments and values are passed without validation +nor interpretation. There is a maximum of 10 arguments. + +The Lua transform function is invoked with these parameters: + + * `input` The buffer provided to the transform + * `arguments` The list of arguments. + +Lua transform functions must return two values [Lua datatypes shown]: + + * `buffer` [Lua string] The return buffer containing the original input buffer or buffer modified by the transform. + * `bytes` [Lua integer] Number of bytes in return buffer. + +This example supplies the HTTP data to a Lua transform and the transform +results are checked with `content`. + +Example:: + + alert http any any -> any any (msg:"Lua Xform example"; flow:established; \ + file.data; luaxform:./lua/lua-transform.lua; content: "abc"; sid: 2;) + + +This example supplies the HTTP data to a Lua transform with with arguments +that specify the offset and byte count for the transform. The resulting +buffer is then checked with a `content` match. + +Example:: + + alert http any any -> any any (msg:"Lua Xform example"; flow:established; \ + file.data; luaxform:./lua/lua-transform.lua, bytes 12, offset 13; content: "abc"; sid: 1;) + + +The following Lua script shows a transform that handles arguments: `bytes` and `offset` and uses +those values (or defaults, if there are no arguments) for applying the uppercase transform to +the buffer. + +.. code-block:: lua + + function init (args) + local needs = {} + return needs + end + + local function get_value(item, key) + if string.find(item, key) then + local _, value = string.match(item, "(%a+)%s*(%d*)") + if value ~= "" then + return tonumber(value) + end + end + + return nil + end + + -- Arguments supported + local bytes_key = "bytes" + local offset_key = "offset" + function transform(input_len, input, argc, args) + local bytes = #input + local offset = 0 + + -- Look for optional bytes and offset arguments + for i, item in ipairs(args) do + local value = get_value(item, bytes_key) + if value ~= nil then + bytes = value + else + local value = get_value(item, offset_key) + if value ~= nil then + offset = value + end + end + end + local str_len = #input + if offset < 0 or offset > str_len then + print("offset is out of bounds: " .. offset) + return nil + end + str_len = str_len - offset + if bytes < 0 or bytes > str_len then + print("invalid bytes " .. bytes .. " or bytes > length " .. bytes .. " length " .. str_len) + return nil + end + local sub = string.sub(input, offset + 1, offset + bytes) + return string.upper(sub), bytes + end -- 2.47.2