]> git.ipfire.org Git - thirdparty/nftables.git/log
thirdparty/nftables.git
5 years agosrc: maps: update data expression dtype based on set
Florian Westphal [Thu, 13 Feb 2020 11:45:55 +0000 (12:45 +0100)] 
src: maps: update data expression dtype based on set

What we want:
-               update @sticky-set-svc-M53CN2XYVUHRQ7UB { ip saddr : 0x00000002 }
what we got:
+               update @sticky-set-svc-M53CN2XYVUHRQ7UB { ip saddr : 0x2000000 [invalid type] }

Reported-by: Serguei Bezverkhi <sbezverk@gmail.com>
Close: https://bugzilla.netfilter.org/show_bug.cgi?id=1405
Signed-off-by: Florian Westphal <fw@strlen.de>
5 years agotests: shell: validate error reporting with include and glob
Pablo Neira Ayuso [Thu, 13 Feb 2020 11:13:11 +0000 (12:13 +0100)] 
tests: shell: validate error reporting with include and glob

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agotests: shell: add test for glob includes
Laurent Fasnacht [Mon, 10 Feb 2020 10:17:20 +0000 (10:17 +0000)] 
tests: shell: add test for glob includes

Including more than MAX_INCLUDE_DEPTH file in one statement should succeed.

This reproduces bug #1243.

Signed-off-by: Laurent Fasnacht <fasnacht@protonmail.ch>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agoscanner: remove parser_state->indesc_idx
Laurent Fasnacht [Mon, 10 Feb 2020 10:17:28 +0000 (10:17 +0000)] 
scanner: remove parser_state->indesc_idx

Now that we have a proper stack implementation, we don't need an
additional counter for the number of buffer state pushed.

Signed-off-by: Laurent Fasnacht <fasnacht@protonmail.ch>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agoscanner: fix indesc_list stack to be in the correct order
Laurent Fasnacht [Mon, 10 Feb 2020 10:17:27 +0000 (10:17 +0000)] 
scanner: fix indesc_list stack to be in the correct order

This fixes the location displayed in error messages.

Signed-off-by: Laurent Fasnacht <fasnacht@protonmail.ch>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agoInclusion depth was computed incorrectly for glob includes.
Laurent Fasnacht [Mon, 10 Feb 2020 10:17:35 +0000 (05:17 -0500)] 
Inclusion depth was computed incorrectly for glob includes.

Signed-off-by: Laurent Fasnacht <fasnacht@protonmail.ch>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agoscanner: remove parser_state->indescs static array
Laurent Fasnacht [Mon, 10 Feb 2020 10:17:24 +0000 (10:17 +0000)] 
scanner: remove parser_state->indescs static array

This static array is redundant with the indesc_list structure, but
is less flexible.

Signed-off-by: Laurent Fasnacht <fasnacht@protonmail.ch>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agoscanner: move indesc list append in scanner_push_indesc
Laurent Fasnacht [Mon, 10 Feb 2020 10:17:22 +0000 (10:17 +0000)] 
scanner: move indesc list append in scanner_push_indesc

Having a single point makes refactoring easier.

Signed-off-by: Laurent Fasnacht <fasnacht@protonmail.ch>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agoscanner: move the file descriptor to be in the input_descriptor structure
Laurent Fasnacht [Mon, 10 Feb 2020 10:17:21 +0000 (10:17 +0000)] 
scanner: move the file descriptor to be in the input_descriptor structure

This prevents a static allocation of file descriptors array, thus allows
more flexibility.

Signed-off-by: Laurent Fasnacht <fasnacht@protonmail.ch>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agoscanner: Extend asteriskstring definition
Phil Sutter [Thu, 6 Feb 2020 11:31:56 +0000 (12:31 +0100)] 
scanner: Extend asteriskstring definition

Accept escaped asterisks also mid-string and as only character.
Especially the latter will help when translating from iptables where
asterisk has no special meaning.

Signed-off-by: Phil Sutter <phil@nwl.cc>
5 years agodoc: nft.8: Mention wildcard interface matching
Phil Sutter [Thu, 6 Feb 2020 11:24:51 +0000 (12:24 +0100)] 
doc: nft.8: Mention wildcard interface matching

Special meaning of asterisk in interface names wasn't described
anywhere.

Signed-off-by: Phil Sutter <phil@nwl.cc>
5 years agodoc: nft.8: Describe element commands in their own section
Phil Sutter [Thu, 6 Feb 2020 16:01:23 +0000 (17:01 +0100)] 
doc: nft.8: Describe element commands in their own section

This unifies the redundant information in sets and maps sections and
also covers 'get' command.

Signed-off-by: Phil Sutter <phil@nwl.cc>
5 years agosrc: compute mnemonic port name much easier
Jan Engelhardt [Fri, 7 Feb 2020 11:43:21 +0000 (12:43 +0100)] 
src: compute mnemonic port name much easier

Signed-off-by: Jan Engelhardt <jengelh@inai.de>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agotests: py: add bit-shift tests.
Jeremy Sowden [Mon, 3 Feb 2020 11:20:23 +0000 (11:20 +0000)] 
tests: py: add bit-shift tests.

Add a couple of Python test-cases for setting the CT mark to a bitwise
expression derived from the packet mark and vice versa.

Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agotests: py: add missing JSON output.
Jeremy Sowden [Mon, 3 Feb 2020 11:20:22 +0000 (11:20 +0000)] 
tests: py: add missing JSON output.

The JSON output was missing for some existing tests.

Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agotests: shell: add bit-shift tests.
Jeremy Sowden [Mon, 3 Feb 2020 11:20:21 +0000 (11:20 +0000)] 
tests: shell: add bit-shift tests.

Add a couple of shell test-cases for setting the CT mark to a bitwise
expression derived from the packet mark and vice versa.

Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agoevaluate: change shift byte-order to host-endian.
Jeremy Sowden [Mon, 3 Feb 2020 11:20:20 +0000 (11:20 +0000)] 
evaluate: change shift byte-order to host-endian.

The byte-order of the righthand operands of the right-shifts generated
for payload and exthdr expressions is big-endian.  However, all right
operands should be host-endian.  Since evaluation of the shift binop
will insert a byte-order conversion to enforce this, change the
endianness in order to avoid the extra operation.

Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agoevaluate: correct variable name.
Jeremy Sowden [Mon, 3 Feb 2020 11:20:19 +0000 (11:20 +0000)] 
evaluate: correct variable name.

Rename the `lshift` variable used to store an right-shift expression to
`rshift`.

Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agoparser: add parenthesized statement expressions.
Jeremy Sowden [Mon, 3 Feb 2020 11:20:18 +0000 (11:20 +0000)] 
parser: add parenthesized statement expressions.

Primary and primary RHS expressions support parenthesized basic and
basic RHS expressions.  However, primary statement expressions do not
support parenthesized basic statement expressions.  Add them.

Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agosrc: Add support for concatenated set ranges
Stefano Brivio [Thu, 30 Jan 2020 00:16:57 +0000 (01:16 +0100)] 
src: Add support for concatenated set ranges

After exporting field lengths via NFTNL_SET_DESC_CONCAT attributes,
we now need to adjust parsing of user input and generation of
netlink key data to complete support for concatenation of set
ranges.

Instead of using separate elements for start and end of a range,
denoting the end element by the NFT_SET_ELEM_INTERVAL_END flag,
as it's currently done for ranges without concatenation, we'll use
the new attribute NFTNL_SET_ELEM_KEY_END as suggested by Pablo. It
behaves in the same way as NFTNL_SET_ELEM_KEY, but it indicates
that the included key represents the upper bound of a range.

For example, "packets with an IPv4 address between 192.0.2.0 and
192.0.2.42, with destination port between 22 and 25", needs to be
expressed as a single element with two keys:

  NFTA_SET_ELEM_KEY: 192.0.2.0 . 22
  NFTA_SET_ELEM_KEY_END: 192.0.2.42 . 25

To achieve this, we need to:

- adjust the lexer rules to allow multiton expressions as elements
  of a concatenation. As wildcards are not allowed (semantics would
  be ambiguous), exclude wildcards expressions from the set of
  possible multiton expressions, and allow them directly where
  needed. Concatenations now admit prefixes and ranges

- generate, for each element in a range concatenation, a second key
  attribute, that includes the upper bound for the range

- also expand prefixes and non-ranged values in the concatenation
  to ranges: given a set with interval and concatenation support,
  the kernel has no way to tell which elements are ranged, so they
  all need to be. For example, 192.0.2.0 . 192.0.2.9 : 1024 is
  sent as:

  NFTA_SET_ELEM_KEY: 192.0.2.0 . 1024
  NFTA_SET_ELEM_KEY_END: 192.0.2.9 . 1024

- aggregate ranges when elements received by the kernel represent
  concatenated ranges, see concat_range_aggregate()

- perform a few minor adjustments where interval expressions
  are already handled: we have intervals in these sets, but
  the set specification isn't just an interval, so we can't
  just aggregate and deaggregate interval ranges linearly

v4: No changes
v3:
 - rework to use a separate key for closing element of range instead of
   a separate element with EXPR_F_INTERVAL_END set (Pablo Neira Ayuso)
v2:
 - reworked netlink_gen_concat_data(), moved loop body to a new function,
   netlink_gen_concat_data_expr() (Phil Sutter)
 - dropped repeated pattern in bison file, replaced by a new helper,
   compound_expr_alloc_or_add() (Phil Sutter)
 - added set_is_nonconcat_range() helper (Phil Sutter)
 - in expr_evaluate_set(), we need to set NFT_SET_SUBKEY also on empty
   sets where the set in the context already has the flag
 - dropped additional 'end' parameter from netlink_gen_data(),
   temporarily set EXPR_F_INTERVAL_END on expressions and use that from
   netlink_gen_concat_data() to figure out we need to add the 'end'
   element (Phil Sutter)
 - replace range_mask_len() by a simplified version, as we don't need
   to actually store the composing masks of a range (Phil Sutter)

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agosrc: Add support for NFTNL_SET_DESC_CONCAT
Stefano Brivio [Thu, 30 Jan 2020 00:16:56 +0000 (01:16 +0100)] 
src: Add support for NFTNL_SET_DESC_CONCAT

To support arbitrary range concatenations, the kernel needs to know
how long each field in the concatenation is. The new libnftnl
NFTNL_SET_DESC_CONCAT set attribute describes this as an array of
lengths, in bytes, of concatenated fields.

While evaluating concatenated expressions, export the datatype size
into the new field_len array, and hand the data over via libnftnl.

Similarly, when data is passed back from libnftnl, parse it into
the set description.

When set data is cloned, we now need to copy the additional fields
in set_clone(), too.

This change depends on the libnftnl patch with title:
  set: Add support for NFTA_SET_DESC_CONCAT attributes

v4: No changes
v3: Rework to use set description data instead of a stand-alone
    attribute
v2: No changes

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agoinclude: resync nf_tables.h cache copy
Stefano Brivio [Thu, 30 Jan 2020 00:16:55 +0000 (01:16 +0100)] 
include: resync nf_tables.h cache copy

Get this header in sync with nf-next as of merge commit
b3a608222336 (5.6-rc1-ish).

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agotests: py: Support testing host binaries
Phil Sutter [Thu, 6 Feb 2020 00:36:01 +0000 (01:36 +0100)] 
tests: py: Support testing host binaries

Support -H/--host option to use host's libnftables.so.1. Alternatively
users may specify a custom library path via -l/--library option.

Signed-off-by: Phil Sutter <phil@nwl.cc>
5 years agotests: monitor: Support testing host's nft binary
Phil Sutter [Wed, 5 Feb 2020 18:48:53 +0000 (19:48 +0100)] 
tests: monitor: Support testing host's nft binary

Add support for -H/--host flag to use 'nft' tool from $PATH instead of
the local one.

Signed-off-by: Phil Sutter <phil@nwl.cc>
5 years agotests: json_echo: Support testing host binaries
Phil Sutter [Fri, 10 Jan 2020 10:19:42 +0000 (11:19 +0100)] 
tests: json_echo: Support testing host binaries

Support -H/--host option to use host's libnftables.so.1. Alternatively
users may specify a custom library path via -l/--library option.

Signed-off-by: Phil Sutter <phil@nwl.cc>
5 years agotests: json_echo: Fix for Python3
Phil Sutter [Thu, 6 Feb 2020 00:21:30 +0000 (01:21 +0100)] 
tests: json_echo: Fix for Python3

The keys() method returns an object which does not support indexing, so
convert it to a list prior to doing so.

Fixes: a35e3a0cdc63a ("tests: json_echo: convert to py3")
Signed-off-by: Phil Sutter <phil@nwl.cc>
5 years agonetlink: add support for handling shift expressions.
Jeremy Sowden [Sun, 19 Jan 2020 22:57:09 +0000 (22:57 +0000)] 
netlink: add support for handling shift expressions.

The kernel supports bitwise shift operations, so add support to the
netlink linearization and delinearization code.  The number of bits (the
righthand operand) is expected to be a 32-bit value in host endianness.

Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agoinclude: update nf_tables.h.
Jeremy Sowden [Sun, 19 Jan 2020 22:57:08 +0000 (22:57 +0000)] 
include: update nf_tables.h.

The kernel UAPI header includes a couple of new bitwise netlink
attributes and an enum.

Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agonetlink_delinearize: remove commented out pr_debug statement.
Jeremy Sowden [Sun, 19 Jan 2020 22:57:05 +0000 (22:57 +0000)] 
netlink_delinearize: remove commented out pr_debug statement.

The statement doesn't compile, so remove it.

Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agonetlink_delinearize: fix typo.
Jeremy Sowden [Sun, 19 Jan 2020 22:57:04 +0000 (22:57 +0000)] 
netlink_delinearize: fix typo.

s/Of/If/ in comment describing function.

Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agosrc: white-space fixes.
Jeremy Sowden [Sun, 19 Jan 2020 22:57:03 +0000 (22:57 +0000)] 
src: white-space fixes.

Remove some trailing white-space and fix some indentation.

Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agoUpdate gitignore.
Jeremy Sowden [Sun, 19 Jan 2020 22:57:02 +0000 (22:57 +0000)] 
Update gitignore.

Add ctags and etags tag files, and Emacs back-up files.

Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agoinclude: nf_tables: correct bitwise header comment.
Jeremy Sowden [Sun, 19 Jan 2020 10:58:01 +0000 (10:58 +0000)] 
include: nf_tables: correct bitwise header comment.

The comment documenting how bitwise expressions work includes a table
which summarizes the mask and xor arguments combined to express the
supported boolean operations.  However, the row for OR:

 mask    xor
 0       x

is incorrect.

  dreg = (sreg & 0) ^ x

is not equivalent to:

  dreg = sreg | x

What the code actually does is:

  dreg = (sreg & ~x) ^ x

Update the documentation to match.

Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agotests: shell: update list of rmmod modules
Pablo Neira Ayuso [Wed, 22 Jan 2020 21:44:32 +0000 (22:44 +0100)] 
tests: shell: update list of rmmod modules

* Missing nft_fib_inet.
* nft_chain_nat_ipv4 and nft_chain_nat_ipv6 became nft_chain_nat.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agotests: shell: set lookup and set update
Pablo Neira Ayuso [Wed, 22 Jan 2020 21:38:56 +0000 (22:38 +0100)] 
tests: shell: set lookup and set update

A simple test to cover set lookup and update in one rule.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agonetlink: Avoid potential NULL-pointer deref in netlink_gen_payload_stmt()
Phil Sutter [Mon, 20 Jan 2020 15:32:40 +0000 (16:32 +0100)] 
netlink: Avoid potential NULL-pointer deref in netlink_gen_payload_stmt()

With payload_needs_l4csum_update_pseudohdr() unconditionally
dereferencing passed 'desc' parameter and a previous check for it to be
non-NULL, make sure to call the function only if input is sane.

Fixes: 68de70f2b3fc6 ("netlink_linearize: fix IPv6 layer 4 checksum mangling")
Signed-off-by: Phil Sutter <phil@nwl.cc>
Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agonetlink: Fix leaks in netlink_parse_cmp()
Phil Sutter [Mon, 20 Jan 2020 13:48:26 +0000 (14:48 +0100)] 
netlink: Fix leaks in netlink_parse_cmp()

This fixes several problems at once:

* Err path would leak expr 'right' in two places and 'left' in one.
* Concat case would leak 'right' by overwriting the pointer. Introduce a
  temporary variable to hold the new pointer.

Fixes: 6377380bc265f ("netlink_delinearize: handle relational and lookup concat expressions")
Signed-off-by: Phil Sutter <phil@nwl.cc>
Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agonetlink: Fix leak in unterminated string deserializer
Phil Sutter [Mon, 20 Jan 2020 12:52:10 +0000 (13:52 +0100)] 
netlink: Fix leak in unterminated string deserializer

Allocated 'mask' expression is not freed before returning to caller,
although it is used temporarily only.

Fixes: b851ba4731d9f ("src: add interface wildcard matching")
Signed-off-by: Phil Sutter <phil@nwl.cc>
Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agoevaluate: better error notice when interval flag is not set on
Pablo Neira Ayuso [Fri, 10 Jan 2020 10:53:22 +0000 (11:53 +0100)] 
evaluate: better error notice when interval flag is not set on

Users get confused with the existing error notice, let's try a different one:

 # nft add element x y { 1.1.1.0/24 }
 Error: You must add 'flags interval' to your set declaration if you want to add prefix elements
 add element x y { 1.1.1.0/24 }
                   ^^^^^^^^^^

Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1380
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Acked-by: Phil Sutter <phil@nwl.cc>
5 years agocache: Fix for doubled output after reset command
Phil Sutter [Tue, 14 Jan 2020 16:25:35 +0000 (17:25 +0100)] 
cache: Fix for doubled output after reset command

Reset command causes a dump of the objects to reset and adds those to
cache. Yet it ignored if the object in question was already there and up
to now CMD_RESET was flagged as NFT_CACHE_FULL.

Tackle this from two angles: First, reduce cache requirements of reset
command to the necessary bits which is table cache. This alone would
suffice if there wasn't interactive mode (and other libnftables users):
A cache containing the objects to reset might be in place already, so
add dumped objects to cache only if they don't exist already.

Signed-off-by: Phil Sutter <phil@nwl.cc>
Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agotests: shell: Search diff tool once and for all
Phil Sutter [Tue, 14 Jan 2020 15:50:35 +0000 (16:50 +0100)] 
tests: shell: Search diff tool once and for all

Instead of calling 'which diff' over and over again, just detect the
tool's presence in run-tests.sh and pass $DIFF to each testcase just
like with nft binary.

Fall back to using 'true' command to avoid the need for any conditional
calling in test cases.

While being at it, unify potential diff calls so that a string
comparison in shell happens irrespective of diff presence.

Signed-off-by: Phil Sutter <phil@nwl.cc>
Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agoxfrm: spi is big-endian
Florian Westphal [Tue, 14 Jan 2020 12:37:28 +0000 (13:37 +0100)] 
xfrm: spi is big-endian

the kernel stores spi in a __be32, so fix up the byteorder annotation.

Signed-off-by: Florian Westphal <fw@strlen.de>
Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agomonitor: Fix output for ranges in anonymous sets
Phil Sutter [Mon, 13 Jan 2020 13:53:24 +0000 (14:53 +0100)] 
monitor: Fix output for ranges in anonymous sets

Previous fix for named interval sets was simply wrong: Instead of
limiting decomposing to anonymous interval sets, it effectively disabled
it entirely.

Since code needs to check for both interval and anonymous bits
separately, introduce set_is_interval() helper to keep the code
readable.

Also extend test case to assert ranges in anonymous sets are correctly
printed by echo or monitor modes. Without this fix, range boundaries are
printed as individual set elements.

Fixes: 5d57fa3e99bb9 ("monitor: Do not decompose non-anonymous sets")
Signed-off-by: Phil Sutter <phil@nwl.cc>
Reviewed-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agotests: monitor: Support running individual test cases
Phil Sutter [Fri, 10 Jan 2020 10:15:45 +0000 (11:15 +0100)] 
tests: monitor: Support running individual test cases

Recognize testcase paths on command line and limit testing on those
only.

Signed-off-by: Phil Sutter <phil@nwl.cc>
Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agomonitor: Fix for use after free when printing map elements
Phil Sutter [Thu, 9 Jan 2020 16:43:11 +0000 (17:43 +0100)] 
monitor: Fix for use after free when printing map elements

When populating the dummy set, 'data' field must be cloned just like
'key' field.

Fixes: 343a51702656a ("src: store expr, not dtype to track data in sets")
Signed-off-by: Phil Sutter <phil@nwl.cc>
Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agomonitor: Do not decompose non-anonymous sets
Phil Sutter [Thu, 9 Jan 2020 12:34:20 +0000 (13:34 +0100)] 
monitor: Do not decompose non-anonymous sets

They have been decomposed already, trying to do that again causes a
segfault. This is a similar fix as in commit 8ecb885589591 ("src:
restore --echo with anonymous sets").

Signed-off-by: Phil Sutter <phil@nwl.cc>
Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agomain: restore --debug
Pablo Neira Ayuso [Thu, 9 Jan 2020 17:16:18 +0000 (18:16 +0100)] 
main: restore --debug

Broken since options are mandatory before commands.

Fixes: fb9cea50e8b3 ("main: enforce options before commands")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agoevaluate: fix expr_set_context call for shift binops.
Jeremy Sowden [Mon, 6 Jan 2020 22:35:10 +0000 (22:35 +0000)] 
evaluate: fix expr_set_context call for shift binops.

expr_evaluate_binop calls expr_set_context for shift expressions to set
the context data-type to `integer`.  This clobbers the byte-order of the
context, resulting in unexpected conversions to NBO.  For example:

  $ sudo nft flush ruleset
  $ sudo nft add table t
  $ sudo nft add chain t c '{ type filter hook output priority mangle; }'
  $ sudo nft add rule t c oif lo tcp dport ssh ct mark set '0x10 | 0xe'
  $ sudo nft add rule t c oif lo tcp dport ssh ct mark set '0xf << 1'
  $ sudo nft list table t
  table ip t {
          chain c {
                  type filter hook output priority mangle; policy accept;
                  oif "lo" tcp dport 22 ct mark set 0x0000001e
                  oif "lo" tcp dport 22 ct mark set 0x1e000000
          }
  }

Replace it with a call to __expr_set_context and set the byteorder to
that of the left operand since this is the value being shifted.

Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agotests: shell: delete flowtable after flushing chain
Pablo Neira Ayuso [Fri, 3 Jan 2020 16:46:24 +0000 (17:46 +0100)] 
tests: shell: delete flowtable after flushing chain

This patch adds a test to cover delete flowtable after chain flush
including two rule references to the flowtable.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agoscanner: incorrect error reporting after file inclusion
Pablo Neira Ayuso [Thu, 2 Jan 2020 15:37:31 +0000 (16:37 +0100)] 
scanner: incorrect error reporting after file inclusion

scanner_pop_buffer() incorrectly sets the current input descriptor. The
state->indesc_idx field actually stores the number of input descriptors
in the stack, decrement it and then update the current input descriptor
accordingly.

Fixes: 60e917fa7cb5 ("src: dynamic input_descriptor allocation")
Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1383
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agometa: add slave device matching
Florian Westphal [Wed, 18 Dec 2019 11:10:41 +0000 (12:10 +0100)] 
meta: add slave device matching

Adds "meta sdif" and "meta sdifname".
Both only work in input/forward hook of ipv4/ipv6/inet family.

Cc: Martin Willi <martin@strongswan.org>
Signed-off-by: Florian Westphal <fw@strlen.de>
5 years agodoc: Fix typo in IGMP section
Benjamin Poirier [Thu, 26 Dec 2019 00:08:37 +0000 (09:08 +0900)] 
doc: Fix typo in IGMP section

Signed-off-by: Benjamin Poirier <benjamin.poirier@gmail.com>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agomain: enforce options before commands
Pablo Neira Ayuso [Fri, 13 Dec 2019 10:32:46 +0000 (11:32 +0100)] 
main: enforce options before commands

This patch turns on POSIXLY_CORRECT on the getopt parser to enforce
options before commands. Users get a hint in such a case:

 # nft list ruleset -a
 Error: syntax error, options must be specified before commands
 nft list ruleset -a
    ^             ~~

This patch recovers 9fc71bc6b602 ("main: Fix for misleading error with
negative chain priority").

Tests have been updated.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agodoc: mention 'typeof' as alternative to 'type' keyword
Florian Westphal [Tue, 10 Dec 2019 14:42:48 +0000 (15:42 +0100)] 
doc: mention 'typeof' as alternative to 'type' keyword

Signed-off-by: Florian Westphal <fw@strlen.de>
5 years agoevaluate: print a hint about 'typeof' syntax on 0 keylen
Florian Westphal [Tue, 10 Dec 2019 14:23:35 +0000 (15:23 +0100)] 
evaluate: print a hint about 'typeof' syntax on 0 keylen

If user says

'type integer; ...' in a set definition, don't just throw an error --
provide a hint that the typeof keyword can be used to provide
the needed size information.

Signed-off-by: Florian Westphal <fw@strlen.de>
5 years agotests: add typeof test cases
Florian Westphal [Fri, 16 Aug 2019 12:22:01 +0000 (14:22 +0200)] 
tests: add typeof test cases

Add sets using unspecific string/integer types, one with
osf name, other with vlan id.  Neither type can be used directly,
as they lack the type size information.

Signed-off-by: Florian Westphal <fw@strlen.de>
5 years agomnl: round up the map data size too
Florian Westphal [Tue, 10 Dec 2019 21:08:44 +0000 (22:08 +0100)] 
mnl: round up the map data size too

Same as key: if the size isn't divisible by BITS_PER_BYTE, we need to
round up, not down.

Without this, you can't store vlan ids in a map, as they are truncated
to 8 bit.

Signed-off-by: Florian Westphal <fw@strlen.de>
5 years agoxfrm: add parse and build userdata interface
Pablo Neira Ayuso [Tue, 17 Dec 2019 17:17:02 +0000 (18:17 +0100)] 
xfrm: add parse and build userdata interface

Add support for meta userdata area.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Florian Westphal <fw@strlen.de>
5 years agofib: add parse and build userdata interface
Pablo Neira Ayuso [Tue, 17 Dec 2019 17:17:01 +0000 (18:17 +0100)] 
fib: add parse and build userdata interface

Add support for meta userdata area.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Florian Westphal <fw@strlen.de>
5 years agort: add parse and build userdata interface
Pablo Neira Ayuso [Tue, 17 Dec 2019 17:17:00 +0000 (18:17 +0100)] 
rt: add parse and build userdata interface

Add support for meta userdata area.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Florian Westphal <fw@strlen.de>
5 years agohash: add parse and build userdata interface
Pablo Neira Ayuso [Tue, 17 Dec 2019 17:16:59 +0000 (18:16 +0100)] 
hash: add parse and build userdata interface

Add support for meta userdata area.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Florian Westphal <fw@strlen.de>
5 years agonumgen: add parse and build userdata interface
Pablo Neira Ayuso [Tue, 17 Dec 2019 17:16:58 +0000 (18:16 +0100)] 
numgen: add parse and build userdata interface

Add support for meta userdata area.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Florian Westphal <fw@strlen.de>
5 years agoct: add parse and build userdata interface
Pablo Neira Ayuso [Tue, 17 Dec 2019 17:16:57 +0000 (18:16 +0100)] 
ct: add parse and build userdata interface

Add support for meta userdata area.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Florian Westphal <fw@strlen.de>
5 years agoosf: add parse and build userdata interface
Pablo Neira Ayuso [Tue, 17 Dec 2019 17:16:56 +0000 (18:16 +0100)] 
osf: add parse and build userdata interface

Add support for meta userdata area.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Florian Westphal <fw@strlen.de>
5 years agosocket: add parse and build userdata interface
Pablo Neira Ayuso [Tue, 17 Dec 2019 17:16:55 +0000 (18:16 +0100)] 
socket: add parse and build userdata interface

Add support for meta userdata area.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Florian Westphal <fw@strlen.de>
5 years agoexthdr: add parse and build userdata interface
Pablo Neira Ayuso [Tue, 17 Dec 2019 17:16:54 +0000 (18:16 +0100)] 
exthdr: add parse and build userdata interface

Add support for meta userdata area.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Florian Westphal <fw@strlen.de>
5 years agoexthdr: add exthdr_desc_id enum and use it
Pablo Neira Ayuso [Tue, 17 Dec 2019 17:16:53 +0000 (18:16 +0100)] 
exthdr: add exthdr_desc_id enum and use it

This allows to identify the exthdr protocol from the userdata area.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Florian Westphal <fw@strlen.de>
5 years agometa: add parse and build userdata interface
Pablo Neira Ayuso [Tue, 17 Dec 2019 17:16:52 +0000 (18:16 +0100)] 
meta: add parse and build userdata interface

Add support for meta userdata area.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Florian Westphal <fw@strlen.de>
5 years agosrc: add "typeof" build/parse/print support
Florian Westphal [Sun, 4 Aug 2019 20:24:22 +0000 (22:24 +0200)] 
src: add "typeof" build/parse/print support

This patch adds two new expression operations to build and to parse the
userdata area that describe the set key and data typeof definitions.

For maps, the grammar enforces either

"type data_type : data_type" or or "typeof expression : expression".

Check both key and data for valid user typeof info first.
If they check out, flag set->key_typeof_valid as true and use it for
printing the key info.

This patch comes with initial support for using payload expressions
with the 'typeof' keyword, followup patches will add support for other
expressions as well.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Florian Westphal <fw@strlen.de>
5 years agoparser: add typeof keyword for declarations
Pablo Neira Ayuso [Tue, 30 Jul 2019 14:16:16 +0000 (16:16 +0200)] 
parser: add typeof keyword for declarations

Add a typeof keyword to automatically use the correct type in set and map
declarations.

table filter {
set blacklist {
typeof ip saddr
}

chain input {
ip saddr @blacklist counter drop
}
}

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Florian Westphal <fw@strlen.de>
5 years agoexpr: add expr_ops_by_type()
Pablo Neira Ayuso [Mon, 16 Dec 2019 12:42:21 +0000 (13:42 +0100)] 
expr: add expr_ops_by_type()

Fetch expression operation from the expression type.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Florian Westphal <fw@strlen.de>
5 years agoproto: add proto_desc_id enumeration
Pablo Neira Ayuso [Mon, 16 Dec 2019 12:42:20 +0000 (13:42 +0100)] 
proto: add proto_desc_id enumeration

This allows to uniquely identify the protocol description.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Florian Westphal <fw@strlen.de>
5 years agosrc: store expr, not dtype to track data in sets
Florian Westphal [Tue, 16 Jul 2019 17:03:55 +0000 (19:03 +0200)] 
src: store expr, not dtype to track data in sets

This will be needed once we add support for the 'typeof' keyword to
handle maps that could e.g. store 'ct helper' "type" values.

Instead of:

set foo {
type ipv4_addr . mark;

this would allow

set foo {
typeof(ip saddr) . typeof(ct mark);

(exact syntax TBD).

This would be needed to allow sets that store variable-sized data types
(string, integer and the like) that can't be used at at the moment.

Adding special data types for everything is problematic due to the
large amount of different types needed.

For anonymous sets, e.g. "string" can be used because the needed size can
be inferred from the statement, e.g.  'osf name { "Windows", "Linux }',
but in case of named sets that won't work because 'type string' lacks the
context needed to derive the size information.

With 'typeof(osf name)' the context is there, but at the moment it won't
help because the expression is discarded instantly and only the data
type is retained.

Signed-off-by: Florian Westphal <fw@strlen.de>
5 years agoparser: add a helper for concat expression handling
Florian Westphal [Wed, 11 Dec 2019 13:31:44 +0000 (14:31 +0100)] 
parser: add a helper for concat expression handling

Cull the repeated copy&paste snippets and add/use a helper for this.

Signed-off-by: Florian Westphal <fw@strlen.de>
5 years agopy: load the SONAME-versioned shared object
Arturo Borrero Gonzalez [Tue, 10 Dec 2019 11:00:45 +0000 (12:00 +0100)] 
py: load the SONAME-versioned shared object

Instruct the python module to load the SONAME versioned shared object.

Normal end-user systems may only have available libnftables.so.1.0.0 and not
libnftables.so which is usually only present in developer systems.

In Debian systems, for example:

 % dpkg -L libnftables1 | grep so.1
 /usr/lib/x86_64-linux-gnu/libnftables.so.1.0.0
 /usr/lib/x86_64-linux-gnu/libnftables.so.1

 % dpkg -L libnftables-dev | grep so
 /usr/lib/x86_64-linux-gnu/libnftables.so

The "1" is not a magic number, is the SONAME of libnftables in the current
version, as stated in Make_global.am.

Reported-by: Michael Biebl <biebl@debian.org>
Signed-off-by: Arturo Borrero Gonzalez <arturo@netfilter.org>
Acked-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agosegtree: don't remove nul-root element from interval set
Pablo Neira Ayuso [Thu, 5 Dec 2019 18:07:16 +0000 (19:07 +0100)] 
segtree: don't remove nul-root element from interval set

Check from the delinearize set element path if the nul-root element
already exists in the interval set. Hence, the element insertion path
skips the implicit nul-root interval insertion.

Under some circunstances, nft bogusly fails to delete the last element
of the interval set and to create an element in an existing empty
internal set. This patch includes a test that reproduces the issue.

Fixes: 4935a0d561b5 ("segtree: special handling for the first non-matching segment")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agodoc: Remove repeated paragraph and fix typo
nl6720 [Sun, 8 Dec 2019 09:25:58 +0000 (11:25 +0200)] 
doc: Remove repeated paragraph and fix typo

[ Florian: Expand "ininterface" to "incoming interface" ]

Signed-off-by: nl6720 <nl6720@gmail.com>
Signed-off-by: Florian Westphal <fw@strlen.de>
5 years agodoc: Clarify conditions under which a reject verdict is permissible
Duncan Roe [Fri, 6 Dec 2019 02:37:12 +0000 (13:37 +1100)] 
doc: Clarify conditions under which a reject verdict is permissible

A phrase like "input chain" is a throwback to xtables documentation.
In nft, chains are containers for rules. They do have a type, but what's
important here is which hook each uses.

Signed-off-by: Duncan Roe <duncan_roe@optusnet.com.au>
Signed-off-by: Florian Westphal <fw@strlen.de>
5 years agobuild: nftables 0.9.3 depends on libnftnl 1.1.5
Pablo Neira Ayuso [Thu, 5 Dec 2019 09:35:19 +0000 (10:35 +0100)] 
build: nftables 0.9.3 depends on libnftnl 1.1.5

nftables 0.9.3 requires libnftnl 1.1.5, otherwise compilation breaks:
https://bugs.gentoo.org/701976.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Acked-by: Phil Sutter <phil@nwl.cc>
5 years agoinclude: add nf_tables_compat.h to tarballs
Pablo Neira Ayuso [Tue, 3 Dec 2019 00:04:38 +0000 (01:04 +0100)] 
include: add nf_tables_compat.h to tarballs

Add it to Makefile.am so make distcheck adds this header to tarballs.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agobuild: Bump version to v0.9.3 v0.9.3
Pablo Neira Ayuso [Mon, 2 Dec 2019 19:34:32 +0000 (20:34 +0100)] 
build: Bump version to v0.9.3

Update release name based on Jazz series, Count Basie's "Topsy":

https://www.youtube.com/watch?v=Up78NJHESKE

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agonetlink: off-by-one write in netdev chain device array
Pablo Neira Ayuso [Mon, 2 Dec 2019 18:47:32 +0000 (19:47 +0100)] 
netlink: off-by-one write in netdev chain device array

==728473== Invalid write of size 8
==728473==    at 0x48960F2: netlink_delinearize_chain (netlink.c:422)
==728473==    by 0x4896252: list_chain_cb (netlink.c:459)
==728473==    by 0x4896252: list_chain_cb (netlink.c:441)
==728473==    by 0x4F2C654: nftnl_chain_list_foreach (chain.c:1011)
==728473==    by 0x489629F: netlink_list_chains (netlink.c:478)
==728473==    by 0x4882303: cache_init_objects (rule.c:177)
==728473==    by 0x4882303: cache_init (rule.c:222)
==728473==    by 0x4882303: cache_update (rule.c:272)
==728473==    by 0x48A7DCE: nft_evaluate (libnftables.c:408)
==728473==    by 0x48A86D9: nft_run_cmd_from_buffer (libnftables.c:449)
==728473==    by 0x10A5D6: main (main.c:338)

Fixes: 3fdc7541fba0 ("src: add multidevice support for netdev chain")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agodoc: fix inconsistency in set statement documentation.
Jeremy Sowden [Sat, 30 Nov 2019 11:30:57 +0000 (11:30 +0000)] 
doc: fix inconsistency in set statement documentation.

The description of the set statement asserts that the set must have been
created with the "dynamic" flag.  However, this is not the case, and it
is contradicted by the following example in which the "dynamic" flag
does not appear.

In fact, one or both of the "dynamic" or the "timeout" flags need to be
used, depending on what the set statement contains.  Amend the
description to explain this more accurately.

Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agoscanner: fix out-of-bound memory write in include_file()
Eric Jallot [Fri, 29 Nov 2019 14:30:39 +0000 (15:30 +0100)] 
scanner: fix out-of-bound memory write in include_file()

Before patch:
 # echo 'include "/tmp/rules.nft"' > /tmp/rules.nft
 # nft -f /tmp/rules.nft
 In file included from /tmp/rules.nft:1:1-25:
                  from /tmp/rules.nft:1:1-25:
 [snip]
                  from /tmp/rules.nft:1:1-25:
 /tmp/rules.nft:1:1-25: Error: Include nested too deeply, max 16 levels
 include "/tmp/rules.nft"
 ^^^^^^^^^^^^^^^^^^^^^^^^^
 double free or corruption (out)
 Aborted (core dumped)

valgrind reports:

==8856== Invalid write of size 8
==8856==    at 0x4E8FCAF: include_file (scanner.l:718)
==8856==    by 0x4E8FEF6: include_glob (scanner.l:793)
==8856==    by 0x4E9985D: scanner_include_file (scanner.l:875)
==8856==    by 0x4E89D7A: nft_parse (parser_bison.y:828)
==8856==    by 0x4E765E1: nft_parse_bison_filename (libnftables.c:394)
==8856==    by 0x4E765E1: nft_run_cmd_from_filename (libnftables.c:497)
==8856==    by 0x40172D: main (main.c:340)

So perform bounds checking on MAX_INCLUDE_DEPTH before writing.

After patch:
 # nft -f /tmp/rules.nft
 In file included from /tmp/rules.nft:1:1-25:
                  from /tmp/rules.nft:1:1-25:
 [snip]
                  from /tmp/rules.nft:1:1-25:
 /tmp/rules.nft:1:1-25: Error: Include nested too deeply, max 16 levels
 include "/tmp/rules.nft"
 ^^^^^^^^^^^^^^^^^^^^^^^^^
 # echo $?
 1

Also:
Update scanner_push_file() function definition accordingly.

Fixes: 32325e3c3fab4 ("libnftables: Store top_scope in struct nft_ctx")
Signed-off-by: Eric Jallot <ejallot@gmail.com>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agosegtree: restore automerge
Pablo Neira Ayuso [Tue, 26 Nov 2019 10:24:16 +0000 (11:24 +0100)] 
segtree: restore automerge

Always close interval in non-anonymous sets unless the auto-merge
feature is set on.

Fixes: a4ec05381261 ("segtree: always close interval in non-anonymous sets")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agosegtree: Fix add and delete of element in same batch
Phil Sutter [Thu, 21 Nov 2019 10:41:24 +0000 (11:41 +0100)] 
segtree: Fix add and delete of element in same batch

The commit this fixes accidentally broke a rather exotic use-case which
is but used in set-simple.t of tests/monitor:

| # nft 'add element t s { 22-25 }; delete element t s { 22-25 }'

Since ranges are now checked for existence in userspace before delete
command is submitted to kernel, the second command above was rejected
because the range in question wasn't present in cache yet. Fix this by
adding new interval set elements to cache after creating the batch job
for them.

Fixes: decc12ec2dc31 ("segtree: Check ranges when deleting elements")
Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agotests/py: Set a fixed timezone in nft-test.py
Phil Sutter [Sat, 16 Nov 2019 21:32:18 +0000 (22:32 +0100)] 
tests/py: Set a fixed timezone in nft-test.py

Payload generated for 'meta time' matches depends on host's timezone and
DST setting. To produce constant output, set a fixed timezone in
nft-test.py. Choose UTC-2 since most payloads are correct then, adjust
the remaining two tests.

Fixes: 0518ea3f70d8c ("tests: add meta time test cases")
Signed-off-by: Phil Sutter <phil@nwl.cc>
Acked-by: Ander Juaristi <a@juaristi.eus>
Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agonft.8: Fix nat family spec position
Phil Sutter [Wed, 27 Nov 2019 12:58:00 +0000 (13:58 +0100)] 
nft.8: Fix nat family spec position

In inet family nat statements, ip/ip6 keyword must come before 'to'
keyword, not after.

Fixes: fbe27464dee45 ("src: add nat support for the inet family")
Cc: Florian Westphal <fw@strlen.de>
Signed-off-by: Phil Sutter <phil@nwl.cc>
Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agonft.8: Describe numgen expression
Phil Sutter [Wed, 27 Nov 2019 17:09:05 +0000 (18:09 +0100)] 
nft.8: Describe numgen expression

Signed-off-by: Phil Sutter <phil@nwl.cc>
Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agofiles: add example secmark config
Christian Göttsche [Sat, 23 Nov 2019 16:22:40 +0000 (17:22 +0100)] 
files: add example secmark config

Signed-off-by: Christian Göttsche <cgzones@googlemail.com>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agosrc: add ability to set/get secmarks to/from connection
Christian Göttsche [Sat, 23 Nov 2019 16:22:39 +0000 (17:22 +0100)] 
src: add ability to set/get secmarks to/from connection

Labeling established and related packets requires the secmark to be stored in the connection.
Add the ability to store and retrieve secmarks like:

    ...
    chain input {
        ...

        # label new incoming packets
        ct state new meta secmark set tcp dport map @secmapping_in

        # add label to connection
        ct state new ct secmark set meta secmark

        # set label for est/rel packets from connection
        ct state established,related meta secmark set ct secmark

        ...
    }
    ...
    chain output {
        ...

        # label new outgoing packets
        ct state new meta secmark set tcp dport map @secmapping_out

        # add label to connection
        ct state new ct secmark set meta secmark

        # set label for est/rel packets from connection
        ct state established,related meta secmark set ct secmark

        ...
        }
    ...

This patch also disallow constant value on the right hand side.

    # nft add rule x y meta secmark 12
    Error: Cannot be used with right hand side constant value
    add rule x y meta secmark 12
                 ~~~~~~~~~~~~ ^^
    # nft add rule x y ct secmark 12
    Error: Cannot be used with right hand side constant value
    add rule x y ct secmark 12
                 ~~~~~~~~~~ ^^
    # nft add rule x y ct secmark set 12
    Error: ct secmark must not be set to constant value
    add rule x y ct secmark set 12
                 ^^^^^^^^^^^^^^^^^

This patch improves 3bc84e5c1fdd ("src: add support for setting secmark").

Signed-off-by: Christian Göttsche <cgzones@googlemail.com>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agomnl: Fix -Wimplicit-function-declaration warnings
Michal Rostecki [Thu, 21 Nov 2019 12:33:32 +0000 (13:33 +0100)] 
mnl: Fix -Wimplicit-function-declaration warnings

This change fixes the following warnings:

mnl.c: In function ‘mnl_nft_flowtable_add’:
mnl.c:1442:14: warning: implicit declaration of function ‘calloc’ [-Wimplicit-function-declaration]
  dev_array = calloc(len, sizeof(char *));
              ^~~~~~
mnl.c:1442:14: warning: incompatible implicit declaration of built-in function ‘calloc’
mnl.c:1442:14: note: include ‘<stdlib.h>’ or provide a declaration of ‘calloc’
mnl.c:1449:2: warning: implicit declaration of function ‘free’ [-Wimplicit-function-declaration]
  free(dev_array);
  ^~~~
mnl.c:1449:2: warning: incompatible implicit declaration of built-in function ‘free’
mnl.c:1449:2: note: include ‘<stdlib.h>’ or provide a declaration of ‘free’

Signed-off-by: Michal Rostecki <mrostecki@opensuse.org>
Acked-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agostatement: make secmark statements idempotent
Christian Göttsche [Wed, 20 Nov 2019 17:43:54 +0000 (18:43 +0100)] 
statement: make secmark statements idempotent

Currently lines like

    ct state new meta secmark set tcp dport map @secmapping_in

becomes

    ct state new secmark name tcp dport map @secmapping_in

which is not correct.

Fixes: 3bc84e5c1fdd ("src: add support for setting secmark")
Signed-off-by: Christian Göttsche <cgzones@googlemail.com>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agoscanner: Introduce numberstring
Phil Sutter [Fri, 15 Nov 2019 18:11:32 +0000 (19:11 +0100)] 
scanner: Introduce numberstring

This token combines decstring and hexstring. The latter two had
identical action blocks (which were not completely trivial), this allows
to merge them.

Signed-off-by: Phil Sutter <phil@nwl.cc>
Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agofiles: Drop shebangs from config files
Phil Sutter [Wed, 6 Nov 2019 20:59:07 +0000 (21:59 +0100)] 
files: Drop shebangs from config files

These are not meant to be executed as is but instead loaded via
'nft -f' - all-in-one.nft even points this out in header comment.
While being at it, drop two spelling mistakes found along the way.

Consequently remove executable bits - being registered in automake as
dist_pkgsysconf_DATA, they're changed to 644 upon installation anyway.

Also there is obviously no need for replacement of nft binary path
anymore, drop that bit from Makefile.am.

Signed-off-by: Phil Sutter <phil@nwl.cc>
Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agofiles: Install sample scripts from files/examples
Phil Sutter [Thu, 7 Nov 2019 11:38:57 +0000 (12:38 +0100)] 
files: Install sample scripts from files/examples

Assuming these are still relevant and useful as a source of inspiration,
install them into DATAROOTDIR/doc/nftables/examples.

Signed-off-by: Phil Sutter <phil@nwl.cc>
Acked-by: Arturo Borrero Gonzalez <arturo@netfilter.org>
Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agotests: shell: set reference from variable definition
Pablo Neira Ayuso [Mon, 18 Nov 2019 14:38:25 +0000 (15:38 +0100)] 
tests: shell: set reference from variable definition

Allow to define variable using set reference, eg.

define x = @z

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agoparser_bison: Avoid set references in odd places
Phil Sutter [Sat, 16 Nov 2019 20:02:07 +0000 (21:02 +0100)] 
parser_bison: Avoid set references in odd places

With set references being recognized by symbol_expr and that being part
of primary_expr as well as primary_rhs_expr, they could basically occur
anywhere while in fact they are allowed only in quite a few spots.

Untangle things a bit by introducing set_ref_expr and adding that only
in places where it is needed to pass testsuites.

Make sure users can define variables as set references, eg.

define xyz = @setref

And allow to use them from set expressions and statements too.

Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agotests: add stateful object update operation test
Fernando Fernandez Mancera [Sat, 2 Nov 2019 21:00:25 +0000 (22:00 +0100)] 
tests: add stateful object update operation test

Signed-off-by: Fernando Fernandez Mancera <ffmancera@riseup.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
5 years agocache: Reduce caching for get command
Phil Sutter [Thu, 14 Nov 2019 14:45:07 +0000 (15:45 +0100)] 
cache: Reduce caching for get command

Introduce a function to distinguish which command object was given and
request only the necessary bits to have sets and their elements
available for 'get element' command.

Signed-off-by: Phil Sutter <phil@nwl.cc>
Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>