Eugene Crosser [Thu, 9 Dec 2021 18:26:06 +0000 (19:26 +0100)]
netlink: Use abort() in case of netlink_abi_error
Library functions should not use exit(), application that uses the
library may contain error handling path, that cannot be executed if
library functions calls exit(). For truly fatal errors, using abort() is
more acceptable than exit().
Signed-off-by: Eugene Crosser <crosser@average.org> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Jeremy Sowden [Sat, 15 Jan 2022 18:27:06 +0000 (18:27 +0000)]
src: add a helper that returns a payload dependency for a particular base
Currently, with only one base and dependency stored this is superfluous,
but it will become more useful when the next commit adds support for
storing a payload for every base.
Remove redundant `ctx->pbase` check.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net> Signed-off-by: Florian Westphal <fw@strlen.de>
Florian Westphal [Sat, 15 Jan 2022 19:00:49 +0000 (20:00 +0100)]
src: silence compiler warnings
cache.c:504:22: warning: ‘chain’ may be used uninitialized in this function [-Wmaybe-uninitialized]
cache.c:504:22: warning: ‘table’ may be used uninitialized in this function [-Wmaybe-uninitialized]
erec.c:128:16: warning: ‘line’ may be used uninitialized in this function [-Wmaybe-uninitialized]
optimize.c:524:9: warning: ‘line’ may be used uninitialized in this function [-Wmaybe-uninitialized]
Fixes: 8ad4056e9182 ("erec: expose print_location() and line_location()") Fixes: afbd102211dc ("src: do not use the nft_cache_filter object from mnl.c") Fixes: fb298877ece2 ("src: add ruleset optimization infrastructure") Signed-off-by: Florian Westphal <fw@strlen.de>
src: 'nft list chain' prints anonymous chains correctly
If the user is requesting a chain listing, e.g. nft list chain x y
and a rule refers to an anonymous chain that cannot be found in the cache,
then fetch such anonymous chain and its ruleset.
Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1577 Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
optimize: merge rules with same selectors into a concatenation
This patch extends the ruleset optimization infrastructure to collapse
several rules with the same selectors into a concatenation.
Transform:
meta iifname eth1 ip saddr 1.1.1.1 ip daddr 2.2.2.3 accept
meta iifname eth1 ip saddr 1.1.1.2 ip daddr 2.2.2.5 accept
meta iifname eth2 ip saddr 1.1.1.3 ip daddr 2.2.2.6 accept
into:
meta iifname . ip saddr . ip daddr { eth1 . 1.1.1.1 . 2.2.2.6, eth1 . 1.1.1.2 . 2.2.2.5 , eth1 . 1.1.1.3 . 2.2.2.6 } accept
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
This patch adds a new -o/--optimize option to enable ruleset
optimization.
You can combine this option with the dry run mode (--check) to review
the proposed ruleset updates without actually loading the ruleset, e.g.
# nft -c -o -f ruleset.test
Merging:
ruleset.nft:16:3-37: ip daddr 192.168.0.1 counter accept
ruleset.nft:17:3-37: ip daddr 192.168.0.2 counter accept
ruleset.nft:18:3-37: ip daddr 192.168.0.3 counter accept
into:
ip daddr { 192.168.0.1, 192.168.0.2, 192.168.0.3 } counter packets 0 bytes 0 accept
This infrastructure collects the common statements that are used in
rules, then it builds a matrix of rules vs. statements. Then, it looks
for common statements in consecutive rules which allows to merge rules.
This ruleset optimization always performs an implicit dry run to
validate that the original ruleset is correct. Then, on a second pass,
it performs the ruleset optimization and add the rules into the kernel
(unless --check has been specified by the user).
From libnftables perspective, there is a new API to enable
this feature:
This patch adds support for the first optimization: Collapse a linear
list of rules matching on a single selector into a set as exposed in the
example above.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
The error reporting infrastructure performs a fseek() on the file
descriptor which does not work in this case since the data from the
descriptor has been already consumed.
This patch adds a new stdin input descriptor to perform this special
handling which consists on re-routing this request through the buffer
functions.
Fixes: 935f82e7dd49 ("Support 'nft -f -' to read from stdin") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
There are several places where we check whether `ctx->pdctx.pbase`
equal to `PROTO_BASE_INVALID` and don't bother trying to free the
dependency if so. However, these checks are redundant.
In `payload_match_expand` and `trace_gen_stmts`, we skip a call to
`payload_dependency_kill`, but that calls `payload_dependency_exists` to check a
dependency exists before doing anything else.
In `ct_meta_common_postprocess`, we skip an open-coded equivalent to
`payload_dependency_kill` which performs some different checks, but the
first is the same: a call to `payload_dependency_exists`.
Therefore, we can drop the redundant checks and simplify the flow-
control in the functions.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net> Signed-off-by: Florian Westphal <fw@strlen.de>
Jeremy Sowden [Tue, 21 Dec 2021 19:36:49 +0000 (19:36 +0000)]
tests: py: fix inet/ip_tcp.t test
Contrary to the comment and expected output, nft does _not_ eliminate
the redundant `ip protocol` expression from the second test. Dependency
elimination requires a higher level expression. `ip saddr` cannot lead
to the elimination of `ip protocol` since they are both L3 expressions.
`tcp dport` cannot because although `ip saddr` and `ip protocol` both
imply that the L3 protocol is `ip`, only protocol matches are stored as
dependencies, so the redundancy is not apparent, and in fact,
`payload_may_dependency_kill` explicitly checks for the combination of
inet, bridge or netdev family, L4 expression and L3 ipv4 or ipv6
dependency and returns false.
Correct the expected output and comment.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net> Signed-off-by: Florian Westphal <fw@strlen.de>
Florian Westphal [Tue, 11 Jan 2022 11:08:59 +0000 (12:08 +0100)]
evaluate: attempt to set_eval flag if dynamic updates requested
When passing no upper size limit, the dynset expression forces
an internal 64k upperlimit.
In some cases, this can result in 'nft -f' to restore the ruleset.
Avoid this by always setting the EVAL flag on a set definition when
we encounter packet-path update attempt in the batch.
Reported-by: Yi Chen <yiche@redhat.com> Suggested-by: Pablo Neira Ayuso <pablo@netfilter.org> Signed-off-by: Florian Westphal <fw@strlen.de>
Jeremy Sowden [Thu, 16 Dec 2021 16:37:20 +0000 (16:37 +0000)]
build: remove scanner.c and parser_bison.c with `maintainer-clean`
automake recommends shipping the output of bison and lex in distribution
tar-balls and runs bison and lex during `make dist` (this has the
advantage that end-users don't need to have bison or lex installed to
compile the software). Accordingly, automake also recommends removing
these files with `make maintainer-clean` and generates rules to do so.
Therefore, remove scanner.c and parser_bison.c from `CLEANFILES`.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Jeremy Sowden [Wed, 15 Dec 2021 18:44:40 +0000 (18:44 +0000)]
build: fix autoconf warnings
autoconf complains about three obsolete macros.
`AC_CONFIG_HEADER` has been superseded by `AC_CONFIG_HEADERS`, so
replace it.
`AM_PROG_LEX` calls `AC_PROG_LEX` with no arguments, but this usage is
deprecated. The only difference between `AM_PROG_LEX` and `AC_PROG_LEX`
is that the former defines `$LEX` as "./build-aux/missing lex" if no lex
is found to ensure a useful error is reported when make is run. How-
ever, the configure script checks that we have a working lex and exits
with an error if none is available, so `$LEX` will never be called and
we can replace `AM_PROG_LEX` with `AC_PROG_LEX`.
`AM_PROG_LIBTOOL` has been superseded by `LT_INIT`, which is already in
configure.ac, so remove it.
We can also replace `AC_DISABLE_STATIC` with an argument to `LT_INIT`.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Jeremy Sowden [Sat, 11 Dec 2021 18:55:25 +0000 (18:55 +0000)]
evaluate: reject: support ethernet as L2 protocol for inet table
When we are evaluating a `reject` statement in the `inet` family, we may
have `ether` and `ip` or `ip6` as the L2 and L3 protocols in the
evaluation context:
The reason it fails is that the ethernet protocol numbers for IPv4 and
IPv6 (`ETH_P_IP` and `ETH_P_IPV6`) do not match `NFPROTO_IPV4` and
`NFPROTO_IPV6`. Add support for the ethernet protocol numbers.
Replace the current `BUG("unsupported family")` error message with
something more informative that tells the user to provide an explicit
reject option.
Jeremy Sowden [Sat, 11 Dec 2021 18:55:23 +0000 (18:55 +0000)]
proto: short-circuit loops over upper protocols
Each `struct proto_desc` contains a fixed-size array of higher layer
protocols. Only the first few are not NULL. Therefore, we can stop
iterating over the array once we reach a NULL member.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
First binop masks out unwanted parts of the 16-bit field.
Second binop needs to left-shift so that lookups in the set will work.
When decoding, the first binop is removed after the exthdr load
has been adjusted accordingly. Constant propagation adjusts the
shift-value to 0 on removal. This change then gets rid of the
shift-by-0 entirely.
After this change, 'frag frag-off @s4' input is shown as-is.
Štěpán Němec [Wed, 1 Dec 2021 11:12:00 +0000 (12:12 +0100)]
tests: shell: better parameters for the interval stack overflow test
Wider testing has shown that 128 kB stack is too low (e.g. for systems
with 64 kB page size), leading to false failures in some environments.
Based on results from a matrix of RHEL 8 and RHEL 9 systems across
x86_64, aarch64, ppc64le and s390x architectures as well as some
anecdotal testing of other Linux distros on x86_64 machines, 400 kB
seems safe: the normal nft stack (which should stay constant during
this test) on all tested systems doesn't exceed 200 kB (stays around
100 kB on typical systems with 4 kB page size), while always growing
beyond 500 kB in the failing case (nftables before baecd1cf2685) with
the increased set size.
Fixes: d8ccad2a2b73 ("tests: cover baecd1cf2685 ("segtree: Fix segfault when restoring a huge interval set")") Signed-off-by: Štěpán Němec <snemec@redhat.com> Signed-off-by: Phil Sutter <phil@nwl.cc>
Its always 0, so remove it.
Looks like this was intended to support variable options that have
array-like members, but so far this isn't implemented, better remove
dead code and implement it properly when such support is needed.
Phil Sutter [Tue, 30 Nov 2021 15:57:54 +0000 (16:57 +0100)]
cache: Filter set list on server side
Fetch either all tables' sets at once, a specific table's sets or even a
specific set if needed instead of iterating over the list of previously
fetched tables and fetching for each, then ignoring anything returned
that doesn't match the filter.
Phil Sutter [Mon, 29 Nov 2021 15:26:44 +0000 (16:26 +0100)]
cache: Filter chain list on kernel side
When operating on a specific chain, add payload to NFT_MSG_GETCHAIN so
kernel returns only relevant data. Since ENOENT is an expected return
code, do not treat this as error.
While being at it, improve code in chain_cache_cb() a bit:
- Check chain's family first, it is a less expensive check than
comparing table names.
- Do not extract chain name of uninteresting chains.
Phil Sutter [Mon, 29 Nov 2021 14:36:45 +0000 (15:36 +0100)]
cache: Filter rule list on kernel side
Instead of fetching all existing rules in kernel's ruleset and filtering
in user space, add payload to the dump request specifying the table and
chain to filter for.
Since list_rule_cb() no longer needs the filter, pass only netlink_ctx
to the callback and drop struct rule_cache_dump_ctx.
Phil Sutter [Mon, 29 Nov 2021 14:28:33 +0000 (15:28 +0100)]
cache: Filter tables on kernel side
Instead of requesting a dump of all tables and filtering the data in
user space, construct a non-dump request if filter contains a table so
kernel returns only that single table.
This should improve nft performance in rulesets with many tables
present.
Florian Westphal [Sun, 21 Nov 2021 22:33:19 +0000 (23:33 +0100)]
exthdr: fix tcpopt_find_template to use length after mask adjustment
Unify binop handling for ipv6 extension header, ip option and tcp option
processing.
Pass the real offset and length expected, not the one used in the kernel.
This was already done for extension headers and ip options, but tcp
option parsing did not do this.
This was fine before because no existing tcp option template
had a non-byte sized member.
With mptcp addition this isn't the case anymore, subtype field is
only 4 bits wide, but tcp option delinearization passed 8bits instead.
Pass the offset and mask delta, just like ip option/ipv6 exthdr.
This makes nft show 'tcp option mptcp subtype 1' instead of
'tcp option mptcp unknown & 240 == 16'.
Florian Westphal [Sun, 21 Nov 2021 22:33:05 +0000 (23:33 +0100)]
scanner: add tcp flex scope
This moves tcp options not used anywhere else (e.g. in synproxy) to a
distinct scope. This will also allow to avoid exposing new option
keywords in the ruleset context.
Phil Sutter [Wed, 10 Mar 2021 18:46:08 +0000 (19:46 +0100)]
netlink_delinearize: Fix for escaped asterisk strings on Big Endian
The original nul-char detection was not functional on Big Endian.
Instead, go a simpler route by exporting the string and working on the
exported data to check for a nul-char and escape a trailing asterisk if
present. With the data export already happening in the caller, fold
escaped_string_wildcard_expr_alloc() into it as well.
Phil Sutter [Wed, 10 Mar 2021 13:38:37 +0000 (14:38 +0100)]
datatype: Fix size of time_type
Used by 'ct expiration', time_type is supposed to be 32bits. Passing a
64bits variable to constant_expr_alloc() causes the value to be always
zero on Big Endian.
Fixes: 0974fa84f162a ("datatype: seperate time parsing/printing from time_type") Signed-off-by: Phil Sutter <phil@nwl.cc>
Phil Sutter [Wed, 10 Mar 2021 10:45:47 +0000 (11:45 +0100)]
meta: Fix hour_type size
In kernel as well as when parsing, hour_type is assumed to be 32bits.
Having the struct datatype field set to 64bits breaks Big Endian and so
does passing a 64bit value and 32 as length to constant_expr_alloc() as
it makes it import the upper 32bits. Fix this by turning 'result' into a
uint32_t and introduce a temporary uint64_t just for the call to
time_parse() which expects that.
Fixes: f8f32deda31df ("meta: Introduce new conditions 'time', 'day' and 'hour'") Signed-off-by: Phil Sutter <phil@nwl.cc>
Phil Sutter [Tue, 9 Mar 2021 20:24:30 +0000 (21:24 +0100)]
meta: Fix {g,u}id_type on Big Endian
Using a 64bit variable to temporarily hold the parsed value works only
on Little Endian. uid_t and gid_t (and therefore also pw->pw_uid and
gr->gr_gid) are 32bit.
To fix this, use uid_t/gid_t for the temporary variable but keep the
64bit one for numeric parsing so values exceeding 32bits are still
detected.
Fixes: e0ed4c45d9ad2 ("meta: relax restriction on UID/GID parsing") Signed-off-by: Phil Sutter <phil@nwl.cc>
Phil Sutter [Thu, 17 Dec 2020 17:19:18 +0000 (18:19 +0100)]
src: Fix payload statement mask on Big Endian
The mask used to select bits to keep must be exported in the same
byteorder as the payload statement itself, also the length of the
exported data must match the number of bytes extracted earlier.
Phil Sutter [Thu, 17 Dec 2020 14:52:03 +0000 (15:52 +0100)]
mnl: Fix for missing info in rule dumps
Commit 0e52cab1e64ab improved error reporting by adding rule's table and
chain names to netlink message directly, prefixed by their location
info. This in turn caused netlink dumps of the rule to not contain table
and chain name anymore. Fix this by inserting the missing info before
dumping and remove it afterwards to not cause duplicated entries in
netlink message.
Phil Sutter [Wed, 17 Mar 2021 19:39:38 +0000 (20:39 +0100)]
exthdr: Fix for segfault with unknown exthdr
Unknown exthdr type with NFT_EXTHDR_F_PRESENT flag set caused
NULL-pointer deref. Fix this by moving the conditional exthdr.desc deref
atop the function and use the result in all cases.
Fixes: e02bd59c4009b ("exthdr: Implement existence check") Signed-off-by: Phil Sutter <phil@nwl.cc>
Phil Sutter [Thu, 4 Feb 2021 14:58:25 +0000 (15:58 +0100)]
tests/py: Avoid duplicate records in *.got files
If payloads don't contain family-specific bits, they may sit in a single
*.payload file for all tested families. In such case, nft-test.py will
consequently write dissenting payloads into a single *.got file. To
avoid the duplicate entries, check if a matching record exists already
before writing it out.
This header is not required to compile nftables with editline, remove
it, this unbreak compilation in several distros which have no symlink
from history.h to editline.h
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
mnl.c: In function ‘mnl_batch_talk’:
mnl.c:417:17: warning: comparison of integer expressions of different signedness: ‘unsigned in’ and ‘long int’ [-Wsign-compare]
if (rcvbufsiz < NFT_MNL_ECHO_RCVBUFF_DEFAULT)
^
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Florian Westphal [Wed, 17 Nov 2021 13:26:21 +0000 (14:26 +0100)]
monitor: do not call interval_map_decompose() for concat intervals
Without this, nft monitor will either print garbage or even segfault
when encountering a concat set because we pass expr->value to libgmp
helpers for concat (non-value) expressions.
Also, for concat case, we need to call concat_range_aggregate() helper.
Add a test case for this. Without this patch, it gives:
Fixes: 50780456a01a ("evaluate: check for missing transport protocol match in nat map with concatenations") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Check family when filtering out listing of tables and sets.
Fixes: 3f1d3912c3a6 ("cache: filter out tables that are not requested") Fixes: 635ee1cad8aa ("cache: filter out sets and maps that are not requested") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Florian Westphal [Thu, 28 Oct 2021 15:36:06 +0000 (17:36 +0200)]
doc: update ct timeout section with the state names
docs are too terse and did not have the list of valid timeout states.
While at it, adjust default stream timeout of udp to 120, this is the
current kernel default.
evaluate: clone variable expression if there is more than one reference
Clone the expression that defines the variable value if there are
multiple references to it in the ruleset. This saves heap memory
consumption in case the variable defines a set with a huge number of
elements.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Do not call alloc_setelem_cache() to build the set element list in
nftnl_set. Instead, translate one single set element expression to
nftnl_set_elem object at a time and use this object to build the netlink
header.
Using a huge test set containing 1.1 million element blocklist, this
patch is reducing userspace memory consumption by 40%.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
tests: py: remove verdict from closing end interval
Kernel does not allow for NFT_SET_ELEM_INTERVAL_END flag and
NFTA_SET_ELEM_DATA. The closing end interval represents a mismatch,
therefore, no verdict can be applied. The existing payload files show
the drop verdict when this is unset (because NF_DROP=0).
This update is required to fix payload warnings in tests/py after
libnftnl's ("set: use NFTNL_SET_ELEM_VERDICT to print verdict").
Fixes: 6671d9d137f6 ("mnl: Set NFTNL_SET_DATA_TYPE before dumping set elements") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Štěpán Němec [Fri, 5 Nov 2021 11:39:11 +0000 (12:39 +0100)]
tests: shell: $NFT needs to be invoked unquoted
The variable has to undergo word splitting, otherwise the shell tries
to find the variable value as an executable, which breaks in cases that 7c8a44b25c22 ("tests: shell: Allow wrappers to be passed as nft command")
intends to support.
Mention this in the shell tests README.
Fixes: d8ccad2a2b73 ("tests: cover baecd1cf2685 ("segtree: Fix segfault when restoring a huge interval set")") Signed-off-by: Štěpán Němec <snemec@redhat.com> Signed-off-by: Phil Sutter <phil@nwl.cc>
Štěpán Němec [Fri, 5 Nov 2021 11:39:10 +0000 (12:39 +0100)]
tests: shell: README: clarify test file name convention
Since commit 4d26b6dd3c4c, test file name suffix no longer reflects
expected exit code in all cases.
Move the sentence "Since they are located with `find', test files can
be put in any subdirectory." to a separate paragraph.
Fixes: 4d26b6dd3c4c ("tests: shell: change all test scripts to return 0") Signed-off-by: Štěpán Němec <snemec@redhat.com> Signed-off-by: Phil Sutter <phil@nwl.cc>
projects / thirdparty / nftables.git / log
