Stefano Brivio [Wed, 19 Aug 2020 22:00:18 +0000 (00:00 +0200)]
tests: sets: Check rbtree overlap detection after tree rotations
Ticket https://bugzilla.netfilter.org/show_bug.cgi?id=1449 showed
an issue with rbtree overlap detection coming from the fact that,
after tree rotations performed as part of tree rebalancing, caused
by deletions, end elements are not necessarily descendants of their
corresponding start elements.
Add single-sized elements, delete every second one of them, and
re-add them (they will always be full overlaps) in order to check
overlap detection after tree rotations.
Port indices used in the sets are pseudo-random numbers generated
with Marsaglia's Xorshift algorithm with triplet (5, 3, 1), chosen
for k-distribution over 16-bit periods, which gives a good
statistical randomness and forces 201 rebalancing operations out of
250 deletions with the chosen seed (1).
Signed-off-by: Stefano Brivio <sbrivio@redhat.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Since 94a945ffa81b ("libnftables: Get rid of explicit cache flushes"),
the cache logic checks for the generation number to refresh the cache.
This breaks interactive mode when listing stateful objects though. This
patch adds a new flag to force a cache refresh when the user requests a
ruleset listing.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
mpz_init_bitmask() overrides the existing memory area:
==19179== 8 bytes in 1 blocks are definitely lost in loss record 1 of 1
==19179== at 0x483577F: malloc (vg_replace_malloc.c:299)
==19179== by 0x489C718: xmalloc (utils.c:36)
==19179== by 0x4B825C5: __gmpz_init2 (in /usr/lib/x86_64-linux-g nu/libgmp.so.10.3.2) f
==19179== by 0x4880239: constant_expr_alloc (expression.c:400)
==19179== by 0x489B8A1: interval_map_decompose (segtree.c:1098)
==19179== by 0x489017D: netlink_list_setelems (netlink.c:1220)
==19179== by 0x48779AC: cache_init_objects (rule.c:170) 5
==19179== by 0x48779AC: cache_init (rule.c:228)
==19179== by 0x48779AC: cache_update (rule.c:279)
==19179== by 0x48A21AE: nft_evaluate (libnftables.c:406)
left-hand side of the interval is leaked when building the range:
==25835== 368 (128 direct, 240 indirect) bytes in 1 blocks are definitely lost in loss record 5 of 5
==25835== at 0x483577F: malloc (vg_replace_malloc.c:299)
==25835== by 0x489B628: xmalloc (utils.c:36)
==25835== by 0x489B6F8: xzalloc (utils.c:65)
==25835== by 0x487E176: expr_alloc (expression.c:45)
==25835== by 0x487F960: mapping_expr_alloc (expression.c:1149)
==25835== by 0x488EC84: netlink_delinearize_setelem (netlink.c:1166)
==25835== by 0x4DC6928: nftnl_set_elem_foreach (set_elem.c:725)
==25835== by 0x488F0D5: netlink_list_setelems (netlink.c:1215)
==25835== by 0x487695C: cache_init_objects (rule.c:170)
==25835== by 0x487695C: cache_init (rule.c:228)
==25835== by 0x487695C: cache_update (rule.c:279)
==25835== by 0x48A10BE: nft_evaluate (libnftables.c:406)
==25835== by 0x48A19B6: nft_run_cmd_from_buffer (libnftables.c:451)
==25835== by 0x10A8E1: main (main.c:487)
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Stefano Brivio [Mon, 3 Aug 2020 14:06:21 +0000 (16:06 +0200)]
tests: 0044interval_overlap_0: Repeat insertion tests with timeout
Mike Dillinger reported issues with insertion of entries into sets
supporting intervals that were denied because of false conflicts with
elements that were already expired. Partial failures would occur to,
leading to the generation of new intervals the user didn't specify,
as only the opening or the closing elements wouldn't be inserted.
The reproducer provided by Mike looks like this:
#!/bin/bash
nft list set ip filter blacklist4-ip-1m
for ((i=1;i<=10;i++)); do
nft add element filter blacklist4-ip-1m {$i.$i.$i.$i}
sleep 1
done
nft list set ip filter blacklist4-ip-1m
which, run in a loop at different intervals, show the different kind
of failures.
Extend the existing test case for overlapping and non-overlapping
intervals to systematically cover sets with a configured timeout.
As reported by Pablo, the test would fail if we keep a one-second
timeout if it runs on a "slow" kernel (e.g. with KASan), using the
libtool wrapper in src/nft as $NFT, because we can't issue 218
commands within one second. To avoid that, introduce an adaptive
timeout based on how many times we can list a single entry with a
fixed one-second timeout.
On a single 2.9GHz AMD Epyc 7351 thread:
test run nft commands/s timeout
- src/nft libtool wrapper, KASan: 68.4s 10 32s
- nft binary, KASan: 5.1s 168 2s
- src/nft libtool wrapper, w/o KASan: 18.3s 37 8s
- nft binary, w/o KASan: 2.4s 719 1s
While at it, fix expectation for insertion of '15-20 . 50-60' (it's
expected to succeed, given the list), and the reason why I didn't
notice: a simple command preceded by ! won't actually result in
the shell exiting, even if it fails. Add some clearer failure reports
too.
v2:
- adjust set timeouts to nft commands/s
- fix checks on expected outcome of insertions and reports
Reported-by: Mike Dillinger <miked@softtalker.com> Signed-off-by: Stefano Brivio <sbrivio@redhat.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Stefano Brivio [Mon, 3 Aug 2020 14:06:39 +0000 (16:06 +0200)]
tests: 0043concatenated_ranges_0: Fix checks for add/delete failures
The test won't stop if we simply precede commands expected to fail
by !. POSIX.1-2017 says:
-e
When this option is on, if a simple command fails for any of
the reasons listed in Consequences of Shell Errors or returns
an exit status value >0, and is not part of the compound list
following a while, until or if keyword, and is not a part of
an AND or OR list, and is not a pipeline preceded by the "!"
reserved word, then the shell will immediately exit.
...but I didn't care about the last part.
Replace those '! nft ...' commands by 'nft ... && exit 1' to actually
detect failures.
As a result, I didn't notice that now, correctly, inserting elements
into a set that contains the same exact element doesn't actually
fail, because nft doesn't pass NLM_F_EXCL on a simple 'add'. Drop
re-insertions from the checks we perform here, overlapping elements
are already covered by other tests.
Fixes: 618393c6b3f2 ("tests: Introduce test for set with concatenated ranges") Signed-off-by: Stefano Brivio <sbrivio@redhat.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Fixes: https://bugzilla.netfilter.org/show_bug.cgi?id=1336 Signed-off-by: Jose M. Guisado Gomez <guigom@riseup.net> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
If the table contains anonymous sets, such as __set0, then delete + add
table might result in nft reusing the existing stale __set0 in the cache.
The problem is that nft gets confused and it reuses the existing stale
__set0 instead of the new anonymous set __set0 with the same name.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
src: remove cache lookups after the evaluation phase
This patch adds a new field to the cmd structure for elements to store a
reference to the set. This saves an extra lookup in the netlink bytecode
generation step.
This patch also allows to incrementally update during the evaluation
phase according to the command actions, which is required by the follow
up ("evaluate: remove table from cache on delete table") bugfix patch.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
evaluate: flush set cache from the evaluation phase
This patch reworks 40ef308e19b6 ("rule: flush set cache before flush
command"). This patch flushes the set cache earlier, from the command
evaluation step.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
nft: rearrange help output to group related options together
It has been reported that nft options are a bit chaotic. With a growing list of options for the nft
CLI, we can do better when presenting them to the user who requests help.
This patch introduces a textual output grouping for options, in 4 groups:
* Options (general) -- common Unix utility options
* Options (operative) -- the options that modify the operative behaviour of nft
* Options (translation) -- output text modifiers for data translation
* Options (parsing) -- output text modifiers for parsing and other operations
There is no behavior change in this patch, is mostly a cosmetic change in the hope that users will
find the nft tool a bit less confusing to use.
Options (general):
-h, help Show this help
-v, version Show version information
-V Show extended version information
Options (ruleset input handling):
-f, file <filename> Read input from <filename>
-i, interactive Read input from interactive CLI
-I, includepath <directory> Add <directory> to the paths searched for include files. Defaul[..]
-c, check Check commands validity without actually applying the changes.
Options (ruleset list formatting):
-a, handle Output rule handle.
-s, stateless Omit stateful information of ruleset.
-t, terse Omit contents of sets.
-S, service Translate ports to service names as described in /etc/services.
-N, reversedns Translate IP addresses to names.
-u, guid Print UID/GID as defined in /etc/passwd and /etc/group.
-n, numeric Print fully numerical output.
-y, numeric-priority Print chain priority numerically.
-p, numeric-protocol Print layer 4 protocols numerically.
-T, numeric-time Print time values numerically.
Options (command output format):
-e, echo Echo what has been added, inserted or replaced.
-j, json Format output in JSON
-d, debug <level [,level...]> Specify debugging level (scanner, parser, eval, netlink, mnl, p[..]
=== 8< ===
While at it, refresh the man page to better reflex this new grouping, and add some missing options.
Joint work with Pablo.
Signed-off-by: Arturo Borrero Gonzalez <arturo@netfilter.org> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Phil Sutter [Wed, 29 Jul 2020 12:21:12 +0000 (14:21 +0200)]
json: Expect refcount increment by json_array_extend()
This function is apparently not "joining" two arrays but rather copying
all items from the second array to the first, leaving the original
reference in place. Therefore it naturally increments refcounts, which
means if used to join two arrays caller must explicitly decrement the
second array's refcount.
evaluate: replace variable expression by the value expression
The variable expression provides the binding between the variable
dereference and the value expression. Replace the variable expression by
the real value expression after the evaluation.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Its possible to add an element to a map, but you can't read it back:
before:
nft add element inet filter test "{ 18.51.100.17 . ad:c1:ac:c0:ce:c0 . 3761 : 0x42 }"
nft get element inet filter test "{ 18.51.100.17 . ad:c1:ac:c0:ce:c0 . 3761 : 0x42 }"
Error: No such file or directory; did you mean map ‘test’ in table inet ‘filter’?
get element inet filter test { 18.51.100.17 . ad:c1:ac:c0:ce:c0 . 3761 : 0x42 }
^^^^
after:
nft get element inet filter test "{ 18.51.100.17 . ad:c1:ac:c0:ce:c0 . 3761 : 0x42 }"
table inet filter {
map test {
type ipv4_addr . ether_addr . inet_service : mark
flags interval,timeout
elements = { 18.51.100.17 . ad:c1:ac:c0:ce:c0 . 3761 : 0x00000042 }
}
}
This patch allows you to group rules in a subchain, e.g.
table inet x {
chain y {
type filter hook input priority 0;
tcp dport 22 jump {
ip saddr { 127.0.0.0/8, 172.23.0.0/16, 192.168.13.0/24 } accept
ip6 saddr ::1/128 accept;
}
}
}
This also supports for the `goto' chain verdict.
This patch adds a new chain binding list to avoid a chain list lookup from the
delinearize path for the usual chains. This can be simplified later on with a
single hashtable per table for all chains.
From the shell, you have to use the explicit separator ';', in bash you
have to escape this:
# nft add rule inet x y tcp dport 80 jump { ip saddr 127.0.0.1 accept\; ip6 saddr ::1 accept \; }
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
table x {
chain y {
ct state invalid log prefix "invalid $test $foo:"
}
}
This patch scans for variables in the log prefix string. The log prefix
expression is a list of constant and variable expression that are
converted into a constant expression from the evaluation phase.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
table inet filter {
chain input {
type filter hook input priority 0; policy drop;
tcp dport {$BASE_ALLOWED_INCOMING_TCP_PORTS, $EXTRA_ALLOWED_INCOMING_TCP_PORTS} ct state new counter accept
}
}
However, disallow this:
define EXTRA_ALLOWED_INCOMING_TCP_PORTS = {}
table inet filter {
chain input {
type filter hook input priority 0; policy drop;
tcp dport {$EXTRA_ALLOWED_INCOMING_TCP_PORTS} ct state new counter accept
}
}
# nft -f x.nft
/tmp/x.nft:6:18-52: Error: Set is empty
tcp dport {$EXTRA_ALLOWED_INCOMING_TCP_PORTS} ct state new counter accept
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Stefano Brivio [Sun, 14 Jun 2020 21:41:37 +0000 (23:41 +0200)]
tests: shell: Allow wrappers to be passed as nft command
The current check on $NFT only allows to directly pass an executable,
so I've been commenting it out locally for a while to run tests with
valgrind.
Instead of using the -x test, run nft without arguments and check the
exit status. POSIX.1-2017, Shell and Utilities volume, par. 2.8.2
("Exit Status for Commands") states:
If a command is not found, the exit status shall be 127. If the
command name is found, but it is not an executable utility, the
exit status shall be 126. Applications that invoke utilities
without using the shell should use these exit status values to
report similar errors.
While this script isn't POSIX-compliant, it requires bash, and any
modern version of bash complies with those exit status requirements.
Also valgrind complies with this.
We need to quote the NFT variable passed to execute the commands in
the main loop and adjust error and informational messages, too.
This way, for example, export NFT="valgrind nft" can be issued to
run tests with valgrind.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Stefano Brivio [Sun, 14 Jun 2020 21:41:57 +0000 (23:41 +0200)]
tests: Run in separate network namespace, don't break connectivity
It might be convenient to run tests from a development branch that
resides on another host, and if we break connectivity on the test
host as tests are executed, we can't run them this way.
If kernel implementation (CONFIG_NET_NS), unshare(1), or Python
bindings for unshare() are not available, warn and continue.
Suggested-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Stefano Brivio <sbrivio@redhat.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
netlink: release dummy rule object from netlink_parse_set_expr()
netlink_parse_set_expr() creates a dummy rule object to reuse the
existing netlink parser. Release the rule object to fix a memleak.
Zap the statement list to avoid a use-after-free since the statement
needs to remain in place after releasing the rule.
Direct leak of 2016 byte(s) in 4 object(s) allocated from:
#0 0x7f7824b26330 in __interceptor_malloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xe9330)
#1 0x7f78245fcebd in xmalloc /home/pablo/devel/scm/git-netfilter/nftables/src/utils.c:36
#2 0x7f78245fd016 in xzalloc /home/pablo/devel/scm/git-netfilter/nftables/src/utils.c:65
#3 0x7f782456f0b5 in rule_alloc /home/pablo/devel/scm/git-netfilter/nftables/src/rule.c:623
Add a test to check for set counters.
SUMMARY: AddressSanitizer: 2016 byte(s) leaked in 4 allocation(s).
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Stefano Brivio [Wed, 27 May 2020 20:51:22 +0000 (22:51 +0200)]
tests: py: Enable anonymous set rule with concatenated ranges in inet/sets.t
Commit 64b9aa3803dd ("tests/py: Add tests involving concatenated
ranges") introduced a rule, commented out, adding an anonymous set
including concatenated ranges. Now that they are properly handled,
we can enable it.
Note that this introduces a new warning. In the output below, '\'
marks newlines I introduced to keep lines short:
which is similar to the existing warning, also introduced by
commit 64b9aa3803dd:
inet/sets.t: WARNING: line 23: \
'add rule inet test-inet input \
ip saddr . ip daddr . tcp dport @set3 accept': \
'ip saddr . ip daddr . tcp dport @set3 accept' mismatches \
'meta nfproto ipv4 ip saddr . ip daddr . tcp dport @set3 accept'
This is mentioned in the commit message for 64b9aa3803dd itself:
* Payload dependency killing ignores the concatenated IP header
expressions on LHS, so rule output is asymmetric.
which means that for family inet, 'meta nfproto ipv4' is added to
the output of the rule, on top of what was passed as input, but not
for families bridge and netdev.
For this reason, it's not possible in this case to specify a single
expected output, differing from the input, and, also,
'meta nfproto ipv4' can only be passed as input for family inet as
it's not relevant for the other families.
As an alternative, we could split the rules from this test into
tests for the corresponding families, as this test case itself
is under the 'inet' directory, but I consider this beyond the scope
of this patchset.
v2: Enable rule in py/inet/sets.t instead of adding a new test in
shell/sets (Phil Sutter)
Stefano Brivio [Wed, 27 May 2020 20:51:21 +0000 (22:51 +0200)]
evaluate: Perform set evaluation on implicitly declared (anonymous) sets
If a set is implicitly declared, set_evaluate() is not called as a
result of cmd_evaluate_add(), because we're adding in fact something
else (e.g. a rule). Expression-wise, evaluation still happens as the
implicit set expression is eventually found in the tree and handled
by expr_evaluate_set(), but context-wise evaluation (set_evaluate())
is skipped, and this might be relevant instead.
This is visible in the reported case of an anonymous set including
concatenated ranges:
# nft add rule t c ip saddr . tcp dport { 192.0.2.1 . 20-30 } accept
BUG: invalid range expression type concat
nft: expression.c:1160: range_expr_value_low: Assertion `0' failed.
Aborted
because we reach do_add_set() without properly evaluated flags and
set description, and eventually end up in expr_to_intervals(), which
can't handle that expression.
Explicitly call set_evaluate() as we add anonymous sets into the
context, and instruct the same function to:
- skip expression-wise set evaluation if the set is anonymous, as
that happens later anyway as part of the general tree evaluation
- skip the insertion in the set cache, as it makes no sense to have
sets that shouldn't be referenced there
For object maps, the allocation of the expression for set->data is
already handled by set_evaluate(), so we can now drop that from
stmt_evaluate_objref_map().
v2:
- skip insertion of set in cache (Pablo Neira Ayuso)
- drop double allocation of expression (and leak of the first
one) for object maps (Pablo Neira Ayuso)
Reported-by: Pablo Neira Ayuso <pablo@netfilter.org> Reported-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Stefano Brivio <sbrivio@redhat.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Michael Braun [Wed, 6 May 2020 09:46:23 +0000 (11:46 +0200)]
datatype: add frag-needed (ipv4) to reject options
This enables to send icmp frag-needed messages using reject target.
I have a bridge with connects an gretap tunnel with some ethernet lan.
On the gretap device I use ignore-df to avoid packets being lost without
icmp reject to the sender of the bridged packet.
Still I want to avoid packet fragmentation with the gretap packets.
So I though about adding an nftables rule like this:
nft insert rule bridge filter FORWARD \
ip protocol tcp \
ip length > 1400 \
ip frag-off & 0x4000 != 0 \
reject with icmp type frag-needed
This would reject all tcp packets with ip dont-fragment bit set that are
bigger than some threshold (here 1400 bytes). The sender would then receive
ICMP unreachable - fragmentation needed and reduce its packet size (as
defined with PMTU).
[ pablo: update tests/py ]
Signed-off-by: Michael Braun <michael-dev@fami-braun.de> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1429 Fixes: f9465cf517cc ("src: add STMT_NAT_F_CONCAT flag and use it") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Stefano Brivio [Sun, 24 May 2020 13:00:07 +0000 (15:00 +0200)]
tests: py: Actually use all available hooks in bridge/chains.t
Despite being explicitly mentioned as available, prerouting and
postrouting hooks are not used, filter-pre and filter-post chains
are both built to hook on input.
Stefano Brivio [Sun, 24 May 2020 12:59:36 +0000 (14:59 +0200)]
build: Fix doc build, restore A2X assignment for doc/Makefile
Commit 4f2813a313ae ("build: Include generated man pages in dist
tarball") skips AC_CHECK_PROG for A2X altogether if doc/nft.8 is
already present.
Now, starting from a clean situation, we can have this sequence:
./configure # doc/nft.8 not there, A2X set in doc/Makefile
make # builds doc/nft.8
./configure # doc/nft.8 is there, A2X left empty in doc/Makefile
make clean # removes doc/nft.8
make
resulting in:
[...]
GEN nft.8
/bin/sh: -L: command not found
make[2]: *** [Makefile:639: nft.8] Error 127
and the only way to get out of this is to issue ./configure again
after make clean, which is rather unexpected.
Instead of skipping AC_CHECK_PROG when doc/nft.8 is present, keep
it and simply avoid returning failure if a2x(1) is not available but
doc/nft.8 was built, so that A2X is properly set in doc/Makefile
whenever needed.
Fixes: 4f2813a313ae ("build: Include generated man pages in dist tarball") Signed-off-by: Stefano Brivio <sbrivio@redhat.com> Acked-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
build: fix tentative generation of nft.8 after disabled doc
Despite doc generation is disabled, the makefile is trying to build it.
$ ./configure --disable-man-doc
$ make
Making all in doc
make[2]: Entering directory '/workdir/build-pkg/workdir/doc'
make[2]: *** No rule to make target 'nft.8', needed by 'all-am'. Stop.
make[2]: Leaving directory '/workdir/build-pkg/workdir/doc'
make[1]: *** [Makefile:479: all-recursive] Error 1
make[1]: Leaving directory '/workdir/build-pkg/workdir'
make: *** [Makefile:388: all] Error 2
Fixes: 4f2813a313ae0 ("build: Include generated man pages in dist tarball") Reported-by: Adan Marin Jacquot <adan.marin@zevenet.com> Signed-off-by: Laura Garcia Liebana <nevola@gmail.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
This new command type results from expanding the set definition in two
commands: One to add the set and another to add the elements. This
results in 1:1 mapping between the command object to the netlink API.
The command is then translated into a netlink message which gets a
unique sequence number. This sequence number allows to correlate the
netlink extended error reporting with the corresponding command.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
The CMD_OBJ_ELEMENTS provides an expression that contains the list of
set elements. This leaves room to introduce CMD_OBJ_SETELEMS in a follow
up patch.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
mnl: fix error rule reporting with missing table/chain and anonymous sets
handle_merge() skips handle location initialization because set name != NULL.
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7f64f1e in erec_print (octx=0x55555555d2c0, erec=0x55555555fcf0, debug_mask=0) at erec.c:95
95 switch (indesc->type) {
(gdb) bt
buf=0x55555555db20 "add rule inet traffic-filter input tcp dport { 22, 80, 443 } accept") at libnftables.c:459
(gdb) p indesc
$1 = (const struct input_descriptor *) 0x0
Closes: http://bugzilla.opensuse.org/show_bug.cgi?id=1171321 Fixes: 086ec6f30c96 ("mnl: extended error support for create command") Reported-by: Jan Engelhardt <jengelh@inai.de> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Phil Sutter [Wed, 13 May 2020 14:29:51 +0000 (16:29 +0200)]
JSON: Improve performance of json_events_cb()
The function tries to insert handles into JSON input for echo option.
Yet there may be nothing to do if the given netlink message doesn't
contain a handle, e.g. if it is an 'add element' command. Calling
seqnum_to_json() is pointless overhead in that case, and if input is
large this overhead is significant. Better wait with that call until
after checking if the message is relevant at all.
Signed-off-by: Phil Sutter <phil@nwl.cc> Acked-by: Eric Garver <eric@garver.life>
evaluate: fix memleak in stmt_evaluate_reject_icmp()
==26297==ERROR: LeakSanitizer: detected memory leaks
c
Direct leak of 512 byte(s) in 4 object(s) allocated from:
#0 0x7f46f8167330 in __interceptor_malloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xe9330)
#1 0x7f46f7b3cf1c in xmalloc /home/pablo/devel/scm/git-netfilter/nftables/src/utils.c:36
#2 0x7f46f7b3d075 in xzalloc /home/pablo/devel/scm/git-netfilter/nftables/src/utils.c:65
#3 0x7f46f7a85760 in expr_alloc /home/pablo/devel/scm/git-netfilter/nftables/src/expression.c:45
#4 0x7f46f7a8915d in constant_expr_alloc /home/pablo/devel/scm/git-netfilter/nftables/src/expression.c:388
#5 0x7f46f7a7bad4 in symbolic_constant_parse /home/pablo/devel/scm/git-netfilter/nftables/src/datatype.c:173
#6 0x7f46f7a7af5f in symbol_parse /home/pablo/devel/scm/git-netfilter/nftables/src/datatype.c:132
#7 0x7f46f7abf2bd in stmt_evaluate_reject_icmp /home/pablo/devel/scm/git-netfilter/nftables./src/evaluate.c:2739
[...]
SUMMARY: AddressSanitizer: 544 byte(s) leaked in 8 allocation(s).
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Direct leak of 256 byte(s) in 2 object(s) allocated from:
#0 0x7f6ce2189330 in __interceptor_malloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xe9330)
#1 0x7f6ce1b1767a in xmalloc /home/pablo/devel/scm/git-netfilter/nftables/src/utils.c:36
#2 0x7f6ce1b177d3 in xzalloc /home/pablo/devel/scm/git-netfilter/nftables/src/utils.c:65
#3 0x7f6ce1a41760 in expr_alloc /home/pablo/devel/scm/git-netfilter/nftables/src/expression.c:45
#4 0x7f6ce1a4dea7 in set_elem_expr_alloc /home/pablo/devel/scm/git-netfilter/nftables/src/expression.c:1278
#5 0x7f6ce1ac2215 in netlink_delinearize_setelem /home/pablo/devel/scm/git-netfilter/nftables/src/netlink.c:1094
#6 0x7f6ce1ac3c16 in list_setelem_cb /home/pablo/devel/scm/git-netfilter/nftables/src/netlink.c:1172
#7 0x7f6ce0198808 in nftnl_set_elem_foreach /home/pablo/devel/scm/git-netfilter/libnftnl/src/set_elem.c:725
Indirect leak of 256 byte(s) in 2 object(s) allocated from:
#0 0x7f6ce2189330 in __interceptor_malloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xe9330)
#1 0x7f6ce1b1767a in xmalloc /home/pablo/devel/scm/git-netfilter/nftables/src/utils.c:36
#2 0x7f6ce1b177d3 in xzalloc /home/pablo/devel/scm/git-netfilter/nftables/src/utils.c:65
#3 0x7f6ce1a41760 in expr_alloc /home/pablo/devel/scm/git-netfilter/nftables/src/expression.c:45
#4 0x7f6ce1a4515d in constant_expr_alloc /home/pablo/devel/scm/git-netfilter/nftables/src/expression.c:388
#5 0x7f6ce1abaf12 in netlink_alloc_value /home/pablo/devel/scm/git-netfilter/nftables/src/netlink.c:354
#6 0x7f6ce1ac17f5 in netlink_delinearize_setelem /home/pablo/devel/scm/git-netfilter/nftables/src/netlink.c:1080
#7 0x7f6ce1ac3c16 in list_setelem_cb /home/pablo/devel/scm/git-netfilter/nftables/src/netlink.c:1172
#8 0x7f6ce0198808 in nftnl_set_elem_foreach /home/pablo/devel/scm/git-netfilter/libnftnl/src/set_elem.c:725
Indirect leak of 16 byte(s) in 1 object(s) allocated from:
#0 0x7f6ce2189720 in __interceptor_realloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xe9720)
#1 0x7f6ce1b1778d in xrealloc /home/pablo/devel/scm/git-netfilter/nftables/src/utils.c:55
#2 0x7f6ce1b1756d in gmp_xrealloc /home/pablo/devel/scm/git-netfilter/nftables/src/gmputil.c:202
#3 0x7f6ce1417059 in __gmpz_realloc (/usr/lib/x86_64-linux-gnu/libgmp.so.10+0x23059)
Indirect leak of 8 byte(s) in 1 object(s) allocated from:
#0 0x7f6ce2189330 in __interceptor_malloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xe9330)
#1 0x7f6ce1b1767a in xmalloc /home/pablo/devel/scm/git-netfilter/nftables/src/utils.c:36
#2 0x7f6ce14105c5 in __gmpz_init2 (/usr/lib/x86_64-linux-gnu/libgmp.so.10+0x1c5c5)
SUMMARY: AddressSanitizer: 536 byte(s) leaked in 6 allocation(s).
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Direct leak of 18 byte(s) in 2 object(s) allocated from:
#0 0x7ff6ee6f9810 in strdup (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x3a810)
#1 0x7ff6ee22666d in xstrdup /home/pablo/devel/scm/git-netfilter/nftables/src/utils.c:75
#2 0x7ff6ee28cce9 in nft_parse /home/pablo/devel/scm/git-netfilter/nftables/src/parser_bison.c:5792
#3 0x4b903f302c8010a (<unknown module>)
Direct leak of 16 byte(s) in 1 object(s) allocated from:
#0 0x7ff6ee7a8330 in __interceptor_malloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xe9330)
#1 0x7ff6ee226578 in xmalloc /home/pablo/devel/scm/git-netfilter/nftables/src/utils.c:36
SUMMARY: AddressSanitizer: 34 byte(s) leaked in 3 allocation(s).
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Direct leak of 4 byte(s) in 1 object(s) allocated from:
#0 0x7f637b64a810 in strdup (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x3a810)
#1 0x7f637b17766d in xstrdup /home/pablo/devel/scm/git-netfilter/nftables/src/utils.c:75
#2 0x7f637b1ddce9 in nft_parse /home/pablo/devel/scm/git-netfilter/nftables/src/parser_bison.c:5792
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Direct leak of 1034 byte(s) in 152 object(s) allocated from:
#0 0x7f7b55f1b810 in strdup (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x3a810)
#1 0x7f7b559597e0 in xstrdup /home/pablo/devel/scm/git-netfilter/nftables/src/utils.c:75
#2 0x7f7b55a494a0 in nft_lex /home/pablo/devel/scm/git-netfilter/nftables/src/scanner.l:641
#3 0x7f7b559cec25 in nft_parse /home/pablo/devel/scm/git-netfilter/nftables/src/parser_bison.c:5792
#4 0x7f7b5597e318 in nft_parse_bison_filename /home/pablo/devel/scm/git-netfilter/nftables/src/libnftables.c:392
#5 0x7f7b5597f864 in nft_run_cmd_from_filename /home/pablo/devel/scm/git-netfilter/nftables/src/libnftables.c:495
#6 0x562a25bbce71 in main /home/pablo/devel/scm/git-netfilter/nftables/src/main.c:457
#7 0x7f7b5457509a in __libc_start_main ../csu/libc-start.c:308
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
This helper function adds a statement at a given position and it updates
the rule statement counter.
This patch fixes this:
flush table bridge test-bridge
add rule bridge test-bridge input vlan id 1 ip saddr 10.0.0.1
rule.c:2870:5: runtime error: index 2 out of bounds for type 'stmt *[*]'
=================================================================
==1043==ERROR: AddressSanitizer: dynamic-stack-buffer-overflow on address 0x7ffdd69c1350 at pc 0x7f1036f53330 bp 0x7ffdd69c1300 sp 0x7ffdd69c12f8
WRITE of size 8 at 0x7ffdd69c1350 thread T0
#0 0x7f1036f5332f in payload_try_merge /home/mbr/nftables/src/rule.c:2870
#1 0x7f1036f534b7 in rule_postprocess /home/mbr/nftables/src/rule.c:2885
#2 0x7f1036fb2785 in rule_evaluate /home/mbr/nftables/src/evaluate.c:3744
#3 0x7f1036fb627b in cmd_evaluate_add /home/mbr/nftables/src/evaluate.c:3982
#4 0x7f1036fbb9e9 in cmd_evaluate /home/mbr/nftables/src/evaluate.c:4462
#5 0x7f10370652d2 in nft_evaluate /home/mbr/nftables/src/libnftables.c:414
#6 0x7f1037065ba1 in nft_run_cmd_from_buffer /home/mbr/nftables/src/libnftables.c:447
Reported-by: Michael Braun <michael-dev@fami-braun.de> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Phil Sutter [Thu, 30 Apr 2020 12:02:44 +0000 (14:02 +0200)]
segtree: Fix get element command with prefixes
Code wasn't aware of prefix elements in interval sets. With previous
changes in place, they merely need to be accepted in
get_set_interval_find() - value comparison and expression duplication is
identical to ranges.
Extend sets/0034get_element_0 test to cover prefixes as well. While
being at it, also cover concatenated ranges.
Phil Sutter [Thu, 30 Apr 2020 11:57:35 +0000 (13:57 +0200)]
segtree: Merge get_set_interval_find() and get_set_interval_end()
Both functions were very similar already. Under the assumption that they
will always either see a range (or start of) that matches exactly or not
at all, reduce complexity and make get_set_interval_find() accept NULL
(left or) right values. This way it becomes a full replacement for
get_set_interval_end().
Phil Sutter [Thu, 30 Apr 2020 11:45:40 +0000 (13:45 +0200)]
segtree: Use expr_clone in get_set_interval_*()
Both functions perform interval set lookups with either start and end or
only start values as input. Interestingly, in practice they either see
values which are not contained or which match an existing range exactly.
Make use of the above and just return a clone of the matching entry
instead of creating a new one based on input data.
Phil Sutter [Tue, 28 Apr 2020 18:54:03 +0000 (20:54 +0200)]
segtree: Fix missing expires value in prefixes
This probable copy'n'paste bug prevented 'expiration' field from being
populated when turning a range into a prefix in
interval_map_decompose(). Consequently, interval sets with timeout did
print expiry value for ranges (such as 10.0.0.1-10.0.0.5) but not
prefixes (10.0.0.0/8, for instance).
Fixes: bb0e6d8a2851b ("segtree: incorrect handling of comments and timeouts with mapping") Signed-off-by: Phil Sutter <phil@nwl.cc>
Michael Braun [Sat, 2 May 2020 10:11:43 +0000 (12:11 +0200)]
main: fix get_optstring truncating output
Without this patch, get_optstring returns optstring = +hvVcf:insNSI:d:aejuy.
After this patch, get_optstring returns optstring = +hvVcf:insNSI:d:aejuypTt
This is due to optstring containing up to two chars per option, thus it was too
short.
Fixes: 906facf31d1d ("main: fix ASAN -fsanitize=address error in get_optstring()") Signed-off-by: Michael Braun <michael-dev@fami-braun.de> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Michael Braun [Fri, 1 May 2020 15:48:18 +0000 (17:48 +0200)]
datatype: fix double-free resulting in use-after-free in datatype_free
nft list table bridge t
table bridge t {
set s4 {
typeof ip saddr . ip daddr
elements = { 1.0.0.1 . 2.0.0.2 }
}
}
=================================================================
==24334==ERROR: AddressSanitizer: heap-use-after-free on address 0x6080000000a8 at pc 0x7fe0e67df0ad bp 0x7ffff83e88c0 sp 0x7ffff83e88b8
READ of size 4 at 0x6080000000a8 thread T0
#0 0x7fe0e67df0ac in datatype_free nftables/src/datatype.c:1110
#1 0x7fe0e67e2092 in expr_free nftables/src/expression.c:89
#2 0x7fe0e67a855e in set_free nftables/src/rule.c:359
#3 0x7fe0e67b2f3e in table_free nftables/src/rule.c:1263
#4 0x7fe0e67a70ce in __cache_flush nftables/src/rule.c:299
#5 0x7fe0e67a71c7 in cache_release nftables/src/rule.c:305
#6 0x7fe0e68dbfa9 in nft_ctx_free nftables/src/libnftables.c:292
#7 0x55f00fbe0051 in main nftables/src/main.c:469
#8 0x7fe0e553309a in __libc_start_main ../csu/libc-start.c:308
#9 0x55f00fbdd429 in _start (nftables/src/.libs/nft+0x9429)
0x6080000000a8 is located 8 bytes inside of 96-byte region [0x6080000000a0,0x608000000100)
freed by thread T0 here:
#0 0x7fe0e6e70fb0 in __interceptor_free (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xe8fb0)
#1 0x7fe0e68b8122 in xfree nftables/src/utils.c:29
#2 0x7fe0e67df2e5 in datatype_free nftables/src/datatype.c:1117
#3 0x7fe0e67e2092 in expr_free nftables/src/expression.c:89
#4 0x7fe0e67a83fe in set_free nftables/src/rule.c:356
#5 0x7fe0e67b2f3e in table_free nftables/src/rule.c:1263
#6 0x7fe0e67a70ce in __cache_flush nftables/src/rule.c:299
#7 0x7fe0e67a71c7 in cache_release nftables/src/rule.c:305
#8 0x7fe0e68dbfa9 in nft_ctx_free nftables/src/libnftables.c:292
#9 0x55f00fbe0051 in main nftables/src/main.c:469
#10 0x7fe0e553309a in __libc_start_main ../csu/libc-start.c:308
previously allocated by thread T0 here:
#0 0x7fe0e6e71330 in __interceptor_malloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xe9330)
#1 0x7fe0e68b813d in xmalloc nftables/src/utils.c:36
#2 0x7fe0e68b8296 in xzalloc nftables/src/utils.c:65
#3 0x7fe0e67de7d5 in dtype_alloc nftables/src/datatype.c:1065
#4 0x7fe0e67df862 in concat_type_alloc nftables/src/datatype.c:1146
#5 0x7fe0e67ea852 in concat_expr_parse_udata nftables/src/expression.c:954
#6 0x7fe0e685dc94 in set_make_key nftables/src/netlink.c:718
#7 0x7fe0e685e177 in netlink_delinearize_set nftables/src/netlink.c:770
#8 0x7fe0e685f667 in list_set_cb nftables/src/netlink.c:895
#9 0x7fe0e4f95a03 in nftnl_set_list_foreach src/set.c:904
SUMMARY: AddressSanitizer: heap-use-after-free nftables/src/datatype.c:1110 in datatype_free
Shadow bytes around the buggy address:
0x0c107fff7fc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c107fff7fd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c107fff7fe0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c107fff7ff0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c107fff8000: fa fa fa fa fd fd fd fd fd fd fd fd fd fd fd fd
=>0x0c107fff8010: fa fa fa fa fd[fd]fd fd fd fd fd fd fd fd fd fd
0x0c107fff8020: fa fa fa fa fd fd fd fd fd fd fd fd fd fd fd fd
0x0c107fff8030: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c107fff8040: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c107fff8050: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c107fff8060: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==24334==ABORTING
Signed-off-by: Michael Braun <michael-dev@fami-braun.de> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Michael Braun [Fri, 1 May 2020 15:48:16 +0000 (17:48 +0200)]
main: fix ASAN -fsanitize=address error in get_optstring()
nft list table bridge t
=================================================================
==28552==ERROR: AddressSanitizer: global-buffer-overflow on address 0x5579c662e816 at pc 0x7fc2803246aa bp 0x7fff495c86f0 sp 0x7fff495c7ea0
WRITE of size 2 at 0x5579c662e816 thread T0
#0 0x7fc2803246a9 in vsprintf (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x546a9)
#1 0x7fc2803249f6 in __interceptor_sprintf (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x549f6)
#2 0x5579c661e7d2 in get_optstring nftables/src/main.c:128
#3 0x5579c66202af in main nftables/src/main.c:315
#4 0x7fc27ea7b09a in __libc_start_main ../csu/libc-start.c:308
#5 0x5579c661e439 in _start (nftables/src/.libs/nft+0x9439)
0x5579c662e816 is located 0 bytes to the right of global variable 'optstring' defined in 'main.c:121:14' (0x5579c662e800) of size 22
0x5579c662e816 is located 42 bytes to the left of global variable 'options' defined in 'main.c:137:23' (0x5579c662e840) of size 672
SUMMARY: AddressSanitizer: global-buffer-overflow (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x546a9) in vsprintf
Shadow bytes around the buggy address:
0x0aafb8cbdcb0: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
0x0aafb8cbdcc0: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
0x0aafb8cbdcd0: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
0x0aafb8cbdce0: f9 f9 f9 f9 00 00 00 00 00 00 00 00 00 00 00 00
0x0aafb8cbdcf0: 00 00 00 00 00 00 00 00 00 f9 f9 f9 f9 f9 f9 f9
=>0x0aafb8cbdd00: 00 00[06]f9 f9 f9 f9 f9 00 00 00 00 00 00 00 00
0x0aafb8cbdd10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0aafb8cbdd20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0aafb8cbdd30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0aafb8cbdd40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0aafb8cbdd50: 00 00 00 00 00 00 00 00 00 00 00 00 f9 f9 f9 f9
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==28552==ABORTING
Fixes: 719e44277f8e ("main: use one data-structure to initialize getopt_long(3) arguments and help.") Signed-of-by: Michael Braun <michael-dev@fami-braun.de> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
rule: fix element cache update in __do_add_setelems()
The set->init and expr arguments might actually refer to the same list
of elements. Skip set element cache update introduced by dd44081d91ce
("segtree: Fix add and delete of element in same batch") otherwise
list_splice_tail_init() actually operates with the same list as
arguments. Valgrind reports this problem as a memleak since the result
of this operation was an empty set element list.
Fixes: dd44081d91ce ("segtree: Fix add and delete of element in same batch") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
This patch invokes interval_map_decompose() with named sets:
==3402== 2,352 (128 direct, 2,224 indirect) bytes in 1 blocks are definitely lost in loss record 9 of 9
==3402== at 0x483577F: malloc (vg_replace_malloc.c:299)
==3402== by 0x48996A8: xmalloc (utils.c:36)
==3402== by 0x4899778: xzalloc (utils.c:65)
==3402== by 0x487CB46: expr_alloc (expression.c:45)
==3402== by 0x487E2A0: mapping_expr_alloc (expression.c:1140)
==3402== by 0x4898AA8: interval_map_decompose (segtree.c:1095)
==3402== by 0x4872BDF: __do_add_setelems (rule.c:1569)
==3402== by 0x4872BDF: __do_add_setelems (rule.c:1559)
==3402== by 0x4877936: do_command (rule.c:2710)
==3402== by 0x489F1CB: nft_netlink.isra.5 (libnftables.c:42)
==3402== by 0x489FB07: nft_run_cmd_from_filename (libnftables.c:508)
==3402== by 0x10A9AA: main (main.c:455)
Fixes: dd44081d91ce ("segtree: Fix add and delete of element in same batch") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
The debugging output displays table and set names as (null). This patch
sets the table and name before displaying the netlink debugging, then
unset them to not break the extended error support.
Fixes: 086ec6f30c96 ("mnl: extended error support for create command") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
projects / thirdparty / nftables.git / log
