]> git.ipfire.org Git - thirdparty/nftables.git/log
thirdparty/nftables.git
2 years agonetlink: statify __netlink_gen_data()
Pablo Neira Ayuso [Fri, 9 Dec 2022 10:53:56 +0000 (11:53 +0100)] 
netlink: statify __netlink_gen_data()

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agoMakefile: Create LZMA-compressed dist-files
Phil Sutter [Wed, 7 Dec 2022 23:45:28 +0000 (00:45 +0100)] 
Makefile: Create LZMA-compressed dist-files

Use a more modern alternative to bzip2.

Suggested-by: Jan Engelhardt <jengelh@inai.de>
Suggested-by: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Phil Sutter <phil@nwl.cc>
2 years agoevaluate: do not crash on runaway number of concatenation components
Pablo Neira Ayuso [Thu, 8 Dec 2022 01:25:19 +0000 (02:25 +0100)] 
evaluate: do not crash on runaway number of concatenation components

Display error message in case user specifies more data components than
those defined by the concatenation of selectors.

 # cat example.nft
 table ip x {
        chain y {
                type filter hook prerouting priority 0; policy drop;
                ip saddr . meta mark { 1.2.3.4 . 0x00000100 . 1.2.3.6-1.2.3.8 } accept
        }
 }
 # nft -f example.nft
 example.nft:4:3-22: Error: too many concatenation components
                ip saddr . meta mark { 1.2.3.4 . 0x00000100 . 1.2.3.6-1.2.3.8 } accept
                ~~~~~~~~~~~~~~~~~~~~   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Without this patch, nft crashes:

==464771==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60d000000418 at pc 0x7fbc17513aa5 bp 0x7ffc73d33c90 sp 0x7ffc73d33c88
READ of size 8 at 0x60d000000418 thread T0
    #0 0x7fbc17513aa4 in expr_evaluate_concat /home/pablo/devel/scm/git-netfilter/nftables/src/evaluate.c:1348
    #1 0x7fbc1752a9da in expr_evaluate /home/pablo/devel/scm/git-netfilter/nftables/src/evaluate.c:2476
    #2 0x7fbc175175e2 in expr_evaluate_set_elem /home/pablo/devel/scm/git-netfilter/nftables/src/evaluate.c:1504
    #3 0x7fbc1752aa22 in expr_evaluate /home/pablo/devel/scm/git-netfilter/nftables/src/evaluate.c:2482
    #4 0x7fbc17512cb5 in list_member_evaluate /home/pablo/devel/scm/git-netfilter/nftables/src/evaluate.c:1310
    #5 0x7fbc17518ca0 in expr_evaluate_set /home/pablo/devel/scm/git-netfilter/nftables/src/evaluate.c:1590
    [...]

Fixes: 64bb3f43bb96 ("src: allow to use typeof of raw expressions in set declaration")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agonetlink: swap byteorder of value component in concatenation of intervals
Pablo Neira Ayuso [Thu, 8 Dec 2022 00:35:05 +0000 (01:35 +0100)] 
netlink: swap byteorder of value component in concatenation of intervals

Commit 1017d323cafa ("src: support for selectors with different byteorder with
interval concatenations") was incomplete.

Switch byteorder of singleton values in a set that contains
concatenation of intervals. This singleton value is actually represented
as a range in the kernel.

After this patch, if the set represents a concatenation of intervals:

- EXPR_F_INTERVAL denotes the lhs of the interval.
- EXPR_F_INTERVAL_END denotes the rhs of the interval (this flag was
  already used in this way before this patch).

If none of these flags are set on, then the set contains concatenations
of singleton values (no interval flag is set on), in such case, no
byteorder swap is required.

Update tests/shell and tests/py to cover the use-case breakage reported
by Eric.

Fixes: 1017d323cafa ("src: support for selectors with different byteorder with interval concatenations")
Reported-by: Eric Garver <eric@garver.life>
Tested-by: Eric Garver <eric@garver.life>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agotests: py: missing json for different byteorder selector with interval concatenation
Pablo Neira Ayuso [Wed, 7 Dec 2022 21:15:12 +0000 (22:15 +0100)] 
tests: py: missing json for different byteorder selector with interval concatenation

Add missing json output, otherwise -j reports an error.

Fixes: 1017d323cafa ("src: support for selectors with different byteorder with interval concatenations")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agoscanner: munch full comment lines
Pablo Neira Ayuso [Tue, 6 Dec 2022 21:59:55 +0000 (22:59 +0100)] 
scanner: munch full comment lines

Munch lines full comment lines, regular expression matches lines that
start by space or tab, then # follows, finally anything including one
single line break.

Call reset_pos() to ensure error reporting location is not puzzled.

Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1196
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agodoc: statements: fwd supports for sending packets via neighbouring layer
Pablo Neira Ayuso [Tue, 6 Dec 2022 17:02:25 +0000 (18:02 +0100)] 
doc: statements: fwd supports for sending packets via neighbouring layer

Document ability to forward packets through neighbour layer added in
30d45266bf38 ("expr: extend fwd statement to support address and family").

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agodoc: payload-expression.txt: Mention that 'ih' exists
Harald Welte [Tue, 6 Dec 2022 14:03:33 +0000 (15:03 +0100)] 
doc: payload-expression.txt: Mention that 'ih' exists

Back in commit b67abc51ba6f ("src: raw payload match and mangle on inner
header / payload data") a new payload expression 'ih' was added, but the
documentation wasn't updated accordingly.

Let's at least mention in the man page that it exists at all.

Signed-off-by: Harald Welte <laforge@gnumonks.org>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agojson: fix 'add flowtable' command
Alex Forster [Fri, 2 Dec 2022 03:35:01 +0000 (21:35 -0600)] 
json: fix 'add flowtable' command

In `json_parse_cmd_add_flowtable`, the format arguments passed to `json_unpack` are incorrect: the object key name ("dev") is not provided.

Fixes: da6cb40177da ("parser_json: permit empty device list")
Signed-off-by: Alex Forster <aforster@cloudflare.com>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agosrc: support for selectors with different byteorder with interval concatenations
Pablo Neira Ayuso [Wed, 23 Nov 2022 16:59:21 +0000 (17:59 +0100)] 
src: support for selectors with different byteorder with interval concatenations

Assuming the following interval set with concatenation:

 set test {
typeof ip saddr . meta mark
flags interval
 }

then, the following rule:

 ip saddr . meta mark @test

requires bytecode that swaps the byteorder for the meta mark selector in
case the set contains intervals and concatenations.

 inet x y
   [ meta load nfproto => reg 1 ]
   [ cmp eq reg 1 0x00000002 ]
   [ payload load 4b @ network header + 12 => reg 1 ]
   [ meta load mark => reg 9 ]
   [ byteorder reg 9 = hton(reg 9, 4, 4) ]  <----- this is required !
   [ lookup reg 1 set test dreg 0 ]

This patch updates byteorder_conversion() to add the unary expression
that introduces the byteorder expression.

Moreover, store the meta mark range component of the element tuple in
the set in big endian as it is required for the range comparisons. Undo
the byteorder swap in the netlink delinearize path to listing the meta
mark values accordingly.

Update tests/py to validate that byteorder expression is emitted in the
bytecode. Update tests/shell to validate insertion and listing of a
named map declaration.

A similar commit 806ab081dc9a ("netlink: swap byteorder for
host-endian concat data") already exists in the tree to handle this for
strings with prefix (e.g. eth*).

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agoWarn for tables with compat expressions in rules
Phil Sutter [Tue, 11 Oct 2022 16:46:55 +0000 (18:46 +0200)] 
Warn for tables with compat expressions in rules

While being able to "look inside" compat expressions using nft is a nice
feature, it is also (yet another) pitfall for unaware users, deceiving
them into assuming interchangeability (or at least compatibility)
between iptables-nft and nft.

In reality, which involves 'nft list ruleset | nft -f -', any correctly
translated compat expressions will turn into native nftables ones not
understood by (the version of) iptables-nft which created them in the
first place. Other compat expressions will vanish, potentially
compromising the firewall ruleset.

Emit a warning (as comment) to give users a chance to stop and
reconsider before shooting their own foot.

Signed-off-by: Phil Sutter <phil@nwl.cc>
2 years agomonitor: missing cache and set handle initialization
Pablo Neira Ayuso [Fri, 11 Nov 2022 10:07:28 +0000 (11:07 +0100)] 
monitor: missing cache and set handle initialization

This leads to a crash when adding stateful expressions to sets:

netlink.c:928:38: runtime error: member access within null pointer of type 'struct nft_ctx'
AddressSanitizer:DEADLYSIGNAL
=================================================================
==13781==ERROR: AddressSanitizer: SEGV on unknown address 0x0000000000d0 (pc 0x7fc96fc2b6b2 bp 0x7ffc0e26b080 sp 0x7ffc0e26b020 T0)
==13781==The signal is caused by a READ memory access.
==13781==Hint: address points to the zero page.
    #0 0x7fc96fc2b6b2 in table_cache_find /home/pablo/devel/scm/git-netfilter/nftables/src/cache.c:456
    #1 0x7fc96fd244d4 in netlink_parse_set_expr /home/pablo/devel/scm/git-netfilter/nftables/src/netlink_delinearize.c:1857
    #2 0x7fc96fcf1b4d in netlink_delinearize_set /home/pablo/devel/scm/git-netfilter/nftables/src/netlink.c:928
    #3 0x7fc96fd41966 in netlink_events_cache_addset /home/pablo/devel/scm/git-netfilter/nftables/src/monitor.c:649

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agodoc: no reset support for limit
Pablo Neira Ayuso [Wed, 9 Nov 2022 14:59:38 +0000 (15:59 +0100)] 
doc: no reset support for limit

Remove reset command, this not supported for ratelimit.

Fixes: eff2d606d20d ("doc: document a few reset commands supported by the parser")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agodoc: document a few reset commands supported by the parser
Pablo Neira Ayuso [Wed, 9 Nov 2022 10:17:34 +0000 (11:17 +0100)] 
doc: document a few reset commands supported by the parser

The following are missing in the manpage:

 *reset counters* ['family']
 *reset quotas* ['family']
 *reset counters* ['family'] *table* 'table'
 *reset quotas* ['family'] *table* 'table'

While at it, expand type to the supported stateful objects.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agooptimize: handle prefix and range when merging into set + concatenation
Pablo Neira Ayuso [Sat, 5 Nov 2022 15:51:10 +0000 (16:51 +0100)] 
optimize: handle prefix and range when merging into set + concatenation

The following ruleset fails to be merged using set + concatenation:

  meta iifname eth1 ip saddr 1.1.1.2 ip daddr 2.2.3.0/24 accept
  meta iifname eth1 ip saddr 1.1.1.2 ip daddr 2.2.4.0-2.2.4.10 accept

hitting the following assertion:

  nft: optimize.c:585: __merge_concat_stmts: Assertion `0' failed.
  Abort

This patch also updates tests/shell.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agopayload: do not kill dependency for proto_unknown
Pablo Neira Ayuso [Mon, 31 Oct 2022 10:35:31 +0000 (11:35 +0100)] 
payload: do not kill dependency for proto_unknown

Unsupported meta match on layer 4 protocol sets on protocol context to
proto_unknown, handle anything coming after it as a raw expression in
payload_expr_expand().

Moreover, payload_dependency_kill() skips dependency removal if protocol
is unknown, so raw payload expression leaves meta layer 4 protocol
remains in place.

Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1641
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agonetlink_delinearize: complete payload expression in payload statement
Pablo Neira Ayuso [Mon, 31 Oct 2022 10:35:21 +0000 (11:35 +0100)] 
netlink_delinearize: complete payload expression in payload statement

Call payload_expr_complete() to complete payload expression in payload
statement, otherwise expr->payload.desc is set to proto_unknown.

Call stmt_payload_binop_postprocess() introduced by 50ca788ca4d0
("netlink: decode payload statment") if payload_expr_complete() fails to
provide a protocol description (eg. ip dscp).

Follow up patch does not allow to remove redundant payload dependency if
proto_unknown is used to deal with the raw payload expression case.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agotests: shell: deletion from interval concatenation
Pablo Neira Ayuso [Mon, 17 Oct 2022 11:52:43 +0000 (13:52 +0200)] 
tests: shell: deletion from interval concatenation

Deleting item from concatenated set stops working at least in 5.15.64.
Add test to cover this use case.

Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1638
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agonetlink_delinearize: do not transfer binary operation to non-anonymous sets
Pablo Neira Ayuso [Wed, 12 Oct 2022 10:50:26 +0000 (12:50 +0200)] 
netlink_delinearize: do not transfer binary operation to non-anonymous sets

Michael Braun says:

This results for nft list ruleset in
  nft: netlink_delinearize.c:1945: binop_adjust_one: Assertion `value->len >= binop->right->len' failed.

This is due to binop_adjust_one setting value->len to left->len, which
is shorther than right->len.

Additionally, it does not seem correct to alter set elements from parsing a
rule, so remove that part all together.

Reported-by: Michael Braun <michael-dev@fami-braun.de>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agorule: do not display handle for implicit chain
Pablo Neira Ayuso [Fri, 7 Oct 2022 08:04:06 +0000 (10:04 +0200)] 
rule: do not display handle for implicit chain

Implicit chains do not allow for incremental updates, do not display rule
handle since kernel refuses to update an implicit chain which is already
bound.

Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1615
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agoparser_bison: display too many levels of nesting error
Pablo Neira Ayuso [Fri, 7 Oct 2022 07:55:59 +0000 (09:55 +0200)] 
parser_bison: display too many levels of nesting error

Instead of hitting this assertion:

 nft: parser_bison.y:70: open_scope: Assertion `state->scope < array_size(state->scopes) - 1' failed.
 Aborted

this is easier to trigger with implicit chains where one level of
nesting from the existing chain scope is supported.

Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1615
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agoevaluate: datatype memleak after binop transfer
Pablo Neira Ayuso [Tue, 4 Oct 2022 23:36:52 +0000 (01:36 +0200)] 
evaluate: datatype memleak after binop transfer

The following ruleset:

ip version vmap { 4 : jump t3, 6 : jump t4 }

results in a memleak.

expr_evaluate_shift() overrides the datatype which results in a datatype
memleak after the binop transfer that triggers a left-shift of the
constant (in the map).

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agoevaluate: bogus datatype assertion in binary operation evaluation
Pablo Neira Ayuso [Tue, 4 Oct 2022 23:36:44 +0000 (01:36 +0200)] 
evaluate: bogus datatype assertion in binary operation evaluation

Use datatype_equal(), otherwise dynamically allocated datatype fails
to fulfill the datatype pointer check, triggering the assertion:

 nft: evaluate.c:1249: expr_evaluate_binop: Assertion `expr_basetype(left) == expr_basetype(right)' failed.

Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1636
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agomonitor: Sanitize startup race condition
Phil Sutter [Wed, 28 Sep 2022 21:26:42 +0000 (23:26 +0200)] 
monitor: Sanitize startup race condition

During startup, 'nft monitor' first fetches the current ruleset and then
keeps this cache up to date based on received events. This is racey, as
any ruleset changes in between the initial fetch and the socket opening
are not recognized.

This script demonstrates the problem:

| #!/bin/bash
|
| while true; do
|  nft flush ruleset
|  iptables-nft -A FORWARD
| done &
| maniploop=$!
|
| trap "kill $maniploop; kill \$!; wait" EXIT
|
| while true; do
|  nft monitor rules >/dev/null &
|  sleep 0.2
|  kill $!
| done

If the table add event is missed, the rule add event callback fails to
deserialize the rule and calls abort().

Avoid the inconvenient program exit by returning NULL from
netlink_delinearize_rule() instead of aborting and make callers check
the return value.

Signed-off-by: Phil Sutter <phil@nwl.cc>
2 years agotests: py: add vlan test case for ip/inet family
Florian Westphal [Tue, 27 Sep 2022 13:41:33 +0000 (15:41 +0200)] 
tests: py: add vlan test case for ip/inet family

before fixup, this failed with:

line 4: 'add rule ip test-ip4 input vlan id 1': '[ payload load 2b @ link header + 12 => reg 1 ]' mismatches '[ payload load 2b @ link header + 0 => reg 1 ]'

... because the auto-dependency did not add the preceeding ethernet
header, so vlan was using the wrong offset.

Note than vlan id match in inet input families will only work if header
removal was disabled, i.e.

... add link vethin1 name vethin1.3 type vlan id 3 reorder_hdr off

otherwise, kernel will strip the vlan tag and interface appears as
a normal ethernet interface.

Signed-off-by: Florian Westphal <fw@strlen.de>
2 years agoevaluate: add ethernet header size offset for implicit vlan dependency
Florian Westphal [Tue, 27 Sep 2022 12:16:15 +0000 (14:16 +0200)] 
evaluate: add ethernet header size offset for implicit vlan dependency

'vlan id 1'

must also add a ethernet header dep, else nft fetches the payload from
header offset 0 instead of 14.

Reported-by: Yi Chen <yiche@redhat.com>
Signed-off-by: Florian Westphal <fw@strlen.de>
2 years agodoc: mention vlan matching in ip/ip6/inet families
Florian Westphal [Tue, 27 Sep 2022 10:28:21 +0000 (12:28 +0200)] 
doc: mention vlan matching in ip/ip6/inet families

It only works if vlan_reorder is turned off to disable the vlan tag
removal.

Signed-off-by: Florian Westphal <fw@strlen.de>
2 years agoevaluate: allow implicit ether -> vlan dep
Florian Westphal [Thu, 11 Aug 2022 07:10:55 +0000 (09:10 +0200)] 
evaluate: allow implicit ether -> vlan dep

nft add rule inet filter input vlan id 2
Error: conflicting protocols specified: ether vs. vlan

Refresh the current dependency after superseding the dummy
dependency to make this work.

Signed-off-by: Florian Westphal <fw@strlen.de>
2 years agodoc, src: make some spelling and grammatical improvements
Jeremy Sowden [Thu, 22 Sep 2022 08:00:42 +0000 (09:00 +0100)] 
doc, src: make some spelling and grammatical improvements

Fix a couple of spelling mistakes:

  'expresion' -> 'expression'

and correct some non-native usages:

  'allows to' -> 'allows one to'

Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Florian Westphal <fw@strlen.de>
2 years agosegtree: fix decomposition of unclosed intervals containing address prefixes
Jeremy Sowden [Sun, 18 Sep 2022 17:22:12 +0000 (18:22 +0100)] 
segtree: fix decomposition of unclosed intervals containing address prefixes

The code which decomposes unclosed intervals doesn't check for prefixes.  This
leads to incorrect output for sets which contain these.  For example,

  # nft -f - <<END
  table ip t {
    chain c {
      ip saddr 192.0.0.0/2 drop
      ip saddr 10.0.0.0/8 drop
      ip saddr { 192.0.0.0/2, 10.0.0.0/8 } drop
    }
  }
  table ip6 t {
    chain c {
      ip6 saddr ff00::/8 drop
      ip6 saddr fe80::/10 drop
      ip6 saddr { ff00::/8, fe80::/10 } drop
    }
  }
  END
  # nft list table ip6 t
  table ip6 t {
    chain c {
      ip6 saddr ff00::/8 drop
      ip6 saddr fe80::/10 drop
      ip6 saddr { fe80::/10, ff00::-ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff } drop
    }
  }
  # nft list table ip t
  table ip t {
    chain c {
      ip saddr 192.0.0.0/2 drop
      ip saddr 10.0.0.0/8 drop
      ip saddr { 10.0.0.0/8, 192.0.0.0-255.255.255.255 } drop
    }
  }

Instead of treating the final unclosed interval as a special case, reuse the
code which correctly handles closed intervals.

Add a shell test-case.

Link: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1018156
Fixes: 86b965bdab8d ("segtree: fix decomposition of unclosed intervals")
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Florian Westphal <fw@strlen.de>
2 years agosegtree: refactor decomposition of closed intervals
Jeremy Sowden [Sun, 18 Sep 2022 17:22:11 +0000 (18:22 +0100)] 
segtree: refactor decomposition of closed intervals

Move the code in `interval_map_decompose` which adds a new closed
interval to the set into a separate function.  In addition to the moving
of the code, there is one other change: `compound_expr_add` is called
once, after the main conditional, instead of being called in each
branch.

Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Florian Westphal <fw@strlen.de>
2 years agoevaluate: un-break rule insert with intervals
Florian Westphal [Tue, 20 Sep 2022 13:26:07 +0000 (15:26 +0200)] 
evaluate: un-break rule insert with intervals

'rule inet dscpclassify dscp_match  meta l4proto { udp }  th dport { 3478 }  th sport { 3478-3497, 16384-16387 } goto ct_set_ef'
works with 'nft add', but not 'nft insert', the latter yields: "BUG: unhandled op 4".

Fixes: 81e36530fcac ("src: replace interval segment tree overlap and automerge")
Signed-off-by: Florian Westphal <fw@strlen.de>
Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agodoc: add nft_ctx_add_var() and nft_ctx_clear_vars() docs
Fernando Fernandez Mancera [Mon, 12 Sep 2022 10:52:25 +0000 (12:52 +0200)] 
doc: add nft_ctx_add_var() and nft_ctx_clear_vars() docs

Add missing documentation for nft_ctx_add_var() and nft_ctx_clear_vars()
functions.

Signed-off-by: Fernando Fernandez Mancera <ffmancera@riseup.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agopy: support variables management and fix formatting
Fernando Fernandez Mancera [Mon, 12 Sep 2022 10:52:24 +0000 (12:52 +0200)] 
py: support variables management and fix formatting

Add nft_ctx_add_var() and nft_ctx_clear_vars() support through add_var() and
clear_vars(). Also, fix some functions documentation and drop unnecesary
comments.

In addition, modify get_dry_run() to return the previous value set. This is
needed to be consistent with the rest of the python API.

Link: https://bugzilla.netfilter.org/show_bug.cgi?id=1591
Signed-off-by: Fernando Fernandez Mancera <ffmancera@riseup.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agopy: extend python API to support libnftables API
Peter Collinson [Mon, 12 Sep 2022 10:52:23 +0000 (12:52 +0200)] 
py: extend python API to support libnftables API

Allows py/nftables.py to support full mapping to the libnftables API. The
changes allow python code to talk in text to the kernel rather than just
using json. The Python API can now also use dry run to test changes.

Link: https://bugzilla.netfilter.org/show_bug.cgi?id=1591
Signed-off-by: Peter Collinson <pc@hillside.co.uk>
Signed-off-by: Fernando Fernandez Mancera <ffmancera@riseup.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agojson: add stateful object comment support
Fernando Fernandez Mancera [Sat, 10 Sep 2022 22:11:14 +0000 (00:11 +0200)] 
json: add stateful object comment support

When listing a stateful object with JSON support, the comment was ignored.

Output example:

{
  "counter": {
    "family": "inet",
    "name": "mycounter",
    "table": "t",
    "handle": 1,
    "comment": "my comment in counter",
    "packets": 0,
    "bytes": 0
  }
}

Link: https://bugzilla.netfilter.org/show_bug.cgi?id=1611
Signed-off-by: Fernando Fernandez Mancera <ffmancera@riseup.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agojson: add secmark object reference support
Fernando Fernandez Mancera [Sat, 10 Sep 2022 07:59:48 +0000 (09:59 +0200)] 
json: add secmark object reference support

The secmark object reference requires a json parser function and it was
missing. In addition, extends the shell testcases.

Link: https://bugzilla.netfilter.org/show_bug.cgi?id=1630
Signed-off-by: Fernando Fernandez Mancera <ffmancera@riseup.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agoinclude: resync nf_tables.h cache copy
Pablo Neira Ayuso [Wed, 14 Sep 2022 14:48:12 +0000 (16:48 +0200)] 
include: resync nf_tables.h cache copy

Get this header in sync with nf-next as of 6.0-rc.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agojson: fix empty statement list output in sets and maps
Fernando Fernandez Mancera [Sun, 4 Sep 2022 17:18:26 +0000 (19:18 +0200)] 
json: fix empty statement list output in sets and maps

JSON output of sets and map should not include the statements list if is
empty. The statement output should be stateless also.

In addition, removes duplicated code.

Fixes: 07958ec53830 ("json: add set statement list support")
Fixes: e66f3187d891 ("json: add table map statement support")
Signed-off-by: Fernando Fernandez Mancera <ffmancera@riseup.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agojson: fix json schema version verification
Fernando Fernandez Mancera [Fri, 2 Sep 2022 13:35:06 +0000 (15:35 +0200)] 
json: fix json schema version verification

nft should ignore malformed or missing entries of `json_schema_version` but
check the value when it is integer.

Link: https://bugzilla.netfilter.org/show_bug.cgi?id=1490
Fixes: 49e0f1dc6e52 ("JSON: Add metainfo object to all output")
Signed-off-by: Fernando Fernandez Mancera <ffmancera@riseup.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agojson: add table map statement support
Fernando Fernandez Mancera [Fri, 2 Sep 2022 10:52:04 +0000 (12:52 +0200)] 
json: add table map statement support

When listing a map with statements with JSON support, the statement list were
ignored.

Output example:

{
  "map": {
    "family": "ip",
    "name": "m",
    "table": "t",
    "type": "ipv4_addr",
    "handle": 1,
    "map": "mark",
    "stmt": [
      {
        "counter": {
          "packets": 0,
          "bytes": 0
        }
      }
    ]
  }
}

Link: https://bugzilla.netfilter.org/show_bug.cgi?id=1588
Signed-off-by: Fernando Fernandez Mancera <ffmancera@riseup.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agorule: check address family in set collapse
Derek Hageman [Thu, 1 Sep 2022 16:10:41 +0000 (10:10 -0600)] 
rule: check address family in set collapse

498a5f0c219d added collapsing of set operations in different commands.
However, the logic is currently too relaxed.  It is valid to have a
table and set with identical names on different address families.
For example:

  table ip a {
    set x {
      type inet_service;
    }
  }
  table ip6 a {
      set x {
        type inet_service;
      }
  }
  add element ip a x { 1 }
  add element ip a x { 2 }
  add element ip6 a x { 2 }

The above currently results in nothing being added to the ip6 family
table due to being collapsed into the ip table add. Prior to
498a5f0c219d the set add would work. The fix is simply to check the
family in addition to the table and set names before allowing a
collapse.

[ Add testcase to tests/shell --pablo ]

Fixes: 498a5f0c219d ("rule: collapse set element commands")
Signed-off-by: Derek Hageman <hageman@inthat.cloud>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agojson: add set statement list support
Fernando Fernandez Mancera [Thu, 1 Sep 2022 10:31:43 +0000 (12:31 +0200)] 
json: add set statement list support

When listing a set with statements with JSON support, the statements were
ignored.

Output example:

{
  "set": {
    "op": "add",
    "elem": {
      "payload": {
        "protocol": "ip",
        "field": "saddr"
      }
    },
    "stmt": [
      {
        "limit": {
          "rate": 10,
          "burst": 5,
          "per": "second"
        }
      },
      {
        "counter": {
          "packets": 0,
          "bytes": 0
        }
      }
    ],
    "set": "@my_ssh_meter"
  }
}

Link: https://bugzilla.netfilter.org/show_bug.cgi?id=1495
Signed-off-by: Fernando Fernandez Mancera <ffmancera@riseup.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agotests/py: missing userdata in netlink payload
Pablo Neira Ayuso [Wed, 31 Aug 2022 14:56:04 +0000 (16:56 +0200)] 
tests/py: missing userdata in netlink payload

Since libnftnl's 212479ad2c92 ("rule, set_elem: fix printing of user
data"), userdata is missing in netlink payload printing via --debug.
Update tests/py/ip6/srh.t.payload to silence warning.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agosrc: allow burst 0 for byte ratelimit and use it as default
Pablo Neira Ayuso [Tue, 30 Aug 2022 14:51:35 +0000 (16:51 +0200)] 
src: allow burst 0 for byte ratelimit and use it as default

Packet-based limit burst is set to 5, as in iptables. However,
byte-based limit burst adds to the rate to calculate the bucket size,
and this is also sets this to 5 (... bytes in this case). Update it to
use zero byte burst by default instead.

This patch also updates manpage to describe how the burst value
influences the kernel module's token bucket in each of the two modes.
This documentation update is based on original text by Phil Sutter.

Adjust tests/py to silence warnings due to mismatching byte burst.

Fixes: 285baccfea46 ("src: disallow burst 0 in ratelimits")
Acked-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agoexpr: update EXPR_MAX and add missing comments
Florian Westphal [Tue, 23 Aug 2022 10:51:52 +0000 (12:51 +0200)] 
expr: update EXPR_MAX and add missing comments

WHen flagcmp and catchall expressions got added the EXPR_MAX definition
wasn't changed.

Should have no impact in practice however, this value is only checked to
prevent crash when old nft release is used to list a ruleset generated
by a newer nft release and a unknown 'typeof' expression.

v2: Pablo suggested to add EXPR_MAX into enum so its easier to spot.
Adding __EXPR_MAX + define EXPR_MAX (__EXPR_MAX - 1) causes '__EXPR_MAX
not handled in switch' warnings, hence the 'EXPR_MAX =' solution.

Signed-off-by: Florian Westphal <fw@strlen.de>
2 years agoerec: Dump locations' expressions only if set
Phil Sutter [Tue, 30 Aug 2022 13:00:52 +0000 (15:00 +0200)] 
erec: Dump locations' expressions only if set

Calling netlink_dump_expr() with a NULL pointer leads to segfault within
libnftnl. Internal ("fake") locations such as 'netlink_location' don't
have an expression assigned so expect this and skip the call. Simple
reproducer (list ruleset with netlink debugging as non-root):

| $ nft -d netlink list ruleset

Reported-by: François Rigault <frigo@amadeus.com>
Signed-off-by: Phil Sutter <phil@nwl.cc>
2 years agodoc: nft.8: Add missing '-T' in synopsis
Phil Sutter [Tue, 12 Apr 2022 16:29:58 +0000 (18:29 +0200)] 
doc: nft.8: Add missing '-T' in synopsis

Originally a typo in fixed commit: It added 't' in lower case, but
meanwhile '-t' was added as well.

Fixes: f8f32deda31df ("meta: Introduce new conditions 'time', 'day' and 'hour'")
Signed-off-by: Phil Sutter <phil@nwl.cc>
2 years agooptimize: expand implicit set element when merging into concatenation
Pablo Neira Ayuso [Mon, 29 Aug 2022 11:46:21 +0000 (13:46 +0200)] 
optimize: expand implicit set element when merging into concatenation

Generalize the existing code to deal with implicit sets. When merging a
ruleset like the following:

udp dport 128 iifname "foo" #1
        udp dport { 67, 123 } iifname "bar" #2

into a concatenation of statements, the following expansion need to
be done for rule #2:

67 . "bar"
123 . "bar"

The expansion logic consists of cloning the existing concatenation being
built and then append each element in the implicit set. A list of
ongoing concatenations being built is maintained, so further expansions
are also supported.

Extend test to cover for this use-case.

Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1628
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agosrc: Don't parse string as verdict in map
Xiao Liang [Fri, 19 Aug 2022 02:40:23 +0000 (10:40 +0800)] 
src: Don't parse string as verdict in map

In verdict map, string values are accidentally treated as verdicts.

For example:

table t {
    map foo {
        type ipv4_addr : verdict
        elements = {
            192.168.0.1 : bar
        }
    }
    chain output {
        type filter hook output priority mangle;
        ip daddr vmap @foo
    }
}

Though "bar" is not a valid verdict (should be "jump bar" or something),
the string is taken as the element value. Then NFTA_DATA_VALUE is sent
to the kernel instead of NFTA_DATA_VERDICT. This would be rejected by
recent kernels. On older ones (e.g. v5.4.x) that don't validate the
type, a warning can be seen when the rule is hit, because of the
corrupted verdict value:

[5120263.467627] WARNING: CPU: 12 PID: 303303 at net/netfilter/nf_tables_core.c:229 nft_do_chain+0x394/0x500 [nf_tables]

Indeed, we don't parse verdicts during evaluation, but only chain names,
which is of type string rather than verdict. For example, "jump $var" is
a verdict while "$var" is a string.

Fixes: c64457cff967 ("src: Allow goto and jump to a variable")
Signed-off-by: Xiao Liang <shaw.leon@gmail.com>
Signed-off-by: Florian Westphal <fw@strlen.de>
2 years agooptimize: check for mergeable rules
Pablo Neira Ayuso [Tue, 9 Aug 2022 20:45:21 +0000 (22:45 +0200)] 
optimize: check for mergeable rules

Rules that are equal need to have at least one mergeable statement.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agooptimize: merging concatenation is unsupported
Pablo Neira Ayuso [Tue, 9 Aug 2022 20:18:14 +0000 (22:18 +0200)] 
optimize: merging concatenation is unsupported

Existing concatenation cannot be merge at this stage, skip them
otherwise this assertion is hit:

 nft: optimize.c:434: rule_build_stmt_matrix_stmts: Assertion `k >= 0' failed

Extend existing test to cover this.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agotests: shell: check for a tainted kernel
Florian Westphal [Thu, 11 Aug 2022 11:19:44 +0000 (13:19 +0200)] 
tests: shell: check for a tainted kernel

If a test case results in a kernel taint (WARN splat for example), make
sure the test script indicates this.

Signed-off-by: Florian Westphal <fw@strlen.de>
2 years agoconcat with dynamically sized fields like vlan id
Michael Braun [Thu, 28 Jul 2022 09:36:44 +0000 (11:36 +0200)] 
concat with dynamically sized fields like vlan id

This enables commands like

 nft set bridge t s4 '{typeof vlan id . ip daddr; elements = { 3567 .
1.2.3.4 }; }'

Which would previously fail with
  Error: can not use variable sized data types (integer) in concat
  expressions

Signed-off-by: Michael Braun <michael-dev@fami-braun.de>
Signed-off-by: Florian Westphal <fw@strlen.de>
2 years agobuild: Bump version to 1.0.5 v1.0.5
Pablo Neira Ayuso [Tue, 9 Aug 2022 18:44:51 +0000 (20:44 +0200)] 
build: Bump version to 1.0.5

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agotests/py: disable arp family for queue statement
Pablo Neira Ayuso [Tue, 9 Aug 2022 08:55:15 +0000 (10:55 +0200)] 
tests/py: disable arp family for queue statement

Kernel commit:

  commit 47f4f510ad586032b85c89a0773fbb011d412425
  Author: Florian Westphal <fw@strlen.de>
  Date:   Tue Jul 26 19:49:00 2022 +0200

    netfilter: nft_queue: only allow supported familes and hooks

restricts supported families, excluding arp.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agometa: don't use non-POSIX formats in strptime()
Jo-Philipp Wich [Mon, 8 Aug 2022 22:18:42 +0000 (00:18 +0200)] 
meta: don't use non-POSIX formats in strptime()

The current strptime() invocations in meta.c use the `%F` format which
is not specified by POSIX and thus unimplemented by some libc flavors
such as musl libc.

Replace all occurrences of `%F` with an equivalent `%Y-%m-%d` format
in order to be able to properly parse user supplied dates in such
environments.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2 years agosrc: allow anon set concatenation with ether and vlan
Florian Westphal [Mon, 25 Jul 2022 19:34:52 +0000 (21:34 +0200)] 
src: allow anon set concatenation with ether and vlan

vlan id uses integer type (which has a length of 0).

Using it was possible, but listing would assert:
python: mergesort.c:24: concat_expr_msort_value: Assertion `ilen > 0' failed.

There are two reasons for this.
First reason is that the udata/typeof information lacks the 'vlan id'
part, because internally this is 'payload . binop(payload AND mask)'.

binop lacks an udata store.  It makes little sense to store it,
'typeof' keyword expects normal match syntax.

So, when storing udata, store the left hand side of the binary
operation, i.e. the load of the 2-byte key.

With that resolved, delinerization could work, but concat_elem_expr()
would splice 12 bits off the elements value, but it should be 16 (on
a byte boundary).

Signed-off-by: Florian Westphal <fw@strlen.de>
2 years agoevaluate: search stacked header list for matching payload dep
Florian Westphal [Mon, 25 Jul 2022 18:02:28 +0000 (20:02 +0200)] 
evaluate: search stacked header list for matching payload dep

"ether saddr 0:1:2:3:4:6 vlan id 2" works, but reverse fails:

"vlan id 2 ether saddr 0:1:2:3:4:6" will give
Error: conflicting protocols specified: vlan vs. ether

After "proto: track full stack of seen l2 protocols, not just cumulative offset",
we have a list of all l2 headers, so search those to see if we had this
proto base in the past before rejecting this.

Reported-by: Eric Garver <eric@garver.life>
Signed-off-by: Florian Westphal <fw@strlen.de>
2 years agonetlink_delinearize: also postprocess OP_AND in set element context
Florian Westphal [Mon, 1 Aug 2022 11:03:18 +0000 (13:03 +0200)] 
netlink_delinearize: also postprocess OP_AND in set element context

Pablo reports:
add rule netdev nt y update @macset { vlan id timeout 5s }

listing still shows the raw expression:
 update @macset { @ll,112,16 & 0xfff timeout 5s }

so also cover the 'set element' case.

Reported-by: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Florian Westphal <fw@strlen.de>
2 years agotests: add a test case for ether and vlan listing
Florian Westphal [Mon, 25 Jul 2022 17:31:22 +0000 (19:31 +0200)] 
tests: add a test case for ether and vlan listing

before this patch series, test fails dump validation:
-               update @macset { ether saddr . vlan id timeout 5s } counter packets 0 bytes 0
-               ether saddr . vlan id @macset
+               update @macset { @ll,48,48 . @ll,112,16 & 0xfff timeout 5s } counter packets 0 bytes 0
+               @ll,48,48 . @ll,112,16 & 0xfff @macset

Signed-off-by: Florian Westphal <fw@strlen.de>
2 years agodebug: dump the l2 protocol stack
Florian Westphal [Mon, 25 Jul 2022 14:42:23 +0000 (16:42 +0200)] 
debug: dump the l2 protocol stack

Previously we used to print the cumulative size of the headers,
update this to print the tracked l2 stack.

Signed-off-by: Florian Westphal <fw@strlen.de>
2 years agoproto: track full stack of seen l2 protocols, not just cumulative offset
Florian Westphal [Mon, 25 Jul 2022 12:32:13 +0000 (14:32 +0200)] 
proto: track full stack of seen l2 protocols, not just cumulative offset

For input, a cumulative size counter of all pushed l2 headers is enough,
because we have the full expression tree available to us.

For delinearization we need to track all seen l2 headers, else we lose
information that we might need at a later time.

Consider:

rule netdev nt nc set update ether saddr . vlan id

during delinearization, the vlan proto_desc replaces the ethernet one,
and by the time we try to split the concatenation apart we will search
the ether saddr offset vs. the templates for proto_vlan.

This replaces the offset with an array that stores the protocol
descriptions seen.

Then, if the payload offset is larger than our description, search the
l2 stack and adjust the offset until we're within the expected offset
boundary.

Reported-by: Eric Garver <eric@garver.life>
Signed-off-by: Florian Westphal <fw@strlen.de>
2 years agonetlink_delinearize: postprocess binary ands in concatenations
Florian Westphal [Tue, 14 Jun 2022 19:56:48 +0000 (21:56 +0200)] 
netlink_delinearize: postprocess binary ands in concatenations

Input:
update ether saddr . vlan id timeout 5s @macset
ether saddr . vlan id @macset

Before this patch, gets rendered as:
update @macset { @ll,48,48 . @ll,112,16 & 0xfff timeout 5s }
@ll,48,48 . @ll,112,16 & 0xfff @macset

After this, listing will show:
update @macset { @ll,48,48 . vlan id timeout 5s }
@ll,48,48 . vlan id @macset

The @ll, ... is due to vlan description replacing the ethernet one,
so payload decode fails to take the concatenation apart (the ethernet
header payload info is matched vs. vlan template).

This will be adjusted by a followup patch.

Signed-off-by: Florian Westphal <fw@strlen.de>
2 years agonetlink_delinearize: allow postprocessing on concatenated elements
Florian Westphal [Tue, 14 Jun 2022 19:57:58 +0000 (21:57 +0200)] 
netlink_delinearize: allow postprocessing on concatenated elements

Currently there is no case where the individual expressions inside a
mapped concatenation need to be munged.

However, to support proper delinearization for an input like
'rule netdev nt nc set update ether saddr . vlan id timeout 5s @macset'

we need to allow this.

Right now, this gets listed as:

update @macset { @ll,48,48 . @ll,112,16 & 0xfff timeout 5s }

because the ethernet protocol is replaced by vlan beforehand,
so we fail to map @ll,48,48 to a vlan protocol.

Likewise, we can't map the vlan info either because we cannot
cope with the 'and' operation properly, nor is it removed.

Prepare for this by deleting and re-adding so that we do not
corrupt the linked list.

After this, the list can be safely changed and a followup patch
can start to delete/reallocate expressions.

Signed-off-by: Florian Westphal <fw@strlen.de>
3 years agoparser_json: fix device parsing in netdev family
Pablo Neira Ayuso [Mon, 1 Aug 2022 14:15:08 +0000 (16:15 +0200)] 
parser_json: fix device parsing in netdev family

json_unpack() function is not designed to take a pre-allocated buffer.

Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1612
Fixes: 3fdc7541fba0 ("src: add multidevice support for netdev chain")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
3 years agosrc: proto: support DF, LE PHB, VA for DSCP
Oleksandr Natalenko [Mon, 11 Jul 2022 10:47:09 +0000 (12:47 +0200)] 
src: proto: support DF, LE PHB, VA for DSCP

Add a couple of aliases for well-known DSCP values.

As per RFC 4594, add "df" as an alias of "cs0" with 0x00 value.

As per RFC 5865, add "va" for VOICE-ADMIT with 0x2c value.

As per RFC 8622, add "lephb" for Lower-Effort Per-Hop Behavior with 0x01 value.

tc-cake(8) in diffserv8 mode would benefit from having "lephb" defined since
it corresponds to "Tin 0".

https://www.iana.org/assignments/dscp-registry/dscp-registry.xhtml

Signed-off-by: Oleksandr Natalenko <oleksandr@natalenko.name>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
3 years agodoc: Document limitations of ipsec expression with xfrm_interface
Phil Sutter [Thu, 23 Jun 2022 15:49:20 +0000 (17:49 +0200)] 
doc: Document limitations of ipsec expression with xfrm_interface

Point at a possible solution to match IPsec info of locally generated
traffic routed to an xfrm-type interface.

Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Florian Westphal <fw@strlen.de>
3 years agocache: report an error message if cache initialization fails
Pablo Neira Ayuso [Mon, 18 Jul 2022 15:17:37 +0000 (17:17 +0200)] 
cache: report an error message if cache initialization fails

cache initialization failure (which should not ever happen) is not
reported to the user.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
3 years agocache: validate handle string length
Pablo Neira Ayuso [Mon, 18 Jul 2022 14:18:33 +0000 (16:18 +0200)] 
cache: validate handle string length

Maximum supported string length for handle is NFT_NAME_MAXLEN, report an
error if user is exceeding this limit.

By validating from the cache evaluation phase, input is validated for the
native and json parsers.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
3 years agocache: prepare nft_cache_evaluate() to return error
Pablo Neira Ayuso [Mon, 18 Jul 2022 13:56:00 +0000 (15:56 +0200)] 
cache: prepare nft_cache_evaluate() to return error

Move flags as parameter reference and add list of error messages to prepare
for sanity checks.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
3 years agorule: crash when uncollapsing command with unexisting table or set
Pablo Neira Ayuso [Thu, 7 Jul 2022 13:11:35 +0000 (15:11 +0200)] 
rule: crash when uncollapsing command with unexisting table or set

If ruleset update refers to an unexisting table or set, then
cmd->elem.set is NULL.

Fixes: 498a5f0c219d ("rule: collapse set element commands")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
3 years agocache: release pending rules when chain binding lookup fails
Pablo Neira Ayuso [Wed, 6 Jul 2022 11:21:34 +0000 (13:21 +0200)] 
cache: release pending rules when chain binding lookup fails

If the implicit chain is not in the cache, release pending rules in
ctx->list and report EINTR to let the cache core retry to populate a
consistent cache.

Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1402
Fixes: c330152b7f77 ("src: support for implicit chain bindings")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
3 years agoevaluate: report missing interval flag when using prefix/range in concatenation
Pablo Neira Ayuso [Wed, 29 Jun 2022 16:40:00 +0000 (18:40 +0200)] 
evaluate: report missing interval flag when using prefix/range in concatenation

If set declaration is missing the interval flag, and user specifies an
element with either prefix or range, then bail out.

Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1592
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
3 years agoscanner: allow prefix in ip6 scope
Florian Westphal [Wed, 6 Jul 2022 21:49:21 +0000 (23:49 +0200)] 
scanner: allow prefix in ip6 scope

'ip6 prefix' is valid syntax, so make sure scanner recognizes it
also in ip6 context.

Also add test case.

Fixes: a67fce7ffe7e ("scanner: nat: Move to own scope")
Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1619
Signed-off-by: Florian Westphal <fw@strlen.de>
3 years agosegtree: fix map listing with interface wildcard
Pablo Neira Ayuso [Mon, 27 Jun 2022 10:54:23 +0000 (12:54 +0200)] 
segtree: fix map listing with interface wildcard

 # nft -f - <<'EOF'
 table inet filter {
    chain INPUT {
        iifname vmap {
            "eth0" : jump input_lan,
            "wg*" : jump input_vpn
        }
    }
    chain input_lan {}
    chain input_vpn {}
 }
 EOF
 # nft list ruleset
 nft: segtree.c:578: interval_map_decompose: Assertion `low->len / 8 > 0' failed.

Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1617
Fixes: 5e393ea1fc0a ("segtree: add string "range" reversal support")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
3 years agoscanner: don't pop active flex scanner scope
Florian Westphal [Thu, 23 Jun 2022 17:56:19 +0000 (19:56 +0200)] 
scanner: don't pop active flex scanner scope

Currently we can pop a flex scope that is still active, i.e. the
scanner_pop_start_cond() for the scope has not been done.

Example:
  counter ipsec out ip daddr 192.168.1.2 counter name "ipsec_out"

Here, parser fails because 'daddr' is parsed as STRING, not as DADDR token.

Bug is as follows:
COUNTER changes scope to COUNTER. (COUNTER).
Next, IPSEC scope gets pushed, stack is: COUNTER, IPSEC.

Then, the 'COUNTER' scope close happens.  Because active scope has changed,
we cannot pop (we would pop the 'ipsec' scope in flex).
The pop operation gets delayed accordingly.

Next, IP gets pushed, stack is: COUNTER, IPSEC, IP, plus the information
that one scope closure/pop was delayed.

Then, the IP scope is closed.  Because a pop operation was delayed, we pop again,
which brings us back to COUNTER state.

This is bogus: The pop operation CANNOT be done yet, because the ipsec scope
is still open, but the existing code lacks the information to detect this.

After popping the IP scope, we must remain in IPSEC scope until bison
parser calls scanner_pop_start_cond(, IPSEC).

This adds a counter per flex scope so that we can detect this case.
In above case, after the IP scope gets closed, the "new" (previous)
scope (IPSEC) will be treated as active and its close is attempted again
on the next call to scanner_pop_start_cond().

After this patch, transition in above rule is:

push counter (COUNTER)
push IPSEC (COUNTER, IPSEC)
pop COUNTER (delayed: COUNTER, IPSEC, pending-pop for COUNTER),
push IP (COUNTER, IPSEC, IP, pending-pop for COUNTER)
pop IP (COUNTER, IPSEC, pending-pop for COUNTER)
parse DADDR (we're in IPSEC scope, its valid token)
pop IPSEC (pops all remaining scopes).

We could also resurrect the commit:
"scanner: flags: move to own scope", the test case passes with the
new scope closure logic.

Fixes: bff106c5b277 ("scanner: add support for scope nesting")
Signed-off-by: Florian Westphal <fw@strlen.de>
3 years agoparser: add missing synproxy scope closure
Florian Westphal [Thu, 23 Jun 2022 16:28:14 +0000 (18:28 +0200)] 
parser: add missing synproxy scope closure

Fixes: 232f2c3287fc ("scanner: synproxy: Move to own scope")
Signed-off-by: Florian Westphal <fw@strlen.de>
3 years agotests/py: Add a test for failing ipsec after counter
Phil Sutter [Thu, 23 Jun 2022 14:28:42 +0000 (16:28 +0200)] 
tests/py: Add a test for failing ipsec after counter

This is a bug in parser/scanner due to scoping:

| Error: syntax error, unexpected string, expecting saddr or daddr
| add rule ip ipsec-ip4 ipsec-forw counter ipsec out ip daddr 192.168.1.2
|                                                       ^^^^^

Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Florian Westphal <fw@strlen.de>
3 years agoevaluate: fix segfault when adding elements to invalid set
Peter Tirsek [Sun, 26 Jun 2022 05:47:07 +0000 (00:47 -0500)] 
evaluate: fix segfault when adding elements to invalid set

Adding elements to a set or map with an invalid definition causes nft to
segfault. The following nftables.conf triggers the crash:

    flush ruleset
    create table inet filter
    set inet filter foo {}
    add element inet filter foo { foobar }

Simply parsing and checking the config will trigger it:

    $ nft -c -f nftables.conf.crash
    Segmentation fault

The error in the set/map definition is correctly caught and queued, but
because the set is invalid and does not contain a key type, adding to it
causes a NULL pointer dereference of set->key within setelem_evaluate().

I don't think it's necessary to queue another error since the underlying
problem is correctly detected and reported when parsing the definition
of the set. Simply checking the validity of set->key before using it
seems to fix it, causing the error in the definition of the set to be
reported properly. The element type error isn't caught, but that seems
reasonable since the key type is invalid or unknown anyway:

    $ ./nft -c -f ~/nftables.conf.crash
    /home/pti/nftables.conf.crash:3:21-21: Error: set definition does not specify key
    set inet filter foo {}
                        ^

[ Add tests to cover this case --pablo ]

Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1597
Signed-off-by: Peter Tirsek <peter@tirsek.com>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
3 years agomnl: store netlink error location for set elements
Pablo Neira Ayuso [Mon, 27 Jun 2022 08:20:46 +0000 (10:20 +0200)] 
mnl: store netlink error location for set elements

Store set element location in the per-command netlink error location
array.  This allows for fine grain error reporting when adding and
deleting elements.

 # nft -f test.nft
 test.nft:5:4-20: Error: Could not process rule: File exists
                        00:01:45:09:0b:26 : drop,
                        ^^^^^^^^^^^^^^^^^

test.nft contains a large map with one redundant entry.

Thus, users do not have to find the needle in the stack.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
3 years agosrc: remove NFT_NLATTR_LOC_MAX limit for netlink location error reporting
Pablo Neira Ayuso [Mon, 27 Jun 2022 08:16:48 +0000 (10:16 +0200)] 
src: remove NFT_NLATTR_LOC_MAX limit for netlink location error reporting

Set might have more than 16 elements, use a runtime array to store
netlink error location.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
3 years agoparser_bison: fix error location for set elements
Pablo Neira Ayuso [Mon, 27 Jun 2022 08:15:30 +0000 (10:15 +0200)] 
parser_bison: fix error location for set elements

opt_newline causes interfere since it points to the previous line.
Refer to set element key for error reporting.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
3 years agointervals: check for EXPR_F_REMOVE in case of element mismatch
Pablo Neira Ayuso [Thu, 23 Jun 2022 16:41:21 +0000 (18:41 +0200)] 
intervals: check for EXPR_F_REMOVE in case of element mismatch

If auto-merge is disable and element to be deleted finds no exact
matching, then bail out.

Fixes: 3e8d934e4f72 ("intervals: support to partial deletion with automerge")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
3 years agointervals: fix crash when trying to remove element in empty set
Pablo Neira Ayuso [Thu, 23 Jun 2022 12:20:17 +0000 (14:20 +0200)] 
intervals: fix crash when trying to remove element in empty set

The set deletion routine expects an initialized set, otherwise it crashes.

Fixes: 3e8d934e4f72 ("intervals: support to partial deletion with automerge")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
3 years agonetlink_delinearize: memleak when parsing concatenation data
Pablo Neira Ayuso [Thu, 23 Jun 2022 18:07:38 +0000 (20:07 +0200)] 
netlink_delinearize: memleak when parsing concatenation data

netlink_get_register() clones the expression in the register,
release after using it.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
3 years agolibnftables: release top level scope
Pablo Neira Ayuso [Fri, 17 Jun 2022 17:33:53 +0000 (19:33 +0200)] 
libnftables: release top level scope

Otherwise bogus variable redefinition are reported via -o/--optimize:

  redefinition.conf:5:8-21: Error: redefinition of symbol 'interface_inet'
  define interface_inet = enp5s0
         ^^^^^^^^^^^^^^

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
3 years agooptimize: limit statement is not supported yet
Pablo Neira Ayuso [Fri, 17 Jun 2022 17:03:05 +0000 (19:03 +0200)] 
optimize: limit statement is not supported yet

Revert support for limit statement, the limit statement is stateful and
it applies a ratelimit per rule, transformation for merging rules with
the limit statement needs to use anonymous sets with statements.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
3 years agooptimize: assume verdict is same when rules have no verdict
Pablo Neira Ayuso [Fri, 17 Jun 2022 16:51:40 +0000 (18:51 +0200)] 
optimize: assume verdict is same when rules have no verdict

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
3 years agooptimize: only merge OP_IMPLICIT and OP_EQ relational
Pablo Neira Ayuso [Fri, 17 Jun 2022 16:17:49 +0000 (18:17 +0200)] 
optimize: only merge OP_IMPLICIT and OP_EQ relational

Add test to cover this case.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
3 years agotests: shell: run -c -o on ruleset
Pablo Neira Ayuso [Fri, 17 Jun 2022 16:10:19 +0000 (18:10 +0200)] 
tests: shell: run -c -o on ruleset

Just run -o/--optimize on a ruleset.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
3 years agooptimize: add unsupported statement
Pablo Neira Ayuso [Fri, 17 Jun 2022 15:49:59 +0000 (17:49 +0200)] 
optimize: add unsupported statement

Do not try to merge rules with unsupported statements. This patch adds a
dummy unsupported statement which is included in the statement
collection and the rule vs statement matrix.

When looking for possible rule mergers, rules using unsupported
statements are discarded, otherwise bogus rule mergers might occur.

Note that __stmt_type_eq() already returns false for unsupported
statements.

Add a test using meta mark statement, which is not yet supported.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
3 years agooptimize: add hash expression support
Pablo Neira Ayuso [Fri, 17 Jun 2022 16:05:15 +0000 (18:05 +0200)] 
optimize: add hash expression support

Extend expr_cmp() to compare hash expressions used in relational.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
3 years agooptimize: add numgen expression support
Pablo Neira Ayuso [Fri, 17 Jun 2022 16:02:56 +0000 (18:02 +0200)] 
optimize: add numgen expression support

Extend expr_cmp() to compare numgen expressions used in relational.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
3 years agooptimize: add binop expression support
Pablo Neira Ayuso [Fri, 17 Jun 2022 15:48:43 +0000 (17:48 +0200)] 
optimize: add binop expression support

Do recursive call using left expression in the binop expression tree to
search for the primary expression.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
3 years agooptimize: add fib expression support
Pablo Neira Ayuso [Fri, 17 Jun 2022 15:48:32 +0000 (17:48 +0200)] 
optimize: add fib expression support

Extend expr_cmp() to compare fib expressions used in relational.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
3 years agooptimize: add xfrm expression support
Pablo Neira Ayuso [Fri, 17 Jun 2022 15:47:53 +0000 (17:47 +0200)] 
optimize: add xfrm expression support

Extend expr_cmp() to compare xfrm expressions used in relational.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
3 years agooptimize: add osf expression support
Pablo Neira Ayuso [Fri, 17 Jun 2022 15:47:15 +0000 (17:47 +0200)] 
optimize: add osf expression support

Extend expr_cmp() to compare osf expressions used in relational.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
3 years agooptimize: fix verdict map merging
Pablo Neira Ayuso [Fri, 17 Jun 2022 15:42:58 +0000 (17:42 +0200)] 
optimize: fix verdict map merging

Skip comparison when collecting the statement and building the rule vs
statement matrix. Compare verdict type when merging rules.

When infering rule mergers, honor the STMT_VERDICT with map (ie. vmap).

Fixes: 561aa3cfa8da ("optimize: merge verdict maps with same lookup key")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
3 years agooptimize: fix reject statement
Pablo Neira Ayuso [Fri, 17 Jun 2022 15:28:00 +0000 (17:28 +0200)] 
optimize: fix reject statement

Add missing code to the statement collection routine. Compare reject
expressions when available. Add tests/shell.

Fixes: fb298877ece2 ("src: add ruleset optimization infrastructure")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>