]> git.ipfire.org Git - thirdparty/nftables.git/log
thirdparty/nftables.git
6 years agoevaluate: use-after-free in implicit set
Pablo Neira Ayuso [Fri, 7 Jun 2019 10:37:00 +0000 (12:37 +0200)] 
evaluate: use-after-free in implicit set

 # cat example.nft
 table inet test {
        chain test {
                ip daddr { 2.2.2.2, 4.4.4.4} counter accept
        }
 }
 # valgrind nft -f example.nft

valgrind reports:

==2272== Invalid read of size 4
==2272==    at 0x4E612A5: expr_free (expression.c:86)
==2272==    by 0x4E58EA2: set_free (rule.c:367)
==2272==    by 0x4E612DA: expr_destroy (expression.c:79)
==2272==    by 0x4E612DA: expr_free (expression.c:93)
==2272==    by 0x4E612DA: expr_destroy (expression.c:79)
==2272==    by 0x4E612DA: expr_free (expression.c:93)
==2272==    by 0x4E5D7E7: stmt_free (statement.c:50)
==2272==    by 0x4E5D8B7: stmt_list_free (statement.c:60)
==2272==    by 0x4E590FF: rule_free (rule.c:610)
==2272==    by 0x4E5C094: cmd_free (rule.c:1420)
==2272==    by 0x4E7E7EF: nft_run_cmd_from_filename (libnftables.c:490)
==2272==    by 0x109A53: main (main.c:310)
==2272==  Address 0x65d94c8 is 56 bytes inside a block of size 128 free'd
==2272==    at 0x4C2CDDB: free (vg_replace_malloc.c:530)
==2272==    by 0x4E6143C: mapping_expr_destroy (expression.c:966)
==2272==    by 0x4E612DA: expr_destroy (expression.c:79)
==2272==    by 0x4E612DA: expr_free (expression.c:93)
==2272==    by 0x4E5D7E7: stmt_free (statement.c:50)
==2272==    by 0x4E5D8B7: stmt_list_free (statement.c:60)
==2272==    by 0x4E590FF: rule_free (rule.c:610)
==2272==    by 0x4E5C094: cmd_free (rule.c:1420)
==2272==    by 0x4E7E7EF: nft_run_cmd_from_filename (libnftables.c:490)
==2272==    by 0x109A53: main (main.c:310)
==2272==  Block was alloc'd at
==2272==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==2272==    by 0x4E79248: xmalloc (utils.c:36)
==2272==    by 0x4E7932D: xzalloc (utils.c:65)
==2272==    by 0x4E60690: expr_alloc (expression.c:45)
==2272==    by 0x4E68B1D: payload_expr_alloc (payload.c:159)
==2272==    by 0x4E91013: nft_parse (parser_bison.y:4242)
==2272==    by 0x4E7E722: nft_parse_bison_filename (libnftables.c:374)
==2272==    by 0x4E7E722: nft_run_cmd_from_filename (libnftables.c:471)
==2272==    by 0x109A53: main (main.c:310)

Fixes: cc7b37d18a68 ("src: Interpret OP_NEQ against a set as OP_LOOKUP")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agorule: ensure cache consistency
Pablo Neira Ayuso [Thu, 6 Jun 2019 12:09:45 +0000 (14:09 +0200)] 
rule: ensure cache consistency

Check for generation ID after the cache is populated. In case of
interference, release the inconsistent cache and retry.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agosrc: generation ID is 32-bit long
Pablo Neira Ayuso [Fri, 7 Jun 2019 09:06:56 +0000 (11:06 +0200)] 
src: generation ID is 32-bit long

Update mnl_genid_get() to return 32-bit long generation ID. Add
nft_genid_u16() which allows us to catch ruleset updates from the
netlink dump path via 16-bit long nfnetlink resource ID field.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agolibnftables: Drop cache in error case
Phil Sutter [Tue, 4 Jun 2019 17:31:51 +0000 (19:31 +0200)] 
libnftables: Drop cache in error case

If a transaction is rejected by the kernel (for instance due to a
semantic error), cache contents are potentially invalid. Release the
cache in that case to avoid the inconsistency.

The problem is easy to reproduce in an interactive session:

| nft> list ruleset
| table ip t {
|  chain c {
|  }
| }
| nft> flush ruleset; add rule ip t c accept
| Error: No such file or directory
| flush ruleset; add rule ip t c accept
|                            ^
| nft> list ruleset
| nft>

Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agosrc: Fix cache_flush() in cache_needs_more() logic
Phil Sutter [Tue, 4 Jun 2019 17:31:49 +0000 (19:31 +0200)] 
src: Fix cache_flush() in cache_needs_more() logic

Commit 34a20645d54fa enabled cache updates depending on command causing
it. As a side-effect, this disabled measures in cache_flush() preventing
a later cache update. Re-establish this by setting cache->cmd in
addition to cache->genid after dropping cache entries.

While being at it, set cache->cmd in cache_release() as well. This
shouldn't be necessary since zeroing cache->genid should suffice for
cache_update(), but better be consistent (and future-proof) here.

Fixes: eeda228c2d17 ("src: update cache if cmd is more specific")
Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agosrc: single cache_update() call to build cache before evaluation
Pablo Neira Ayuso [Tue, 4 Jun 2019 10:02:18 +0000 (12:02 +0200)] 
src: single cache_update() call to build cache before evaluation

This patch allows us to make one single cache_update() call. Thus, there
is not need to rebuild an incomplete cache from the middle of the batch
processing.

Note that nft_run_cmd_from_filename() does not need a full netlink dump
to build the cache anymore, this should speed nft -f with incremental
updates and very large rulesets.

cache_evaluate() calculates the netlink dump to populate the cache that
this batch needs.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agosrc: Display parser and evaluate errors in one shot
Pablo Neira Ayuso [Tue, 4 Jun 2019 09:03:06 +0000 (11:03 +0200)] 
src: Display parser and evaluate errors in one shot

This patch restores 61236968b7a1 ("parser: evaluate commands immediately
after parsing") following a different approach.

In this patch, the evaluation phase is done if the parsing phase fails,
hence the user gets parsing and evaluation errors in one shot, which is
the purpose of 61236968b7a1.

Note that evaluation errors are now shown after parser errors, the example
available in 61236968b7a1 displays with this patch the following error:

 # nft -f /tmp/bad.nft
 /tmp/bad.nft:3:32-32: Error: syntax error, unexpected newline
 add rule filter input tcp dport
                                ^
 /tmp/bad.nft:5:37-41: Error: syntax error, unexpected dport, expecting end of file or newline or semicolon
 add rule filter input tcp dport tcp dport
                                     ^^^^^
 /tmp/bad.nft:4:33-35: Error: datatype mismatch, expected internet network service, expression has type Internet protocol
 add rule filter input tcp dport tcp
                       ~~~~~~~~~ ^^^

So evaluation pointing to line 4 happens after line error reporting
generated by the parser that points to line 3, while 61236968b7a1 was
showing errors per line in order. As a future work, we can sort the
error reporting list to restore exactly the same behaviour.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agosrc: perform evaluation after parsing
Pablo Neira Ayuso [Wed, 5 Jun 2019 15:07:42 +0000 (17:07 +0200)] 
src: perform evaluation after parsing

Since 61236968b7a1 ("parser: evaluate commands immediately after
parsing"), evaluation is invoked from the parsing phase in order to
improve error reporting.

However, this approach is problematic from the cache perspective since
we don't know if a full or partial netlink dump from the kernel is
needed. If the number of objects in the kernel is significant, the
netlink dump operation to build the cache may significantly slow down
commands.

This patch moves the evaluation phase after the parsing phase as a
preparation update to allow for a better strategy to build the cache.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agosrc: dynamic input_descriptor allocation
Pablo Neira Ayuso [Wed, 5 Jun 2019 09:56:11 +0000 (11:56 +0200)] 
src: dynamic input_descriptor allocation

This patch introduces the input descriptor list, that stores the
existing input descriptor objects. These objects are now dynamically
allocated and release from scanner_destroy() path.

Follow up patches that decouple the parsing and the evaluation phases
require this for error reporting as described by b14572f72aac ("erec:
Fix input descriptors for included files"), this patch partially reverts
such partial.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agotests/py: Support JSON validation
Phil Sutter [Mon, 27 May 2019 11:36:42 +0000 (13:36 +0200)] 
tests/py: Support JSON validation

Introduce a new flag -s/--schema to nft-test.py which enables validation
of any JSON input and output against our schema.

Make use of traceback module to get more details if validation fails.

Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agopy: Implement JSON validation in nftables module
Phil Sutter [Mon, 27 May 2019 11:36:41 +0000 (13:36 +0200)] 
py: Implement JSON validation in nftables module

Using jsonschema it is possible to validate any JSON input to make sure
it formally conforms with libnftables JSON API requirements.

Implement a simple validator class for use within a new Nftables class
method 'json_validate' and ship a minimal schema definition along with
the package.

Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agomnl: Simplify mnl_batch_talk()
Phil Sutter [Fri, 31 May 2019 14:17:43 +0000 (16:17 +0200)] 
mnl: Simplify mnl_batch_talk()

By mimicking mnl_nft_event_listener() code, mnl_batch_talk() may be
simplified quite a bit:

* Turn the conditional loop into an unconditional one.
* Call select() at loop start, which merges the two call sites.
* Check readfds content after select() returned instead of in loop
  condition - if fd is not set, break to return error state stored in
  'err' variable.
* Old code checked that select() return code is > 0, but that was
  redundant: if FD_ISSET() returns true, select return code was 1.
* Move 'nlh' helper variable definition into error handling block, it is
  not used outside of it.

Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agomnl: Initialize fd_set before select(), not after
Phil Sutter [Fri, 31 May 2019 14:17:42 +0000 (16:17 +0200)] 
mnl: Initialize fd_set before select(), not after

Calling FD_SET() in between return of select() and call to FD_ISSET()
effectively renders the whole thing useless: FD_ISSET() will always
return true no matter what select() actually did.

Fixes: a72315d2bad47 ("src: add rule batching support")
Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agotests/shell: Test large transaction with echo output
Phil Sutter [Wed, 29 May 2019 13:13:46 +0000 (15:13 +0200)] 
tests/shell: Test large transaction with echo output

This reliably triggered ENOBUFS condition in mnl_batch_talk(). With the
past changes, it passes even after increasing the number of rules to
300k.

Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agoerec: remove double \n on error when internal_netlink is used
Pablo Neira Ayuso [Thu, 30 May 2019 10:49:30 +0000 (12:49 +0200)] 
erec: remove double \n on error when internal_netlink is used

Remove double empty line linebreak when printing internal errors.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agomnl: mnl_batch_talk() returns -1 on internal netlink errors
Pablo Neira Ayuso [Wed, 29 May 2019 18:40:39 +0000 (20:40 +0200)] 
mnl: mnl_batch_talk() returns -1 on internal netlink errors

Display an error in case internal netlink plumbing hits problems.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agomnl: estimate receiver buffer size
Pablo Neira Ayuso [Wed, 29 May 2019 18:23:23 +0000 (20:23 +0200)] 
mnl: estimate receiver buffer size

Set a receiver buffer size based on the number of commands and the
average message size, this is useful for the --echo option in order to
avoid ENOBUFS errors.

On the kernel side, each skbuff consumes truesize from the socket queue
(although it uses NLMSG_GOODSIZE to allocate it), which is approximately
four times the estimated size per message that we get in turn for each
echo message to ensure enough receiver buffer space.

We could also explore increasing the buffer and retry if
mnl_nft_socket_sendmsg() hits ENOBUFS if we ever hit this problem again.

Reported-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agomnl: add mnl_nft_batch_to_msg()
Pablo Neira Ayuso [Thu, 30 May 2019 10:21:28 +0000 (12:21 +0200)] 
mnl: add mnl_nft_batch_to_msg()

This function transforms the batch into a msghdr object.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agomnl: call mnl_set_sndbuffer() from mnl_batch_talk()
Pablo Neira Ayuso [Wed, 29 May 2019 16:24:12 +0000 (18:24 +0200)] 
mnl: call mnl_set_sndbuffer() from mnl_batch_talk()

Instead of mnl_nft_socket_sendmsg(), just a cleanup.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agomnl: mnl_set_rcvbuffer() skips buffer size update if it is too small
Pablo Neira Ayuso [Thu, 30 May 2019 10:52:49 +0000 (12:52 +0200)] 
mnl: mnl_set_rcvbuffer() skips buffer size update if it is too small

Check for existing buffer size, if this is larger than the newer buffer
size, skip this size update.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agomnl: add mnl_set_rcvbuffer() and use it
Pablo Neira Ayuso [Wed, 29 May 2019 15:27:04 +0000 (17:27 +0200)] 
mnl: add mnl_set_rcvbuffer() and use it

This new function allows us to set the netlink receiver buffer.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agotests: replace single element sets
Pablo Neira Ayuso [Fri, 24 May 2019 19:12:56 +0000 (21:12 +0200)] 
tests: replace single element sets

Add at least two elements to sets.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agotests: json_echo: convert to py3
Shekhar Sharma [Tue, 28 May 2019 00:36:53 +0000 (06:06 +0530)] 
tests: json_echo: convert to py3

This patch converts the run-test.py file to run on both python3 and python2.

Signed-off-by: Shekhar Sharma <shekhar250198@gmail.com>
Acked-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agoparser_json: Fix and simplify verdict expression parsing
Phil Sutter [Mon, 27 May 2019 11:37:00 +0000 (13:37 +0200)] 
parser_json: Fix and simplify verdict expression parsing

Parsing of the "target" property was flawed in two ways:

* The value was extracted twice. Drop the first unconditional one.
* Expression allocation required since commit f1e8a129ee428 was broken,
  The expression was allocated only if the property was not present.

Fixes: f1e8a129ee428 ("src: Introduce chain_expr in jump and goto statements")
Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agosrc: add cache_is_complete() and cache_is_updated()
Pablo Neira Ayuso [Fri, 24 May 2019 18:46:18 +0000 (20:46 +0200)] 
src: add cache_is_complete() and cache_is_updated()

Just a few functions to help clarify cache update logic.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Acked-by: Phil Sutter <phil@nwl.cc>
6 years agosrc: Allow goto and jump to a variable
Fernando Fernandez Mancera [Fri, 24 May 2019 13:06:50 +0000 (15:06 +0200)] 
src: Allow goto and jump to a variable

This patch introduces the use of nft input files variables in 'jump' and 'goto'
statements, e.g.

define dest = ber

add table ip foo
add chain ip foo bar {type filter hook input priority 0;}
add chain ip foo ber
add rule ip foo ber counter
add rule ip foo bar jump $dest

table ip foo {
        chain bar {
                type filter hook input priority filter; policy accept;
                jump ber
        }

        chain ber {
                counter packets 71 bytes 6664
        }
}

Signed-off-by: Fernando Fernandez Mancera <ffmancera@riseup.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agosrc: Introduce chain_expr in jump and goto statements
Fernando Fernandez Mancera [Fri, 24 May 2019 13:06:47 +0000 (15:06 +0200)] 
src: Introduce chain_expr in jump and goto statements

Introduce expressions as a chain in jump and goto statements.
This is going to be used to support variables as a chain in the
following patches.

Signed-off-by: Fernando Fernandez Mancera <ffmancera@riseup.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agosrc: support for arp sender and target ethernet and IPv4 addresses
Pablo Neira Ayuso [Wed, 22 May 2019 20:06:16 +0000 (22:06 +0200)] 
src: support for arp sender and target ethernet and IPv4 addresses

 # nft add table arp x
 # nft add chain arp x y { type filter hook input priority 0\; }
 # nft add rule arp x y arp saddr ip 192.168.2.1 counter

Testing this:

 # ip neigh flush dev eth0
 # ping 8.8.8.8
 # nft list ruleset
 table arp x {
        chain y {
                type filter hook input priority filter; policy accept;
                arp saddr ip 192.168.2.1 counter packets 1 bytes 46
        }
 }

You can also specify hardware sender address, eg.

 # nft add rule arp x y arp saddr ether aa:bb:cc:aa:bb:cc drop counter

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agosrc: update cache if cmd is more specific
Eric Garver [Wed, 22 May 2019 19:44:04 +0000 (21:44 +0200)] 
src: update cache if cmd is more specific

If we've done a partial fetch of the cache and the genid is the same the
cache update will be skipped without fetching the needed items. This
change flushes the cache if the new request is more specific than the
current cache - forcing a cache update which includes the needed items.

Introduces a simple scoring system which reflects how
cache_init_objects() looks at the current command to decide if it is
finished already or not. Then use that in cache_needs_more(): If current
command's score is higher than old command's, cache needs an update.

Fixes: 816d8c7659c1 ("Support 'add/insert rule index <IDX>'")
Signed-off-by: Eric Garver <eric@garver.life>
Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agoRevert "tests: shell: avoid single-value anon sets"
Pablo Neira Ayuso [Fri, 24 May 2019 18:55:27 +0000 (20:55 +0200)] 
Revert "tests: shell: avoid single-value anon sets"

This reverts commit b7459b0c854fc7a0d6cd86151b81035a8edf8e63.

6 years agoRevert "tests: py: remove single-value-anon-set test cases"
Pablo Neira Ayuso [Fri, 24 May 2019 18:54:37 +0000 (20:54 +0200)] 
Revert "tests: py: remove single-value-anon-set test cases"

This reverts commit d03bcb669c0c645190df9bd166f53380bcac7862.

6 years agotests: py: remove single-value-anon-set test cases
Florian Westphal [Fri, 17 May 2019 10:46:31 +0000 (12:46 +0200)] 
tests: py: remove single-value-anon-set test cases

future change will rewrite all single-element anon sets to a cmp op.
Retain a few test cases to later check that the rewrite is correct, but
remove all others.

Signed-off-by: Florian Westphal <fw@strlen.de>
6 years agotests: shell: avoid single-value anon sets
Florian Westphal [Fri, 17 May 2019 11:57:23 +0000 (13:57 +0200)] 
tests: shell: avoid single-value anon sets

Future change is going to auto-change them to simple compare
ops rather than lookup in set with only one element.

Signed-off-by: Florian Westphal <fw@strlen.de>
6 years agoinclude: refresh nf_tables.h cached copy
Pablo Neira Ayuso [Sun, 12 May 2019 19:08:25 +0000 (21:08 +0200)] 
include: refresh nf_tables.h cached copy

Fetch fix for NFT_LOGLEVEL_MAX.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agosrc: use definitions in include/linux/netfilter/nf_tables.h
Pablo Neira Ayuso [Sun, 12 May 2019 18:52:43 +0000 (20:52 +0200)] 
src: use definitions in include/linux/netfilter/nf_tables.h

Use NFT_LOGLEVEL_* definitions in UAPI.

Make an internal definition of NFT_OSF_F_VERSION, this was originally
defined in the UAPI header in the initial patch version, however, this
is not available anymore.

Add a bison rule to deal with the timeout case.

Otherwise, compilation breaks.

Fixes: d3869cae9d62 ("include: refresh nf_tables.h cached copy")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agopy: fix missing decode/encode of strings
Eric Garver [Fri, 10 May 2019 12:29:47 +0000 (08:29 -0400)] 
py: fix missing decode/encode of strings

When calling ffi functions, if the string is unicode we need to convert
to utf-8. Then convert back for any output we receive.

Fixes: 586ad210368b7 ("libnftables: Implement JSON parser")
Signed-off-by: Eric Garver <eric@garver.life>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agoinclude: refresh nf_tables.h cached copy
Pablo Neira Ayuso [Sun, 12 May 2019 17:54:41 +0000 (19:54 +0200)] 
include: refresh nf_tables.h cached copy

Refresh it to fetch what we have in 5.2-rc.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agoRevert "proto: support for draft-ietf-tsvwg-le-phb-10.txt"
Pablo Neira Ayuso [Thu, 9 May 2019 15:07:07 +0000 (17:07 +0200)] 
Revert "proto: support for draft-ietf-tsvwg-le-phb-10.txt"

This reverts commit 55715486efba424e97361c81d8d47e854f45a5a6.

This breaks tests/py.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agotests/py: Fix JSON expected output for icmpv6 code values
Phil Sutter [Thu, 9 May 2019 11:35:45 +0000 (13:35 +0200)] 
tests/py: Fix JSON expected output for icmpv6 code values

Reverse translation is happening for values which are known, even if
they are part of a range. In contrast to standard output, this is OK
because in JSON lower and upper bounds are properties and there is no
ambiguity if names contain a dash.

Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agotests/py: Fix JSON expected output after expr merge change
Phil Sutter [Thu, 9 May 2019 11:35:44 +0000 (13:35 +0200)] 
tests/py: Fix JSON expected output after expr merge change

Looks like original patch missed this one.

Fixes: 88ba0c92754d8 ("tests: fix up expected payloads after expr merge change")
Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agoparser_json: Fix ct timeout object support
Phil Sutter [Thu, 9 May 2019 11:35:42 +0000 (13:35 +0200)] 
parser_json: Fix ct timeout object support

Seems like it wasn't possible to add or list ct timeout objects.

Fixes: c82a26ebf7e9f ("json: Add ct timeout support")
Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agojson: Fix tproxy support regarding latest changes
Phil Sutter [Thu, 9 May 2019 11:35:41 +0000 (13:35 +0200)] 
json: Fix tproxy support regarding latest changes

Family may be specified also if no address is given at the same time,
make parser/printer tolerant to that. Also fix for missing/incorrect
JSON equivalents in tests/py.

While being at it, fix two issues in non-JSON tests:

* Ruleset is printed in numeric mode, so use 'l4proto 6' instead of
  'l4proto tcp' in rules to avoid having to specify expected output for
  that unrelated bit.

* In ip and ip6 family tables, family parameter is not deserialized on
  output.

Fixes: 3edb96200690b ("parser_bison: missing tproxy syntax with port only for inet family")
Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agotests/py: Fix JSON equivalents of osf tests
Phil Sutter [Thu, 9 May 2019 11:35:40 +0000 (13:35 +0200)] 
tests/py: Fix JSON equivalents of osf tests

Fixes: 12adf747a3f62 ("tests: py: add osf tests with versions")
Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agonetlink: Fix printing of zero-length prefixes
Phil Sutter [Thu, 9 May 2019 11:35:39 +0000 (13:35 +0200)] 
netlink: Fix printing of zero-length prefixes

When delinearizing, an all-zero mask didn't qualify as prefix. Therefore
a statement:

| ip daddr 0.0.0.0/0

would be printed as:

| ip daddr & 0.0.0.0 == 0.0.0.0

To fix this, expr_mask_is_prefix() must return true if the initial 1-bit
search fails (the given value must be zero in this case). Additionally,
a shortcut is needed in conversion algorithm of expr_mask_to_prefix()
to not turn the zero prefix into a 1 by accident.

Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agoparser_json: Fix igmp support
Phil Sutter [Thu, 9 May 2019 11:35:38 +0000 (13:35 +0200)] 
parser_json: Fix igmp support

Parser didn't know about that protocol, also testsuite bits were
missing.

Fixes: bad27ca386276 ("src: add igmp support")
Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agojson: Support nat in inet family
Phil Sutter [Thu, 9 May 2019 11:35:37 +0000 (13:35 +0200)] 
json: Support nat in inet family

Add the missing bits to JSON parser, printer, man page and testsuite.

Fixes: fbe27464dee45 ("src: add nat support for the inet family")
Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agodoc: Review man page synopses
Phil Sutter [Tue, 7 May 2019 13:23:50 +0000 (15:23 +0200)] 
doc: Review man page synopses

Fix use of font typefaces:

- *bold* for terminals
- 'italic' for non-terminals
- plain for meta-characters

Apart from that:

* Variable definitions require an equals sign
* 'auto-merge' option in set spec does not take a parameter
* List header fields in payload expressions instead of unexplained
  placeholder
* Introduce non-terminals in some places to avoid repetitions or clarify
  syntax
* Fix syntax for ip6 header expresssion example
* Reorganize ct expression synopsis into four parts:
  1) direction not allowed
  2) direction optional
  3) direction mandatory
  4) direction and family mandatory
* Add missing 'version' keyword to osf expression
* Clarify verdict statements example topic
* Add synopses for payload and exthdr statements
* Fix typo: differv -> diffserv
* Reorganize reject statement synopsis to point out which code type
  is required for which type arg
* Counter statement requires either one of 'packets' or 'bytes' args or
  both, none is an invalid variant
* Limit statement accepts a unit in burst, too
* Improve language in limit statement description a bit

Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Florian Westphal <fw@strlen.de>
6 years agopy: Fix gitignore of lib/ directory
Phil Sutter [Tue, 7 May 2019 13:21:45 +0000 (15:21 +0200)] 
py: Fix gitignore of lib/ directory

Pattern is not a PCRE one but merely a shell glob. Hence 'lib.*' matches
only 'lib.' prefix, not also 'lib'.

Fixes: bf9653667a39e ("python: installation of binding via make install")
Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Florian Westphal <fw@strlen.de>
6 years agogmputil: Add missing header for va_list
Rosen Penev [Fri, 3 May 2019 18:49:11 +0000 (11:49 -0700)] 
gmputil: Add missing header for va_list

Otherwise it errors with unknown type name when using uClibc.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agosrc: use UDATA defines from libnftnl
Phil Sutter [Thu, 25 Apr 2019 12:56:54 +0000 (14:56 +0200)] 
src: use UDATA defines from libnftnl

Userdata attribute names have been added to libnftnl, use them instead
of the local copy.

While being at it, rename udata_get_comment() in netlink_delinearize.c
and the callback it uses since the function is specific to rules. Also
integrate the existence check for NFTNL_RULE_USERDATA into it along with
the call to nftnl_rule_get_data().

Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agoparser_json: fix crash on add rule to bad references
Eric Garver [Wed, 1 May 2019 16:34:45 +0000 (12:34 -0400)] 
parser_json: fix crash on add rule to bad references

Pass the location via the handle so the error leg in
rule_translate_index() can reference it. Applies to invalid references
to tables, chains, and indexes.

Fixes: 586ad210368b ("libnftables: Implement JSON parser")
Signed-off-by: Eric Garver <eric@garver.life>
Acked-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agoparser_json: fix off by one index on rule add/replace
Eric Garver [Wed, 1 May 2019 16:25:37 +0000 (12:25 -0400)] 
parser_json: fix off by one index on rule add/replace

We need to increment the index by one just as the CLI does.

Fixes: 586ad210368b7 ("libnftables: Implement JSON parser")
Signed-off-by: Eric Garver <eric@garver.life>
Acked-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agoproto: support for draft-ietf-tsvwg-le-phb-10.txt
Loganaden Velvindron [Wed, 1 May 2019 13:23:15 +0000 (17:23 +0400)] 
proto: support for draft-ietf-tsvwg-le-phb-10.txt

Upcoming dscp codepoint for background traffic of low precendence
such as bulk data transfers with low priority in time, non time-critical
backups, larger software updates, web search engines while gathering
information from web servers and so on.

Signed-off-by: Loganaden Velvindron <logan@cyberstorm.mu>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agoparser_json: default to unspecified l3proto for ct helper/timeout
Eric Garver [Thu, 2 May 2019 16:20:57 +0000 (12:20 -0400)] 
parser_json: default to unspecified l3proto for ct helper/timeout

As per the man page, if the user does not specify the l3proto it should
be derived from the table family.

Fixes: 586ad210368b ("libnftables: Implement JSON parser")
Signed-off-by: Eric Garver <eric@garver.life>
Acked-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agosrc: fix double free on xt stmt destruction
Florian Westphal [Tue, 23 Apr 2019 11:18:05 +0000 (13:18 +0200)] 
src: fix double free on xt stmt destruction

'nft monitor' dies with:
*** Error in `/sbin/nft': double free or corruption (fasttop): 0x000055f8ba57b750 ***

... when the iptables-nft test suite is running in parallel, because
xfree(stmt->xt.name) gets called twice.

Fixes: 4ac11b890fe870 ("src: missing destroy function in statement definitions")
Signed-off-by: Florian Westphal <fw@strlen.de>
Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agoparser_json: Fix parser for list maps command
Phil Sutter [Thu, 25 Apr 2019 12:59:43 +0000 (14:59 +0200)] 
parser_json: Fix parser for list maps command

Wrong callback in json_parse_cmd_list() for "maps" key.

Fixes: 586ad210368b7 ("libnftables: Implement JSON parser")
Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agoparser_json: Fix typo in ct timeout policy parser
Phil Sutter [Thu, 25 Apr 2019 12:59:42 +0000 (14:59 +0200)] 
parser_json: Fix typo in ct timeout policy parser

Parser expects an object in policy property and json_is_object() returns
true if given parameter is an object. The check was inverse by accident.

Fixes: c82a26ebf7e9f ("json: Add ct timeout support")
Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agotests/py: Fix error messages in chain_delete()
Phil Sutter [Thu, 25 Apr 2019 12:59:41 +0000 (14:59 +0200)] 
tests/py: Fix error messages in chain_delete()

Adding string and chain object is an illegal operation in Python.
Instead concatenate with cmd string since that contains all required
information already.

Fixes: 820fd08b5f1d4 ("tests/py: Review print statements in nft-test.py")
Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agotests: monitor: Adjust to changed events ordering
Phil Sutter [Thu, 25 Apr 2019 12:59:40 +0000 (14:59 +0200)] 
tests: monitor: Adjust to changed events ordering

When replacing a rule, kernel nowadays seems to report rule add event
before rule delete one. Since both events belong to the same
transaction, this is harmless per definition and merely needs adjustment
in expected output.

Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agoparser_json: Disallow ct helper as type to map to
Phil Sutter [Thu, 11 Apr 2019 10:38:51 +0000 (12:38 +0200)] 
parser_json: Disallow ct helper as type to map to

When creating a map, users may either map dtype:dtype or dtype:object.
In the second case, only counter, quota, limit and secmark is allowed by
bison, but JSON parser wasn't as strict, allowing ct helper as well.
Remove that to avoid undefined behaviour.

Fixes: 586ad210368b7 ("libnftables: Implement JSON parser")
Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agoparser_json: fix segfault in translating string to nft object
Laura Garcia Liebana [Thu, 11 Apr 2019 08:59:40 +0000 (10:59 +0200)] 
parser_json: fix segfault in translating string to nft object

A segmentation fault is produced when applying an input JSON file
like the following:

{"nftables": [
{ "add":
{"map":
{"family": "ip",
"name": "persistencia",
"table": "nftlb",
"type": "ipv4_addr",
"map": "mark",
"size": 65535,
"flags": ["timeout"],
"timeout": 44
}
}
}
]}

The captured error is:

 Program received signal SIGSEGV, Segmentation fault.
 #1  0x00007ffff7f734f9 in string_to_nft_object (str=0x55555555f410
  "mark") at parser_json.c:2513
 2513 if (!strcmp(str, obj_tbl[i]))

The obj_tbl array is allocated with the maximum element index even
if lower indexes are not populated, so it produces null pointer
items.

This patch ensures that the maximum number of possible indexes
but also the element is not comparing a null pointer.

Signed-off-by: Laura Garcia Liebana <nevola@gmail.com>
Signed-off-by: Florian Westphal <fw@strlen.de>
6 years agosegtree: fix memleak in interval_map_decompose()
Pablo Neira Ayuso [Tue, 9 Apr 2019 22:37:04 +0000 (00:37 +0200)] 
segtree: fix memleak in interval_map_decompose()

Do not inconditionally hold reference to start interval.

The handling depends on what kind of range expression we need to build,
either no range at all, a prefix or a plain range. Depending on the
case, we need to partially clone what we need from the expression to
avoid use-after-free.

This fixes valgrind reports that look like this, when listing rulesets:

==30018== 2,057,984 (1,028,992 direct, 1,028,992 indirect) bytes in 8,039 blocks are definitely lost in loss record 76 of 83
==30018==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==30018==    by 0x4E75978: xmalloc (utils.c:36)
==30018==    by 0x4E75A5D: xzalloc (utils.c:65)
==30018==    by 0x4E5CEC0: expr_alloc (expression.c:45)
==30018==    by 0x4E5D610: mapping_expr_alloc (expression.c:985)
==30018==    by 0x4E6A068: netlink_delinearize_setelem (netlink.c:810)
==30018==    by 0x5B51320: nftnl_set_elem_foreach (set_elem.c:673)
==30018==    by 0x4E6A2D5: netlink_list_setelems (netlink.c:864)
==30018==    by 0x4E56C76: cache_init_objects (rule.c:166)
==30018==    by 0x4E56C76: cache_init (rule.c:216)
==30018==    by 0x4E56C76: cache_update (rule.c:243)
==30018==    by 0x4E64530: cmd_evaluate_list (evaluate.c:3503)
==30018==    by 0x4E64530: cmd_evaluate (evaluate.c:3880)
==30018==    by 0x4E7D12F: nft_parse (parser_bison.y:798)
==30018==    by 0x4E7AB56: nft_parse_bison_buffer (libnftables.c:349)
==30018==    by 0x4E7AB56: nft_run_cmd_from_buffer (libnftables.c:394)

Reported-by: Václav Zindulka <vaclav.zindulka@tlapnet.cz>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agosrc: memleak in expressions
Pablo Neira Ayuso [Tue, 9 Apr 2019 20:58:27 +0000 (22:58 +0200)] 
src: memleak in expressions

Fix memleak in set element and hash expressions destroy path.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agosrc: add nat support for the inet family
Florian Westphal [Fri, 1 Feb 2019 23:36:51 +0000 (00:36 +0100)] 
src: add nat support for the inet family

consider a simple ip6 nat table:

table ip6 nat { chain output {
  type nat hook output priority 0; policy accept;
  dnat to dead:2::99
}

Now consider same ruleset, but using 'table inet nat':
nft now lacks context to determine address family to parse 'to $address'.

This adds code to make the following work:

table inet nat { [ .. ]
  # detect af from network protocol context:
  ip6 daddr dead::2::1 dnat to dead:2::99

  # use new dnat ip6 keyword:
  dnat ip6 to dead:2::99
  }

On list side, the keyword is only shown in the inet family, else the
short version (dnat to ...) is used as the family is redundant when the
table already mandates the ip protocol version supported.

Address mismatches such as

table ip6 { ..
dnat ip to 1.2.3.4

are detected/handled during the evaluation phase.

Signed-off-by: Florian Westphal <fw@strlen.de>
Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agofiles: pf.os: merge the signatures splitted by version
Fernando Fernandez Mancera [Wed, 27 Mar 2019 10:38:01 +0000 (11:38 +0100)] 
files: pf.os: merge the signatures splitted by version

In order to be able to identify the OS version we need to merge the signatures
split by version. eg.

65535:64:1:60:M*,N,W1,N,N,T: FreeBSD:4.7-4.11::FreeBSD 4.7-5.2
65535:64:1:60:M*,N,W1,N,N,T: FreeBSD:5.0-5.2::FreeBSD 4.7-5.2

65535:64:1:60:M*,N,W1,N,N,T: FreeBSD:4.7-5.2::FreeBSD 4.7-5.2

Fingerprints need to be unique to make this fit into the set/map
infrastructure for exact matches. Having multiples fingerprints with
same signature is a problem, since it forces users to add multiple
rules.

Signed-off-by: Fernando Fernandez Mancera <ffmancera@riseup.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agofiles: osf: update pf.os with newer OS fingerprints
Fernando Fernandez Mancera [Wed, 27 Mar 2019 10:38:00 +0000 (11:38 +0100)] 
files: osf: update pf.os with newer OS fingerprints

After notice that some fingerprints are outdated we have updated the most common
of them.

Signed-off-by: Fernando Fernandez Mancera <ffmancera@riseup.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agodoc: add osf version option to man page
Fernando Fernandez Mancera [Wed, 27 Mar 2019 10:37:59 +0000 (11:37 +0100)] 
doc: add osf version option to man page

Signed-off-by: Fernando Fernandez Mancera <ffmancera@riseup.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agotests: py: add osf tests with versions
Fernando Fernandez Mancera [Wed, 27 Mar 2019 10:37:58 +0000 (11:37 +0100)] 
tests: py: add osf tests with versions

Signed-off-by: Fernando Fernandez Mancera <ffmancera@riseup.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agojson: osf: add version json support
Fernando Fernandez Mancera [Wed, 27 Mar 2019 10:37:57 +0000 (11:37 +0100)] 
json: osf: add version json support

Signed-off-by: Fernando Fernandez Mancera <ffmancera@riseup.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agoosf: add version fingerprint support
Fernando Fernandez Mancera [Wed, 27 Mar 2019 10:37:56 +0000 (11:37 +0100)] 
osf: add version fingerprint support

Add support for version fingerprint in "osf" expression. Example:

table ip foo {
chain bar {
type filter hook input priority filter; policy accept;
osf ttl skip name "Linux"
osf ttl skip version "Linux:4.20"
}
}

Signed-off-by: Fernando Fernandez Mancera <ffmancera@riseup.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agodoc: update nft list plural form parameters
Fernando Fernandez Mancera [Fri, 5 Apr 2019 15:36:55 +0000 (17:36 +0200)] 
doc: update nft list plural form parameters

Signed-off-by: Fernando Fernandez Mancera <ffmancera@riseup.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agosrc: missing destroy function in statement definitions
Pablo Neira Ayuso [Fri, 5 Apr 2019 13:35:36 +0000 (15:35 +0200)] 
src: missing destroy function in statement definitions

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agoparser_bison: type_identifier string memleak
Pablo Neira Ayuso [Fri, 5 Apr 2019 12:09:56 +0000 (14:09 +0200)] 
parser_bison: type_identifier string memleak

==6297== 24 bytes in 3 blocks are definitely lost in loss record 2 of 13
==6297==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==6297==    by 0x56193B9: strdup (strdup.c:42)
==6297==    by 0x4E758BD: xstrdup (utils.c:75)
==6297==    by 0x4E7F9D3: nft_parse (parser_bison.y:1895)
==6297==    by 0x4E7AAE1: nft_parse_bison_filename (libnftables.c:370)
==6297==    by 0x4E7AAE1: nft_run_cmd_from_filename (libnftables.c:438)
==6297==    by 0x109A33: main (main.c:310)

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agoct: use nft_print() instead of printf()
Pablo Neira Ayuso [Fri, 5 Apr 2019 13:36:48 +0000 (15:36 +0200)] 
ct: use nft_print() instead of printf()

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agoevaluate: improve error reporting in tproxy with inet family
Pablo Neira Ayuso [Thu, 4 Apr 2019 11:23:04 +0000 (13:23 +0200)] 
evaluate: improve error reporting in tproxy with inet family

 # nft add rule inet filter divert ip daddr 0.0.0.0/0 meta l4proto tcp tproxy to :2000
 Error: conflicting protocols specified: ip vs. unknown. You must specify ip or ip6 family in tproxy statement
 add rule inet filter divert ip daddr 0.0.0.0/0 meta l4proto tcp tproxy to :2000
                             ~~~~~~~~                            ^^^^^^^^^^^^^^^

instead of:

 # nft add rule inet filter divert ip daddr 0.0.0.0/0 meta l4proto tcp tproxy to :2000
 Error: Conflicting network layer protocols.
 add rule inet filter divert ip daddr 0.0.0.0/0 meta l4proto tcp tproxy to :2000
                                                                 ^^^^^^^^^^^^^^^

Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1310
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agoparser_bison: missing tproxy syntax with port only for inet family
Pablo Neira Ayuso [Wed, 3 Apr 2019 21:40:04 +0000 (23:40 +0200)] 
parser_bison: missing tproxy syntax with port only for inet family

 # nft add rule inet filter divert ip daddr 0.0.0.0/0 meta l4proto tcp tproxy ip to :2000
 Error: syntax error, unexpected colon
 add rule inet filter divert ip daddr 0.0.0.0/0 meta l4proto tcp tproxy ip to :2000
                                                                              ^

Syntax with no protocol for tproxy complains with:

 # nft add rule inet filter divert ip daddr 0.0.0.0/0 meta l4proto tcp tproxy to :2000
 Error: Conflicting network layer protocols.
 add rule inet filter divert ip daddr 0.0.0.0/0 meta l4proto tcp tproxy to :2000
                                                                 ^^^^^^^^^^^^^^^

Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1310
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agodoc: Add minimal description of (v)map statements
Phil Sutter [Tue, 2 Apr 2019 13:36:42 +0000 (15:36 +0200)] 
doc: Add minimal description of (v)map statements

Although quite useful, these were missing in man page. Content loosely
based on wiki documentation.

Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agoparser_json: Rewrite echo support
Phil Sutter [Tue, 2 Apr 2019 13:34:43 +0000 (15:34 +0200)] 
parser_json: Rewrite echo support

Instead of guessing which object to update with retrieved handle,
introduce a list containing struct cmd <-> json_t associations. Upon
batch commit, allocated cmd objects are assigned a unique netlink
sequence number. Monitor events contain that number as well, so they may
be associated to the cmd object which triggered them. Using
json_cmd_assoc list the event may in turn be associated to the input's
JSON object which should receive the handle value.

This also fixes incorrect behaviour if JSON input contained "insert"
commands.

Fixes: bb32d8db9a125 ("JSON: Add support for echo option")
Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agoevaluate: skip binary transfer for named sets
Pablo Neira Ayuso [Tue, 26 Mar 2019 12:09:21 +0000 (13:09 +0100)] 
evaluate: skip binary transfer for named sets

Set may be empty, content might be yet unknown, we cannot do any
transfer in this case.

Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1327
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agosrc: use 'flow add' syntax
Pablo Neira Ayuso [Wed, 20 Mar 2019 10:32:58 +0000 (11:32 +0100)] 
src: use 'flow add' syntax

As discussed during NFWS 2018. Old syntax is stilled allowed.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agobuild: missing misspell.h in Makefile.am
Pablo Neira Ayuso [Wed, 20 Mar 2019 10:33:49 +0000 (11:33 +0100)] 
build: missing misspell.h in Makefile.am

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agoconfigure.ac: Clean up AC_ARG_{WITH, ENABLE} invocations, s/==/=/
Luis Ressel [Sun, 17 Mar 2019 18:24:22 +0000 (19:24 +0100)] 
configure.ac: Clean up AC_ARG_{WITH, ENABLE} invocations, s/==/=/

* AC_ARG_ENABLE implicitly defines enable_debug; there's no point in
  performing extra work just to define with_debug with an identical
  value.

* The same applies to with_xtables and with_libxtables.

* The AS_IF block in the `AC_ARG_ENABLE([man-doc], ...` invocation is
  essentially a noop. All it does is to set enable_man_doc to `yes` if
  has a value that matches neither `yes` nor `no`. (This could happen if
  a user calls `configure --enable-man-doc=foo`, but that'd be a user
  error which we don't need to handle.)

* The correct operator for equality tests in `test` is `=`. Some
  implementations also support `==`, but this is not portable.

Signed-off-by: Luis Ressel <aranea@aixah.de>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agoconfigure.ac: Fix a2x check
Luis Ressel [Sun, 17 Mar 2019 17:19:11 +0000 (18:19 +0100)] 
configure.ac: Fix a2x check

* If enable_man_doc is set, but a2x can't be found, configure should
  fail instead of silently disabling man page creation.

* The AS_IF block checking $need_a2x is never active (need_a2x has been
  removed from configure.ac in 13e44a608 and a277479dc).

* AC_CHECK_PROG(VAR, ...) is a noop if VAR is already set, allowing the
  user to explicitly specify the (path to the) binary in VAR. Adjust the
  AS_IF check to account for this.

Signed-off-by: Luis Ressel <aranea@aixah.de>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agosrc: file descriptor leak in include_file()
Pablo Neira Ayuso [Fri, 15 Mar 2019 10:31:50 +0000 (11:31 +0100)] 
src: file descriptor leak in include_file()

File that contains the ruleset is never closed, track open files through
the nft_ctx object and close them accordingly.

Reported-by: Václav Zindulka <vaclav.zindulka@tlapnet.cz>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agoparser_bison: no need for statement separator for ct object commands
Pablo Neira Ayuso [Thu, 14 Mar 2019 10:04:37 +0000 (11:04 +0100)] 
parser_bison: no need for statement separator for ct object commands

Otherwise, this forces user to place a double semi-colon to skip a
parser error in a multi-line commands:

 # nft add "ct helper ip filter test { type \"ftp\" protocol tcp; };add rule filter test ct helper set \"ftp\""
 Error: syntax error, unexpected add, expecting end of file or newline or semicolon
 add ct helper ip filter test { type "ftp" protocol tcp; };add rule filter test ct helper set "ftp"
                                                           ^^^

Reported-by: Laura Garcia <nevola@gmail.com>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agotests: shell: bogus EBUSY on helper deletion from transaction
Pablo Neira Ayuso [Thu, 14 Mar 2019 09:41:45 +0000 (10:41 +0100)] 
tests: shell: bogus EBUSY on helper deletion from transaction

Reported-by: Laura Garcia <nevola@gmail.com>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agotests: shell: bogus ENOENT on element deletion in interval set
Pablo Neira Ayuso [Tue, 12 Mar 2019 11:31:04 +0000 (12:31 +0100)] 
tests: shell: bogus ENOENT on element deletion in interval set

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agotests: shell: bogus EBUSY in set deletion after flush
Pablo Neira Ayuso [Fri, 8 Mar 2019 13:31:50 +0000 (14:31 +0100)] 
tests: shell: bogus EBUSY in set deletion after flush

flush chain ip filter group_7933

Removes all rules, including references to set 'group_7933', however:

delete map ip filter group_7933

results in:

delete.nft:6:1-32: Error: Could not process rule: Device or resource busy
delete map ip filter group_7933
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agotests: add test case for anon set abort.
Florian Westphal [Thu, 7 Mar 2019 19:43:56 +0000 (20:43 +0100)] 
tests: add test case for anon set abort.

This triggers kernel crash in 5.0, see
https://bugzilla.netfilter.org/show_bug.cgi?id=1325

Signed-off-by: Florian Westphal <fw@strlen.de>
6 years agoevaluate: misleading error reporting with sets and maps
Pablo Neira Ayuso [Wed, 6 Mar 2019 10:56:59 +0000 (11:56 +0100)] 
evaluate: misleading error reporting with sets and maps

When trying to list a map content, if set is used, nft reports:

 # nft list set filter group_7933
 Error: No such file or directory; did you mean set ‘group_7933’ in table ip ‘filter’?
 list set filter group_7933
                 ^^^^^^^^^^

Which is confusing in case user wants to list an existing map:

 # nft list map filter group_7933
 table ip filter {
        map group_7933 {
                type ipv4_addr : classid
                flags interval
                elements = { 10.4.22.0/24 : 1:c7cb }
        }
 }

Instead, give a hint to user that probably wants to list a map, not a set:

 # nft list set filter group_7933
 Error: No such file or directory; did you mean map ‘group_7933’ in table ip ‘filter’?
 list set filter group_7933
                 ^^^^^^^^^^

Fixes: 285bb67a11ad ("src: introduce simple hints on incorrect set")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agosegtree: add missing non-matching segment to set in flat representation
Pablo Neira Ayuso [Tue, 5 Mar 2019 23:51:03 +0000 (00:51 +0100)] 
segtree: add missing non-matching segment to set in flat representation

 # cat test.nft
 add set x y { type ipv4_addr; }
 add element x y { 10.0.24.0/24 }
 # nft -f test.nft
 # nft delete element x y { 10.0.24.0/24 }

bogusly returns -ENOENT. The non-matching segment (0.0.0.0 with end-flag
set on) is not added to the set in the example above.

This patch also adds a test to cover this case.

Fixes: 4935a0d561b5 ("segtree: special handling for the first non-matching segment")
Reported-by: Václav Zindulka <vaclav.zindulka@tlapnet.cz>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agosegtree: remove dummy debug_octx
Pablo Neira Ayuso [Tue, 5 Mar 2019 12:13:52 +0000 (13:13 +0100)] 
segtree: remove dummy debug_octx

Breaks custom-defined configuration in library mode, ie. user may want
to store output in a file, instead of stderr.

Fixes: 35f6cd327c2e ("src: Pass stateless, numeric, ip2name and handle variables as structure members.")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agosegtree: fix crash when debug mode is active
Florian Westphal [Mon, 4 Mar 2019 00:06:05 +0000 (01:06 +0100)] 
segtree: fix crash when debug mode is active

We must set output_fp to sensible filep, else crash.

Reported-by: Václav Zindulka <vaclav.zindulka@tlapnet.cz>
Signed-off-by: Florian Westphal <fw@strlen.de>
6 years agodoc: update goto/jump help text
Florian Westphal [Fri, 22 Feb 2019 16:33:09 +0000 (17:33 +0100)] 
doc: update goto/jump help text

pointed out by redsh on #netfilter: uses "of" instead of "or".
While at it, explain continue and update return statement.

Signed-off-by: Florian Westphal <fw@strlen.de>
Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agoparser_json: Respect base chain priority
Phil Sutter [Tue, 26 Feb 2019 21:13:42 +0000 (22:13 +0100)] 
parser_json: Respect base chain priority

Priority value was parsed but not assigned to allocated chain object.

Fixes: 586ad210368b7 ("libnftables: Implement JSON parser")
Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agojson: Fix memleaks in echo support
Phil Sutter [Tue, 26 Feb 2019 21:13:41 +0000 (22:13 +0100)] 
json: Fix memleaks in echo support

When extracting netlink message data for populating JSON objects with
handles, allocated nftnl objects were not freed. Though since freeing
these objects also frees retrieved string attributes, copy them using
strdupa() which takes care of memory deallocation upon function return.
This is ideal since these strings are used only to find the right JSON
object to insert the handle into.

Fixes: bb32d8db9a125 ("JSON: Add support for echo option")
Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agoparser_json: Use xstrdup() when parsing rule comment
Phil Sutter [Tue, 26 Feb 2019 21:13:40 +0000 (22:13 +0100)] 
parser_json: Use xstrdup() when parsing rule comment

Use xstrdup() instead of plain strdup() for consistency (and implicit
ENOMEM checking).

Fixes: 586ad210368b7 ("libnftables: Implement JSON parser")
Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agoparser_json: Duplicate chain name when parsing jump verdict
Phil Sutter [Tue, 26 Feb 2019 21:13:39 +0000 (22:13 +0100)] 
parser_json: Duplicate chain name when parsing jump verdict

Since verdict expression frees the chain name, pass a newly allocated
string to it. Otherwise double free happens because json_decref() frees
the string property value as well.

Fixes: d1057a5feb5fd ("JSON: Simplify verdict statement parsing")
Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agolibnftables: Print errors before freeing commands
Phil Sutter [Tue, 26 Feb 2019 21:13:38 +0000 (22:13 +0100)] 
libnftables: Print errors before freeing commands

Commands may contain data printed by an error record, so make sure
cmd_free() is not called before erec_print_list() has returned.

Fixes: 778de37d82e7b ("libnftables: Keep cmds list outside of parser_state")
Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
6 years agotests: shell: flush after rule deletion
Pablo Neira Ayuso [Fri, 15 Feb 2019 10:05:43 +0000 (11:05 +0100)] 
tests: shell: flush after rule deletion

Flush after rule deletion should hit no ENOENT.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>