The relational expression allows recursion from both sides, this doesn't
allow us to know what hand side the input is coming from. This patch
adds a new expr_rhs rule that specifies what can be found on the
constant side of the relational.
Besides making it easier to understand what is actually supported, this
allows us to use reserve words both as constant and statements. This is
used by the following patch to allow to use redirect as constant from
the icmp payload match.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Arturo Borrero [Fri, 11 Dec 2015 10:10:19 +0000 (11:10 +0100)]
tests/: add shell test-suite
This new test-suite is intended to perform tests of higher level than
the other reggresion test-suite.
It can run arbitrary executables which can perform any test apart of testing
the nft syntax or netlink code (which is what the regression tests does).
To run the test suite (as root):
% cd tests/shell
% ./run-tests.sh
Test files are executables files with the pattern <<name_N>>, where N is the
expected return code of the executable. Since they are located with `find',
test-files can be spreaded in any sub-directories.
You can turn on a verbose execution by calling:
% ./run-tests.sh -v
Before each call to the test-files, `nft flush ruleset' will be called.
Also, test-files will receive the environment variable $NFT which contains the
path to the nftables binary being tested.
You can pass an arbitrary $NFT value as well:
% NFT=../../src/nft ./run-tests.sh
Signed-off-by: Arturo Borrero Gonzalez <arturo.borrero.glez@gmail.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
netlink_delinearize: add previous statement to rule_pp_ctx
564b0e7c13f9 ("netlink_delinearize: postprocess expression before range
merge") crashes nft when the previous statement is removed via
payload_dependency_kill() as this pointer is not valid anymore.
Move the pointer to the previous statement to rule_pp_ctx and invalidate
it when required.
Reported-by: "Pablo M. Bermudo Garay" <pablombg@gmail.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org> Reported-by: "Pablo M. Bermudo Garay" <pablombg@gmail.com>
Update bitfield definitions to match according to the way they are
expressed in RFC and IEEE specifications.
This required a bit of update for c3f0501 ("src: netlink_linearize:
handle sub-byte lengths").
>From the linearize step, to calculate the shift based on the bitfield
offset, we need to obtain the length of the word in bytes:
len = round_up(expr->len, BITS_PER_BYTE);
Then, we substract the offset bits and the bitfield length.
shift = len - (offset + expr->len);
From the delinearize, payload_expr_trim() needs to obtain the real
offset through:
off = round_up(mask->len, BITS_PER_BYTE) - mask_len;
For vlan id (offset 12), this gets the position of the last bit set in
the mask (ie. 12), then we substract the length we fetch in bytes (16),
so we obtain the real bitfield offset (4).
Then, we add that to the original payload offset that was expressed in
bytes:
payload_offset += off;
Note that payload_expr_trim() now also adjusts the payload expression to
its real length and offset so we don't need to propagate the mask
expression.
Reported-by: Patrick McHardy <kaber@trash.net> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
The tests script suffers a crash when a rule test line is malformed
(e.g. if expected result is missing). This commit fixes these crashes
and now the line is skipped and a warning is printed.
While at it, fix a malformed test line too.
Signed-off-by: Pablo M. Bermudo Garay <pablombg@gmail.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
In the test files, some lines defining tables was commented out with a
minus "-" sign, also used to mark broken rules. This commit replaces
these signs with actual comments "#" and removes the code that handled
the situation.
Signed-off-by: Pablo M. Bermudo Garay <pablombg@gmail.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
During tests execution, some *.payload.got files may be generated. To
avoid annoyances, this commit adds the pattern to .gitignore. Also, the
file "dup.t.payload.got", that was presumably included by mistake, has
been deleted.
Signed-off-by: Pablo M. Bermudo Garay <pablombg@gmail.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
We have to clone the payload expression before attaching it to the lhs
of the relational expression, this payload expression is located at the
lhs of the binary operation that is released thereafter.
Fixes: 39f15c2 ("nft: support listing expressions that use non-byte header fields") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
tests: regression: allow to run tests from anywhere
Since 357d8cf "tests: use the src/nft binary instead of $PATH one", the
tests script needs to be executed from nftables repository root. Now
the script can be run from any location and also checks the binary
existence.
To run a single test file, the path must be relative from the directory
where you launch the script.
Signed-off-by: Pablo M. Bermudo Garay <pablombg@gmail.com> Signed-off-by: Florian Westphal <fw@strlen.de>
Python interpreter doesn't like mixed indentation. So in order to
prevent future problems, this commit replace some tabs found in the
script with space indentation.
Signed-off-by: Pablo M. Bermudo Garay <pablombg@gmail.com> Signed-off-by: Florian Westphal <fw@strlen.de>
We need to reallocate the constant expression with the right expression
length when evaluating the string. Otherwise the linearization step
generates a wrong comparison on big endian. We cannot do this any
earlier since we don't know the maximum string length for this datatype
at the parsing stage.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Contrary to iptables, we use the asterisk character '*' as wildcard.
# nft --debug=netlink add rule test test iifname eth\*
ip test test
[ meta load iifname => reg 1 ]
[ cmp eq reg 1 0x00687465 ]
Note that this generates an optimized comparison without bitwise.
In case you want to match a device that contains an asterisk, you have
to escape the asterisk, ie.
# nft add rule test test iifname eth\\*
The wildcard string handling occurs from the evaluation step, where we
convert from:
relational
/ \
/ \
meta value
oifname eth*
to:
relational
/ \
/ \
meta prefix
ofiname
As Patrick suggested, this not actually a wildcard but a prefix since it
only applies to the string when placed at the end.
More comments:
* This relaxes the left->size > right->size from netlink_parse_cmp()
for strings since the optimization that this patch applies may now
result in bogus errors.
* This patch can be later on extended to apply a similar optimization to
payload expressions when:
expr->len % BITS_PER_BYTE == 0
For meta and ct, the kernel checks for the exact length of the attributes
(it expects integer 32 bits) so we can't do it unless we relax that.
* Wildcard strings are not supported from sets and maps yet. Error
reporting is not very good at this stage since expr_evaluate_prefix()
doesn't have enough context (ctx->set is NULL, the set object is
currently created later after evaluating the lhs and rhs of the
relational). I'll be following up on this later.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
evaluate: check if table and chain exists when adding rules
Assuming a table 'test' that contains a chain 'test':
# nft add rule test1 test2 counter
<cmdline>:1:1-28: Error: Could not process rule: Table 'test1' does not exist
add rule test1 test2 counter
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# nft add rule test test2 counter
<cmdline>:1:1-27: Error: Could not process rule: Chain 'test2' does not exist
add rule test test2 counter
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
rule: `list sets' only displays declaration, not definition
# nft list sets
table ip nat {
set libssh {
type ipv4_addr
}
}
table inet filter {
set set0 {
type inet_service
flags constant
}
set set1 {
type inet_service
flags constant
}
set set2 {
type icmpv6_type
flags constant
}
}
So in case you want to inspect the definition, you have to use `list set'
and the specific set that you want to inspect:
# nft list set inet filter set0
table inet filter {
set set0 {
type inet_service
flags constant
elements = { 2200, ssh}
}
}
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org> Acked-by: Arturo Borrero Gonzalez <arturo.borrero.glez@gmail.com>
Several fixes:
* handles are printed last
* simplify space games (an extra space was being printed)
* comments are shown with `nft monitor' as well (missing before this patch)
Before this patch:
% nft list ruleset -a
[...]
chain test {
iifname eth0 # handle 1 comment "test"
}
[...]
% nft list ruleset
[...]
chain test {
iifname eth0 comment "test"
^^
}
[...]
% nft monitor &
% nft add rule test test iifname eth0 comment "test"
add rule test test iifname eth0
After this patch:
% nft list ruleset -a
chain test {
iifname eth0 comment "test" # handle 1
^
}
% nft monitor -a &
% nft add rule test test iifname eth0 comment "test"
add rule test test iifname eth0 comment "test" # handle 1
Signed-off-by: Arturo Borrero Gonzalez <arturo.borrero.glez@gmail.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
netlink: cmp: shift rhs constant if lhs offset doesn't start on byte boundary
if we have payload(someoffset) == 42, then shift 42 in case someoffset
doesn't start on a byte boundary.
We already insert a mask instruction to only load those bits into
the register that we were interested in, but the cmp will fail without
also adjusting rhs accordingly.
Needs additional patch in reverse direction to undo the shift again
when dumping ruleset.
payload: disable payload merge if offsets are not on byte boundary.
... because it doesn't work, we attempt to merge it into wrong
place, we would have to merge the second value at a specific location.
F.e. vlan hdr 4094 gives us
0xfe0f
Merging in the CFI should yield 0xfe1f, but the constant merging
doesn't know how to achive that; at the moment 'vlan id 4094'
and 'vlan id 4094 vlan cfi 1' give same result -- 0xfe0f.
For now just turn off the optimization step unless everything is
byte divisible (the common case).
nft: allow stacking vlan header on top of ethernet
currently 'vlan id 42' or even 'vlan type ip' doesn't work since
we expect ethernet header but get vlan.
So if we want to add another protocol header to the same base, we
attempt to figure out if the new header can fit on top of the existing
one (i.e. proto_find_num gives a protocol number when asking to find
link between the two).
We also annotate protocol description for eth and vlan with the full
header size and track the offset from the current base.
Otherwise, 'vlan type ip' fetches the protocol field from mac header
offset 0, which is some mac address.
Instead, we must consider full size of ethernet header.
Pablo reported test failures because the order of returned set entries
is not deterministic.
This sorts set elements before comparision.
Patrick suggested to move ordering into libnftnl (since we could f.e.
also get duplicate entries due to how netlink dumps work), but thats a bit
more work. Hence this quick workaround.
Reported-by: Pablo Neira Ayuso <pablo@netfilter.org> Signed-off-by: Florian Westphal <fw@strlen.de>
Adapt the nftables code to use the new symbols in libnftnl. This patch contains
quite some renaming to reserve the nft_ prefix for our high level library.
Explicitly request libnftnl 1.0.5 at configure stage.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
evaluate: use existing table object from evaluation context
Skip table object lookup if we are in the context of table declaration already,
ctx->table already points to the right table we have to use during the
evalution. Otherwise, a list corruption occurs when using the wrong table
object when it already exists in the kernel.
mnl: rework netlink socket receive path for events
This patch reworks two aspects of the netlink socket event receive path:
1) In case of ENOBUFS, stay in the loop to keep receiving messages. The tool
displays a message so the user knows that we got lost event messages.
2) Rise the default size of the receive socket buffer up to 16 MBytes to reduce
chances of hitting ENOBUFS. Asumming that the netlink event message size is
~150 bytes, we can bear with ~111848 rules without message loss.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
tests: display error when trying to run tests out of the root directory
Since 357d8cfcceb2 ("tests: use the src/nft binary instead of $PATH one"), the
tests fail if you try to run them if you are not under the root directory of
the nftables repository.
Display an error so I don't forget I have to do it like this.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
tests: sets: don't include listing in payload tests
Since e715f6d1241c ("netlink: don't call netlink_dump_*() from listing
functions with --debug=netlink"), there is no debugging from the listing path.
Thus, we can remove the set line from the test files.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
netlink: don't call netlink_dump_*() from listing functions with --debug=netlink
Now that we always retrieve the object list to build a cache before executing
the command, this results in fully listing of existing objects in the kernel.
This is confusing when adding a simple rule, so better not to call
netlink_dump_*() from listing functions.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
evaluate: display error on unexisting chain when listing
nft list chain ip test output
<cmdline>:1:1-25: Error: Could not process rule: Chain 'output' does not exist
list chain ip test output
^^^^^^^^^^^^^^^^^^^^^^^^^
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
The only remaining caller that needs this is netlink_dump_ruleset(), that is
used to export the ruleset using markup representation. We can remove it and
handle this from do_command_export() now that we have a centralized point to
build up the object cache.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
This patch adds set objects to the cache if they don't exist in the kernel, so
they can be referenced from this batch. This occurs from the evaluation step.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
This patch populates the cache only once through netlink_list_sets() during
evaluation. As a result, there is a single call to netlink_list_sets().
After this change, we can rid of get_set(). This function was fine by the time
we had no transaction support, but this doesn't work for set objects that are
declared in this batch, so inquiring the kernel doesn't help since they are not
yet available.
As a result from this update, the monitor code gets simplified quite a lot
since it can rely of the set cache. Moreover, we can now validate that the
table and set exists from evaluation path.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Add declared table objects to the cache, thus we can refer to objects that
come in this batch but that are not yet available in the kernel. This happens
from the evaluation step.
Get rid of code that is doing this from the later do_command_*() stage.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
.. to make sure that later support to match header elements that have odd
(non-byte aligned) lengths/offsets doesn't erronously eliminate explicitly
added binops while searching expressions for implicit binops.
netlink_delinearize: meta l4proto range printing broken on 32bit
Florian Westphal says:
09565a4b1ed4863d44c4509a93c50f44efd12771 ("netlink_delinearize: consolidate
range printing") causes nft to segfault on 32bit machine when printing l4proto
ranges.
The problem is that meta_expr_pctx_update() assumes that right is a value, but
after this change it can also be a range.
Thus, expr->value contents are undefined (its union). On x86_64 this is also
broken but by virtue of struct layout and pointer sizes, value->_mp_size will
almost always be 0 so mpz_get_uint8() returns 0.
But on x86-32 _mp_size will be huge value (contains expr->right pointer of
range), so we crash in libgmp.
Pablo says:
We shouldn't call pctx_update(), before the transformation we had
there a expr->op == { OP_GT, OP_GTE, OP_LT, OP_LTE }. So we never
entered that path as the assert in payload_expr_pctx_update()
indicates.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org> Tested-by: Florian Westphal <fw@strlen.de>
Note that nft_parse() returns 1 on parsing errors and 0 + state->errs on
evaluation problems, so return -1 as other functions do here to pass up the
error to the main routine.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>