Signed-off-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Garming Sam <garming@catalyst.net.nz>
samdb: test for schemainfo update with relax control
Currently schema info's revision field isn't incremented if relax
control is present. This is so that no increment is done during
provision, but we need the relax control in other situations where the
increment is desired. This patch adds a failing test to expose the
problem.
Signed-off-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Andrew Bartlett [Thu, 4 Apr 2019 21:46:50 +0000 (10:46 +1300)]
ldb_kv: Skip @ records early in a search full scan
@ records like @IDXLIST are only available via a base search on the specific name
but the method by which they were excluded was expensive, after the unpack the
DN is exploded and ldb_match_msg_error() would reject it for failing to match the
scope.
This uses the fact that @ records have the DN=@ prefix on their TDB/LMDB key
to quickly exclude them from consideration.
Signed-off-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Apr 10 06:23:39 UTC 2019 on sn-devel-144
Swen Schillig [Mon, 18 Mar 2019 14:25:54 +0000 (15:25 +0100)]
ctdb-test: Adding test case to verify queue resizeing
If a data packet arrives which exceeds the queue's current buffer size,
the buffer needs to be increased to hold the full packet. Once the packet
is processed the buffer size should be decreased to its standard size again.
This test case verifies this process.
Signed-off-by: Swen Schillig <swen@linux.ibm.com> Reviewed-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Christof Schmitt <cs@samba.org>
Autobuild-User(master): Christof Schmitt <cs@samba.org>
Autobuild-Date(master): Wed Apr 10 00:17:37 UTC 2019 on sn-devel-144
Andrew Bartlett [Thu, 14 Mar 2019 05:20:06 +0000 (18:20 +1300)]
CVE-2019-3870 pysmbd: Move umask manipuations as close as possible to users
Umask manipulation was added to pysmbd with e146fe5ef96c1522175a8e81db15d1e8879e5652 in 2012
and init_files_struct was split out in 747c3f1fb379bb68cc7479501b85741493c05812 in 2018 for
Samba 4.9. (It was added to assist the smbd.create_file() routine used in the backup and
restore tools, which needed to write files with full metadata).
This in turn avoids leaving init_files_struct() without resetting the umask to
the original, saved, value.
Per umask(2) this is required before open() and mkdir() system calls (along
side other file-like things such as those for Unix domain socks and FIFOs etc).
Therefore for safety and clarify the additional 'belt and braces' umask
manipuations elsewhere are removed.
mkdir() will be protected by a umask() bracket, for correctness, in the next patch.
Tim Beale [Fri, 15 Mar 2019 02:20:21 +0000 (15:20 +1300)]
CVE-2019-3870 tests: Extend smbd tests to check for umask being overwritten
The smbd changes the umask - if the code fails to restore the umask to
what it was, then this is very bad. Add an extra check to every
smbd-related test that the umask at the end of the test is the same as
what it was at the beginning (i.e. if the smbd code changed the umask
then it correctly restored the value afterwards).
As the selftest sets the umask for all tests to zero, it makes it hard
to detect this problem, so the test setUp() needs to set it to something
else first.
This extra checking is added to the setUp()/tearDown() so that it
applies to all test-cases. However, any failure that occur with this
approach will not be able to be known-failed.
Andrew Bartlett [Mon, 8 Apr 2019 02:04:08 +0000 (14:04 +1200)]
selftest: Correct name of flapping smb2.notify test
Signed-off-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Mon Apr 8 03:09:42 UTC 2019 on sn-devel-144
Aaron Haslett [Wed, 20 Mar 2019 00:52:16 +0000 (13:52 +1300)]
ldb: version 2.0.0
* Version bump for adding index_format_fn to the schema syntax structure.
* Range index support added, allowing <= and >= operations to be indexed
* Improved reindex performance by setting the in-memory TDB hash size correctly
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Garming Sam [Wed, 3 Apr 2019 03:12:35 +0000 (16:12 +1300)]
ldb: tests for <= and >= integer indexing with duplicates
We need to make sure that duplicates are correctly returned (uSNChanged
for instance is UNIQUE but, we should be able to index on attributes
which are not unique).
Signed-off-by: Garming Sam <garming@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Aaron Haslett [Mon, 11 Mar 2019 03:39:13 +0000 (16:39 +1300)]
ldb: tests for <= and >= integer indexing
Testing max, min and negative values for indexed 32 and 64 bit types.
This has to be done in two different files because the 64 bit type is
LDB_SYNTAX_INTEGER which is implemented at the ldb level, while the 32
bit is added in the ldb-samba module. Schema syntax binding added for
ldb-samba.
We also need to make sure that full scans are not invoked for LMDB.
Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz> Signed-off-by: Garming Sam <garming@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Garming Sam [Thu, 4 Apr 2019 21:22:28 +0000 (10:22 +1300)]
schema_syntax: Add comments for our index format functions
We had to devise our own scheme for writing integers in a human readable
format which also sorted correctly numerically. This might look a bit
confusing to outsiders, so here's a large comment as a peace offering.
Pair-programmed-with: Tim Beale <timbeale@catalyst.net.nz>
Signed-off-by: Garming Sam <garming@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Aaron Haslett [Thu, 14 Mar 2019 05:05:23 +0000 (18:05 +1300)]
ldb: activating <= and >= indexing for integers
Activating <= and >= mdb indexing in samba for int32 and int64 attributes by:
1. Adding index_format_fn to LDB_SYNTAX_SAMBA_INT32 in ldb_samba
2. Cloning the 64bit LDB_SYNTAX_INTEGER type as LDB_SYNTAX_ORDERED_INTEGER
3. Adding index_format_fn to the new type
4. Modifying LargeInteger use the new type in samba schema
5. Bumping the index version to trigger reindexing
Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz> Signed-off-by: Garming Sam <garming@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Garming Sam [Thu, 4 Apr 2019 02:02:15 +0000 (15:02 +1300)]
ldb_kv_index: Add a giant comment in regards to index_format_fn
The reason we needed it in the first place was that the original
canonicalize is being used for non-index functions and it never produced
the right order originally (at least for integers).
Signed-off-by: Garming Sam <garming@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Garming Sam [Thu, 4 Apr 2019 01:36:08 +0000 (14:36 +1300)]
ldb_kv_index: Make the edge keys slightly cleaner and generic
It makes no difference in our standard case because \0 will always go
before any value for our index_format_fn, but this is better for
correctness (in case we do mess up our NUL terminations elsewhere).
Signed-off-by: Garming Sam <garming@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Aaron Haslett [Mon, 4 Mar 2019 06:06:31 +0000 (19:06 +1300)]
ldb: <= and >= indexed searching
Full implementation of <= and >= indexed searching using iterate_range
backend operation. Adds index_format_fn to ldb_schema_syntax so
requires an ABI version bump. The function must be provided for any
type for which <= and >= indexing is required, and must return a
lexicographically ordered canonicalization of a value. This causes
index entries to be written in correct order to the database, so
iterate_range on the index DNs can be used.
ldb_kv_index_key is modified to return an index DN with attribute name
but without value if an empty value is provided. This is needed for
constructing keys that match the beginning or end of an index DN range.
Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz> Signed-off-by: Garming Sam <garming@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Garming Sam [Sun, 3 Mar 2019 23:50:24 +0000 (12:50 +1300)]
lmdb: iterate_range implementation
Adding iterate_range to LDB API and implementing in LMDB. This
operation takes a start_key and end_key and returns all records between
the two, inclusive of both. This will be used to implementing indexing
for <= and >= expressions.
Signed-off-by: Garming Sam <garming@catalyst.net.nz> Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Sat Apr 6 11:51:55 UTC 2019 on sn-devel-144
Signed-off-by: Christof Schmitt <cs@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Apr 6 06:08:42 UTC 2019 on sn-devel-144
memcache: Increase size of default memcache to 512k
With the fixed accounting of talloc objects, the default cache size
needs to increase. The exact increase required depends on the workloads,
going form 256k to 512k seems like a reasonable guess.
memcache: Properly track the size of talloc objects
With memcache_add_talloc, the talloc object becomes part of the pool and
the memcache_element stores a pointer to the talloc object. The
size of the the talloc object was not used when tracking the used space,
allowing the cache to grow larger than defined in the memcache_init
call.
Fix this by adding the size of the talloc object to the used space.
Also record the initial size of the talloc object for proper adjustment
of the used space in the cache later. This is in case the size of the
talloc object is modified while being owned by the cache (e.g.
allocating talloc child objects). This should never happen, but better
be safe than ending up with a broken cache usage counter.
The min_pwd_age and max_pwd_age parameters are both optional and default
to None. However, if we just set the max-pwd-age, then the check
'min_pwd_age >= max_pwd_age' will throw a Python exception because it's
trying to compare an int to NoneType (min_pwd_age). This works on Python 2
but is a problem on Python 3.
We could just add a check that min_pwd_age is not None, but that defeats
the point of having the check if you're only setting either the min or
max age indepedently.
This patch gets the current min/max password age from the DB (in ticks).
If either setting is changed, the ticks will be updated. Then at the end
we check the min is still less than the max (to do this, we convert the
ticks back to days in the interests of readability).
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Apr 5 08:03:08 UTC 2019 on sn-devel-144
Douglas Bagnall [Thu, 4 Apr 2019 02:08:18 +0000 (15:08 +1300)]
ldb_kv_search: avoid handling uninitialised dn
If ldb_kv_filter_attrs() fails, we don't know that the dn of filtered_msg
is OK.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Apr 5 05:46:55 UTC 2019 on sn-devel-144
Douglas Bagnall [Wed, 3 Apr 2019 21:36:37 +0000 (10:36 +1300)]
perf-tests: rename paged search test for regex disambiguation
We like to use "TESTS=medley" for the old ad_dc_medley.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Garming Sam <garming@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
rpc: Convert npa_tstream.c to use tstream_u32_read_send
This avoids a bit of code duplication. Overall the last two commits
add a few lines, but that also contains the header file and another GPL
header for tstream_u32_read.c.
Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Apr 5 01:00:48 UTC 2019 on sn-devel-144
Volker Lendecke [Thu, 21 Mar 2019 18:41:28 +0000 (19:41 +0100)]
rpc: Add tstream_u32_read
In npa_tstream.c we have two next_vector functions reading a big
endian uin32_t length and then the blob described by the length. This
factors that next_vector out into a central routine.
Why? I'll add another NPA protocol in the future, and this would add
yet another two copies of that next_vector code
Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
Gary Lockyer [Mon, 1 Apr 2019 03:49:38 +0000 (16:49 +1300)]
python join: Set index transaction cache size.
The default value is too small for joining a large domain. So we specify a
size of 200,000 which is suitable for domains with up to 100,000 users.
At a later date this could be added as a parameter to the join, but
200,000 should be suitable for now.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Apr 4 07:45:03 UTC 2019 on sn-devel-144
Gary Lockyer [Mon, 1 Apr 2019 03:33:52 +0000 (16:33 +1300)]
lib ldb key_value: Set index cache size on open
Set the default index cache from the passed option
"transaction_index_cache_size" on open. This allows the default cache
size to be overridden when processing large transactions i.e. joining a
large domain.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Gary Lockyer [Mon, 1 Apr 2019 02:27:32 +0000 (15:27 +1300)]
lib ldb key_value: Add get_size method
Add the get_size method to the ldb_key_value layer, this will allow the
reindexing code to get an estimate of the number of records in the
database.
The lmdb backend returns an accurate count of the number of records in
the database withe the mdb_env_stat call.
The tdb backend does not provide a low cost method to determine the
number of records on the database. It does provide a tdb_summary call
however this this walks the entire database.
So for tdb we use the map size divided by 500, this over estimates the counts
for small domains, but the extra memory allocated for the cache should
not be significant.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Gary Lockyer [Mon, 1 Apr 2019 01:10:10 +0000 (14:10 +1300)]
lib ldb key_value: Pass index cache size
Pass the index cache size to ldb_kv_index_transaction_start. This will
allow it to be set for reindex and join operations, where the current
defaults result in a significant performance penalty on large databases.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Gary Lockyer [Mon, 1 Apr 2019 00:12:20 +0000 (13:12 +1300)]
lib ldb key_value: Remove index cache lazy initialisation
Remove the lazy initialisation of the index cache. This make setting
the size of the cache for re-indexing easier, which will be done in
later commits.
Performance testing shows that the removal of lazy initialisation makes
no appreciable difference to performance.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
s3-libnet_join: allow fallback to NTLMSSP auth in libnet_join
When a non-DNS and non-default admin domain is provided during the join
sometimes we might not be able to kinit with 'user@SHORTDOMAINNAME'
(e.g. when the winbind krb5 locator is not installed). In that case lets
fallback to NTLMSSP, like we do in winbind.
Philipp Gesang [Tue, 12 Mar 2019 14:43:42 +0000 (15:43 +0100)]
python/samba: ignore encoding errors while reading files
Provisioning fails on C locale due to the Unicode quotes in ldif
data. Patch read_and_sub_file() to read the files as UTF-8.
Signed-off-by: Philipp Gesang <philipp.gesang@intra2net.com> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Wed Apr 3 11:33:38 UTC 2019 on sn-devel-144
lib:util: Include talloc_keep_secret.h in samba_util.h
Signed-off-by: Andreas Schneider <asn@samba.org> Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Wed Apr 3 07:50:03 UTC 2019 on sn-devel-144
The "oLschema2ldif" program contained multiple bugs triggered by
malformed inputs:
* Iteration beyond list of recognized dsdb syntax OIDs when value wasn't
found (bug 9567)
* NULL pointer dereference when input didn't define a name
* Heap buffer overflows for unterminated token values
Signed-off-by: Michael Hanselmann <public@hansmi.ch> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Then adapted to use ARRAY_SIZE() consistently as suggested by
metze.
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Apr 3 02:43:07 UTC 2019 on sn-devel-144
Gary Lockyer [Wed, 27 Mar 2019 00:52:37 +0000 (13:52 +1300)]
prefork tests: disable restart tests MIT
The prefork restart tests use the kdc to test restart back off. These
tests need to use a process that has not been restarted to ensure that
the process restart counters are sitting at 0 when the tests start.
As MIT kerberos is experimental the heimdal builds will still ensure
that this functionality is tested.
At some stage the echo server should be converted to run in pre-fork and
this process can then be used to test the restart backoff.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Björn Baumbach [Fri, 29 Mar 2019 15:36:03 +0000 (16:36 +0100)]
s4:rpc_server: add missing newline to error debug message
Signed-off-by: Björn Baumbach <bb@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Apr 3 00:13:10 UTC 2019 on sn-devel-144
waf: Simplify condition for undefined symbol detection
There's no need to check for OpenBSD twice.
Signed-off-by: Michael Hanselmann <public@hansmi.ch> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Noel Power <npower@samba.org>
Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Tue Apr 2 17:28:40 UTC 2019 on sn-devel-144
Kristján Valur [Tue, 12 Mar 2019 09:33:15 +0000 (09:33 +0000)]
make some auth functions return an NTSTATUS like other similar functions for better diagnostics.
Signed-off-by: Kristján Valur <kristjan@rvx.is> Reviewed-by: Andreas Schneider <asn@samba.org> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Apr 2 02:12:48 UTC 2019 on sn-devel-144
Pair-Programmed-With: Andreas Schneider <asn@samba.org> Signed-off-by: Guenther Deschner <gd@samba.org> Signed-off-by: Andreas Schneider <asn@samba.org> Reviewed-by: Ralph Boehme <slow@samba.org> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This got lost somehow during refactoring. This is still viable
information when trying to figure out what is going wrong when
authenticating a user over NTLMSSP.
Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@samba.org>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Sun Mar 31 11:47:44 UTC 2019 on sn-devel-144
Martin Schwenke [Wed, 20 Mar 2019 10:19:49 +0000 (21:19 +1100)]
ctdb-scripts: Allow load_system_config() to take multiple alternatives
The situation for NFS config has got more complicated and is probably
broken in statd-callout on Debian-like systems at the moment. Allow
several alternative configuration names to be tried. Stop after the
first that is found and loaded.
projects / thirdparty / samba.git / log
