Gary Lockyer [Fri, 21 Jun 2019 02:32:08 +0000 (14:32 +1200)]
lib ldb key value: use TALLOC_FREE() per README.Coding
Signed-off-by: Andrew Bartlett <abartlet@samba.org> Signed-off-by: Gary Lockyer <gary@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Jun 21 05:45:02 UTC 2019 on sn-devel-184
Gary Lockyer [Wed, 6 Mar 2019 02:28:45 +0000 (15:28 +1300)]
lib ldb key value: fix index buffering
As a performance enhancement the key value layer maintains a cache of
the index records, which is written to disk as part of a prepare commit.
This patch adds an extra cache at the operation layer to ensure that the
cached indexes remain consistent in the event of an operation failing.
Add test to test for index corruption in a failed modify.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org> Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Gary Lockyer [Fri, 21 Jun 2019 02:52:24 +0000 (14:52 +1200)]
lib ldb key value: Remove check_parent from ldb_kv_index_idxptr()
The callers will soon have two possible parents for this pointer, so
we need to remove this check, which was added out of caution given
the rather strange pattern of putting an active memory pointer into a
TDB (as a hash map).
That is, the only callers that did call this with "true" would
have to call this with "false", so just remove the complexity.
Signed-off-by: Andrew Bartlett <abartlet@samba.org> Signed-off-by: Gary Lockyer <gary@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Use the nested transaction support added to the key value back ends to
make key value operations atomic. This will ensure that rename
operation failures, which delete the original record and add a new
record, leave the database in a consistent state.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Gary Lockyer [Wed, 6 Mar 2019 21:18:00 +0000 (10:18 +1300)]
lib ldb key value backends: Add nested txn support
Add limited nested transaction support to the back ends to make the key value
operations atomic (for those back ends that support nested transactions).
Note: that only the lmdb backend currently supports nested transactions.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Andrew Bartlett [Wed, 19 Jun 2019 17:11:41 +0000 (05:11 +1200)]
provision: Suggest "minimal-responses yes;" by default
This improves Samba AD DC performance as a DNS server dramatically, because NS records do not
need to be looked up and there is less risk the response will have to fall back
to TCP, doubling the cost again.
Signed-off-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Jun 21 00:52:19 UTC 2019 on sn-devel-184
Assertions should only be used when there's absolutely no recovery or to
verify data structure invariants. In this case the supplied registry
hive file may have a malformed block with a size of zero. Such a block
should not terminate the whole program.
Signed-off-by: Michael Hanselmann <public@hansmi.ch> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Volker Lendecke [Thu, 20 Jun 2019 18:58:42 +0000 (20:58 +0200)]
smbd: Remove struct blocking_lock_record
Forgot that one in the last patches
Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Jun 20 23:39:34 UTC 2019 on sn-devel-184
Volker Lendecke [Thu, 20 Jun 2019 11:42:12 +0000 (13:42 +0200)]
vfs: Remove SMB_VFS_BRL_CANCEL_WINDOWS
This is not called anymore, bump the VFS version number in a separate
commit
Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Jun 20 18:34:20 UTC 2019 on sn-devel-184
Volker Lendecke [Thu, 13 Jun 2019 10:58:02 +0000 (03:58 -0700)]
smbd: Ping dbwrap_watch on locking.tdb for in smbd_do_unlocking()
smbd_smb1_do_locks_send() watches the file's locking.tdb record for
changes, like the oplock code does. Unlocking a byte range thus must
trigger a retry.
With the share mode cache get_existing_share_mode_lock() is pretty
cheap. We have to write out the share mode record with the current
code, but an obvious optimization will be a share_mode_do_locked doing
all this without actually unmarshalling the locking.tdb entry. And --
there's precedence for this pattern in downgrade_lease()...
Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
Volker Lendecke [Thu, 13 Jun 2019 10:38:57 +0000 (03:38 -0700)]
smbd: Add smbd_smb1_do_locks_send/recv()
This contains the SMB1-specific brlock logic. Right now our core
brlock code has specialized code to deal with pending locks. For
dealing with pending requests waiting for something nowadays we
recommend to use tevent_req. This code also provides the basis to
remove the SMB1 special case handling from the core logic, isolating
protocol features specific to SMB1 and not exposed in SMB2 in
SMB1-specific code.
The core brlock code will not see blocking locks anymore. Instead, the
code in this patch will always immediately fail and take care of the
retries and timeouts.
Retries are implemented by a dbwrap_watch_record_send on the
corresponding locking.tdb entry. A later commit will make unlocks
trigger wakeups there. I chose locking.tdb and not brlock itself to
simplify the implementation. We already have oplock break watchers on
locking.tdb, this will only add one more. This might lead to spurious
wakeups, but they are taken care of by careful retries. An advantage
of doing that is the implicit handling of a killed blocker PID through
dbwrap_watch, obsoleting brl_revalidate.
Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
Volker Lendecke [Thu, 20 Jun 2019 10:42:23 +0000 (12:42 +0200)]
smbd: Add some paranoia against NULL dereference
Quite a few callers set "psmblctx" to NULL, and I could not really
follow 100% that brl_lock only assigns that in the blocking lock
case. Too many layers :-)
Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
Volker Lendecke [Thu, 20 Jun 2019 10:05:30 +0000 (12:05 +0200)]
smbd: Add "blocker_pid" to brl_lock()
Soon we will wait on a conflicting lock to become free via
dbwrap_watched_watch_send. That routine can take a server_id that
blocks us, watching it to go away. To use that, we need to know which
PID it is that blocks us.
Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
Volker Lendecke [Wed, 19 Jun 2019 15:50:54 +0000 (17:50 +0200)]
torture3: Run a blocking lock&x call with a subsequent read
Samba aborts the read&x after a blocked, but eventually successful
locking&x call. Both Windows and source4/ntvfs do the read properly,
source3/smbd does not. With later code, this will become possible much
easier. Lets see if it's worth it given that we've got away with this
forever.
Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
Volker Lendecke [Fri, 14 Jun 2019 11:39:04 +0000 (04:39 -0700)]
smbd: Add "lock_flav" to smbd_do_unlocking()
The next commits will pass all direct "do_unlock" calls through
smbd_do_unlocking(). Why? Unlocking will later on require that we take
the share mode lock for the file in question while the unlock is
happening, and this should be in one central place.
Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
Signed-off-by: Björn Baumbach <bb@samba.org> Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Björn Baumbach <bb@sernet.de>
Autobuild-Date(master): Thu Jun 20 17:14:50 UTC 2019 on sn-devel-184
Douglas Bagnall [Fri, 17 May 2019 02:42:24 +0000 (14:42 +1200)]
CVE-2019-12436 dsdb/paged_results: ignore successful results without messages
So that we don't dereference result->msgs[0] when it doesn't exist.
This can happen when the object has changed in such a way that it no
longer matches the original search query.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(master): Wed Jun 19 08:16:39 UTC 2019 on sn-devel-184
Signed-off-by: Andreas Schneider <asn@samba.org> Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Tue Jun 18 18:10:40 UTC 2019 on sn-devel-184
Lukas Slebodnik [Wed, 12 Jun 2019 10:27:04 +0000 (12:27 +0200)]
wafsamba: Use native waf timer
__main__:1: DeprecationWarning: time.clock has been deprecated in Python 3.3
and will be removed from Python 3.8: use time.perf_counter
or time.process_time instead
Björn Baumbach [Wed, 12 Jun 2019 19:02:43 +0000 (21:02 +0200)]
selftest: add tests for samba-tool ntacl changedomsid
Signed-off-by: Björn Baumbach <bb@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Björn Baumbach <bb@sernet.de>
Autobuild-Date(master): Tue Jun 18 16:54:22 UTC 2019 on sn-devel-184
Björn Baumbach [Tue, 11 Jun 2019 13:11:20 +0000 (15:11 +0200)]
samba-tool: add ntacl changedomsid command
This tool is meant to locally change all entries in acl_xattr when the
machine's SID has accidentially changed or the data set has been copied
to another box either via backup/restore or rsync.
Signed-off-by: Björn Baumbach <bb@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
Björn Baumbach [Wed, 12 Jun 2019 19:16:25 +0000 (21:16 +0200)]
python/ntacls: use correct "state directory" smb.conf option instead of "state dir"
samba-tool ntacl get testfile --xattr-backend=tdb --use-ntvfs Fixes: Unknown parameter encountered: "state dir" Signed-off-by: Björn Baumbach <bb@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
Volker Lendecke [Thu, 13 Jun 2019 19:21:09 +0000 (12:21 -0700)]
smbd: Deprecate "blocking locks" parameter
This parameter adds complexity to our brlock implementation that I don't think
is justified. Not a lot of complexity, but if we would want to really support
it we'd need tests. Instead of doing those, I think removing the parameter is
the better choice.
Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Jun 18 14:47:08 UTC 2019 on sn-devel-184
Volker Lendecke [Thu, 30 May 2019 09:42:16 +0000 (11:42 +0200)]
smbd: Slightly simplify reply_lockingX()
Make the data dependencies more obvious. "locks" and "ulocks" are used
only for short time spans. Any sane compiler will coalesce them to
just one stack variable.
Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Tue Jun 18 06:47:05 UTC 2019 on sn-devel-184
Gary Lockyer [Wed, 12 Jun 2019 20:49:16 +0000 (08:49 +1200)]
lib ldb ldb_key_value: csbuild unused parm ldb_kv
Fixes csbuild error.
Error: COMPILER_WARNING:
lib/ldb/ldb_key_value/ldb_kv_search.c: scope_hint: In
function ‘search_func’ lib/ldb/ldb_key_value/ldb_kv_search.c:308:47:
warning: unused parameter ‘ldb_kv’ [-Wunused-parameter]
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz> Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu Jun 13 08:27:12 UTC 2019 on sn-devel-184
Gary Lockyer [Fri, 7 Jun 2019 03:19:25 +0000 (15:19 +1200)]
lib ldb ldb_key_value: csbuild fix signed unsigned compare
Fixes csbuild error.
Error: COMPILER_WARNING:
lib/ldb/ldb_key_value/ldb_kv_index.c: scope_hint: In function
‘ldb_kv_index_add1’
lib/ldb/ldb_key_value/ldb_kv_index.c:2601:15: warning: comparison of
integer expressions of different signedness: ‘int’ and ‘unsigned int’
[-Wsign-compare]
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz> Reviewed-by: Andreas Schneider <asn@samba.org>