Volker Lendecke [Wed, 24 Oct 2018 12:31:34 +0000 (14:31 +0200)]
tdb: Make record deletion circular-chain safe
Before this patch we had 3 loops walking a hash chain to delete
records:
tdb_do_delete() to find the predecessor of the record that was to be
deleted. tdb_count_dead(), the name says it all and tdb_purge_dead()
to give back all dead records from a chain to the freelist.
This patch introduces tdb_trim_dead that walks a hash chain just
once. While it does so it counts the number of dead records, and all
records beyond tdb->max_dead_records are moved to the freelist.
Normal record deletion now works by always marking a record as dead in
step 1 and then calling tdb_trim_dead. This is made safe against
circular chains by doing the slow chain walk only in the case when we
did not delete a dead record during our walk.
It changes our dynamics a bit:
When deleting a record with non-zero max_dead_records, now we always
leave that number of records around when deleting, doing a blocking
lock on the freelist when we found too many dead records.
Previously when exceeding max_dead_records we wiped all dead records
to start accumulating them from scratch, assuming we could lock the
freelist in a nonblocking fashion.
The net effect for an uncontended freelist is the same: In
tdb_allocate() we still completely hand over all dead records to the
freelist when we could lock it, it just happens later than without
this patch.
This means for a lightly loaded system we will potentially leave more
dead records around in databases like locking.tdb. However, on a
heavily loaded system we become more predictable: If the freelist is
so heavily contended that across many deletes we can't get hold of it,
previously we accumulated more dead records than max_dead_records
would allow. This is a really lowlevel tradeoff that is likely hard to
measure, but to me becoming more deterministic without sacrificing too
much parallelism (we keep more dead records around) is worth trying.
Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Oct 30 02:48:38 CET 2018 on sn-devel-144
Volker Lendecke [Tue, 23 Oct 2018 11:40:34 +0000 (13:40 +0200)]
tdb: Don't delete dead records in traverse
The next commit will change the handling of dead records, removing the
"tdb_do_delete" function. As traverses should not happen in normal
operations, dead records from them should be rare, and relying on
traverses to remove them is a very bad idea IMHO.
Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Fri, 19 Oct 2018 10:15:42 +0000 (12:15 +0200)]
vfs_fruit: remove check for number of xattrs from ad_convert_xattr
Turns out that there exist AppleDouble files with an extended FinderInfo
entry that includes the xattr marshall buffer, but the count of xattrs
in the buffer is just zero.
We do want to discard this extended FinderInfo entry and convert it to a
simple fixed size FinderInfo entry, so remove the check.
Douglas Bagnall [Fri, 26 Oct 2018 07:25:59 +0000 (20:25 +1300)]
python: do not use "is" for string equality
This is not always going to work, and is not guaranteed to be
consistent even between minor versions.
Here is a simple counterexample:
>>> a = 'hello'
>>> a is 'hello'
True
>>> a is 'hello'.lower()
False
>>> a == a.lower()
True
Possibly it always works for the empty string, but we cannot rely
on that.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Mon Oct 29 23:13:36 CET 2018 on sn-devel-144
Douglas Bagnall [Thu, 25 Oct 2018 09:09:59 +0000 (22:09 +1300)]
python/samdb: properly use property()
Python's property() function works like this:
property([getter[, setter[, delete[, doc]]]])
but we have been forgetting the delete function, or rather setting it
to be a string. A string is not callable and is unlikely to succeed at
deleting the property.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andreas Schneider <asn@samba.org>
Signed-off-by: David Mulder <dmulder@suse.com> Reviewed-by: Ralph Böhme <slow@samba.org> Reviewed-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Sun Oct 28 00:35:35 CEST 2018 on sn-devel-144
Christian Ambach [Tue, 23 Oct 2018 18:05:04 +0000 (20:05 +0200)]
s3:utils/smbget fix recursive download with empty source directories
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13199 Signed-off-by: Christian Ambach <ambi@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Fri Oct 26 09:58:07 CEST 2018 on sn-devel-144
Douglas Bagnall [Thu, 11 Oct 2018 22:52:49 +0000 (11:52 +1300)]
pytests/samba3sam: unshadow and fix a search_non_mapped test
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Noel Power <npower@samba.org>
Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Fri Oct 26 00:50:37 CEST 2018 on sn-devel-144
Douglas Bagnall [Thu, 11 Oct 2018 03:15:43 +0000 (16:15 +1300)]
python/tests/kcc_utils: disambiguate/unshadow a test
Pair-programmed-with: Garming Sam <garming@catalyst.net.nz> Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Noel Power <npower@samba.org>
s3:smbd: Move a variable declaration closer to its use
Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu Oct 25 21:44:17 CEST 2018 on sn-devel-144
Volker Lendecke [Mon, 22 Oct 2018 15:18:43 +0000 (17:18 +0200)]
tdbtorture: No transaction with mutexes
Right now we don't do transactions with mutexed tdbs. tdbtorture -m
locks up. I haven't really investigated why that is the case. The lockup
confused me quite a bit until I figured out it works fine as long as it
does not do transactions, which is all we need right now.
Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org>
Volker Lendecke [Mon, 22 Oct 2018 08:10:24 +0000 (10:10 +0200)]
tdb: Remove "USE_RIGHT_MERGES" code
This has not been activated by default for ages and can be very
inefficient. With check_merge_with_left_record() we have an
alternative that will merge freelist records while we walk it
anyway. This has reduced fragmentation significantly
Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org>
Noel Power [Thu, 18 Oct 2018 08:49:24 +0000 (09:49 +0100)]
CI/Autobuild: Remove samba-none-env-py3 test
We now run a purepython3 none-env test, later when the whole
build is running under python3 we will resurrect this job
but as (samba-none-env-py2) for python2
Signed-off-by: Noel Power <noel.power@suse.com> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Oct 23 09:10:22 CEST 2018 on sn-devel-144
Noel Power [Tue, 16 Oct 2018 18:25:26 +0000 (19:25 +0100)]
script: Add new (temporary) autobuild task for none-env tests.
Ideally we want all the tests to run under python3 by default (no
special task for this) and then convert the existing '-py3' tasks
to run the python tests with python3.
However at the moment the convertion process is not ready to do this,
for a while we need to run separate autobuild tasks for this.
Signed-off-by: Noel Power <noel.power@suse.com> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
projects / thirdparty / samba.git / log
