Douglas Bagnall [Thu, 8 Aug 2024 23:20:38 +0000 (11:20 +1200)]
s4:drs:test:getncchanges: add a timeout failure
In the next commit we are going to add tests in which the client
modifies the highwatermark in a way that resets replication (on Samba
only). After that we'll fix it.
If we leave the test in an eternal loop, the commit history will not
be bisectable, so we are temporarily going to turn long waits into
failures.
For us to reach the statement ‘if (0 < len1)’, ‘len1’ must be equal to
‘len2’, and they must not both be equal to zero. That cannot be the case
if ‘len1’ is equal to zero, and therefore the ‘else’ branch cannot be
reached.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
David Mulder [Tue, 27 Aug 2024 15:32:43 +0000 (09:32 -0600)]
bootstrap: Migrate to Rocky8
Signed-off-by: David Mulder <dmulder@samba.org> Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Wed Aug 28 00:24:35 UTC 2024 on atb-devel-224
Anoop C S [Mon, 5 Aug 2024 13:21:49 +0000 (18:51 +0530)]
source3/wscript: Introduce auto mode to build ceph vfs modules
Use 'auto' mode as the default for building ceph vfs modules so that an
explicit --enable-cephfs can reliably fail in the absence of required
dependencies.
Signed-off-by: Anoop C S <anoopcs@samba.org> Reviewed-by: Guenther Deschner <gd@samba.org>
Autobuild-User(master): Anoop C S <anoopcs@samba.org>
Autobuild-Date(master): Tue Aug 27 06:18:51 UTC 2024 on atb-devel-224
Martin Schwenke [Tue, 13 Aug 2024 11:55:34 +0000 (21:55 +1000)]
s3:tests: Fix spelling error
Signed-off-by: Martin Schwenke <mschwenke@ddn.com> Reviewed-by: Anoop C S <anoopcs@samba.org>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Thu Aug 22 10:38:09 UTC 2024 on atb-devel-224
Shachar Sharon [Tue, 20 Aug 2024 09:06:40 +0000 (12:06 +0300)]
vfs_ceph_new: add missing newline in debug-logging
Commit d00f20f3 ("vfs_ceph_new: debug-log upon libcephfs low-level
calls") introduced debug-logging before each call to libcephfs low-level
APIs. Unfortunately, one of the logging messages missed the terminating
newline ('\n') character.
Signed-off-by: Shachar Sharon <ssharon@redhat.com> Reviewed-by: John Mulligan <jmulligan@samba.org> Reviewed-by: Anoop C S <anoopcs@samba.org>
Autobuild-User(master): Anoop C S <anoopcs@samba.org>
Autobuild-Date(master): Wed Aug 21 14:18:07 UTC 2024 on atb-devel-224
Volker Lendecke [Tue, 20 Aug 2024 13:13:52 +0000 (15:13 +0200)]
libsmb: Remove cli_state->rap_error
I could not find any caller which uses rap_error outside of the
routine implementing the rap client call itself.
Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Wed Aug 21 12:49:45 UTC 2024 on atb-devel-224
Signed-off-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Wed Aug 21 09:18:00 UTC 2024 on atb-devel-224
s3:smbd: let mkdir_internal() try VFS_RENAME_HOW_NO_REPLACE first
With renameat2(RENAME_NOREPLACE) being available
it's even better, as we don't even have the short
window where the incomplete directory is visible
to others.
The flow will be this:
tmp_name = ".::TMPNAME:D:$PID:client_name"
mkdirat(tmp_name, mode=client_mode);
prepare_acls(tmp_name);
renameat2(tmp_name, client_name, NOREPLACE);
if (EEXIST) {
unlinkat(tmp_name);
return EEXIST;
}
if (EINVAL) {
/* fallback if NOREPLACE is not supported */
mkdirat(client_name, mode=0);
if (EEXIST) {
unlinkat(tmp_name);
return EEXIST;
}
renameat(tmp_name, client_name);
}
s3:smbd: let mkdir_internal() work more atomically using a temporary name
Currently we do this in mkdir_internal():
mkdirat(client_name);
if (EEXIST) {
return EEXIST;
}
prepare_acls(client_name);
Note 'prepare_acls()' is a placeholder for the complex steps
it is doing to prepare the directory. During these steps
we have the problem that other clients already see
the directory and are able to create files or subdirectories
in it and these may not inherit the correct acls as
the their parent directory is not created completely.
I think I found a good strategie even without relying on
renameat2(RENAME_NOREPLACE).
So instead of having a long windows during prepare_acls,
we just have a short window between mkdirat(client_name, mode=0)
and renameat(tmp_name, client_name);
And in that short window the directory with the client_name
has a mode of 0, so it's not possible for other clients
to create files or subdirs in it.
As the mkdirat(client_name, mode=0) still catches
EEXIST the race where two clients try to create
the same client_name is closed as before,
so we don't need any other protection.
Following patches will make use of renameat2(RENAME_NOREPLACE),
but this already a very good improvement.
This reproduces a race where one client creates
a directory and other clients see it before
the directory is fully setup including the correct
permissions and similar things.
We have a DENY ACE for SEC_DIR_ADD_FILE, which means
that files can't be created. This is set on
a base directory 'mkdir_visible'.
Then we have a lot of async loops trying to create
a file called 'mkdir_visible\dir\file_NR'. These loop
as fast as possible expecting OBJECT_PATH_NOT_FOUND,
because 'mkdir_visible\dir' is not there.
Then we send a create for 'mkdir_visible\dir' and
expect that to work.
This should turn the 'mkdir_visible\dir\file_NR' loop
into getting ACCESS_DENIED, because the
DENY ACE for SEC_DIR_ADD_FILE should be inherited
before 'mkdir_visible\dir' is visible to other clients.
Because of the complex steps in mkdir_internal(),
smbd allows the creation 'mkdir_visible\dir\file_NR',
as 'mkdir_visible\dir' is already visible after the
mkdirat(), before the DENY ACE is inherited.
'samba-tool domain provision -d10' fails if the included file does not
exist:
lpcfg_load: refreshing parameters from /etc/samba/smb.conf
Processing section "[global]"
Can't find include file /etc/samba/usershares.conf
pm_process() returned No
ERROR: Unable to load default file
File "/usr/lib64/python3.12/site-packages/samba/netcmd/domain/provision.py", line 183, in run
lp = sambaopts.get_loadparm()
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.12/site-packages/samba/getopt.py", line 282, in get_loadparm
self._lp.load_default()
Signed-off-by: Pavel Filipenský <pfilipensky@samba.org> Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Wed Aug 21 00:04:19 UTC 2024 on atb-devel-224
Martin Schwenke [Sat, 29 Jun 2024 02:25:59 +0000 (12:25 +1000)]
ctdb-scripts: Avoid flapping NFS services at startup
If an NFS service check is set to, say, unhealthy_after=2 then it will
always switch from the (default startup) unhealthy state to healthy,
even if there is a fatal problem. If all services/scripts appear OK
then the node will become healthy. When the counter hits the limit it
will return to unhealthy. This is misleading.
Instead, never use the counter at startup, until the service becomes
healthy. This stops services flapping unhealthy-healthy-unhealthy.
A side-effect is that a service that starts in a broken state will
never be restarted to try to fix the problem. This makes sense. The
counting and restarting really exist to deal with problems that might
occur under load. The first monitor events occur before public IPs
are hosted, so there can be no load. If a service doesn't start
reliably the first time then the admin probably wants to know about
it.
nfs_iterate_test() is updated to run an initial monitor event to mark
the services as healthy. This initialises the counter so it can be
used for the important part of the test. Passing the -i option avoids
running the extra monitor event, so the first iteration will be the
initial monitor event.
Signed-off-by: Martin Schwenke <mschwenke@ddn.com> Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Sat, 29 Jun 2024 09:24:25 +0000 (19:24 +1000)]
ctdb-scripts: Make initial statistics output empty
This makes initial failure to retrieve statistics less likely to
result in a statistics change. To help with this, statistics
retrieval stderr now goes to the log - only stdout goes to the file.
This means that the test code for checking statistics changes needs to
be redone to actually run the statistics command and check. As with
rpcinfo output, this output needs to behave as deterministically in
the test code as it done in the event script.
Signed-off-by: Martin Schwenke <mschwenke@ddn.com> Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Sun, 30 Jun 2024 00:35:09 +0000 (10:35 +1000)]
ctdb-scripts: Only consider statistics on timeout
Checking statistics is only really relevant to timeouts. That is, if
an rpcinfo times out it is worth checking if the service making
progress. If the RPC service is not registered then the statistics
don't need to be checked because they shouldn't be changing.
The 2 previously added tests added to check statistics progress now
behave identically and fail on all iterations. To support testing
with "timeouts", an optional TIMEOUT flag can now be added to the RPC
service passed to nfs_iterate_test(). 2 new tests are added to
exercise the new behaviour.
The 2 new "if" statements in nfs_iterate_test() could be combined.
However, a subsequent commit would split them and would be more
difficult to read.
Signed-off-by: Martin Schwenke <mschwenke@ddn.com> Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Thu, 4 Jul 2024 01:10:59 +0000 (11:10 +1000)]
ctdb-tests: Make NFS RPC monitoring tests consistent
Update the remaining RPC monitoring tests to use nfs_iterate_test(),
depending on it to set results. This makes all RPC monitoring tests
consistent, so they will all benefit from future improvements.
Signed-off-by: Martin Schwenke <mschwenke@ddn.com> Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Fri, 5 Jul 2024 00:46:30 +0000 (10:46 +1000)]
ctdb-tests: Simplify handling of statistics change
Handling this across two different functions led to insanity, so
simplify.
The handling of unhealthy_after when $_numfails = 0 implicitly causes
the node to be healthy. This is how the "rpcinfo succeeds" case
works. Doing it this way for statistics makes this patch easier to
read. The implicit behaviour will go away in the next patch.
Signed-off-by: Martin Schwenke <mschwenke@ddn.com> Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Fri, 9 Aug 2024 01:20:06 +0000 (11:20 +1000)]
ctdb-tcp: Use path_rundir_append() to construct lock_path
The current constant value doesn't respect CTDB_TEST_MODE/CTDB_BASE.
Instead use the path module to allow automatic listening in test mode
with local daemons.
A single node can be tested with local daemons, using something like:
The trick is that commenting out the node address in ctdb.conf means
the chosen node address is the first one from the nodes file that
allows bind/listen. In this case it is the only line.
The following ensures that automatic listening works for a node that
isn't the first:
Note that the first address isn't local on this host, so will always
fail.
So, doing the above and starting both nodes yields...
...
$ tests/local_daemons.sh foo start 1
$ sleep 3; tests/local_daemons.sh foo start 0
$ tests/local_daemons.sh foo print-log all | grep -i 'chose\|bind'
[...] node.1 ctdbd[26351]: ctdb chose network address 127.0.0.1:4379
[...] node.0 ctdbd[26438]: ctdb_tcp_listen_addr: Failed to bind() to socket - Address already in use (98)
[...] node.0 ctdbd[26438]: Unable to bind to any node address - giving up
... as expected.
It would be nice to add tests for this, but we don't really have
infrastructure for that. At least manual testing shows, for the
obvious cases, the previous commits didn't break anything. :-)
Signed-off-by: Martin Schwenke <mschwenke@ddn.com> Reviewed-by: Volker Lendecke <vl@samba.org>
Martin Schwenke [Fri, 26 Jul 2024 00:49:16 +0000 (10:49 +1000)]
ctdb-daemon: Remove a use of ctdb_errstr()
Code to setup the transport is about to be cleaned up, including
removing uses of ctdb_set_error(), so avoid logging a NULL pointer or
some other old error.
Signed-off-by: Martin Schwenke <mschwenke@ddn.com> Reviewed-by: Volker Lendecke <vl@samba.org>
projects / thirdparty / samba.git / log
