Volker Lendecke [Thu, 7 Nov 2019 14:26:01 +0000 (15:26 +0100)]
ctdb-tcp: Close inflight connecting TCP sockets after fork
Commit c68b6f96f26 changed the talloc hierarchy such that outgoing TCP sockets
while sitting in the async connect() syscall are not freed via
ctdb_tcp_shutdown() anymore, they are hanging off a longer-running structure.
Free this structure as well.
If an outgoing TCP socket leaks into a long-running child process (possibly the
recovery daemon), this connection will never be closed as seen by the
destination node. Because with recent changes incoming connections will not be
accepted as long as any incoming connection is alive, with that socket leak
into the recovery daemon we will never again be able to successfully connect to
the node that is affected by this leak. Further attempts to connect will be
discarded by the destination as long as the recovery daemon keeps this socket
alive.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14175
RN: Avoid communication breakdown on node reconnect
Signed-off-by: Martin Schwenke <martin@meltin.net> Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit a6d99d9e5c5bc58e6d56be7a6c1dbc7c8d1a882f)
Autobuild-User(v4-9-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-9-test): Wed Nov 20 14:58:33 UTC 2019 on sn-devel-144
Martin Schwenke [Tue, 29 Oct 2019 04:29:11 +0000 (15:29 +1100)]
ctdb-tcp: Avoid orphaning the TCP incoming queue
CTDB's incoming queue handling does not check whether an existing
queue exists, so can overwrite the pointer to the queue. This used to
be harmless until commit c68b6f96f26664459187ab2fbd56767fb31767e0
changed the read callback to use a parent structure as the callback
data. Instead of cleaning up an orphaned queue on disconnect, as
before, this will now free the new queue.
At first glance it doesn't seem possible that 2 incoming connections
from the same node could be processed before the intervening
disconnect. However, the incoming connections and disconnect occur on
different file descriptors. The queue can become orphaned on node A
when the following sequence occurs:
1. Node A comes up
2. Node A accepts an incoming connection from node B
3. Node B processes a timeout before noticing that outgoing the queue is writable
4. Node B tears down the outgoing connection to node A
5. Node B initiates a new connection to node A
6. Node A accepts an incoming connection from node B
Node A processes then the disconnect of the old incoming connection
from (2) but tears down the new incoming connection from (6). This
then occurs until the originally affected node is restarted.
However, due to the number of outgoing connection attempts and
associated teardowns, this induces the same behaviour on the
corresponding incoming queue on all nodes that node A attempts to
connect to. Therefore, other nodes become affected and need to be
restarted too.
As a result, the whole cluster probably needs to be restarted to
recover from this situation.
The problem can occur any time CTDB is started on a node.
The fix is to avoid accepting new incoming connections when a queue
for incoming connections is already present. The connecting node will
simply retry establishing its outgoing connection.
Karolin Seeger [Thu, 24 Oct 2019 10:37:23 +0000 (12:37 +0200)]
VERSION: Disable GIT_SNAPSHOT for the 4.9.15 release.
* Bug 14071: CVE-2019-10218: Client code can return filenames containing path
separators.
* Bug 12438: CVE-2019-14833: Samba AD DC check password script does not receive
the full password.
* Bug 14040: CVE-2019-14847: User with "get changes" permission can crash AD DC LDAP
server via dirsync.
Karolin Seeger [Thu, 24 Oct 2019 10:36:15 +0000 (12:36 +0200)]
WHATSNEW: Add release notes for Samba 4.9.15.
* Bug 14071: CVE-2019-10218: Client code can return filenames containing path
separators.
* Bug 12438: CVE-2019-14833: Samba AD DC check password script does not receive
the full password.
* Bug 14040: CVE-2019-14847: User with "get changes" permission can crash AD DC LDAP
server via dirsync.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
(cherry picked from commit 23f72c4d712f8d1fec3d67a66d477709d5b0abe2)
Björn Baumbach [Tue, 6 Aug 2019 14:32:32 +0000 (16:32 +0200)]
CVE-2019-14833 dsdb: send full password to check password script
utf8_len represents the number of characters (not bytes) of the
password. If the password includes multi-byte characters it is required
to write the total number of bytes to the check password script.
Otherwise the last bytes of the password string would be ignored.
Therefore we rename utf8_len to be clear what it does and does
not represent.
Signed-off-by: Isaac Boukris <iboukris@redhat.com> Reviewed-by: Andreas Schneider <asn@samba.org> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Sat Oct 12 15:51:42 UTC 2019 on sn-devel-184
Autobuild-User(v4-9-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-9-test): Wed Oct 16 16:47:14 UTC 2019 on sn-devel-144
Signed-off-by: Isaac Boukris <iboukris@redhat.com> Reviewed-by: Andreas Schneider <asn@samba.org> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Isaac Boukris <iboukris@redhat.com> Reviewed-by: Andreas Schneider <asn@samba.org> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
Jeremy Allison [Mon, 26 Aug 2019 18:22:35 +0000 (11:22 -0700)]
s3/4: libsmbclient test. Test using smbc_telldir/smbc_lseekdir with smbc_readdir/smbc_readdirplus/smbc_getdents.
Ensure that for file access you can mix any of these
three access methods for directory entries and the
returned names/structs stay in sync across telldir/seekdir
changes.
Note: Unlike the current documentation, the utf16 string
is not null-terminated, that matches Windows Server 1903
as a client.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14055
RN: Add the target server name of SMB 3.1.1 connections
as a hint to load balancers or servers with "multi-tenancy"
support.
s3:ldap: Fix join with don't exists machine account
Add check for requested replies of existing machine object during join
machine to domain. This solves regression fail during join with error:
"None of the information to be translated has been translated."
https://bugzilla.samba.org/show_bug.cgi?id=14007
Reviewed-by: Guenther Deschner <gd@samba.org> Reviewed-by: Alexander Bokovoy <ab@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Günther Deschner <gd@samba.org>
Autobuild-Date(master): Wed Sep 4 17:02:37 UTC 2019 on sn-devel-184
Ralph Boehme [Fri, 12 Jul 2019 08:49:13 +0000 (10:49 +0200)]
ctdb: fix compilation on systems with glibc robust mutexes
On older systems like SLES 11 without POSIX robust mutexes, but with glib robust
mutexes where all the functions are available but have a "_np" suffix,
compilation fails in:
ctdb/tests/src/test_mutex_raw.c.239.o: In function `worker':
/root/samba-4.10.6/bin/default/../../ctdb/tests/src/test_mutex_raw.c:129: undefined reference to `pthread_mutex_consistent'
ctdb/tests/src/test_mutex_raw.c.239.o: In function `main':
/root/samba-4.10.6/bin/default/../../ctdb/tests/src/test_mutex_raw.c:285: undefined reference to `pthread_mutex_consistent'
/root/samba-4.10.6/bin/default/../../ctdb/tests/src/test_mutex_raw.c:332: undefined reference to `pthread_mutexattr_setrobust'
/root/samba-4.10.6/bin/default/../../ctdb/tests/src/test_mutex_raw.c:363: undefined reference to `pthread_mutex_consistent'
collect2: ld returned 1 exit status
This could be fixed by using libreplace system/threads.h instead of pthreads.h
directly, but as there has been a desire to keep test_mutex_raw.c standalone and
compilable without other external depenencies then libc and libpthread, make the
tool developer build only. This should get the average user over the cliff.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14038
RN: Fix compiling ctdb on older systems lacking POSIX robust mutexes
Signed-off-by: Ralph Boehme <slow@samba.org> Reviewed-by: Martin Schwenke <martin@meltin.net>
(cherry picked from commit f5388f97792ac2d7962950dad91aaf8ad49bceaa)
Autobuild-User(v4-9-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-9-test): Thu Sep 5 16:12:34 UTC 2019 on sn-devel-144
Martin Schwenke [Tue, 13 Aug 2019 04:45:33 +0000 (14:45 +1000)]
ctdb-tests: Clear deleted record via recovery instead of vacuuming
This test has been flapping because sometimes the record is not
vacuumed within the expected time period, perhaps even because the
check for the record can interfere with vacuuming. However, instead
of waiting for vacuuming the record can be cleared by doing a
recovery. This should be much more reliable.
Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Wed Aug 21 13:06:57 UTC 2019 on sn-devel-184
Martin Schwenke [Mon, 29 Jul 2019 06:45:07 +0000 (16:45 +1000)]
ctdb-tests: Don't retrieve the VNN map from target node for notlmaster
Use the VNN map from the node running node_has_status().
This means that
wait_until_node_has_status 1 notlmaster 10 0
will run "ctdb status" on node 0 and check (for up to 10 seconds) if
node 1 is in the VNN map.
If the LMASTER capability has been dropped on node 1 then the above
will wait for the VNN map to be updated on node 0. This will happen
as part of the recovery that is triggered by the change of LMASTER
capability. The next command will then only be able to attach to
$TESTDB after the recovery is complete thus guaranteeing a sane state
for the test to continue.
This stops simple/79_volatile_db_traverse.sh from going into recovery
during the traverse or at some other inconvenient time.
Martin Schwenke [Mon, 19 Aug 2019 11:48:04 +0000 (21:48 +1000)]
ctdb-daemon: Make node inactive in the NODE_STOP control
Currently some of this is supported by a periodic check in the
recovery daemon's main_loop(), which notices the flag change, sets
recovery mode active and freezes databases. If STOP_NODE returns
immediately then the associated recovery can complete and the node can
be continued before databases are actually frozen.
Instead, immediately do all of the things that make a node inactive.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14087
RN: Stop "ctdb stop" from completing before freezing databases
Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Tue Aug 20 08:32:27 UTC 2019 on sn-devel-184
Karolin Seeger [Wed, 28 Aug 2019 07:44:22 +0000 (09:44 +0200)]
WHATSNEW: Add release notes for Samba 4.9.13.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14035
CVE-2019-10197 [SECURITY][EMBARGOED] permissions check deny can allow user to
escape from the share.
CVE-2019-10197: smbd: split change_to_user_impersonate() out of change_to_user_internal()
This makes sure we always call chdir_current_service() even
when we still impersonated the user. Which is important
in order to run the SMB* request within the correct working directory
and only if the user has permissions to enter that directory.
It makes sure we always update conn->lastused_count
in chdir_current_service() for each request.
Note that vfs_ChDir() (called from chdir_current_service())
maintains its own cache and avoids calling SMB_VFS_CHDIR()
if possible.
It means we still avoid syscalls if we get a multiple requests
for the same session/tcon tuple.
CVE-2019-10197: smbd: make sure that change_to_user_internal() always resets current_user.done_chdir
We should not leave current_user.done_chdir as true if we didn't call
chdir_current_service() with success.
This caused problems in when calling vfs_ChDir() in pop_conn_ctx() when
chdir_current_service() worked once on one share but later failed on another
share.
Martin Schwenke [Tue, 13 Aug 2019 07:08:43 +0000 (17:08 +1000)]
ctdb-tcp: Mark node as disconnected if incoming connection goes away
To make it easy to pass the node data to the upcall, the private data
for ctdb_tcp_read_cb() needs to be changed from tnode to node.
RN: Avoid marking a node as connected before it can receive packets
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14084
Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Fri Aug 16 22:50:35 UTC 2019 on sn-devel-184
Martin Schwenke [Fri, 9 Aug 2019 05:33:05 +0000 (15:33 +1000)]
ctdb-tcp: Only mark a node connected if both directions are up
Nodes are currently marked as up if the outgoing connection is
established. However, if the incoming connection is not yet
established then this node could send a request where the replying
node can not queue its reply. Wait until both directions are up
before marking a node as connected.
Martin Schwenke [Thu, 15 Aug 2019 05:57:31 +0000 (15:57 +1000)]
ctdb-tcp: Create outbound queue when the connection becomes writable
Since commit ddd97553f0a8bfaada178ec4a7460d76fa21f079
ctdb_queue_send() doesn't queue a packet if the connection isn't yet
established (i.e. when fd == -1). So, don't bother creating the
outbound queue during initialisation but create it when the connection
becomes writable.
Now the presence of the queue indicates that the outbound connection
is up.
Martin Schwenke [Fri, 9 Aug 2019 05:06:34 +0000 (15:06 +1000)]
ctdb-tcp: Rename fd -> out_fd
in_fd is coming soon.
Fix coding style violations in the affected and adjacent lines.
Modernise some debug macros and make them more consistent (e.g. drop
logging of errno when strerror(errno) is already logged.
Martin Schwenke [Thu, 8 Aug 2019 06:20:44 +0000 (16:20 +1000)]
ctdb-daemon: Add function ctdb_ip_to_node()
This is the core logic from ctdb_ip_to_pnn(), so re-implement that
that function using ctdb_ip_to_node().
Something similar (ctdb_ip_to_nodeid()) was recently removed in commit 010c1d77cd7e192b1fff39b7b91fccbdbbf4a786 because it wasn't required.
Now there is a use case.
Martin Schwenke [Fri, 21 Jun 2019 19:53:15 +0000 (05:53 +1000)]
ctdb-daemon: Replace function ctdb_ip_to_nodeid() with ctdb_ip_to_pnn()
Node ID is a poorly defined concept, indicating the slot in the node
map where the IP address was found. This signed value also ends up
compared to num_nodes, which is unsigned, producing unwanted warnings.
Just return the PNN because this what both callers really want.
Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 010c1d77cd7e192b1fff39b7b91fccbdbbf4a786)
Signed-off-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> 56400153c8c7052fe319f273c30c6d59556102dc was:
Signed-off-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Garming Sam <garming@catalyst.net.nz>
(cherry picked from commit 09f2a187b3d8c161e2c11588499b3256a9dbcc95)
Signed-off-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Garming Sam <garming@catalyst.net.nz>
(cherry picked from commit 40ca8ed5a152ae7c5ec039649c09a037a20a4143)
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
(cherry picked from commit 561b654bc5bc2f5e614c5c2ab378193ca94d481a)
Douglas Bagnall [Sat, 27 Oct 2018 22:12:48 +0000 (11:12 +1300)]
s4/scripting/*: py3 compatible print
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Noel Power <noel.power@suse.com>
(cherry picked from commit 78f5b6e3999a0bf4a118df36a2aabcb696049792)
Andrew Bartlett [Tue, 16 Jul 2019 06:13:48 +0000 (18:13 +1200)]
ldb: release ldb 1.4.8
* Check for errors from ldb_unpack_data() in ldb_tdb (bug 13959)
* Check for new pack formats during startup (bug 13977)
* Make ldbdump print out pack format info and keys so we have
low level visibility for testing in python (for bug 13978)
Aaron Haslett [Mon, 20 May 2019 04:19:51 +0000 (16:19 +1200)]
ldb: ldbdump key and pack format version comments
For testing we need to know the actual KV level key of records and each
record's pack format version. This patch makes ldbdump add comments with
that info. We will parse it out in python tests.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed May 22 05:58:17 UTC 2019 on sn-devel-184
Aaron Haslett [Fri, 10 May 2019 06:10:51 +0000 (18:10 +1200)]
ldb: baseinfo pack format check on init
We will be adding a new packing format in forthcoming commits and there
may be more versions in the future. We need to make sure the database
contains records in a format we know how to read and write.
Done by fetching the @BASEINFO record and reading the first 4
bytes which contain the packing format version.
NOTE: Configure with --abi-check-disable to build this commit. This
patch is part of a set of LDB ABI changes, and the version update is
done on the last commit.
Andrew Bartlett [Wed, 22 May 2019 04:38:08 +0000 (16:38 +1200)]
ldb: Fix segfault parsing new pack formats
We need to check for the errors given by ldb_unpack() et al by preserving
the error code from kv_ctx->parser() called by tdb_parse_record() in
ltdb_parse_record().
Otherwise we will silently accept corrupt records and segfault later.
Likewise new pack formats will confuse the parser but not be
detected except by the incomplete struct ldb_message.
With this patch, the user will see a message like:
Invalid data for index DN=@BASEINFO
Failed to connect to 'st/ad_dc/private/sam.ldb' with backend 'tdb': Unable to load ltdb cache records for backend 'ldb_tdb backend'
Failed to connect to st/ad_dc/private/sam.ldb - Unable to load ltdb cache records for backend 'ldb_tdb backend'
This can be refined in the future by a specific check for
pack format versions in a higher caller, but this much is
needed regardless to detect corrupt records.
Signed-off-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Garming Sam <garming@catalyst.net.nz>
(cherry picked from commit a3101b9704f554a350493553336cbbbd7d4ae02e)
Signed-off-by: Anoop C S <anoopcs@redhat.com> Reviewed-by: Guenther Deschner <gd@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Aug 20 19:25:28 UTC 2019 on sn-devel-184
Autobuild-User(v4-9-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-9-test): Mon Aug 26 13:26:08 UTC 2019 on sn-devel-144
vfs_gpfs: Implement special case for denying owner access to ACL
In GPFS, it is not possible to deny ACL or attribute access through a
SPECIAL_OWNER entry. The best that can be done is mapping this to a
named user entry, as this one can at least be stored in an ACL. The same
cannot be done for inheriting SPECIAL_OWNER entries, as these represent
CREATOR OWNER entries, and the limitation of not being able to deny
owner access to ACL or attributes remains.
nfs4_acls: Use correct owner information for ACL after owner change
After a chown, the cached stat data is obviously no longer valid. The
code in smb_set_nt_acl_nfs4 checked the file correctly, but did only use
a local buffer for the stat data. So later checks of the stat buffer
under the fsp->fsp_name->st would still see the old information.
Fix this by removing the local stat buffer and always update the one
under fsp->fsp_name->st.
nfs4_acls: Add test for merging duplicates when mapping from NFS4 ACL to DACL
The previous patch introduced merging of duplicates on the mapping path
from NFS4 ACL entries to DACL entries. Add a testcase to verify the
expected behavior of this codepath.
nfs4_acls: Remove duplicate entries when mapping from NFS4 ACL to DACL
The previous patch added an additional entry for IDMAP_TYPE_BOTH. When
mapping back to a DACL, there should be no additional entry. Add a loop
that will check and remove entries that are exact duplicates.
nfs4_acls: Add additional owner entry when mapping to NFS4 ACL with IDMAP_TYPE_BOTH
With IDMAP_TYPE_BOTH, all entries have to be mapped to group entries.
In order to have the file system reflect the owner permissions in the
POSIX modebits, create a second entry for the user. This will be mapped
to the "special owner" entry.
The previous patch introduced a pointer to a local variable to reduce
the amount of lines changed. Remove that pointer and adjust all usage
accordingly.