John Mulligan [Thu, 6 Jun 2024 14:00:10 +0000 (10:00 -0400)]
ctdb-conf: add ctdb_read_nodes_cmd function
Add ctdb_read_nodes_cmd a function that works similarly to
ctdb_read_nodes_file but reads the nodes list from the stdout of a
subprocess instead of a file in the file system.
Signed-off-by: John Mulligan <jmulligan@redhat.com> Reviewed-by: Martin Schwenke <martin@meltin.net>
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Andreas Schneider <asn@samba.org> Signed-off-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Günther Deschner <gd@samba.org>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Tue Aug 6 00:42:56 UTC 2024 on atb-devel-224
Anoop C S [Fri, 2 Aug 2024 05:40:28 +0000 (11:10 +0530)]
vfs_ceph_new: Unconditionally use ceph_select_filesystem
Currently we don't have an explicit check for the presence of
ceph_select_filesystem() libcephfs API as it is always found to
be present with the minimum ceph version that is supported with
Samba right now. Therefore under this assumption directly call
ceph_select_filesystem() without any #ifdefs. Please note that
this change is already part of vfs_ceph via ef0068cd.
ref: https://gitlab.com/samba-team/samba/-/merge_requests/3715
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Anoop C S <anoopcs@samba.org> Reviewed-by: Guenther Deschner <gd@samba.org>
Autobuild-User(master): Anoop C S <anoopcs@samba.org>
Autobuild-Date(master): Mon Aug 5 16:06:47 UTC 2024 on atb-devel-224
Signed-off-by: Pavel Filipenský <pfilipensky@samba.org> Reviewed-by: Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Pavel Filipensky <pfilipensky@samba.org>
Autobuild-Date(master): Mon Aug 5 13:29:25 UTC 2024 on atb-devel-224
Signed-off-by: Jo Sutton <josutton@catalyst.net.nz> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Sun Aug 4 01:35:55 UTC 2024 on atb-devel-224
Jo Sutton [Tue, 30 Jul 2024 04:55:17 +0000 (16:55 +1200)]
lib:cmdline: Load network interfaces in _samba_cmdline_load_config_s4()
This makes the samba.tests.domain_backup tests start working again when
they are run standalone. Without the load_interfaces() call,
smb_sysvol_conn() fails to make a connection to the sysvol share.
Signed-off-by: Jo Sutton <josutton@catalyst.net.nz> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Direct leak of 102 byte(s) in 1 object(s) allocated from:
#0 0x7f35322fc7d7 in malloc ../../../../libsanitizer/asan/asan_malloc_linux.cpp:69
#1 0x7f3531e43bc2 in __talloc_with_prefix ../../lib/talloc/talloc.c:783
#2 0x7f3531e45034 in __talloc ../../lib/talloc/talloc.c:825
#3 0x7f3531e45034 in __talloc_strlendup ../../lib/talloc/talloc.c:2454
#4 0x7f3531e45034 in talloc_strdup ../../lib/talloc/talloc.c:2470
#5 0x7f352f90264b in smbcli_parse_unc ../../source4/libcli/cliconnect.c:269
#6 0x55fbf83aa207 in torture_parse_target ../../source4/torture/smbtorture.c:192
#7 0x55fbf83ae031 in main ../../source4/torture/smbtorture.c:744
#8 0x7f352ca2a1ef in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
Signed-off-by: Andreas Schneider <asn@samba.org> Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Thu Aug 1 16:55:43 UTC 2024 on atb-devel-224
smbd: Respect per-user hide and veto files with parametric options
For my taste this is a nicer configuration syntax than
/../username1/file1/../username2/file2/
Is this too expensive? I don't think so. The scanning only happens an
tcon time, and it only walks the parametric options. If this turns out
to be a performance problem, we should think about smarter data
structures for parametric options instead of just a linked list of
string triples for everything.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=15688 Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
lib: Factor out append_namearray from set_namearray
We'll have to add to an existing namearray soon.
This turns one talloc_array() into a set of reallocs. This is slower,
but set_namearray is only used for smb.conf entries where we don't
expect hundreds or more entries to add. I've done this to avoid array
length calculations, but if it turns out to be too slow we can get
smarter again.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=15688 Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
Shachar Sharon [Sun, 23 Jun 2024 11:57:10 +0000 (14:57 +0300)]
vfs_ceph_new: use low-level APIs for xattr ops
Implement extended-attributes operations using libcephfs' low-level
APIs. Whenever possible, use the open file-handle from fsp-extension to
resolve inode-reference and user-permissions. Otherwise, resolve both
on-the-fly.
Shachar Sharon [Thu, 20 Jun 2024 19:46:52 +0000 (22:46 +0300)]
vfs_ceph_new: use low-level APIs for linkat
Implement link operations using libcephfs' low-level APIs. Requires two
phase operation: resolve (by-lookup) reference to inode and then do the
actual (hard) link operation using parent dir-inode reference to the
locally-cached inode.
Shachar Sharon [Wed, 19 Jun 2024 14:52:45 +0000 (17:52 +0300)]
vfs_ceph_new: use low-level APIs for read/write
Implement read/write IO operations using libcephfs' low-level APIs.
Requires open ceph Fh* associated with fsp (extension) to complete both
pread/pwrite as well as async I/O operations.
Shachar Sharon [Wed, 26 Jun 2024 10:46:54 +0000 (13:46 +0300)]
vfs_ceph_new: use low-level APIs for symlink/readlink
Implement unlinkat using libcephfs low-level APIs. For readlink
operation need to resolve child inode by-lookup and then used the inode
reference for the actual low-level readlink.
Shachar Sharon [Wed, 19 Jun 2024 11:27:24 +0000 (14:27 +0300)]
vfs_ceph_new: use low-level APIs for unlinkat
Implement unlinkat using libcephfs low-level APIs. Operate using parent
directory's open file-handle. When flags has AT_REMOVEDIR bit set call
low-level rmdir; otherwise, do normal unlink.
Shachar Sharon [Wed, 19 Jun 2024 10:59:53 +0000 (13:59 +0300)]
vfs_ceph_new: use low-level APIs for fntimes
Implement fntimes hook using libcephfs' low-level APIs. Convert
smb_file_time to ceph_statx plus proper field mask on-the-fly upon
issuing low-level call to libcephfs.
Shachar Sharon [Wed, 19 Jun 2024 09:48:14 +0000 (12:48 +0300)]
vfs_ceph_new: use low-level APIs for fchown/fchmod
Use libcephfs' low-level APIs to implement 'fchown' and 'fchmod' using
open file-handle. If fsp does not have an open cephfs Fh reference,
set errno to EBADF and return -1 to VFS.
Error handling in the case of 'ceph_readdir' is done by setting 'errno'
deep within libcephfs code. In case of error, emit proper debug message
and re-update errno to avoid possible over-write by logging mechanism.
Shachar Sharon [Mon, 24 Jun 2024 10:33:05 +0000 (13:33 +0300)]
vfs_ceph_new: use low-level APIs for readdir ops
Implement readdir and rewinddir operations using libcephfs' low-level
APIs. Casts the opaque DIR pointer into struct vfs_ceph_dirp (the first
member of struct vfs_ceph_fh) to resolve the ceph_dir_result pointer
which libcephfs expects for readdir operations.
Shachar Sharon [Tue, 18 Jun 2024 14:20:59 +0000 (17:20 +0300)]
vfs_ceph_new: use low-level APIs for fdopendir
Implement fdopendir using libcephfs low-level API and cached (via fsp)
open file-handle. Embed the result within cached vfs_ceph_fh so it may
be used properly by closedir.
Shachar Sharon [Tue, 18 Jun 2024 12:02:52 +0000 (15:02 +0300)]
vfs_ceph_new: use low-level APIs for open/close
Implement openat, close and closedir and hooks using libcephfs'
low-level APIs. Cache the open Fh* from libcephfs and its related
meta-data using VFS fsp-extension mechanism.
Upon open-create of new vfs_ceph_fh store the caller credentials
(ceph's UserPerm*) within the same context object for subsequent calls.
In addition, provide a "pseudo" fd numbering which is reported back to
VFS layer and used as debugging hints.
Shachar Sharon [Tue, 18 Jun 2024 09:58:52 +0000 (12:58 +0300)]
vfs_ceph_new: ref cephmount_cached entry in handle->data
Allow direct access to ceph-mount cached-entry via 'handle->data'
private pointer. Required in order to allow more complex cached-state
with each cephfs mount. Users should now use the local-helper function
'cmount_of' to access the underlying ceph_mount_info.
Shachar Sharon [Mon, 17 Jun 2024 13:59:05 +0000 (16:59 +0300)]
vfs_ceph_new: use low-level APIs for lchown
Use libcephfs' low-level API ceph_ll_setattr to implement VFS lchown_fn
hook. Use to standard pattern of iget/iput to allow operation by Inode
reference.
Shachar Sharon [Mon, 17 Jun 2024 09:11:18 +0000 (12:11 +0300)]
vfs_ceph_new: use low-level APIs for stat
Start migrating to libcephfs' low-level APIs, using explicit Inode*
reference. Implement the VFS 'stat' hook using a ceph_ll_getattr
function, encapsulated with a pair of iget/iput to hold a
pinned-to-cache Inode* instance.
Upon calling to libcephfs this new code crates and destroys on-the-fly
a Ceph UserPerm instance based on the uig, gid and groups from
'handle->conn->session_info->unix_token'. This logic ensures that the
correct caller-credentials are passed-on to cephfs (instead of those
set upon connection-creation in legacy 'vfs_ceph.c').
Shachar Sharon [Sun, 16 Jun 2024 11:50:08 +0000 (14:50 +0300)]
vfs_ceph_new: use low-level APIs for disk_free
Start using libcephfs low-level APIs: get reference to root inode and
use it to query statfs. Requires an explicit put-inode to avoid resource
leakage by libcephfs.
Shachar Sharon [Thu, 13 Jun 2024 12:54:48 +0000 (15:54 +0300)]
vfs_ceph_new: next iteration of samba-to-cephfs bridge
Defined new module 'vfs_ceph_new.c' which serves as a place holder for
the next development phase of the bridge between samba's VFS layer and
libcephfs. Begin with a module which is almost identical to existing
'vfs_ceph.c', except for hooks-names prefix which is 'vfs_ceph_' in
order to make clear distinction from existing code base. Following
commits will also switch to low-level APIs.
* Add tdbdump -x option to output all data as hex values
* Add missing overflow check for num_values in pytdb.c
* Remove Py2 related tests
* Update times in tdb_transaction_commit per fd, not per name
* Fix compilation with TDB_TRACE=1
* Allow tracing of internal tdb
Signed-off-by: Jule Anger <janger@samba.org> Signed-off-by: Stefan Metzmacher <metze@samba.org>
Douglas Bagnall [Sun, 21 Jul 2024 06:04:49 +0000 (18:04 +1200)]
ldb:kv_index: use subtransaction_cancel in transaction_cancel
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Sat Jul 27 23:51:44 UTC 2024 on atb-devel-224
Douglas Bagnall [Mon, 22 Jul 2024 10:22:15 +0000 (22:22 +1200)]
ldb:kv_index: realloc away old dn list
We can't just free it, because has the GUID index list as a child, and
these are shared by the new dn list (from the subtransaction we are
committing). But if the dn list is long and the main transaction is
long-lived, we can save a lot of memory by turning this dn list into
an almost empty node in the talloc tree. This returns us to roughly
the situation we had prior to the last commit.
For example, with the repro.sh script on bug 15590 in indexes mode
with 10000 rules, The last 3 commits use this much memory at the end
of an unusually large transaction:
full talloc report on 'struct ldb_context' (total 4012222 bytes in 90058 blocks)
full talloc report on 'struct ldb_context' (total 2405482219 bytes in 90058 blocks)
full talloc report on 'struct ldb_context' (total 4282195 bytes in 90058 blocks)
That is, the last commit increased usage 500 fold, and this commit
brings it back to normal.
Douglas Bagnall [Tue, 25 Jun 2024 23:05:49 +0000 (11:05 +1200)]
ldb_kv_index: dn_list load sub transaction can re-use keys
We don't want to modify the original list, but we can reuse the keys
if we treat them as immutable and don't free them. That makes it a lot
quicker if there are many keys (i.e. where an index is useful) and may
sub-transactions. In particular, it avoids O(n²) talloc_memdups.
A removed comment that says "We have to free the top level index
memory otherwise we would leak", and this will be addressed in the
next commit.
Douglas Bagnall [Tue, 9 Jul 2024 02:47:25 +0000 (14:47 +1200)]
ldb_kv_cache: always initialise dn_list.strict
The strict flag is only read in list intersection, so most of the time
it doesn't matter whether it is set because that path is not used.
Nevertheless seeing it set to all kinds of values is distracting.
The undefined behaviour has likely been hidden from static analysis
because the structure is passed through the in-memory tdb before use.
Incorrect true values will have disabled an optimisation but not
caused the wrong result.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Stefan Metzmacher <metze@samba.org>
Douglas Bagnall [Wed, 10 Jul 2024 02:35:28 +0000 (14:35 +1200)]
tdb: fix compilation with TDB_TRACE=1
../../lib/tdb/common/tdb.c: In function ‘tdb_trace_record’:
../../lib/tdb/common/tdb.c:1224:22: error: ‘snprintf’ output truncated before the last format character [-Werror=format-truncation=]
1224 | p += snprintf(p, 2, %02x, rec.dptr[i]);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../lib/tdb/common/tdb.c:1224:22: note: ‘snprintf’ output 3 bytes into a destination of size 2
cc1: all warnings being treated as errors
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Pavel Filipenský <pfilipensky@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Pavel Filipensky <pfilipensky@samba.org>
Autobuild-Date(master): Fri Jul 26 18:16:15 UTC 2024 on atb-devel-224
Admin should use this script in smb.conf parameter 'sync machine
password script' in clustered samba
TODO: onnode will update the keytab on all connected nodes, so the
update will happen on the triggering node twice. This can be improved in
the future.
projects / thirdparty / samba.git / log
