Shachar Sharon [Sun, 23 Jun 2024 11:57:10 +0000 (14:57 +0300)]
vfs_ceph_new: use low-level APIs for xattr ops
Implement extended-attributes operations using libcephfs' low-level
APIs. Whenever possible, use the open file-handle from fsp-extension to
resolve inode-reference and user-permissions. Otherwise, resolve both
on-the-fly.
Shachar Sharon [Thu, 20 Jun 2024 19:46:52 +0000 (22:46 +0300)]
vfs_ceph_new: use low-level APIs for linkat
Implement link operations using libcephfs' low-level APIs. Requires two
phase operation: resolve (by-lookup) reference to inode and then do the
actual (hard) link operation using parent dir-inode reference to the
locally-cached inode.
Shachar Sharon [Wed, 19 Jun 2024 14:52:45 +0000 (17:52 +0300)]
vfs_ceph_new: use low-level APIs for read/write
Implement read/write IO operations using libcephfs' low-level APIs.
Requires open ceph Fh* associated with fsp (extension) to complete both
pread/pwrite as well as async I/O operations.
Shachar Sharon [Wed, 26 Jun 2024 10:46:54 +0000 (13:46 +0300)]
vfs_ceph_new: use low-level APIs for symlink/readlink
Implement unlinkat using libcephfs low-level APIs. For readlink
operation need to resolve child inode by-lookup and then used the inode
reference for the actual low-level readlink.
Shachar Sharon [Wed, 19 Jun 2024 11:27:24 +0000 (14:27 +0300)]
vfs_ceph_new: use low-level APIs for unlinkat
Implement unlinkat using libcephfs low-level APIs. Operate using parent
directory's open file-handle. When flags has AT_REMOVEDIR bit set call
low-level rmdir; otherwise, do normal unlink.
Shachar Sharon [Wed, 19 Jun 2024 10:59:53 +0000 (13:59 +0300)]
vfs_ceph_new: use low-level APIs for fntimes
Implement fntimes hook using libcephfs' low-level APIs. Convert
smb_file_time to ceph_statx plus proper field mask on-the-fly upon
issuing low-level call to libcephfs.
Shachar Sharon [Wed, 19 Jun 2024 09:48:14 +0000 (12:48 +0300)]
vfs_ceph_new: use low-level APIs for fchown/fchmod
Use libcephfs' low-level APIs to implement 'fchown' and 'fchmod' using
open file-handle. If fsp does not have an open cephfs Fh reference,
set errno to EBADF and return -1 to VFS.
Error handling in the case of 'ceph_readdir' is done by setting 'errno'
deep within libcephfs code. In case of error, emit proper debug message
and re-update errno to avoid possible over-write by logging mechanism.
Shachar Sharon [Mon, 24 Jun 2024 10:33:05 +0000 (13:33 +0300)]
vfs_ceph_new: use low-level APIs for readdir ops
Implement readdir and rewinddir operations using libcephfs' low-level
APIs. Casts the opaque DIR pointer into struct vfs_ceph_dirp (the first
member of struct vfs_ceph_fh) to resolve the ceph_dir_result pointer
which libcephfs expects for readdir operations.
Shachar Sharon [Tue, 18 Jun 2024 14:20:59 +0000 (17:20 +0300)]
vfs_ceph_new: use low-level APIs for fdopendir
Implement fdopendir using libcephfs low-level API and cached (via fsp)
open file-handle. Embed the result within cached vfs_ceph_fh so it may
be used properly by closedir.
Shachar Sharon [Tue, 18 Jun 2024 12:02:52 +0000 (15:02 +0300)]
vfs_ceph_new: use low-level APIs for open/close
Implement openat, close and closedir and hooks using libcephfs'
low-level APIs. Cache the open Fh* from libcephfs and its related
meta-data using VFS fsp-extension mechanism.
Upon open-create of new vfs_ceph_fh store the caller credentials
(ceph's UserPerm*) within the same context object for subsequent calls.
In addition, provide a "pseudo" fd numbering which is reported back to
VFS layer and used as debugging hints.
Shachar Sharon [Tue, 18 Jun 2024 09:58:52 +0000 (12:58 +0300)]
vfs_ceph_new: ref cephmount_cached entry in handle->data
Allow direct access to ceph-mount cached-entry via 'handle->data'
private pointer. Required in order to allow more complex cached-state
with each cephfs mount. Users should now use the local-helper function
'cmount_of' to access the underlying ceph_mount_info.
Shachar Sharon [Mon, 17 Jun 2024 13:59:05 +0000 (16:59 +0300)]
vfs_ceph_new: use low-level APIs for lchown
Use libcephfs' low-level API ceph_ll_setattr to implement VFS lchown_fn
hook. Use to standard pattern of iget/iput to allow operation by Inode
reference.
Shachar Sharon [Mon, 17 Jun 2024 09:11:18 +0000 (12:11 +0300)]
vfs_ceph_new: use low-level APIs for stat
Start migrating to libcephfs' low-level APIs, using explicit Inode*
reference. Implement the VFS 'stat' hook using a ceph_ll_getattr
function, encapsulated with a pair of iget/iput to hold a
pinned-to-cache Inode* instance.
Upon calling to libcephfs this new code crates and destroys on-the-fly
a Ceph UserPerm instance based on the uig, gid and groups from
'handle->conn->session_info->unix_token'. This logic ensures that the
correct caller-credentials are passed-on to cephfs (instead of those
set upon connection-creation in legacy 'vfs_ceph.c').
Shachar Sharon [Sun, 16 Jun 2024 11:50:08 +0000 (14:50 +0300)]
vfs_ceph_new: use low-level APIs for disk_free
Start using libcephfs low-level APIs: get reference to root inode and
use it to query statfs. Requires an explicit put-inode to avoid resource
leakage by libcephfs.
Shachar Sharon [Thu, 13 Jun 2024 12:54:48 +0000 (15:54 +0300)]
vfs_ceph_new: next iteration of samba-to-cephfs bridge
Defined new module 'vfs_ceph_new.c' which serves as a place holder for
the next development phase of the bridge between samba's VFS layer and
libcephfs. Begin with a module which is almost identical to existing
'vfs_ceph.c', except for hooks-names prefix which is 'vfs_ceph_' in
order to make clear distinction from existing code base. Following
commits will also switch to low-level APIs.
* Add tdbdump -x option to output all data as hex values
* Add missing overflow check for num_values in pytdb.c
* Remove Py2 related tests
* Update times in tdb_transaction_commit per fd, not per name
* Fix compilation with TDB_TRACE=1
* Allow tracing of internal tdb
Signed-off-by: Jule Anger <janger@samba.org> Signed-off-by: Stefan Metzmacher <metze@samba.org>
Douglas Bagnall [Sun, 21 Jul 2024 06:04:49 +0000 (18:04 +1200)]
ldb:kv_index: use subtransaction_cancel in transaction_cancel
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Sat Jul 27 23:51:44 UTC 2024 on atb-devel-224
Douglas Bagnall [Mon, 22 Jul 2024 10:22:15 +0000 (22:22 +1200)]
ldb:kv_index: realloc away old dn list
We can't just free it, because has the GUID index list as a child, and
these are shared by the new dn list (from the subtransaction we are
committing). But if the dn list is long and the main transaction is
long-lived, we can save a lot of memory by turning this dn list into
an almost empty node in the talloc tree. This returns us to roughly
the situation we had prior to the last commit.
For example, with the repro.sh script on bug 15590 in indexes mode
with 10000 rules, The last 3 commits use this much memory at the end
of an unusually large transaction:
full talloc report on 'struct ldb_context' (total 4012222 bytes in 90058 blocks)
full talloc report on 'struct ldb_context' (total 2405482219 bytes in 90058 blocks)
full talloc report on 'struct ldb_context' (total 4282195 bytes in 90058 blocks)
That is, the last commit increased usage 500 fold, and this commit
brings it back to normal.
Douglas Bagnall [Tue, 25 Jun 2024 23:05:49 +0000 (11:05 +1200)]
ldb_kv_index: dn_list load sub transaction can re-use keys
We don't want to modify the original list, but we can reuse the keys
if we treat them as immutable and don't free them. That makes it a lot
quicker if there are many keys (i.e. where an index is useful) and may
sub-transactions. In particular, it avoids O(n²) talloc_memdups.
A removed comment that says "We have to free the top level index
memory otherwise we would leak", and this will be addressed in the
next commit.
Douglas Bagnall [Tue, 9 Jul 2024 02:47:25 +0000 (14:47 +1200)]
ldb_kv_cache: always initialise dn_list.strict
The strict flag is only read in list intersection, so most of the time
it doesn't matter whether it is set because that path is not used.
Nevertheless seeing it set to all kinds of values is distracting.
The undefined behaviour has likely been hidden from static analysis
because the structure is passed through the in-memory tdb before use.
Incorrect true values will have disabled an optimisation but not
caused the wrong result.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Stefan Metzmacher <metze@samba.org>
Douglas Bagnall [Wed, 10 Jul 2024 02:35:28 +0000 (14:35 +1200)]
tdb: fix compilation with TDB_TRACE=1
../../lib/tdb/common/tdb.c: In function ‘tdb_trace_record’:
../../lib/tdb/common/tdb.c:1224:22: error: ‘snprintf’ output truncated before the last format character [-Werror=format-truncation=]
1224 | p += snprintf(p, 2, %02x, rec.dptr[i]);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../lib/tdb/common/tdb.c:1224:22: note: ‘snprintf’ output 3 bytes into a destination of size 2
cc1: all warnings being treated as errors
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Pavel Filipenský <pfilipensky@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Pavel Filipensky <pfilipensky@samba.org>
Autobuild-Date(master): Fri Jul 26 18:16:15 UTC 2024 on atb-devel-224
Admin should use this script in smb.conf parameter 'sync machine
password script' in clustered samba
TODO: onnode will update the keytab on all connected nodes, so the
update will happen on the triggering node twice. This can be improved in
the future.
s3:lib: Merge library trusts_util into library ads
Function trust_pw_change() originally from library trustis_util was updated
to call functionality from ads library. This would introduce circular
dependency between the two libraries. To avoid it, trusts_util is merged
into ads.
Ralph Boehme [Fri, 2 Feb 2024 14:14:27 +0000 (15:14 +0100)]
WHATSNEW.txt: document "veto files" and "hide files"
Signed-off-by: Ralph Boehme <slow@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Fri Jul 26 11:10:42 UTC 2024 on atb-devel-224
Ralph Boehme [Fri, 2 Feb 2024 07:10:54 +0000 (08:10 +0100)]
smbd: return errors from token_contains_name()
Invalid names in "valid users", "invalid users", "read list", "write list",
"veto files" and "hide files" are logged and ignored, but a failure to contact
winbind or a DC from winbind, or a memory allocation failure, now all trigger a
failure of the tree connect.
Manually tested with smbclient with the following hack in winbindd:
Ralph Boehme [Fri, 9 Feb 2024 09:53:35 +0000 (10:53 +0100)]
s3/passdb: factor out lookup_name_internal()
lookup_name() becomes a thin wrapper around lookup_name_internal(). Prepares for
adding more callers to lookup_name_internal() that are interested in the
NTSTATUS return value to distinguish between NAME_NOT_FOUND lookup results and
real errors.
Signed-off-by: Ralph Boehme <slow@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
Ralph Boehme [Wed, 6 Mar 2024 13:23:45 +0000 (14:23 +0100)]
s3/rpc_client: fix handling of NT_STATUS_SOME_NOT_MAPPED
In this case names that couldn't be resolved will be have a NULL sid pointer
which would trigger a crash in sid_copy().
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org> Signed-off-by: Ralph Boehme <slow@samba.org> Signed-off-by: Stefan Metzmacher <metze@samba.org>
Ralph Boehme [Fri, 16 Feb 2024 15:44:57 +0000 (16:44 +0100)]
winbindd: let LookupNames return NT_STATUS_OK and SID_NAME_UNKNOWN for unmapped names
Previously LookupNames would fail if a name could not be translated, so winbindd
clients like libwbclient couldn't differentiate between not being able to talk
to a DC and just an unkown name.
As a visible change this alters
$ bin/wbinfo -n Idontexist
failed to call wbcLookupName: WBC_ERR_DOMAIN_NOT_FOUND
Could not lookup name Idontexist
to
$ bin/wbinfo -n Idontexist
failed to call wbcLookupName: WBC_ERR_SOME_NOT_MAPPED
Could not lookup name Idontexist
Signed-off-by: Ralph Boehme <slow@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
Ralph Boehme [Wed, 7 Feb 2024 09:18:13 +0000 (10:18 +0100)]
smbd: maintain veto_list and hide_list in the vuid cache
Prepares for adding per-user support to both options. I don't think it makes
sense also adding per-user support to "veto oplock files" and "aio write behind"
so I'm ignoring those.
Signed-off-by: Ralph Boehme <slow@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
projects / thirdparty / samba.git / log
