Martin Schwenke [Thu, 15 May 2025 04:01:16 +0000 (14:01 +1000)]
ctdb-daemon: Run "startipreallocate" event in SHUTDOWN runstate
Even though all nodes may be shutting down there is still a very small
window for a race when multiple nodes are shut down. For simplicity,
assume 2 nodes. Assume the shutdowns of nodes are staggered, which is
usual because they're usually initiated by a loop (e.g. onnode -p all
ctdb shutdown). Although commands can continue in parallel, some
commands are started later than others.
Consider this sequence:
1. Node 0 reaches ctdb_shutdown_takeover() in
ctdb_shutdown_sequence() and a takeover run starts
2. Node 1 has not yet set its runlevel to SHUTDOWN in
ctdb_shutdown_sequence()
3. The leader node asks node 1 which IPs it can host
4. Node 1 replies "all of them"
5. Node 1 now sets its runlevel to SHUTDOWN in
ctdb_shutdown_sequence()
6. The leader node continues with the takeover run, first asking all
nodes to run "startipreallocate"
7. Node 0 runs "startipreallocate", so its NFS server starts grace
8. Node 1 does not run "startipreallocate" because it is not in
RUNNING runstate, so its NFS server does not start grace
9. The leader node continues with the takeover run, first asking all
nodes to run "releaseip" for IPs they can no longer hold
10. Node 0 releases all IPs, since it is SHUTDOWN runstate (so can't
host IPs)
11. As part of this, the NFS server on node 0 releases locks held
against IPs it is releasing
12. A client connected to node 1, where the NFS server is not in
grace, takes ("steals") one of those locks
This client is then permitted to reclaim the lock when nodes are
restarted.
Martin Schwenke [Mon, 12 May 2025 01:33:19 +0000 (11:33 +1000)]
ctdb-daemon: Add failover on shutdown
Without this, NFS servers on other nodes will not go into grace before
this node releases locks. This should also support improved behaviour
for SMB durable file handles.
The timeout is currently a constant 10s. However, it will
subsequently be switched to an option.
Martin Schwenke [Wed, 14 May 2025 06:55:51 +0000 (16:55 +1000)]
ctdb-protocol: Add CTDB server SRVID range
Normally, communication from other components to ctdbd is done via
controls. However, there are contexts where receiving SRVID messages
in ctdbd makes sense, such as replies to outgoing SRVID messages.
Martin Schwenke [Wed, 21 May 2025 12:17:42 +0000 (22:17 +1000)]
ctdb-daemon: Avoid aborting during early shutdown
An early shutdown can put ctdbd into SHUTDOWN runstate before ctdbd
has completed all early initialisation. Some of the start-time
transitions then attempt to set the runstate to FIRST_RECOVERY or
RUNNING, which would make the runstate go backwards, so ctdbd aborts.
Upcoming changes cause ctdbd shutdown to take longer, so the problem
will become more likely. With those changes, this can be
unreliably (50% of the time?) triggered by:
since it does an early shutdown due to a version mismatch.
Avoid this by noticing when the runstate is SHUTDOWN and refusing to
continue with subsequent early initialisation steps, which aren't
needed when shutting down.
Earlier runstate transitions do not seems likely to cause an abort
during early shutdown. The following:
sees ctdbd already into FIRST_RECOVERY before the shutdown is
processed.
The change to ctdb_run_startup() probably isn't strictly necessary.
There will be no abort in this case. ctdb_shutdown_sequence() will
always run the "shutdown" event and then stop the event daemon, so it
doesn't seem possible that services could be left running. However,
we might as well avoid running the "startup" event when shutting down,
even if only to avoid confusing logs.
Ultimately, it seems like some redesign would be needed to avoid this
in a more predictable manner, rather than responding when an early
initialisation step inconveniently completes during shutdown. For
example, hanging a lot of the start-time event handling off a common
talloc context, could allow it to be cancelled with a single
TALLOC_FREE(). However, a change like that would involve a lot of
analysis to ensure that the talloc hierarchy is correct and there is
no change of free'd pointers being dereferenced. So, we're probably
better off just keeping this issue in mind during a broader redesign.
Use the same logic from shadow_copy2 module to always prepend the
connectpath to the relative snapshot path so as to return converted
path corresponding to the file's share root.
Please note that with the current working directory staying at the
connectpath level we are safe to prefix it to the smb_filename. In
other words it seems we never get past the connectpath internally
during normal file system operations via chdir(). Since all relative
paths are now based on dirfsp we could constitute absolute path by
prepending the connectpath to full_path_from_dirfsp_atname() output
ignoring the current working directory.
Signed-off-by: Anoop C S <anoopcs@samba.org> Reviewed-by: David Disseldorp <ddiss@samba.org>
Autobuild-User(master): Anoop C S <anoopcs@samba.org>
Autobuild-Date(master): Wed Apr 30 11:32:59 UTC 2025 on atb-devel-224
Anoop C S [Tue, 4 Mar 2025 09:39:33 +0000 (15:09 +0530)]
vfs_ceph_snapshots: Use full path from dirfsp at smb_fname
In ceph_snap_gmt_openat() we hand in the incoming smb_fname as it is
to ceph_snap_gmt_strip_snapshot() which is then passed on to derive
the actual snapshot path using ceph_snap_gmt_convert(). But this can
go wrong in ceph_snap_gmt_convert_dir() while opening the snapdir.
Unless we constitute the full path from dirfsp at the first place we
always end up opening the snapdir from the parent directory with
OpenDir().
For example with dirfsp("foobar") and smb_fname("shift.txt"), we open
snapdir from share root because parent is calculated as empty string
via ceph_snap_get_parent_path(). Instead we could construct the full
path from dirfsp using full_path_from_dirfsp_atname() to ensure we
don't open the wrong snapdir.
Since we have access to the twrp token at VFS layer it doesn't make
much sense to make use of ceph_snap_gmt_strip_snapshot() in openat.
We could instead directly act based on already available twrp token
avoiding an extra copy of incoming smb_filename.
Signed-off-by: Pavel Filipenský <pfilipensky@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Wed Apr 23 12:14:17 UTC 2025 on atb-devel-224
Signed-off-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org>
(cherry picked from commit 783ca9dc434bd1d18b762185ee936fcbcf292124)
vfs_wide_links hides symlinks from the rest of smbd, and it implicitly
follows symlinks. Also, O_PATH will expose symlinks to the rest of
smbd, remove that.
We also need to do this for posix paths, as deep inside
rename_internals we want to avoid case-insensitive lookups by setting
SMB_FILENAME_POSIX_PATH.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Jennifer Sutton <jennifersutton@catalyst.net.nz>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Tue Mar 25 05:21:49 UTC 2025 on atb-devel-224
Douglas Bagnall [Fri, 14 Mar 2025 04:45:18 +0000 (17:45 +1300)]
samba-tool gpo: better entities check copes with new lines
Per https://www.w3.org/TR/xml/#sec-entity-decl (and MS references)
there is always some whitespace between '<!ENTITY' and the name, and
between the name and whatever is next. Also, it is valid XML to have
newlines inside entity declarations, like this:
<!ENTITY
bubble
"*S-1-5-113"
>
We used to create such files, so we should allow them.
There is a kind of entity that has '%' before the name, and there are
non-ascii names, which we continue not to support.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Jennifer Sutton <jennifersutton@catalyst.net.nz>
(cherry picked from commit 6107656ebc8d092b2c1907940b2486ab0265aad9)
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Jennifer Sutton <jennifersutton@catalyst.net.nz>
(cherry picked from commit 65751f2562f98bd7fd0734dc00784e6395d76322)
This meant we were testing nothing because the assertions are all that
the files are the same -- though the only affected check is one in
test_backup_restore_generalize().
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Jennifer Sutton <jennifersutton@catalyst.net.nz>
(cherry picked from commit 6b619b568f6661d3a5f0701cdfaf1e1e4943ff6f)
Andreas Hasenack [Tue, 18 Feb 2025 15:43:46 +0000 (12:43 -0300)]
python:netcmd:gpo: fix crash when updating an MOTD GPO
When the policy exists already, there is no exception and the code
tries to use the "data" variable, but it doesn't exist because it was
only defined in the exception handling.
Signed-off-by: Andreas Hasenack <andreas.hasenack@canonical.com> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Jennifer Sutton <jennifersutton@catalyst.net.nz>
(cherry picked from commit e87e20c04d90292e3a5caac8ea3105b16f948ed3)
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Jennifer Sutton <jennifersutton@catalyst.net.nz>
(cherry picked from commit 969cb41e06247949c3992cab25e824795204e31e)
Signed-off-by: Ralph Boehme <slow@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit c36cc2b6720a2cfe54ce52a500dc499418e27e34)
Signed-off-by: Ralph Boehme <slow@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
(backported from commit dc03a06ffcc79d0818ae4a36fe3f2df705144138)
[slow@samba.org: conflict due to removed delayed write time handling]
[slow@samba.org: conflict due to filename_convert_dirfsp_rel()]
Signed-off-by: Ralph Boehme <slow@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
(backported from commit 393379fc9c726eb781fd1bfb3a70ea2802739aff)
[slow@samba.org: conflict due to removed delayed write time handling]
Signed-off-by: Ralph Boehme <slow@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 56bb20c87a733ab8f7efedd881ea0ecaf51b2ba8)
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org> Signed-off-by: Ralph Boehme <slow@samba.org> Signed-off-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 678f28c1af7c160ffdcb0e4baa0a7d4b9906f2e5)
Signed-off-by: Ralph Boehme <slow@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 8f9387ceb5c94c7db92ab342e33c64b858c301b1)
Signed-off-by: Ralph Boehme <slow@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 4d680b6c17ee7674b9686aec2b69038f89e1989a)
Pair-Programmed-With: Ralph Boehme <slow@samba.org> Signed-off-by: Ralph Boehme <slow@samba.org> Signed-off-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 0c4c430c50e15d591a0d871a5f3e59e8be0d0a83)
Ralph Boehme [Wed, 29 Jan 2025 05:13:44 +0000 (06:13 +0100)]
smbd: use share_mode_do_locked_brl()
Fix a deadlock that can happen if two clients happen to open and byte-range-lock
two different files whos record in locking.tdb and brlock.tdb happen to sit on
the same hashchain.
The deadlock was introduced by commit 680c7907325b433856ac1dd916ab63e671fbe4ab. Before, we used share_mode_do_locked()
in do_lock() which meant we acquired a chainlock on locking.tdb before getting a
chainlock on brlock.tdb via brl_get_locks_for_locking(), so the TDB chainlock
order invariant was always uphold.
The following race between specific client requests lead to the deadlock.
Client A) issues a byte-range-lock request on a file:
B1) glock and chainlock locking.tdb (via _share_mode_entry_prepare_lock())
B2) attempt to chainlock brlock.tdb (via file_has_brlocks())
[2]
The glock from A1 is per record and hence doesn't synchronize with the glock
from B1 as it is for a different file and hence a different record, subsequently
A2 and A3 violate the lock order constraint
To avoid the chainlock lock order violation in the second client we modify the
br-lock code to not take the brlock.tdb chainlock from step A2 via
br_get_locks() for the whole time we process the request. Instead we just fetch
the br-locks via br_get_locks_readonly(), so when running into
contend_level2_oplocks_begin_default() to check for leases and looking into
locking.tdb we don't hold a brlock.tdb chainlock.
Or im simpler terms, we only ever take at most one low-level TDB chainlock at a
time:
Byte-range-lock code calls share_mode_do_locked_brl(..., cb_fn, ...):
1) chainlock locking.tdb
2) glock locking.tdb (via share_mode_do_locked_vfs_allowed())
3) chainunlock locking.tdb
4) share_mode_do_locked_brl_fn() -> brl_get_locks_readonly_parse():
a) chainlock brlock.tdb
b) parse record and store in-memory copy
c) chainunlock brlock.tdb
5) run cb_fn()
6) chainlock brlock.tdb:
a) br_lck->record = dbwrap_fetch_locked(brlock_db, ...)
b) store modifed br_lck from 5) via byte_range_lock_flush()
7) chainunlock brlock.tdb
8) chainlock locking.tdb
9) gunlock locking.tdb
10) chainunlock locking.tdb
All access to brlock.tdb is synchronized correctly via glocks on the locking.tdb
record of the file (step 3)), so operations still appear atomic to clients.
As a result of using share_mode_do_locked_brl(), the functions do_[un]lock() ->
brl_[un]lock() now loop over the same br_lck object in memory, avoiding
repeatedly fetching and storing the locks per loop.
[1]
Full SBT:
#0 0x00007fffa0cecbb0 in __pthread_mutex_lock_full () from /lib64/glibc-hwcaps/power9/libpthread-2.28.so
#1 0x00007fffa0a73cf8 in chain_mutex_lock (m=<optimized out>, m@entry=0x7fff9ae071b0, waitflag=<optimized out>, waitflag@entry=true) at ../../lib/tdb/common/mutex.c:182
#2 0x00007fffa0a7432c in tdb_mutex_lock (tdb=0x1543ba120, rw=<optimized out>, off=<optimized out>, len=<optimized out>, waitflag=<optimized out>, pret=0x7fffd7df3858) at ../../lib/tdb/common/mutex.c:234
#3 0x00007fffa0a6812c in fcntl_lock (waitflag=<optimized out>, len=1, off=376608, rw=0, tdb=0x1543ba120) at ../../lib/tdb/common/lock.c:200
#4 tdb_brlock (tdb=0x1543ba120, rw_type=<optimized out>, offset=<optimized out>, len=1, flags=<optimized out>) at ../../lib/tdb/common/lock.c:200
#5 0x00007fffa0a68af8 in tdb_nest_lock (flags=<optimized out>, ltype=0, offset=<optimized out>, tdb=0x1543ba120) at ../../lib/tdb/common/lock.c:390
#6 tdb_nest_lock (tdb=0x1543ba120, offset=<optimized out>, ltype=<optimized out>, flags=<optimized out>) at ../../lib/tdb/common/lock.c:336
#7 0x00007fffa0a69088 in tdb_lock_list (tdb=0x1543ba120, list=<optimized out>, ltype=<optimized out>, waitflag=<optimized out>) at ../../lib/tdb/common/lock.c:482
#8 0x00007fffa0a69198 in tdb_lock (tdb=0x1543ba120, list=<optimized out>, ltype=<optimized out>) at ../../lib/tdb/common/lock.c:500
#9 0x00007fffa0a64b50 in tdb_find_lock_hash (tdb=<optimized out>, tdb@entry=0x1543ba120, key=..., hash=<optimized out>, locktype=<optimized out>, locktype@entry=0, rec=<optimized out>, rec@entry=0x7fffd7df3ab0) at ../../lib/tdb/common/tdb.c:165
#10 0x00007fffa0a64ed0 in tdb_parse_record (tdb=0x1543ba120, key=..., parser=0x7fffa0e74470 <db_ctdb_ltdb_parser>, private_data=0x7fffd7df3b18) at ../../lib/tdb/common/tdb.c:329
#11 0x00007fffa0e74cbc in db_ctdb_ltdb_parse (db=<optimized out>, private_data=0x7fffd7df3b70, parser=0x7fffa0e76470 <db_ctdb_parse_record_parser_nonpersistent>, key=...) at ../../source3/lib/dbwrap/dbwrap_ctdb.c:170
#12 db_ctdb_try_parse_local_record (ctx=ctx@entry=0x1543d4580, key=..., state=state@entry=0x7fffd7df3b70) at ../../source3/lib/dbwrap/dbwrap_ctdb.c:1385
#13 0x00007fffa0e76024 in db_ctdb_parse_record (db=<optimized out>, key=..., parser=0x7fffa1313910 <dbwrap_watched_parse_record_parser>, private_data=0x7fffd7df3c08) at ../../source3/lib/dbwrap/dbwrap_ctdb.c:1425
#14 0x00007fffa0884760 in dbwrap_parse_record (db=<optimized out>, key=..., parser=<optimized out>, private_data=<optimized out>) at ../../lib/dbwrap/dbwrap.c:454
#15 0x00007fffa1313ab4 in dbwrap_watched_parse_record (db=0x1543a7160, key=..., parser=0x7fffa13187d0 <g_lock_dump_fn>, private_data=0x7fffd7df3ce8) at ../../source3/lib/dbwrap/dbwrap_watch.c:783
#16 0x00007fffa0884760 in dbwrap_parse_record (db=<optimized out>, key=..., parser=<optimized out>, private_data=<optimized out>) at ../../lib/dbwrap/dbwrap.c:454
#17 0x00007fffa131c004 in g_lock_dump (ctx=<error reading variable: value has been optimized out>, key=..., fn=0x7fffa14f3d70 <fsp_update_share_mode_flags_fn>, private_data=0x7fffd7df3dd8) at ../../source3/lib/g_lock.c:1653
#18 0x00007fffa14f434c in share_mode_g_lock_dump (key=..., fn=0x7fffa14f3d70 <fsp_update_share_mode_flags_fn>, private_data=0x7fffd7df3dd8) at ../../source3/locking/share_mode_lock.c:96
#19 0x00007fffa14f8d44 in fsp_update_share_mode_flags (fsp=0x15433c550) at ../../source3/locking/share_mode_lock.c:1181
#20 file_has_read_lease (fsp=0x15433c550) at ../../source3/locking/share_mode_lock.c:1207
#21 0x00007fffa15ccc98 in contend_level2_oplocks_begin_default (type=<optimized out>, fsp=0x15433c550) at ../../source3/smbd/smb2_oplock.c:1282
#22 smbd_contend_level2_oplocks_begin (fsp=0x15433c550, type=<optimized out>) at ../../source3/smbd/smb2_oplock.c:1338
#23 0x00007fffa0dd0b54 in contend_level2_oplocks_begin (fsp=<optimized out>, type=<optimized out>) at ../../source3/lib/smbd_shim.c:72
#24 0x00007fffa14ecfd0 in brl_lock_windows_default (br_lck=0x154421330, plock=0x7fffd7df4250) at ../../source3/locking/brlock.c:457
#25 0x00007fffa150b70c in vfswrap_brl_lock_windows (handle=<optimized out>, br_lck=<optimized out>, plock=<optimized out>) at ../../source3/modules/vfs_default.c:3424
#26 0x00007fffa1561910 in smb_vfs_call_brl_lock_windows (handle=<optimized out>, br_lck=<optimized out>, plock=<optimized out>) at ../../source3/smbd/vfs.c:2686
#27 0x00007fff9c0a7350 in smb_time_audit_brl_lock_windows (handle=<optimized out>, br_lck=0x154421330, plock=0x7fffd7df4250) at ../../source3/modules/vfs_time_audit.c:1740
#28 0x00007fffa1561910 in smb_vfs_call_brl_lock_windows (handle=<optimized out>, br_lck=<optimized out>, plock=<optimized out>) at ../../source3/smbd/vfs.c:2686
#29 0x00007fffa14ed410 in brl_lock (br_lck=0x154421330, smblctx=3102281601, pid=..., start=0, size=18446744073709551615, lock_type=<optimized out>, lock_flav=WINDOWS_LOCK, blocker_pid=0x7fffd7df4540, psmblctx=0x7fffd7df4558) at ../../source3/locking/brlock.c:1004
#30 0x00007fffa14e7b18 in do_lock_fn (lck=<optimized out>, private_data=0x7fffd7df4508) at ../../source3/locking/locking.c:271
#31 0x00007fffa14fcd94 in _share_mode_do_locked_vfs_allowed (id=..., fn=0x7fffa14e7a60 <do_lock_fn>, private_data=0x7fffd7df4508, location=<optimized out>) at ../../source3/locking/share_mode_lock.c:2927
#32 0x00007fffa14e918c in do_lock (fsp=0x15433c550, req_mem_ctx=<optimized out>, req_guid=<optimized out>, smblctx=<optimized out>, count=18446744073709551615, offset=0, lock_type=<optimized out>, lock_flav=<optimized out>, pblocker_pid=0x7fffd7df46f0,
psmblctx=0x7fffd7df46d8) at ../../source3/locking/locking.c:335
#33 0x00007fffa155381c in smbd_do_locks_try (fsp=0x15433c550, num_locks=<optimized out>, locks=0x1543bc310, blocker_idx=0x7fffd7df46d6, blocking_pid=0x7fffd7df46f0, blocking_smblctx=0x7fffd7df46d8) at ../../source3/smbd/blocking.c:46
#34 0x00007fffa159dc90 in smbd_smb2_lock_try (req=req@entry=0x1543bc080) at ../../source3/smbd/smb2_lock.c:590
#35 0x00007fffa159ee8c in smbd_smb2_lock_send (in_locks=<optimized out>, in_lock_count=1, in_lock_sequence=<optimized out>, fsp=0x15433c550, smb2req=0x1543532e0, ev=0x154328120, mem_ctx=0x1543532e0) at ../../source3/smbd/smb2_lock.c:488
#36 smbd_smb2_request_process_lock (req=0x1543532e0) at ../../source3/smbd/smb2_lock.c:150
#37 0x00007fffa158a368 in smbd_smb2_request_dispatch (req=0x1543532e0) at ../../source3/smbd/smb2_server.c:3515
#38 0x00007fffa158c540 in smbd_smb2_io_handler (fde_flags=<optimized out>, xconn=0x154313f30) at ../../source3/smbd/smb2_server.c:5112
#39 smbd_smb2_connection_handler (ev=<optimized out>, fde=<optimized out>, flags=<optimized out>, private_data=<optimized out>) at ../../source3/smbd/smb2_server.c:5150
#40 0x00007fffa1198b2c in tevent_common_invoke_fd_handler (fde=0x1543670f0, flags=<optimized out>, removed=0x0) at ../../lib/tevent/tevent_fd.c:158
#41 0x00007fffa11a2b9c in epoll_event_loop (tvalp=0x7fffd7df4b28, epoll_ev=0x1543b4e80) at ../../lib/tevent/tevent_epoll.c:730
#42 epoll_event_loop_once (ev=<optimized out>, location=<optimized out>) at ../../lib/tevent/tevent_epoll.c:946
#43 0x00007fffa11a0090 in std_event_loop_once (ev=0x154328120, location=0x7fffa1668db8 "../../source3/smbd/smb2_process.c:2158") at ../../lib/tevent/tevent_standard.c:110
#44 0x00007fffa119744c in _tevent_loop_once (ev=0x154328120, location=0x7fffa1668db8 "../../source3/smbd/smb2_process.c:2158") at ../../lib/tevent/tevent.c:823
#45 0x00007fffa1197884 in tevent_common_loop_wait (ev=<optimized out>, location=<optimized out>) at ../../lib/tevent/tevent.c:950
#46 0x00007fffa119ffc0 in std_event_loop_wait (ev=0x154328120, location=0x7fffa1668db8 "../../source3/smbd/smb2_process.c:2158") at ../../lib/tevent/tevent_standard.c:141
#47 0x00007fffa1197978 in _tevent_loop_wait (ev=<optimized out>, location=<optimized out>) at ../../lib/tevent/tevent.c:971
#48 0x00007fffa15737fc in smbd_process (ev_ctx=0x154328120, msg_ctx=<optimized out>, sock_fd=<optimized out>, interactive=<optimized out>) at ../../source3/smbd/smb2_process.c:2158
#49 0x000000011db5c554 in smbd_accept_connection (ev=0x154328120, fde=<optimized out>, flags=<optimized out>, private_data=<optimized out>) at ../../source3/smbd/server.c:1150
#50 0x00007fffa1198b2c in tevent_common_invoke_fd_handler (fde=0x1543ac2d0, flags=<optimized out>, removed=0x0) at ../../lib/tevent/tevent_fd.c:158
#51 0x00007fffa11a2b9c in epoll_event_loop (tvalp=0x7fffd7df4f98, epoll_ev=0x154328350) at ../../lib/tevent/tevent_epoll.c:730
#52 epoll_event_loop_once (ev=<optimized out>, location=<optimized out>) at ../../lib/tevent/tevent_epoll.c:946
#53 0x00007fffa11a0090 in std_event_loop_once (ev=0x154328120, location=0x11db60b50 "../../source3/smbd/server.c:1499") at ../../lib/tevent/tevent_standard.c:110
#54 0x00007fffa119744c in _tevent_loop_once (ev=0x154328120, location=0x11db60b50 "../../source3/smbd/server.c:1499") at ../../lib/tevent/tevent.c:823
#55 0x00007fffa1197884 in tevent_common_loop_wait (ev=<optimized out>, location=<optimized out>) at ../../lib/tevent/tevent.c:950
#56 0x00007fffa119ffc0 in std_event_loop_wait (ev=0x154328120, location=0x11db60b50 "../../source3/smbd/server.c:1499") at ../../lib/tevent/tevent_standard.c:141
#57 0x00007fffa1197978 in _tevent_loop_wait (ev=<optimized out>, location=<optimized out>) at ../../lib/tevent/tevent.c:971
#58 0x000000011db58c54 in smbd_parent_loop (parent=<optimized out>, ev_ctx=0x154328120) at ../../source3/smbd/server.c:1499
#59 main (argc=<optimized out>, argv=<optimized out>) at ../../source3/smbd/server.c:2258
[2]
Full SBT:
#0 0x00007fffa0cecbb0 in __pthread_mutex_lock_full () from /lib64/glibc-hwcaps/power9/libpthread-2.28.so
#1 0x00007fffa0a73cf8 in chain_mutex_lock (m=<optimized out>, m@entry=0x7fff9b3a71b0, waitflag=<optimized out>, waitflag@entry=true) at ../../lib/tdb/common/mutex.c:182
#2 0x00007fffa0a7432c in tdb_mutex_lock (tdb=0x1543c6900, rw=<optimized out>, off=<optimized out>, len=<optimized out>, waitflag=<optimized out>, pret=0x7fffd7df2e28) at ../../lib/tdb/common/mutex.c:234
#3 0x00007fffa0a6812c in fcntl_lock (waitflag=<optimized out>, len=1, off=376608, rw=0, tdb=0x1543c6900) at ../../lib/tdb/common/lock.c:200
#4 tdb_brlock (tdb=0x1543c6900, rw_type=<optimized out>, offset=<optimized out>, len=1, flags=<optimized out>) at ../../lib/tdb/common/lock.c:200
#5 0x00007fffa0a68af8 in tdb_nest_lock (flags=<optimized out>, ltype=0, offset=<optimized out>, tdb=0x1543c6900) at ../../lib/tdb/common/lock.c:390
#6 tdb_nest_lock (tdb=0x1543c6900, offset=<optimized out>, ltype=<optimized out>, flags=<optimized out>) at ../../lib/tdb/common/lock.c:336
#7 0x00007fffa0a69088 in tdb_lock_list (tdb=0x1543c6900, list=<optimized out>, ltype=<optimized out>, waitflag=<optimized out>) at ../../lib/tdb/common/lock.c:482
#8 0x00007fffa0a69198 in tdb_lock (tdb=0x1543c6900, list=<optimized out>, ltype=<optimized out>) at ../../lib/tdb/common/lock.c:500
#9 0x00007fffa0a64b50 in tdb_find_lock_hash (tdb=<optimized out>, tdb@entry=0x1543c6900, key=..., hash=<optimized out>, locktype=<optimized out>, locktype@entry=0, rec=<optimized out>, rec@entry=0x7fffd7df3080) at ../../lib/tdb/common/tdb.c:165
#10 0x00007fffa0a64ed0 in tdb_parse_record (tdb=0x1543c6900, key=..., parser=0x7fffa0e74470 <db_ctdb_ltdb_parser>, private_data=0x7fffd7df30e8) at ../../lib/tdb/common/tdb.c:329
#11 0x00007fffa0e74cbc in db_ctdb_ltdb_parse (db=<optimized out>, private_data=0x7fffd7df3140, parser=0x7fffa0e76470 <db_ctdb_parse_record_parser_nonpersistent>, key=...) at ../../source3/lib/dbwrap/dbwrap_ctdb.c:170
#12 db_ctdb_try_parse_local_record (ctx=ctx@entry=0x154328fc0, key=..., state=state@entry=0x7fffd7df3140) at ../../source3/lib/dbwrap/dbwrap_ctdb.c:1385
#13 0x00007fffa0e76024 in db_ctdb_parse_record (db=<optimized out>, key=..., parser=0x7fffa14ec820 <brl_get_locks_readonly_parser>, private_data=0x7fffd7df3218) at ../../source3/lib/dbwrap/dbwrap_ctdb.c:1425
#14 0x00007fffa0884760 in dbwrap_parse_record (db=<optimized out>, key=..., parser=<optimized out>, private_data=<optimized out>) at ../../lib/dbwrap/dbwrap.c:454
#15 0x00007fffa14ef5bc in brl_get_locks_readonly (fsp=0x1543d01e0) at ../../source3/locking/brlock.c:1884
#16 0x00007fffa1546968 in file_has_brlocks (fsp=0x1543d01e0) at ../../source3/smbd/open.c:2232
#17 delay_for_oplock (pgranted=<synthetic pointer>, poplock_type=<synthetic pointer>, first_open_attempt=<optimized out>, create_disposition=1, have_sharing_violation=false, lck=0x7fffd7df3ce8, lease=0x0, oplock_request=0, fsp=0x1543d01e0) at ../../source3/smbd/open.c:2749
#18 handle_share_mode_lease (pgranted=<synthetic pointer>, poplock_type=<synthetic pointer>, first_open_attempt=<optimized out>, lease=0x0, oplock_request=0, share_access=7, access_mask=131201, create_disposition=1, lck=0x7fffd7df3ce8, fsp=0x1543d01e0) at ../../source3/smbd/open.c:2865
#19 check_and_store_share_mode (first_open_attempt=<optimized out>, lease=0x0, oplock_request=0, share_access=7, access_mask=131201, create_disposition=1, lck=0x7fffd7df3ce8, req=0x154414800, fsp=0x1543d01e0) at ../../source3/smbd/open.c:3333
#20 open_ntcreate_lock_add_entry (lck=0x7fffd7df3ce8, keep_locked=0x7fffd7df3ad0, private_data=0x7fffd7df3cc8) at ../../source3/smbd/open.c:3688
#21 0x00007fffa14f6248 in share_mode_entry_prepare_lock_fn (glck=0x7fffd7df35b8, cb_private=0x7fffd7df3a88) at ../../source3/locking/share_mode_lock.c:2978
#22 0x00007fffa1317680 in g_lock_lock_cb_run_and_store (cb_state=cb_state@entry=0x7fffd7df35b8) at ../../source3/lib/g_lock.c:597
#23 0x00007fffa1319df8 in g_lock_lock_simple_fn (rec=0x7fffd7df3798, value=..., private_data=0x7fffd7df39a0) at ../../source3/lib/g_lock.c:1212
#24 0x00007fffa13160e0 in dbwrap_watched_do_locked_fn (backend_rec=<optimized out>, backend_value=..., private_data=0x7fffd7df3768) at ../../source3/lib/dbwrap/dbwrap_watch.c:458
#25 0x00007fffa0884e48 in dbwrap_do_locked (db=<optimized out>, key=..., fn=0x7fffa1316080 <dbwrap_watched_do_locked_fn>, private_data=0x7fffd7df3768) at ../../lib/dbwrap/dbwrap.c:602
#26 0x00007fffa1315274 in dbwrap_watched_do_locked (db=0x1543a7160, key=..., fn=0x7fffa1319ca0 <g_lock_lock_simple_fn>, private_data=0x7fffd7df39a0) at ../../source3/lib/dbwrap/dbwrap_watch.c:480
#27 0x00007fffa0884d60 in dbwrap_do_locked (db=<optimized out>, key=..., fn=<optimized out>, private_data=<optimized out>) at ../../lib/dbwrap/dbwrap.c:582
#28 0x00007fffa131b458 in g_lock_lock (ctx=0x1543cc630, key=..., type=<optimized out>, timeout=..., cb_fn=0x7fffa14f6190 <share_mode_entry_prepare_lock_fn>, cb_private=0x7fffd7df3a88) at ../../source3/lib/g_lock.c:1267
#29 0x00007fffa14fd060 in _share_mode_entry_prepare_lock (prepare_state=0x7fffd7df3cc8, id=..., servicepath=<optimized out>, smb_fname=<optimized out>, old_write_time=<optimized out>, fn=<optimized out>, private_data=0x7fffd7df3cc8, location=0x7fffa165b880 "../../source3/smbd/open.c:4292") at ../../source3/locking/share_mode_lock.c:3033
#30 0x00007fffa15491e0 in open_file_ntcreate (conn=conn@entry=0x154382050, req=req@entry=0x154414800, access_mask=<optimized out>, access_mask@entry=131201, share_access=share_access@entry=7, create_disposition=create_disposition@entry=1, create_options=create_options@entry=0, new_dos_attributes=<optimized out>, new_dos_attributes@entry=128, oplock_request=oplock_request@entry=0, lease=<optimized out>, lease@entry=0x0, private_flags=<optimized out>, private_flags@entry=0, parent_dir_fname=<optimized out>, smb_fname_atname=<optimized out>, pinfo=<optimized out>, pinfo@entry=0x7fffd7df3f1c, fsp=<optimized out>, fsp@entry=0x1543d01e0) at ../../source3/smbd/open.c:4286
#31 0x00007fffa154b94c in create_file_unixpath (conn=conn@entry=0x154382050, req=req@entry=0x154414800, dirfsp=dirfsp@entry=0x15439a7f0, smb_fname=smb_fname@entry=0x154416300, access_mask=access_mask@entry=131201, share_access=share_access@entry=7, create_disposition=create_disposition@entry=1, create_options=create_options@entry=0, file_attributes=file_attributes@entry=128, oplock_request=<optimized out>, oplock_request@entry=0, lease=<optimized out>, lease@entry=0x0, allocation_size=allocation_size@entry=0, private_flags=private_flags@entry=0, sd=sd@entry=0x0, ea_list=ea_list@entry=0x0, result=result@entry=0x7fffd7df4168, pinfo=pinfo@entry=0x7fffd7df4160) at ../../source3/smbd/open.c:6290
#32 0x00007fffa154dfac in create_file_default (conn=0x154382050, req=0x154414800, dirfsp=0x15439a7f0, smb_fname=0x154416300, access_mask=<optimized out>, share_access=<optimized out>, create_disposition=<optimized out>, create_options=<optimized out>, file_attributes=128, oplock_request=0, lease=0x0, allocation_size=0, private_flags=0, sd=0x0, ea_list=0x0, result=0x1544144e8, pinfo=0x1544144fc, in_context_blobs=0x7fffd7df4798, out_context_blobs=0x154414710) at ../../source3/smbd/open.c:6609
#33 0x00007fffa150972c in vfswrap_create_file (handle=<optimized out>, req=<optimized out>, dirfsp=<optimized out>, smb_fname=<optimized out>, access_mask=<optimized out>, share_access=<optimized out>, create_disposition=<optimized out>, create_options=<optimized out>, file_attributes=128, oplock_request=0, lease=0x0, allocation_size=0, private_flags=0, sd=0x0, ea_list=0x0, result=0x1544144e8, pinfo=0x1544144fc, in_context_blobs=0x7fffd7df4798, out_context_blobs=0x154414710) at ../../source3/modules/vfs_default.c:776
#34 0x00007fffa1559cbc in smb_vfs_call_create_file (handle=<optimized out>, req=<optimized out>, dirfsp=<optimized out>, smb_fname=<optimized out>, access_mask=<optimized out>, share_access=<optimized out>, create_disposition=<optimized out>, create_options=<optimized out>, file_attributes=128, oplock_request=0, lease=0x0, allocation_size=0, private_flags=0, sd=0x0, ea_list=0x0, result=0x1544144e8, pinfo=0x1544144fc, in_context_blobs=0x7fffd7df4798, out_context_blobs=0x154414710) at ../../source3/smbd/vfs.c:1560
#35 0x00007fff9c0a9ec4 in smb_time_audit_create_file (handle=0x154426820, req=0x154414800, dirfsp=0x15439a7f0, fname=0x154416300, access_mask=<optimized out>, share_access=<optimized out>, create_disposition=<optimized out>, create_options=<optimized out>, file_attributes=128, oplock_request=0, lease=0x0, allocation_size=0, private_flags=0, sd=0x0, ea_list=0x0, result_fsp=0x1544144e8, pinfo=0x1544144fc, in_context_blobs=0x7fffd7df4798, out_context_blobs=0x154414710) at ../../source3/modules/vfs_time_audit.c:634
#36 0x00007fffa1559cbc in smb_vfs_call_create_file (handle=<optimized out>, req=<optimized out>, dirfsp=<optimized out>, smb_fname=<optimized out>, access_mask=<optimized out>, share_access=<optimized out>, create_disposition=<optimized out>, create_options=<optimized out>, file_attributes=128, oplock_request=0, lease=0x0, allocation_size=0, private_flags=0, sd=0x0, ea_list=0x0, result=0x1544144e8, pinfo=0x1544144fc, in_context_blobs=0x7fffd7df4798, out_context_blobs=0x154414710) at ../../source3/smbd/vfs.c:1560
#37 0x00007fffa1597aa8 in smbd_smb2_create_send (in_context_blobs=..., in_name=0x154413ca0, in_create_options=<optimized out>, in_create_disposition=<optimized out>, in_share_access=<optimized out>, in_file_attributes=<optimized out>, in_desired_access=<optimized out>, in_impersonation_level=<optimized out>, in_oplock_level=<optimized out>, smb2req=0x154413770, ev=0x154328120, mem_ctx=0x154413770) at ../../source3/smbd/smb2_create.c:1115
#38 smbd_smb2_request_process_create (smb2req=0x154413770) at ../../source3/smbd/smb2_create.c:291
#39 0x00007fffa158a628 in smbd_smb2_request_dispatch (req=0x154413770) at ../../source3/smbd/smb2_server.c:3485
#40 0x00007fffa158c540 in smbd_smb2_io_handler (fde_flags=<optimized out>, xconn=0x154313f30) at ../../source3/smbd/smb2_server.c:5112
#41 smbd_smb2_connection_handler (ev=<optimized out>, fde=<optimized out>, flags=<optimized out>, private_data=<optimized out>) at ../../source3/smbd/smb2_server.c:5150
#42 0x00007fffa1198b2c in tevent_common_invoke_fd_handler (fde=0x15435add0, flags=<optimized out>, removed=0x0) at ../../lib/tevent/tevent_fd.c:158
#43 0x00007fffa11a2b9c in epoll_event_loop (tvalp=0x7fffd7df4b28, epoll_ev=0x1543b4e80) at ../../lib/tevent/tevent_epoll.c:730
#44 epoll_event_loop_once (ev=<optimized out>, location=<optimized out>) at ../../lib/tevent/tevent_epoll.c:946
#45 0x00007fffa11a0090 in std_event_loop_once (ev=0x154328120, location=0x7fffa1668db8 "../../source3/smbd/smb2_process.c:2158") at ../../lib/tevent/tevent_standard.c:110
#46 0x00007fffa119744c in _tevent_loop_once (ev=0x154328120, location=0x7fffa1668db8 "../../source3/smbd/smb2_process.c:2158") at ../../lib/tevent/tevent.c:823
#47 0x00007fffa1197884 in tevent_common_loop_wait (ev=<optimized out>, location=<optimized out>) at ../../lib/tevent/tevent.c:950
#48 0x00007fffa119ffc0 in std_event_loop_wait (ev=0x154328120, location=0x7fffa1668db8 "../../source3/smbd/smb2_process.c:2158") at ../../lib/tevent/tevent_standard.c:141
#49 0x00007fffa1197978 in _tevent_loop_wait (ev=<optimized out>, location=<optimized out>) at ../../lib/tevent/tevent.c:971
#50 0x00007fffa15737fc in smbd_process (ev_ctx=0x154328120, msg_ctx=<optimized out>, sock_fd=<optimized out>, interactive=<optimized out>) at ../../source3/smbd/smb2_process.c:2158
#51 0x000000011db5c554 in smbd_accept_connection (ev=0x154328120, fde=<optimized out>, flags=<optimized out>, private_data=<optimized out>) at ../../source3/smbd/server.c:1150
#52 0x00007fffa1198b2c in tevent_common_invoke_fd_handler (fde=0x1543ac2d0, flags=<optimized out>, removed=0x0) at ../../lib/tevent/tevent_fd.c:158
#53 0x00007fffa11a2b9c in epoll_event_loop (tvalp=0x7fffd7df4f98, epoll_ev=0x154328350) at ../../lib/tevent/tevent_epoll.c:730
#54 epoll_event_loop_once (ev=<optimized out>, location=<optimized out>) at ../../lib/tevent/tevent_epoll.c:946
#55 0x00007fffa11a0090 in std_event_loop_once (ev=0x154328120, location=0x11db60b50 "../../source3/smbd/server.c:1499") at ../../lib/tevent/tevent_standard.c:110
#56 0x00007fffa119744c in _tevent_loop_once (ev=0x154328120, location=0x11db60b50 "../../source3/smbd/server.c:1499") at ../../lib/tevent/tevent.c:823
#57 0x00007fffa1197884 in tevent_common_loop_wait (ev=<optimized out>, location=<optimized out>) at ../../lib/tevent/tevent.c:950
#58 0x00007fffa119ffc0 in std_event_loop_wait (ev=0x154328120, location=0x11db60b50 "../../source3/smbd/server.c:1499") at ../../lib/tevent/tevent_standard.c:141
#59 0x00007fffa1197978 in _tevent_loop_wait (ev=<optimized out>, location=<optimized out>) at ../../lib/tevent/tevent.c:971
#60 0x000000011db58c54 in smbd_parent_loop (parent=<optimized out>, ev_ctx=0x154328120) at ../../source3/smbd/server.c:1499
#61 main (argc=<optimized out>, argv=<optimized out>) at ../../source3/smbd/server.c:2258
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org> Signed-off-by: Ralph Boehme <slow@samba.org> Signed-off-by: Stefan Metzmacher <metze@samba.org>
(backported from commit 2eef298ff4c5baf15c7d29c65fb021dbed5b0a93)
[slow@samba.org: changed argument of share_mode_watch_send()]
[slow@samba.org: small context change in vfs_fruit]
Signed-off-by: Ralph Boehme <slow@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 2772f147c9b13cd2160181c4f7905b54ab765054)
Signed-off-by: Ralph Boehme <slow@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 3a0c6e99de4377f44bc29766b6ceb79040caed9f)
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org> Signed-off-by: Ralph Boehme <slow@samba.org> Signed-off-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit e17fb732c89f8b34de00904383044de3c4f85bd0)
Signed-off-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit c9c04c7d75dee0c3e6e843b581624a3852042057)
Pair-Programmed-With: Ralph Boehme <slow@samba.org> Signed-off-by: Ralph Boehme <slow@samba.org> Signed-off-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 94e7cbcc32b73e4d56e7209e04d22d4270a6eb5b)
Ralph Boehme [Thu, 9 Jan 2025 11:27:43 +0000 (12:27 +0100)]
smbtorture: add test "open-brlock-deadlock"
smbtorture reproducer for bug 15767. As it needs a very specific setup that
can't easily be done in selftest, the test is only executed when manually called
with
Signed-off-by: Ralph Boehme <slow@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 7c60498cee7dca5770d4d1f623c472d585ae9cae)
Signed-off-by: Ralph Boehme <slow@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 7eb135c42d530a16e80e165d9e8e99d920797f12)
Signed-off-by: Anoop C S <anoopcs@samba.org> Reviewed-by: Guenther Deschner <gd@samba.org>
Autobuild-User(master): Günther Deschner <gd@samba.org>
Autobuild-Date(master): Mon Mar 17 20:48:55 UTC 2025 on atb-devel-224
Signed-off-by: Ralph Boehme <slow@samba.org> Reviewed-by: Björn Jacke <bjacke@samba.org> Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Wed Mar 12 01:32:30 UTC 2025 on atb-devel-224
Xavi Hernandez [Tue, 4 Mar 2025 11:48:41 +0000 (12:48 +0100)]
vfs_ceph_new: detect case sensitivity in CephFS
CephFS has recently added support for case insensitive access to the
file system. This modification detects whether the shared volume is case
sensitive or not and reports the FILE_CASE_SENSITIVE_SEARCH capability
accordingly.
Anoop C S [Tue, 17 Sep 2024 18:28:50 +0000 (23:58 +0530)]
vfs_glusterfs: Retrieve fs capabilities using vfs_get_fs_capabilities
vfs_glusterfs is supposed to be the last entry when listed with
other vfs modules. This is due to the fact that the connection path
is not local to the server but relative to the virtual remote file
system beneath it. Especially SMB_VFS_FS_CAPABILITIES implementation
from vfs_default is likely to return incorrect results based on the
connection path assumed to be local to the server which might not be
the case with glusterfs module stacked. Therefore it doesn't make sense
to pass through any vfs interface implementations further down the line
to vfs_default.
Instead make use of get_fs_capabilties to start with already known fs
capabilties from connect phase.
Signed-off-by: Anoop C S <anoopcs@samba.org> Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Mon Oct 7 13:36:11 UTC 2024 on atb-devel-224
Anoop C S [Thu, 3 Oct 2024 06:59:58 +0000 (12:29 +0530)]
vfs_ceph_new: Populate fs capabilities within vfs_ceph_statvfs
SMB_VFS_STATVFS implementation for vfs_ceph_new failed to fill in the
FsCapabilities field for vfs_statvfs_struct. Insert the minimum
required values for defining the capabilties of a ceph file system.
Anoop C S [Tue, 17 Sep 2024 18:20:25 +0000 (23:50 +0530)]
vfs_ceph: Retrieve fs capabilties using vfs_get_fs_capabilities
vfs_ceph is supposed to be the last entry when listed with other vfs
modules. This is due to the fact that the connection path is not local
to the server but relative to the virtual remote file system beneath
it. Especially SMB_VFS_FS_CAPABILITIES implementation from vfs_default
is likely to return incorrect results based on the connection path
assumed to be local to the server which might not be the case with
ceph module stacked. Therefore it doesn't make sense to pass through
any vfs interface implementations further down the line to vfs_default.
Instead make use of get_fs_capabilties to start with already known fs
capabilties from connect phase.
Anoop C S [Mon, 23 Sep 2024 09:25:59 +0000 (14:55 +0530)]
vfs_ceph: Populate fs capabilities within cephwrap_statvfs
SMB_VFS_STATVFS implementation for vfs_ceph failed to fill in the
FsCapabilities field for vfs_statvfs_struct. Insert the minimum
required values for defining the capabilties of a ceph file system.
Anoop C S [Tue, 24 Sep 2024 05:36:58 +0000 (11:06 +0530)]
s3/smbd: Add a helper to fetch fs capabilities
We create a new helper function to retrieve the fs capabilties via
STATVFS call. Additionally set other capabilities based on specific
parametric options. This is 99.9% taken from SMB_VFS_FS_CAPABILITIES
implementation of vfs_default in preparation to be called from any
required vfs module.
Ralph Boehme [Thu, 6 Jun 2024 13:38:16 +0000 (15:38 +0200)]
smbd: consolidate fs capabilities code in vfswrap_fs_capabilities()
This ensures the values we return via SMB_FS_ATTRIBUTE_INFORMATION is the same
we use internally via conn->fs_capabilities.
This deliberately preserves existing behaviour as much as possible and leaves
possible improvements as a future excercize. Particularily FILE_VOLUME_QUOTAS is
already set insided SMB_VFS_STATVFS() depending on backend filesystem flags
which is probably the correct way to do it instead of just setting the
capability when Samba was built with quota support.
Signed-off-by: Ralph Boehme <slow@samba.org> Reviewed-by: David Disseldorp <ddiss@samba.org>
(cherry picked from commit deeca8ff176e7e9082cc0fcf48d7f981de2523c5)
Anoop C S [Tue, 25 Feb 2025 12:10:13 +0000 (17:40 +0530)]
vfs_ceph_new: Do not resolve by inode number
CephFS snapshots within snap directory shares the same inode number from
its parent. Until unless we resolve by name we may incorrectly point at
an inode which is not a snapshot directory. Therefore to be functionally
correct we avoid resolving by inode number but proper name.
Signed-off-by: Anoop C S <anoopcs@samba.org> Reviewed-by: Guenther Deschner <gd@samba.org>
Autobuild-User(master): Günther Deschner <gd@samba.org>
Autobuild-Date(master): Fri Mar 7 18:20:47 UTC 2025 on atb-devel-224
Anoop C S [Mon, 24 Feb 2025 08:30:56 +0000 (14:00 +0530)]
vfs_ceph_new: Handle absolute path in vfs_ceph_ll_walk
It can very well be the case that the incoming path is absolute in
nature which breaks the assumption inside vfs_ceph_ll_walk that it
is within the current working directory. Instead perform a check to
see whether the path includes current working directory path in its
components and accordingly trim it to make it relative in nature.
Shachar Sharon [Tue, 1 Oct 2024 09:09:40 +0000 (12:09 +0300)]
vfs_ceph_new: use libcephfs nonblocking API for async-io ops
Use libcephfs non-blocking API (ceph_ll_nonblocking_readv_writev[1]) in
combination with smb VFS async hooks ({pread,pwrite,fsync}_send/_recv).
Fills libcephfs' struct ceph_ll_io_info with single iovec and
submit/complete the operation asynchronously on libcephfs side, with
corresponding tevent schedule-immediate upon completion on smbd side.
Control nonblocking/normal I/O mode via config parameter. The common
parts of async I/O (with/without HAVE_CEPH_ASYNCIO) are united.
Specifically, use same struct vfs_ceph_aio_state and common code via
helper function for all async I/O hooks. When HAVE_CEPH_ASYNCIO
is True _and_ config option 'asyncio = yes' use libcephfs asynchronous
I/O API. Otherwise, fake async operation using normal blocking APIs.
Anoop C S [Mon, 28 Oct 2024 05:43:10 +0000 (11:13 +0530)]
vfs_ceph_new: Remove unused symbol for ceph_readdir
ce459ddbcd0f32252858a7640f6871057eb14645 recently switched the readdir
implementation to use ceph_readdir_r(). Thus ceph_readdir() is
unnecessarily loaded which is no longer used.
Signed-off-by: Anoop C S <anoopcs@samba.org> Reviewed-by: John Mulligan <jmulligan@redhat.com>
Autobuild-User(master): Anoop C S <anoopcs@samba.org>
Autobuild-Date(master): Sun Nov 3 11:07:23 UTC 2024 on atb-devel-224
Anoop C S [Mon, 5 Aug 2024 13:21:49 +0000 (18:51 +0530)]
source3/wscript: Introduce auto mode to build ceph vfs modules
Use 'auto' mode as the default for building ceph vfs modules so that an
explicit --enable-cephfs can reliably fail in the absence of required
dependencies.
Signed-off-by: Anoop C S <anoopcs@samba.org> Reviewed-by: Guenther Deschner <gd@samba.org>
Autobuild-User(master): Anoop C S <anoopcs@samba.org>
Autobuild-Date(master): Tue Aug 27 06:18:51 UTC 2024 on atb-devel-224
s3:utils: Remove call of ads_startup() from net_ads_keytab_create()
Calling ads_startup() is not needed in net_ads_keytab_create. Keytab
creation code in sync_pw2keytabs() decides if it needs to talk to DC or
not and connects to AD accordingly.
Fixing this, makes the bug below easier to reproduce using
'net ads keytab create'.
Signed-off-by: Pavel Filipenský <pfilipensky@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Pavel Filipensky <pfilipensky@samba.org>
Autobuild-Date(master): Mon Mar 10 11:09:29 UTC 2025 on atb-devel-224
s3:libads: Make sure that REALM is always added to keytab principals
The code responsible for adding SPNs to keytab should always set the
REALM part. Current code is not adding it for e.g. SPNs synced from AD.
If REALM is missing, krb5_parse_name() will succeed (and add the REALM)
only if the krb5.conf contains libdefaults section with
default_realm set and will fail otherwise. E.g.:
Pair-Programmed-With: Noel Power <noel.power@suse.com>
Signed-off-by: Pavel Filipenský <pfilipensky@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Alexander Bokovoy <ab@samba.org>
Autobuild-User(master): Pavel Filipensky <pfilipensky@samba.org>
Autobuild-Date(master): Sun Mar 9 00:25:08 UTC 2025 on atb-devel-224
Signed-off-by: Pavel Filipenský <pfilipensky@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Alexander Bokovoy <ab@samba.org>
(cherry picked from commit cf34645050df64d6b8c4fa45394c3feebe691e79)
Anoop C S [Thu, 6 Feb 2025 12:20:10 +0000 (17:50 +0530)]
vfs_shadow_copy2: Use VFS interface to derive mount point
shadow_copy2_find_mount_point() does direct stat() calls locally while
trying to automatically detect the mount point. This cannot be always
true as there are virtual file systems like CephFS, GlusterFS etc.
without their share path locally available on the system. Instead use
the VFS interface to make the stat calls hit the underlying file system
irrespective of their local presence in the system.
Signed-off-by: Anoop C S <anoopcs@samba.org> Reviewed-by: Ralph Boehme <slow@samba.org> Reviewed-by: John Mulligan <jmulligan@samba.org>
Autobuild-User(master): Anoop C S <anoopcs@samba.org>
Autobuild-Date(master): Fri Feb 7 06:23:12 UTC 2025 on atb-devel-224
Signed-off-by: Martin Schwenke <mschwenke@ddn.com> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Wed Mar 5 03:40:15 UTC 2025 on atb-devel-224
GNU getopt(3) is by default non-POSIX compliant and accepts options after
positional arguments (unless forced with POSIXLY_CORRECT). This is not portable,
e..g., on FreeBSD. Put options first and then positional arguments.
Signed-off-by: Andrea Venturoli <ml@netfence.it> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Jennifer Sutton <jennifersutton@catalyst.net.nz>
(cherry picked from commit 7e083a6b3a12933b79ef19ccbd4c13bfa0203498)
Autobuild-User(v4-21-test): Jule Anger <janger@samba.org>
Autobuild-Date(v4-21-test): Tue Mar 4 13:36:01 UTC 2025 on atb-devel-224
Douglas Bagnall [Thu, 30 Jan 2025 21:31:32 +0000 (10:31 +1300)]
provision: always use a large transaction index cache
A larger cache costs more per transaction, but makes a large number
of operations within a transaction faster. We expect to be dealing
with the latter case here, regardless of the batch_mode parameter
and the database size. 200000 is chosen because it is also used in
join and schemaupgrade, and should be sufficient in most cases.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Tue Feb 4 22:47:35 UTC 2025 on atb-devel-224
Ralph Boehme [Wed, 29 Jan 2025 14:11:16 +0000 (15:11 +0100)]
mdssvc: support a few more attributes
This adds support for the following Spotlight Metadata Attributes:
_kMDItemFileName (another alias for kMDItemFSName and kMDItemDisplayName)
kMDItemLastUsedDate
kMDItemContentCreationDate
kMDItemLogicalSize (another alias for kMDItemFSSize)
Signed-off-by: Ralph Boehme <slow@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 0ba1a8d77694182058d1c01b54a8759bdf0e28a6)
Autobuild-User(v4-21-test): Jule Anger <janger@samba.org>
Autobuild-Date(v4-21-test): Mon Feb 17 11:04:23 UTC 2025 on atb-devel-224
Ralph Boehme [Wed, 22 Jan 2025 11:34:31 +0000 (12:34 +0100)]
vfs_gpfs: add gpfs:clamp_invalid_times
The timestamp validation added as part of the fix for bug 15151 causes hard
failures for certain clients that seem to use a temporary timestamp initially
when creating files, changing in a later step.
Clamp invalid timestamps to the allowed range 0..UINT32_MAX if
"gpfs:clamp_invalid_times = yes" is set.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Volker Lendecke <vl@samba.org>
(cherry picked from commit 455a0558c89312061f3b9ccaa577a4a60df7ee77)
[bugzilla link added in backport]
Signed-off-by: Pavel Filipenský <pfilipensky@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Pavel Filipensky <pfilipensky@samba.org>
Autobuild-Date(master): Sat Feb 15 19:21:56 UTC 2025 on atb-devel-224
Signed-off-by: Pavel Filipenský <pfilipensky@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit ccc3b2b2fba7b5d223c79bffc0f655490aed19cf)
Signed-off-by: Pavel Filipenský <pfilipensky@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org> Reviewed-by: Alexander Bokovoy <ab@samba.org>
Autobuild-User(master): Pavel Filipensky <pfilipensky@samba.org>
Autobuild-Date(master): Thu Feb 13 18:45:21 UTC 2025 on atb-devel-224
Signed-off-by: Pavel Filipenský <pfilipensky@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org> Reviewed-by: Alexander Bokovoy <ab@samba.org>
(cherry picked from commit 15e191736d3eaba83b2fb4b901e1df2214526b64)
Shachar Sharon [Sun, 6 Oct 2024 10:26:48 +0000 (13:26 +0300)]
vfs_ceph_new: log open-flags upon release-fh
Store the set of open O_XXX flags as part of the referenced file-handle
to allow more verbose debug-logging info upon close. This should ease
the developer's logging analysis where same inode is opened multiple
times but with different flags set.
Signed-off-by: Shachar Sharon <ssharon@redhat.com> Reviewed-by: Anoop C S <anoopcs@samba.org> Reviewed-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 341ff48a5a787ed358a28037b5ec751b7b530e20)
Have more verbose and explicit values in various DBG_DEBUG logging. In
particular, do not use the redundant '__func__' argument as it's info is
already provided via the DBG_ logging macros.
Signed-off-by: Shachar Sharon <ssharon@redhat.com> Reviewed-by: Anoop C S <anoopcs@samba.org> Reviewed-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 73d5503a90e3bd5b1d1cfc6322d7de10955faf6c)
Signed-off-by: Shachar Sharon <ssharon@redhat.com> Reviewed-by: Anoop C S <anoopcs@samba.org> Reviewed-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 2153bedc8eb53e5d9f44ef677bd1781d934abd34)
Shachar Sharon [Sun, 20 Oct 2024 08:50:13 +0000 (11:50 +0300)]
vfs_ceph_new: improve mount cache-entry add
Use boolean return value from cephmount_cache_add, to align code-style
with other caphmount helper functions. Returns false in case of memory
allocation failure, true otherwise (success).
Signed-off-by: Shachar Sharon <ssharon@redhat.com> Reviewed-by: Anoop C S <anoopcs@samba.org> Reviewed-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 866b872cdb2b08a8b5e6a9015cde9b34c4bcdf01)
Use singed int32_t for cached mount-entries reference counting. Define
helper function for inc/dec ref-count which also provides proper
logging. Prefer boolean return-value for 'cephmount_cache_remove' as
'int' is often used as error indicator within the context of libcephfs
and this VFS module.
Signed-off-by: Shachar Sharon <ssharon@redhat.com> Reviewed-by: Anoop C S <anoopcs@samba.org> Reviewed-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 29a9d18d2d21842bb38bcdc6b9e366abac458ed9)
vfs_ceph_new: avoid setting errno in cephmount_cache_update
Do not set 'errno' to ENOENT in cephmount_cache_update. Setting this
errno value upon newly inserted entry may cause vfs_ceph_connect to
change errno to non-zero value even though the final result is OK.
Signed-off-by: Shachar Sharon <ssharon@redhat.com> Reviewed-by: Anoop C S <anoopcs@samba.org> Reviewed-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 1f7581c9edb32a2a63bf032f7e167be8a401c3d5)
vfs_ceph_new: refactor error-case in cephmount_mount_fs
Align code-style of 'cephmount_mount_fs' with rest of the code: use
'goto' for bail-out upon error case (with proper cleanups). For the
common case of successful operation complete execution and return final
value. Added extra debug-logging for good-path case.
Signed-off-by: Shachar Sharon <ssharon@redhat.com> Reviewed-by: Anoop C S <anoopcs@samba.org> Reviewed-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit d9b872afeee4dee49de2e6eb86e4b59e07804363)
Prefer a safe version of ceph_readdir, where the directory entry struct
is allocated by the caller. Use a dynamic-allocated 'struct dirent'
which is associated with a directory vfs_ceph_fh (optional), which is
allocated on-the-fly upon start of READDIR and released at the end or
CLOSEDIR (or unlikely readdir error).
Signed-off-by: Shachar Sharon <ssharon@redhat.com> Reviewed-by: John Mulligan <jmulligan@redhat.com> Reviewed-by: Anoop C S <anoopcs@samba.org>
Autobuild-User(master): Anoop C S <anoopcs@samba.org>
Autobuild-Date(master): Fri Oct 25 10:29:44 UTC 2024 on atb-devel-224
Signed-off-by: Anoop C S <anoopcs@samba.org> Reviewed-by: Guenther Deschner <gd@samba.org>
Autobuild-User(master): Anoop C S <anoopcs@samba.org>
Autobuild-Date(master): Thu Sep 12 16:51:14 UTC 2024 on atb-devel-224
Anoop C S [Thu, 5 Sep 2024 09:46:54 +0000 (15:16 +0530)]
vfs_ceph_new: Pass module config to userperm helpers
userperm helpers will switch to function references instead of direct
invocation of APIs. This would mean the matching config structure is
passed to those helpers.
Anoop C S [Thu, 5 Sep 2024 08:31:37 +0000 (14:01 +0530)]
vfs_ceph_new: Hold a config reference in vfs_ceph_fh
This is required to perform the cleanup when fsp extension destructor is
called as part of VFS_REMOVE_FSP_EXTENSION where mount information and
function references are to be used in upcoming changes.
Anoop C S [Thu, 5 Sep 2024 09:20:48 +0000 (14:50 +0530)]
vfs_ceph_new: Call vfs_ceph_userperm_new with handle->conn
vfs_ceph_userperm_new() only need connection structure from handle
for fetching the current unix token. Therefore modify the signature
to accept just handle->conn.
Pair-Programmed-With: Anoop C S <anoopcs@samba.org> Signed-off-by: Guenther Deschner <gd@samba.org> Signed-off-by: Anoop C S <anoopcs@samba.org> Reviewed-by: Guenther Deschner <gd@samba.org>
(cherry picked from commit e4fc1df4b4486f5eed095135f905bdbf10bb1793)
Pair-Programmed-With: Anoop C S <anoopcs@samba.org> Signed-off-by: Guenther Deschner <gd@samba.org> Signed-off-by: Anoop C S <anoopcs@samba.org> Reviewed-by: Guenther Deschner <gd@samba.org>
(cherry picked from commit 0d2ad13d8e5191f4b01584afa0bcdbf1114042b8)
vfs_ceph_new: Dynamically open library for 'proxy' mode
Use dlopen() to load either of the shared libraries(libcephfs.so or
libcephfs_proxy.so) based on the configuration for 'proxy' module
parameter. Further down the line we will define the required APIs
as function pointers within the config structure.
Pair-Programmed-With: Anoop C S <anoopcs@samba.org> Signed-off-by: Guenther Deschner <gd@samba.org> Signed-off-by: Anoop C S <anoopcs@samba.org> Reviewed-by: Guenther Deschner <gd@samba.org>
(cherry picked from commit 47812a279118befbaeffdd6c81e3d49b071f04c5)
vfs_ceph_new: Introduce new parametric option 'proxy'
Provide early support for consuming yet to come libcephfs proxy[1] for
optimized resource utilization. For better control we make use of an
additional module specific option 'proxy' to specify the intent to load
proxy library. With the default value 'no' a regular cephfs connection
is established. There is also an 'auto' mode which can fall back to the
regular connection if proxy requirements are not met.
vfs_ceph_new: Add a new struct to hold ceph module config
Consolidate all required configuration related data under a dedicated
structure named vfs_ceph_config. As of now it includes the location of
configuration file, file system name, ceph client user id and mount
related information. This is expected to grow in future with more
details as and when required. Apart from that small cleanups are also
done to make code more robust.
Pair-Programmed-With: Anoop C S <anoopcs@samba.org> Signed-off-by: Guenther Deschner <gd@samba.org> Signed-off-by: Anoop C S <anoopcs@samba.org> Reviewed-by: Guenther Deschner <gd@samba.org>
(cherry picked from commit f1d418181d12bd513204cf2b77c37990939e2041)
Shachar Sharon [Mon, 12 Aug 2024 11:45:53 +0000 (14:45 +0300)]
vfs_ceph_new: implement DFS hooks using libcephfs low-level APIs
Refactor the VFS hooks 'create_dfs_pathat_fn' and 'read_dfs_pathat_fn'
in 'vfs_ceph_new.c' to use libcephfs low-level APIs: instead of using
path-based operations (as done in legacy 'vfs_ceph.c') use inode based
operations to create and read msdfs symbolic-links values.
This codepath is also used for provisiong non-local machines into AD
during offlinejoin operations. When creating accounts for non-local
machines we certainly need to be able to use arbitrary hostname other
than lp_netbios_name() (which is used internally by lp_dns_hostname()).
This partly reverts 0e96092c1895ecb41d4064111566b4ada71fe457.
Guenther
Signed-off-by: Guenther Deschner <gd@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu Jan 30 07:35:05 UTC 2025 on atb-devel-224
projects / thirdparty / samba.git / log
