ublk: restore auto buf unregister refcount optimization

Commit 1ceeedb59749 ("ublk: optimize UBLK_IO_UNREGISTER_IO_BUF on daemon
task") optimized ublk request buffer unregistration to use a non-atomic
reference count decrement when performed on the ublk_io's daemon task.
The optimization applied to auto buffer unregistration, which happens as
part of handling UBLK_IO_COMMIT_AND_FETCH_REQ on the daemon task.
However, commit b749965edda8 ("ublk: remove ublk_commit_and_fetch()")
reordered the ublk_sub_req_ref() for the completed request before the
io_buffer_unregister_bvec() call. As a result, task_registered_buffers
is already 0 when io_buffer_unregister_bvec() calls ublk_io_release()
and the non-atomic refcount optimization doesn't apply.
Move the io_buffer_unregister_bvec() call back to before
ublk_need_complete_req() to restore the reference counting optimization.

Signed-off-by: Caleb Sander Mateos <csander@purestorage.com>
Fixes: b749965edda8 ("ublk: remove ublk_commit_and_fetch()")
Reviewed-by: Ming Lei <ming.lei@redhat.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>

diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c
index 8a5a6ba29a1dec61e6903a0f86c9399ebb4f5acd..5efaf53261cecf5b6798d47aed9b83d1e3c16715 100644 (file)
--- a/drivers/block/ublk_drv.c
+++ b/drivers/block/ublk_drv.c
@@ -3334,11 +3334,11 @@ static int ublk_ch_uring_cmd_local(struct io_uring_cmd *cmd,
                io->res = result;
                req = ublk_fill_io_cmd(io, cmd);
                ret = ublk_config_io_buf(ub, io, cmd, addr, &buf_idx);
+               if (buf_idx != UBLK_INVALID_BUF_IDX)
+                       io_buffer_unregister_bvec(cmd, buf_idx, issue_flags);
                compl = ublk_need_complete_req(ub, io);
 
                /* can't touch 'ublk_io' any more */
-               if (buf_idx != UBLK_INVALID_BUF_IDX)
-                       io_buffer_unregister_bvec(cmd, buf_idx, issue_flags);
                if (req_op(req) == REQ_OP_ZONE_APPEND)
                        req->__sector = addr;
                if (compl)