io_uring/zcrx: make scrubbing more reliable

Currently, scrubbing is done once before killing all recvzc requests.
It's fine as those are cancelled and don't return buffers afterwards,
but it'll be more reliable not to rely that much on cancellations.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://patch.msgid.link/c4ea127023494cbbedebd21a2b7ae5ff0448eb95.1779189667.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>

diff --git a/io_uring/zcrx.c b/io_uring/zcrx.c
index 19837e0b5e9189568b9e08af2f130c19b6307bc5..a7eef37868cfd0306f40dca2b49dfcddef7f0cc9 100644 (file)
--- a/io_uring/zcrx.c
+++ b/io_uring/zcrx.c
@@ -985,6 +985,12 @@ void io_unregister_zcrx(struct io_ring_ctx *ctx)
                }
                if (!ifq)
                        break;
+               /*
+                * io_uring can run requests and return buffers to the user
+                * after termination, scrub it again.
+                */
+               if (refcount_read(&ifq->user_refs) == 0)
+                       io_zcrx_scrub(ifq);
                io_put_zcrx_ifq(ifq);
        }