block: cleanup and fix batch completion adding conditions

[ Upstream commit 1f47ed294a2bd577d5ae43e6e28e1c9a3be4a833 ]

The conditions for whether or not a request is allowed adding to a
completion batch are a bit hard to read, and they also have a few
issues. One is that ioerror may indeed be a random value on passthrough,
and it's being checked unconditionally of whether or not the given
request is a passthrough request or not.

Rewrite the conditions to be separate for easier reading, and only check
ioerror for non-passthrough requests. This fixes an issue with bio
unmapping on passthrough, where it fails getting added to a batch. This
both leads to suboptimal performance, and may trigger a potential
schedule-under-atomic condition for polled passthrough IO.

Fixes: f794f3351f26 ("block: add support for blk_mq_end_request_batch()")
Link: https://lore.kernel.org/r/20575f0a-656e-4bb3-9d82-dec6c7e3a35c@kernel.dk
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Sasha Levin <sashal@kernel.org>

diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h
index c5063e0a38a05803ecfeca2a2d8a049db946c364..a53cbe256910434e8a2fd32739037ec3ca4f33f6 100644 (file)
--- a/include/linux/blk-mq.h
+++ b/include/linux/blk-mq.h
@@ -880,12 +880,22 @@ static inline bool blk_mq_add_to_batch(struct request *req,
                                       void (*complete)(struct io_comp_batch *))
 {
        /*
-        * blk_mq_end_request_batch() can't end request allocated from
-        * sched tags
+        * Check various conditions that exclude batch processing:
+        * 1) No batch container
+        * 2) Has scheduler data attached
+        * 3) Not a passthrough request and end_io set
+        * 4) Not a passthrough request and an ioerror
         */
-       if (!iob || (req->rq_flags & RQF_SCHED_TAGS) || ioerror ||
-                       (req->end_io && !blk_rq_is_passthrough(req)))
+       if (!iob)
                return false;
+       if (req->rq_flags & RQF_SCHED_TAGS)
+               return false;
+       if (!blk_rq_is_passthrough(req)) {
+               if (req->end_io)
+                       return false;
+               if (ioerror < 0)
+                       return false;
+       }
 
        if (!iob->complete)
                iob->complete = complete;