dm: clear cloned request bio pointer when last clone bio completes

Stale rq->bio values have been observed to cause double-initialization of
cloned bios in request-based device-mapper targets, leading to
use-after-free and double-free scenarios.

One such case occurs when using dm-multipath on top of a PCIe NVMe
namespace, where cloned request bios are freed during
blk_complete_request(), but rq->bio is left intact. Subsequent clone
teardown then attempts to free the same bios again via
blk_rq_unprep_clone().

The resulting double-free path looks like:

  nvme_pci_complete_batch()
    nvme_complete_batch()
      blk_mq_end_request_batch()
        blk_complete_request()        // called on a DM clone request
          bio_endio()                 // first free of all clone bios
          ...
        rq->end_io()                  // end_clone_request()
          dm_complete_request(tio->orig)
            dm_softirq_done()
              dm_done()
                dm_end_request()
                  blk_rq_unprep_clone()  // second free of clone bios

Fix this by clearing the clone request's bio pointer when the last cloned
bio completes, ensuring that later teardown paths do not attempt to free
already-released bios.

Signed-off-by: Michael Liang <mliang@purestorage.com>
Reviewed-by: Mohamed Khalfella <mkhalfella@purestorage.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Cc: stable@vger.kernel.org

diff --git a/drivers/md/dm-rq.c b/drivers/md/dm-rq.c
index 5e08546696145d279360f2cf62265c857aa30625..923252fb57aec7d8a833ee351052acee6618d2da 100644 (file)
--- a/drivers/md/dm-rq.c
+++ b/drivers/md/dm-rq.c
@@ -109,14 +109,21 @@ static void end_clone_bio(struct bio *clone)
         */
        tio->completed += nr_bytes;
 
+       if (!is_last)
+               return;
+       /*
+        * At this moment we know this is the last bio of the cloned request,
+        * and all cloned bios have been released, so reset the clone request's
+        * bio pointer to avoid double free.
+        */
+       tio->clone->bio = NULL;
+ exit:
        /*
         * Update the original request.
         * Do not use blk_mq_end_request() here, because it may complete
         * the original request before the clone, and break the ordering.
         */
-       if (is_last)
- exit:
-               blk_update_request(tio->orig, BLK_STS_OK, tio->completed);
+       blk_update_request(tio->orig, BLK_STS_OK, tio->completed);
 }
 
 static struct dm_rq_target_io *tio_from_request(struct request *rq)