From: Jens Axboe Date: Mon, 13 Sep 2021 15:20:44 +0000 (-0600) Subject: io-wq: fix wakeup race when adding new work X-Git-Tag: v5.13.19~326 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=08c87b24330edbfdbc8f7fdee17209e78c3f03e2;p=thirdparty%2Fkernel%2Fstable.git io-wq: fix wakeup race when adding new work commit 87df7fb922d18e96992aa5e824aa34b2065fef59 upstream. When new work is added, io_wqe_enqueue() checks if we need to wake or create a new worker. But that check is done outside the lock that otherwise synchronizes us with a worker going to sleep, so we can end up in the following situation: CPU0 CPU1 lock insert work unlock atomic_read(nr_running) != 0 lock atomic_dec(nr_running) no wakeup needed Hold the wqe lock around the "need to wakeup" check. Then we can also get rid of the temporary work_flags variable, as we know the work will remain valid as long as we hold the lock. Cc: stable@vger.kernel.org Reported-by: Andres Freund Signed-off-by: Jens Axboe Signed-off-by: Greg Kroah-Hartman --- diff --git a/fs/io-wq.c b/fs/io-wq.c index c7171d9758968..3de639ddb2d0d 100644 --- a/fs/io-wq.c +++ b/fs/io-wq.c @@ -798,7 +798,7 @@ append: static void io_wqe_enqueue(struct io_wqe *wqe, struct io_wq_work *work) { struct io_wqe_acct *acct = io_work_get_acct(wqe, work); - int work_flags; + bool do_wake; unsigned long flags; /* @@ -811,14 +811,14 @@ static void io_wqe_enqueue(struct io_wqe *wqe, struct io_wq_work *work) return; } - work_flags = work->flags; raw_spin_lock_irqsave(&wqe->lock, flags); io_wqe_insert_work(wqe, work); wqe->flags &= ~IO_WQE_FLAG_STALLED; + do_wake = (work->flags & IO_WQ_WORK_CONCURRENT) || + !atomic_read(&acct->nr_running); raw_spin_unlock_irqrestore(&wqe->lock, flags); - if ((work_flags & IO_WQ_WORK_CONCURRENT) || - !atomic_read(&acct->nr_running)) + if (do_wake) io_wqe_wake_worker(wqe, acct); }