]> git.ipfire.org Git - thirdparty/kernel/linux.git/commit
block: invalidate cached plug timestamp after task switch
authorUsama Arif <usama.arif@linux.dev>
Tue, 16 Jun 2026 14:15:18 +0000 (07:15 -0700)
committerJens Axboe <axboe@kernel.dk>
Tue, 16 Jun 2026 16:07:36 +0000 (10:07 -0600)
commitfad156c2af227f42ca796cbb20ddc354a6dd9932
treeab68093aae12f62ff5f16d34cf027cbc9d9a0a4d
parentfd38b75c4b43295b10d69772a46d1c74dbd6fc81
block: invalidate cached plug timestamp after task switch

blk_time_get_ns() caches ktime_get_ns() in current->plug->cur_ktime
and marks the task with PF_BLOCK_TS. That cache is only valid while the
task keeps running; if the task is switched out, wall-clock time
advances and the cached value must not be reused when the task runs again.

The existing invalidation covers explicit plug flushes through
__blk_flush_plug(), and the schedule() / rtmutex paths through
sched_update_worker(). It does not cover in-kernel preemption paths such
as preempt_schedule(), preempt_schedule_notrace(), and
preempt_schedule_irq(), which enter __schedule(SM_PREEMPT) directly and
return without calling sched_update_worker().

As a result, a task preempted while holding a plug with PF_BLOCK_TS set
can reuse a stale plug->cur_ktime after it is scheduled back in. blk-iocost
then consumes that stale timestamp through ioc_now(), producing stale vnow
values for throttle decisions, and through ioc_rqos_done(), inflating
on-queue time and feeding false missed-QoS samples into vrate
adjustment.

Move the schedule-side invalidation to finish_task_switch(), which runs
for the scheduled-in task after every actual context switch regardless
of which schedule entry point was used. Keep __blk_flush_plug() as the
explicit flush/finish-plug invalidation path, and remove only the
PF_BLOCK_TS handling from sched_update_worker().

Fixes: 06b23f92af87 ("block: update cached timestamp post schedule/preemption")
Cc: stable@vger.kernel.org
Signed-off-by: Usama Arif <usama.arif@linux.dev>
Link: https://patch.msgid.link/20260616141604.328820-3-usama.arif@linux.dev
Signed-off-by: Jens Axboe <axboe@kernel.dk>
include/linux/blkdev.h
kernel/sched/core.c