btrfs: trigger cow fixup via dirty_folio()

btrfs: trigger cow fixup via dirty_folio()

The problem scenario:
If we have a folio mmapped shared and then somebody does a dio read with
that folio as the read destination, then it is possible that the dio
will see a dirty destination page when it starts (and thus skip
dirtying and just GUP pin it) but then while it is doing the read, btrfs
finishes writing it back and by the endio, the folio is clean. In that
case, the dio read must re-dirty the folio with aops->dirty_folio():

btrfs_check_read_bio()
|- __iomap_dio_bio_end_io() from btrfs_bio_end_io()
   |- bio_check_pages_dirty()
      |- bio_dirty_fn()
         |- bio_release_pages(bio, true)
            |- __bio_release_pages(bio, mark_dirty == true)
               |- folio_lock()
               |- folio_mark_dirty()
                  |- aops->dirty_folio()
               |- folio_unlock()

A data block normally moves through writeback as follows:

  TASK
    folio_lock
    write              clean -> dirty bit + delalloc
    folio_unlock
  WRITEBACK
    for-each-dirty-folio:
      folio_lock
      run_delalloc     delalloc consumed  -> dirty bit + OE
      submission       dirty bit consumed -> writeback bit + OE
      folio_unlock
  ENDIO
    endio              OE bytes accounted
    OE finish          writeback -> clean; destroy OE

Three critical invariants that this path maintains are:

  I1. Any dirty block is covered by delalloc xor an ordered extent
  I2. Any dirty block covered by an OE will be submitted into that OE
  I3. Any dirty block already submitted into an OE will not be submitted
      again into the same OE.

These ensure that the block will be written exactly once. It is clear
that not reserving delalloc for the re-dirty case violates I1.

This situation, even without bs < folio_size, has long required btrfs to
fixup such dirty pages during writeback with an asynchronous worker that
is allowed to do this expensive work and writeback does not proceed for
a folio while it is doing this work.

Commit 247e743cbe6e ("Btrfs: Use async helpers to deal with pages that
have been improperly dirtied") introduced the COW fixup to catch exactly
this class at writeback, way back in 2008.

Since then, there have been many advances to prevent most of the causes
of such re-dirtying and we thought we could get away with removing the
annoying cow-fixup in the hope of simplifying writeback for large folio
support.

  Commit b2a9f217ad3f ("btrfs: remove the COW fixup mechanism")
  Commit 4927b141877c ("btrfs: remove folio ordered flag and subpage bitmap")

Since it turns out this assumption was incorrect, as evidenced by the
report and attendant reproducers, we must reintroduce the fixup concept.

This is of course critically further complicated by bs < folio_size. In
that case, rather than just a folio dirty bit, we have a bitmap for the
dirty blocks in the folio. And the (also broken) invariant is:

  I4. folio dirty IFF at least one block bitmap dirty.

The original report of a stall on a misinterpreted empty bitmap is
exactly evidence of a violation of I4.

It is exactly because of bs < folio_size we don't want to simply revert the
removal patches. The original fixup was not properly bs < folio_size
aware, which motivated removal in the first place. So we wish to build a
bs < folio_size aware fixup.

One other important detail from the old design, any normal write that
happens after a re-dirty but before a fixup is racing with the cow fixup
to do the delalloc reservation, therefore it must cancel the fixup state.
If it arrives after the reservation exists, it will be a normal dirty
overwrite. This critically informs the design in a pretty clear way.
fixup requiring re-dirty has folio granularity, while cancellation has
delalloc (block) granularity so while we only ever produce fixup in
chunks of folios, we must be able to clear it in blocks. Therefore we
must track the blocks needing fixup at block granularity.

The obvious way to do this is with a new bitmap in btrfs_folio_state,
but it is desirable to avoid that if possible. Unfortunately, I don't
think it is possible and the reason is subtle and leans on a sort of
extreme reproducer, but I think can be explained relatively succinctly.

Consider a folio whose two halves will land in different ordered extents
(can be accomplished with tricks using nodatasum) and a dio read is
running with it as the shared mmap destination.

1. The front half:
   a. folio comes clean on a normal write
   b. dio read completes into the folio marking it fixup.
   c. a write comes for the previous folio for a range extending into
      this folio, this is a cancellation of the fixup which reserves
      space.
   d. writeback runs on the range *not* overlapping the folio. This half
      remains dirty but is now covered by an OE and is awaiting
      writeback running on its range to be submitted and finish the OE.

2. The back half:
   a. the folio is part of an OE that gets far enough along to clear
      writeback.
   b. dio read completes into the folio marking it fixup.

After this, the folio's front half is dirty in the "normal" sense, it
needs to be submitted to the OE waiting for it. It's a cancelled fixup.
Meanwhile, the second half is a true fresh fixup. So at this point if we
run writeback on this folio, we genuinely can't know what to do without
block level information. If we submit it, we submit unreserved dirty
from the back half. If we don't, we will never finish the OE waiting for
it. So it's either a corruption or a deadlock.

Thus, the full high level design picture:

- btrfs_data_dirty_folio(): For out of band non-reserving dirties,
  mark still-clean blocks inside EOF dirty and set their fixup bits
  (the event carries no range, so every clean block is suspect).
  Already-dirty blocks are covered or pending and are left alone.

- Writeback: skip fixup blocks and enqueue work for them

- writepage_fixup(): for each fixup block do the fixup reservation in a
  worker, after which the blocks can be written back normally.

- Typical reserving write paths cancel fixup state for the ranges they
  cover with btrfs_folio_cancel_fixup()

Link: https://lore.kernel.org/linux-btrfs/20260721191152.101118-1-borntraeger@linux.ibm.com/
Assisted-by: LLM
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>