btrfs: add extra device item checks at mount

btrfs: add extra device item checks at mount

[BUG]
There is a bug report where after a dev-replace, the replace source
device with devid 4 is properly erased (dump tree shows it's the old
devid 4), but the target device is still using devid 0.

When the user tries to mount the fs degraded, the mount failed with the
following errors:

  BTRFS: device fsid 84a1ed4a-365c-45c3-a9ee-a7df525dc3c9 devid 5 transid 1394395 /dev/sda (8:0) scanned by btrfs (261)
  BTRFS: device fsid 84a1ed4a-365c-45c3-a9ee-a7df525dc3c9 devid 6 transid 1394395 /dev/sde (8:64) scanned by btrfs (261)
  BTRFS: device fsid 84a1ed4a-365c-45c3-a9ee-a7df525dc3c9 devid 0 transid 1394395 /dev/sdd (8:48) scanned by btrfs (261)
  BTRFS: device fsid 84a1ed4a-365c-45c3-a9ee-a7df525dc3c9 devid 3 transid 1394395 /dev/sdf (8:80) scanned by btrfs (261)
  BTRFS info (device sdd): first mount of filesystem 84a1ed4a-365c-45c3-a9ee-a7df525dc3c9
  BTRFS info (device sdd): using crc32c (crc32c-intel) checksum algorithm
  BTRFS warning (device sdd): devid 4 uuid 01e2081c-9c2a-4071-b9f4-e1b27e571ff5 is missing
  BTRFS info (device sdd): bdev <missing disk> errs: wr 84994544, rd 15567, flush 65872, corrupt 0, gen 0
  BTRFS info (device sdd): bdev /dev/sdd errs: wr 71489901, rd 0, flush 30001, corrupt 0, gen 0
  BTRFS error (device sdd): replace without active item, run 'device scan --forget' on the target device
  BTRFS error (device sdd): failed to init dev_replace: -117
  BTRFS error (device sdd): open_ctree failed: -117

[CAUSE]
The devid 0 didn't get its devid updated is its own problem, here I'm
only focusing on the mount failure itself.

The mount is not caused by the missing device, as the fs has RAID1C3 for
metadata and RAID10 for data, thus is completely able to tolerate one
missing device.

The device tree shows the dev-replace has properly finished:

        item 7 key (0 DEV_REPLACE 0) itemoff 15931 itemsize 72
                src devid -1 cursor left 11091821199360 cursor right 11091821199360 mode ALWAYS
                state FINISHED write errors 0 uncorrectable read errors 0
		      ^^^^^^^^

And the chunk tree shows there is no devid 0:

  leaf 37980736602112 items 23 free space 12548 generation 1394388 owner CHUNK_TREE
  leaf 37980736602112 flags 0x1(WRITTEN) backref revision 1
  fs uuid 84a1ed4a-365c-45c3-a9ee-a7df525dc3c9
  chunk uuid d074c661-6311-4570-b59f-a5c83fd37f8e
         item 0 key (DEV_ITEMS DEV_ITEM 3) itemoff 16185 itemsize 98
                 devid 3 total_bytes 20000588955648 bytes_used 8282877984768
                 io_align 4096 io_width 4096 sector_size 4096 type 0
                 generation 0 start_offset 0 dev_group 0
                 seek_speed 0 bandwidth 0
                 uuid 0d596b69-fb0d-4031-b4af-a301d0868b8b
                 fsid 84a1ed4a-365c-45c3-a9ee-a7df525dc3c9
         ...

Which shows the first device is devid 3.

But there is indeed /dev/sdd with devid 0:

  superblock: bytenr=65536, device=/dev/sdd
  ---------------------------------------------------------
  csum_type               0 (crc32c)
  csum_size               4
  csum                    0xd4bed87e [match]
  bytenr                  65536
  flags                   0x1
                          ( WRITTEN )
  magic                   _BHRfS_M [match]
  fsid                    84a1ed4a-365c-45c3-a9ee-a7df525dc3c9
  ...
  uuid_tree_generation    1394388
  dev_item.uuid           ee6532ad-5442-45f7-87fb-7703e29ed934
  dev_item.fsid           84a1ed4a-365c-45c3-a9ee-a7df525dc3c9 [match]
  dev_item.type           0
  dev_item.total_bytes    20000588955648
  dev_item.bytes_used     8292541661184
  dev_item.io_align       0
  dev_item.io_width       0
  dev_item.sector_size    0
  dev_item.devid          0 <<<

So this means device scan will register sdd as devid 0 into the fs, then
during btrfs_init_dev_replace(), we located the replace progress item,
found the previous replace is finished, but we still need to check if
the dev-replace target device (devid 0) exists.

If that device exists, we error out showing that error message.

But to be honest the end user may not really remember which device is
the replace target device, thus not sure what to do in the next step.

[ENHANCEMENT]
To make the error more obvious, and tell the end user which devices
should be unregistered:

- Introduce BTRFS_DEV_STATE_ITEM_FOUND flag
  During device item read from the chunk tree, set the flag for each
  found device item.

- Verify there is no device without the above flag during mount
  Even missing device should have that flag set.
  If we found a device without that flag set, it means it's an
  unexpected one and should be rejected.

- More detailed error message on what to do next
  This will show all unexpected devices and tell the end user to use
  'btrfs dev scan --forget' to forget them or remove them before mount.

There is an example dmesg where a device of a valid filesystem is modified to
have devid 0, then try degraded mount:

  BTRFS info (device dm-6): first mount of filesystem 7c873869-844c-4b39-bd75-a96148bf4656
  BTRFS info (device dm-6): using crc32c checksum algorithm
  BTRFS warning (device dm-6): devid 3 uuid b4a9f35b-db42-4ac4-b55a-cbf81d3b9683 is missing
  BTRFS error (device dm-6): devid 0 path /dev/mapper/test-scratch3 is registered but not found in chunk tree
  BTRFS error (device dm-6): please remove above devices or use 'btrfs device scan --forget <dev>' to unregister them before mount
  BTRFS error (device dm-6): open_ctree failed: -117

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>