Kent Overstreet [Wed, 1 Feb 2023 20:45:45 +0000 (15:45 -0500)]
bcachefs: Improve btree node read error path
This ensures that failure to read a btree node error is treated as a
topology error, and returns the correct error so that the topology
repair pass will be run.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Tue, 31 Jan 2023 01:58:43 +0000 (20:58 -0500)]
bcachefs: Better inlining for bch2_alloc_to_v4_mut
This separates out the slowpath into a separate function, and inlines
bch2_alloc_v4_mut into bch2_trans_start_alloc_update(), the main place
it's called.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Sun, 8 Jan 2023 03:55:42 +0000 (22:55 -0500)]
bcachefs: Fix rereplicate when we already have a cached pointer
When we need to add more replicas to an extent, it might be the case
that we already have a replica on every device, but some of them are
cached.
This patch fixes a bug where we'd spin on that extent because the write
path fails to find a device we can allocate from: we allow allocating
from devices that already have cached replicas on them, and change
bch2_data_update_index_update() to drop the cached replica if needed.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Sat, 3 Dec 2022 20:44:54 +0000 (15:44 -0500)]
bcachefs: Fix compat path for old inode formats
Old inode formats don't have all the fields of the current inode format:
when unpacking inodes in the current format we can thus skip zeroing out
the destination buffer, but that doesn't work on for the old formats.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Sun, 8 Jan 2023 05:04:30 +0000 (00:04 -0500)]
bcachefs: Fix bch2_btree_path_traverse_all()
We need to take a ref on a path while we're traversing it: this fixes a
bug with paths getting reused while being traversed, in the key cache
fill code.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Tue, 3 Jan 2023 22:32:16 +0000 (17:32 -0500)]
bcachefs: Plumb saw_error through to btree_err()
The btree node read path has the ability to kick off an asynchronous
btree node rewrite if we saw and corrected an error. Previously this was
only used for errors that caused one of the replicas to be unusable -
this patch plumbs it through to all error paths, so that normal fsck
errors can be corrected.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Wed, 14 Dec 2022 15:39:04 +0000 (10:39 -0500)]
bcachefs: Log more messages in the journal
This patch
- Adds a mechanism for queuing up journal entries prior to the journal
being started, which will be used for early journal log messages
- Adds bch2_fs_log_msg() and improves bch2_trans_log_msg(), which now
take format strings. bch2_fs_log_msg() can be used before or after
the journal has been started, and will use the appropriate mechanism.
- Deletes the now obsolete bch2_journal_log_msg()
- And adds more log messages to the recovery path - messages for
journal/filesystem started, journal entries being blacklisted, and
journal replay starting/finishing.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Wed, 28 Dec 2022 20:17:07 +0000 (15:17 -0500)]
bcachefs: Fix some memcpy() warnings
With CONFIG_FORTIFY_SOURCE, the compiler attempts to warn about mempcys
that extend past struct field boundaries. This results in some spurious
warnings where we use embedded variable length structs, this patch
switches to unsafe_mecpy() to fix the warnings.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Fri, 16 Dec 2022 02:44:32 +0000 (21:44 -0500)]
bcachefs: Fix for long running btree transactions & key cache
While a btree transaction is running, we hold a SRCU read lock on the
btree key cache that prevents btree key cache keys from being freed -
this is so that relock() operations won't access freed memory.
The downside of this is that long running btree transactions prevent
memory from being freed from the key cache. This adds a check in
bch2_trans_begin() - if the transaction has been running longer than 1
second, drop and retake the SRCU read lock and zero out pointers to
unlock key cache paths.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Thu, 24 Nov 2022 01:11:46 +0000 (20:11 -0500)]
bcachefs: Add some unlikely() annotations
Add a few easy unlikely() optimizations. These are mainly worthwhile
because the compiler will (usually) put the branch-not-taken path at the
end of the function, meaning better icache utilization.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Wed, 14 Dec 2022 19:47:42 +0000 (14:47 -0500)]
bcachefs: Recover from blacklisted journal entries
If it so happens that we crash while dirty, meaning we don't have the
superblock clean section, and we erroneously mark a journal entry we
wrote as blacklisted, we won't be able to recover.
This patch fixes this by adding a fallback: if we've got no superblock
clean section, and no non-ignored journal entries, we try the most
recent ignored journal entry.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Tue, 13 Dec 2022 23:19:30 +0000 (18:19 -0500)]
bcachefs: Fix error path in bch2_trans_commit_write_locked()
Previously, we were journalling extra_journal_entries (which is used for
new btree roots, and irreversably mutates system state) before calling
bch2_trans_fs_usage_apply(), which can fail - whoops.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Wed, 23 Nov 2022 23:46:03 +0000 (18:46 -0500)]
bcachefs: bch2_trans_revalidate_updates_in_node()
When we started stashing the key being overwritten in
btree_insert_entry, this introduced a typical iterator invalidation
problem, triggered by btree node splits or resorts.
Previously, dealt with this by unconditionally re-validating those
stashed pointers in the transaction commit path. This patch gets rid of
that by doing it only when needed, in bch2_trans_node_add() or
bch2_trans_node_reinit_iter().
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Sun, 20 Nov 2022 03:39:08 +0000 (22:39 -0500)]
bcachefs: More errcode cleanup
We shouldn't be overloading standard error codes now that we have
provisions for bcachefs-specific errorcodes: this patch converts super.c
and super-io.c to per error site errcodes, with a bit of cleanup.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Fri, 9 Dec 2022 21:22:36 +0000 (16:22 -0500)]
bcachefs: Add a missing bch2_btree_path_traverse() call
bch2_btree_iter_peek_upto() in snapshots mode may need to keep a
btree_path for the insert position, not just the position of the key
we're returning. The code was incorrectly assuming this would be in the
same btree node - we were missing a bch2_btree_path_traverse() call.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Fri, 2 Dec 2022 02:59:25 +0000 (21:59 -0500)]
bcachefs: Simplify journal read path
This just cleans up and simplifies the code that decides where to resume
writing in the journal - when the code was originally written we weren't
saving the precise location of every journal write found.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Fri, 2 Dec 2022 16:45:58 +0000 (11:45 -0500)]
bcachefs: Fix a "no journal entries found" bug
On startup, we need to ensure the first journal entry written is a flush
write: after a clean shutdown we generally don't read the journal, which
means we might be overwriting whatever was there previously, and there
must always be at least one flush entry in the journal or recovery will
fail.
Found by fstests generic/388.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Thu, 1 Dec 2022 16:17:18 +0000 (11:17 -0500)]
bcachefs: Don't error out when just reading the journal
This tweaks the recovery and journal paths so that we don't error out
before we need to: the list_journal command should work, even if we
wouldn't be able to replay successfully.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Tue, 20 Dec 2022 16:13:19 +0000 (11:13 -0500)]
bcachefs: bch2_btree_path_peek_slot_exact()
When we start using the key cache for inodes again, it'll be possible
for bch2_btree_path_peek_slot() to return a key in a different snapshot
with a key cache path.
This isn't what we want when triggers are checking what they're
overwriting, so introduce a new helper for the commit path.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Sat, 26 Nov 2022 09:36:56 +0000 (04:36 -0500)]
bcachefs: extents no longer require special handling for packing
Extent overwrite used to be handled differently, underneath the
journaling layer and within the core btree code. This imposed
restrictions on bkey packing/packed formats, which no longer apply.
This patch deletes those restrictions.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Fri, 25 Nov 2022 23:29:36 +0000 (18:29 -0500)]
bcachefs: Fix BCH_IOCTL_DISK_SET_STATE
- Ensure we print an error message if necessary.
Ideally we'd return the precise error code to userspace and leave
printing the error message to the userspace tool, but we haven't
decided to make our private error codes ABI-stable yet.
- Return standard error code to userspace
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Fri, 25 Nov 2022 21:04:42 +0000 (16:04 -0500)]
bcachefs: Don't set accessed bit on btree node fill
Btree nodes shouldn't have their accessed bit set when entering the
btree cache by being read in from disk - this fixes linear scans
thrashing the cache.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Fri, 25 Nov 2022 05:40:27 +0000 (00:40 -0500)]
bcachefs: Optimize bch2_trans_iter_init()
When flags & btree_id are constants, we can constant fold the entire
calculation of the actual iterator flags - and the whole thing becomes
small enough to inline.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Wed, 23 Nov 2022 23:51:27 +0000 (18:51 -0500)]
bcachefs: Kill some unneeded references to c->flags
This drops some unneeded references to JOURNAL_REPLAY_DONE in c->flags:
we're already mirroring it in btree_trans, we just weren't using it
consistently.
We may want to do this with more flags:
btree_iter.c: unsigned nr = test_bit(BCH_FS_STARTED, &c->flags)
btree_update_leaf.c: if (unlikely(!test_bit(BCH_FS_MAY_GO_RW, &c->flags))) {
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Thu, 24 Nov 2022 01:28:15 +0000 (20:28 -0500)]
bcachefs: Improve bch2_inode_opts_to_opts()
It turns out the *_defined entries of bch_io_opts are only used in one
place - in the xattr get path - and there we immediately convert to a
bch_opts struct, which also has the *_defined entries.
This patch changes bch2_inode_opts_to_opts() to go directly from
bch_inode_unpacked to bch_opts, which is a minor simplification and will
also let us slim down struct bch_io_opts in another patch.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Wed, 23 Nov 2022 23:22:59 +0000 (18:22 -0500)]
bcachefs: Better inlining in bch2_subvolume_get_snapshot()
This provides an inlined version of bch2_subvolume_get() and uses it in
bch2_subvolume_get_snapshot(), since this is the version that's used all
over the place and in fast paths (e.g. IO paths).
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Wed, 23 Nov 2022 03:05:45 +0000 (22:05 -0500)]
bcachefs: Split out __bch2_btree_node_get()
Standard splitting out of the slow path from the fast path of a
function. We may follow this up in another patch with inlining the fast
path into btree_iter.c.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Sun, 20 Nov 2022 02:40:35 +0000 (21:40 -0500)]
bcachefs: Handle last journal write being torn
If the last journal write didn't complete sucessfully due to a torn
write, we'll detect it as a checksum error. In that case, we should just
pretend that journal entry was never written.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Sun, 20 Nov 2022 02:20:58 +0000 (21:20 -0500)]
bcachefs: Improve journal_read() logging
Print out the journal entries we read and will replay as soon as
possible - if we get an error walidating keys it's helpful to know where
it was in the journal.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Fri, 18 Nov 2022 00:07:40 +0000 (19:07 -0500)]
bcachefs: Fix a transaction path overflow
It turns out we need bch2_extent_trim_atomi() even when we're deleting
extents one at a time because it's possible for one reflink_p to
reference arbitrarily many reflink_v extents. This doesn't normally
happen, but the data move path can fragment existing extents in the
background.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Wed, 16 Nov 2022 01:25:08 +0000 (20:25 -0500)]
bcachefs: Error message improvement
- Centralize format strings in bcachefs.h
- Add bch2_fmt_inum_offset() and related helpers
- Switch error messages for inodes to also print out the offset, in
bytes
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Wed, 16 Nov 2022 04:17:55 +0000 (23:17 -0500)]
bcachefs: Fix for_each_btree_key2()
Previously, when we exited from the loop body with a break statement
_ret wouldn't have been assigned to yet, and we could spuriously return
a transaction restart error.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Wed, 16 Nov 2022 02:52:12 +0000 (21:52 -0500)]
bcachefs: Btree split improvement
This improves the bkey_format calculation when splitting btree nodes.
Previously, we'd use a format calculated for the original node for the
lower of the two new nodes.
This was particularly bad on sequential insertions, where we iteratively
split the last btree node, whos format has to include KEY_MAX.
Now, we calculate formats precisely for the keys the two new nodes will
contain. This also should make splitting a bit more efficient, since
we're only copying keys once (from the original node to the new node,
instead of new node, replacement node, then upper split).
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Mon, 14 Nov 2022 03:43:37 +0000 (22:43 -0500)]
bcachefs: Minor dio write path improvements
This switches where we take quota reservations to be per bch_wirte_op
instead of per dio_write, so we can drop the quota reservation in the
same place as we call i_sectors_acct(), and only take/release
ei_quota_lock once.
In the future we'd like ei_quota_lock to not be a mutex, so that we can
avoid punting to process context before deliving write completions in
nocow mode.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Mon, 14 Nov 2022 03:35:55 +0000 (22:35 -0500)]
bcachefs: Quota: Don't allocate memory under lock
The genradix code can handle multiple threads trying to allocate at the
same time - we don't need the genradix_ptr_alloc() call to happen under
a lock.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Mon, 14 Nov 2022 07:22:30 +0000 (02:22 -0500)]
bcachefs: Fix a use after free
This fixes a regression from percpu freedlists in the btree key cache
code: in a rare error path, we were immediately freeing a bkey_cached
that had been used before and should've waited for an SRCU barrier.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Fri, 4 Nov 2022 17:25:57 +0000 (13:25 -0400)]
bcachefs: Factor out two_state_shared_lock
We have a unique lock used for controlling adding to the pagecache: the
lock has two states, where both states are shared - the lock may be held
multiple times for either state - but not both states at the same time.
This is exactly what we need for nocow mode locking, so this patch pulls
it out of fs.c into its own file.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Thu, 3 Nov 2022 04:29:43 +0000 (00:29 -0400)]
bcachefs: Kill BCH_WRITE_FLUSH
BCH_WRITE_FLUSH is a write flag that causes a journal flush. It's only
used in the direct IO path, and this will allow for some consolidation
with the regular fsync path, which will help with the upcoming nocow
mode.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Tue, 1 Nov 2022 08:23:24 +0000 (04:23 -0400)]
bcachefs: Improve __bch2_btree_path_make_mut()
btree_path_copy() doesn't need to call
bch2_btree_path_check_sort_fast() - the newly allocated path will always
be in the correct position, post copy; also delete some redundant
branches from __bch2_btree_path_make_mut().
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>