journal: Recover filtered journal queries after crash truncated writes

generic_array_get() which is used for the unfiltered iteration path in
the previous commit treats a chain pointer that resolves past the end of
the file as the end of the chain. In that case, moving to the missing
array object returns -EADDRNOTAVAIL (or -EBADMSG), and it either stops
(going downwards) or steps back to the previous array (going upwards).

However, generic_array_bisect(), which is used for filtered or seeking
reads does not. On -EADDRNOTAVAIL/-EBADMSG from
journal_file_move_to_object(), it instead returns the error directly to
the caller, which propagates out through
sd_journal_next()/sd_journal_previous() and aborts the query.

The per-data entry array chain has the same issue as the global one,
since n_entries and entry_array_offset are (re)written in place as
entries are linked, and thus after a crash they can reference more
arrays than actually reached the disk. That is to say in practical
terms, a journal recovered for reading by the previous commit could
nevertheless still drop matching entries from `journalctl FIELD=value`,
and a seqnum or time seek into the lost region could fail outright.

Let's give generic_array_bisect() the same tolerance generic_array_get()
already has. That is, when moving to an entry array object fails, treat
the chain as ending at the previous array. This means that the result
matches what generic_array_get() would yield for the same file.

diff --git a/src/libsystemd/sd-journal/journal-file.c b/src/libsystemd/sd-journal/journal-file.c
index 632b3bddbefa4328ab15dc33727ebf904655aa75..b2ee3626840b1abe2de79ac09a047b8204580f30 100644 (file)
--- a/src/libsystemd/sd-journal/journal-file.c
+++ b/src/libsystemd/sd-journal/journal-file.c
@@ -1546,6 +1546,18 @@ static int get_next_hash_offset(
         return 0;
 }
 
+static bool chain_tail_lost(JournalFile *f, int r, uint64_t offset, ObjectType type) {
+        assert(f);
+
+        /* Only accept truncation when reading. On writing it's not possible to know how to safely proceed. */
+        if (journal_file_writable(f) || !IN_SET(r, -EBADMSG, -EADDRNOTAVAIL))
+                return false;
+
+        log_debug_errno(r, "Failed to read %s at offset %" PRIu64 " of %s, treating it as the end of the chain: %m",
+                        journal_object_type_to_string(type), offset, f->path);
+        return true;
+}
+
 int journal_file_find_field_object_with_hash(
                 JournalFile *f,
                 const void *field,
@@ -3099,6 +3111,10 @@ static int generic_array_bisect(
                 uint64_t left, right, k, m, m_original;
 
                 r = journal_file_move_to_object(f, OBJECT_ENTRY_ARRAY, a, &array);
+                if (chain_tail_lost(f, r, a, OBJECT_ENTRY_ARRAY)) {
+                        r = TEST_GOTO_PREVIOUS;
+                        goto previous;
+                }
                 if (r < 0)
                         return r;