lib-index: Fix calling expunge handlers with mdbox

This fixes updating deleted_record_count in dovecot.index.cache files.
Because they were wrong, the cache wasn't always compressed as early as
it should have been.

mdbox uses a separate transaction to commit expunges while mailbox is being
synced. When syncing was finished, tail_offset was updated too early so
mail_index_map(MAIL_INDEX_SYNC_HANDLER_FILE) was always a no-op and expunge
handlers were never called.

There doesn't seem to be any downside to not updating tail_offset early.
sdbox saving also uses such a transaction, but there is no difference in
the resulting dovecot.index.log file. The main worry I had was that
tail_offset wouldn't be updated to point to the end of the log file.
However, this doesn't happen with the old code either. This is because the
extra transaction is external, and tail_offset updating skips over all
external transactions anyway.

diff --git a/src/lib-index/mail-index-sync.c b/src/lib-index/mail-index-sync.c
index 656a804d30e8d9d9ca4a71450c931e6564514150..2e8082b5c2532e7666b7a5c2645ee3246293a220 100644 (file)
--- a/src/lib-index/mail-index-sync.c
+++ b/src/lib-index/mail-index-sync.c
@@ -789,9 +789,27 @@ mail_index_sync_update_mailbox_offset(struct mail_index_sync_ctx *ctx)
                   tail offset. */
                return;
        }
-       /* synced everything, but we might also have committed new
-          transactions. include them also here. */
-       mail_transaction_log_get_head(ctx->index->log, &seq, &offset);
+       /* All changes were synced. During the syncing other transactions may
+          have been created and committed as well. They're expected to be
+          external transactions. These could be at least:
+           - mdbox finishing expunges
+           - mdbox writing to dovecot.map.index (requires tail offset updates)
+           - sdbox appending messages
+
+          If any expunges were committed, tail_offset must not be updated
+          before mail_index_map(MAIL_INDEX_SYNC_HANDLER_FILE) is called.
+          Otherwise expunge handlers won't be called for them.
+
+          We'll require MAIL_INDEX_SYNC_FLAG_UPDATE_TAIL_OFFSET flag for the
+          few places that actually require tail_offset to include the
+          externally committed transactions. Otherwise tail_offset is updated
+          only up to what was just synced. */
+       if ((ctx->flags & MAIL_INDEX_SYNC_FLAG_UPDATE_TAIL_OFFSET) != 0)
+               mail_transaction_log_get_head(ctx->index->log, &seq, &offset);
+       else {
+               mail_transaction_log_view_get_prev_pos(ctx->view->log_view,
+                                                      &seq, &offset);
+       }
        mail_transaction_log_set_mailbox_sync_pos(ctx->index->log, seq, offset);
 
        /* If tail offset has changed, make sure it gets written to