midx.c: protect against disappearing packs

When a packed object is stored in a multi-pack index, but that pack has
racily gone away, the MIDX code simply calls die(), when it could be
returning an error to the caller, which would in turn lead to
re-scanning the pack directory.

A pack can racily disappear, for example, due to a simultaneous 'git
repack -ad',

You can also reproduce this with two terminals, where one is running:

    git init
    while true; do
      git commit -q --allow-empty -m foo
      git repack -ad
      git multi-pack-index write
    done

(in effect, constantly writing new MIDXs), and the other is running:

    obj=$(git rev-parse HEAD)
    while true; do
      echo $obj | git cat-file --batch-check='%(objectsize:disk)' || break
    done

That will sometimes hit the error preparing packfile from
multi-pack-index message, which this patch fixes.

Right now, that path to discovering a missing pack looks something like
'find_pack_entry()' calling 'fill_midx_entry()' and eventually making
its way to call 'nth_midxed_pack_entry()'.

'nth_midxed_pack_entry()' already checks 'is_pack_valid()' and
propagates an error if the pack is invalid. So, this works if the pack
has gone away between calling 'prepare_midx_pack()' and before calling
'is_pack_valid()', but not if it disappears before then.

Catch the case where the pack has already disappeared before
'prepare_midx_pack()' by returning an error in that case, too.

Co-authored-by: Jeff King <peff@peff.net>
Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

diff --git a/midx.c b/midx.c
index f29afc0d2daba3ae9364561be0ea7fe18acc0cc7..62101c76085008cf7320a74f75dece6874c138b8 100644 (file)
--- a/midx.c
+++ b/midx.c
@@ -285,7 +285,7 @@ static int nth_midxed_pack_entry(struct repository *r,
        pack_int_id = nth_midxed_pack_int_id(m, pos);
 
        if (prepare_midx_pack(r, m, pack_int_id))
-               die(_("error preparing packfile from multi-pack-index"));
+               return 0;
        p = m->packs[pack_int_id];
 
        /*

diff --git a/t/t5319-multi-pack-index.sh b/t/t5319-multi-pack-index.sh
index 98c7c9df155e31ea51184a42dfa6013cc4c94214..27c7eb906aed6dc05e32af82e485b53b73dbdeaa 100755 (executable)
--- a/t/t5319-multi-pack-index.sh
+++ b/t/t5319-multi-pack-index.sh
@@ -547,7 +547,7 @@ test_expect_success 'repack --batch-size=0 repacks everything' '
        )
 '
 
-test_expect_success 'load reverse index when missing .idx' '
+test_expect_success 'load reverse index when missing .idx, .pack' '
        git init repo &&
        test_when_finished "rm -fr repo" &&
        (
@@ -560,9 +560,15 @@ test_expect_success 'load reverse index when missing .idx' '
                git multi-pack-index write &&
 
                git rev-parse HEAD >tip &&
+               pack=$(ls .git/objects/pack/pack-*.pack) &&
                idx=$(ls .git/objects/pack/pack-*.idx) &&
 
                mv $idx $idx.bak &&
+               git cat-file --batch-check="%(objectsize:disk)" <tip &&
+
+               mv $idx.bak $idx &&
+
+               mv $pack $pack.bak &&
                git cat-file --batch-check="%(objectsize:disk)" <tip
        )
 '