index-pack: retain child bases in delta cache

When resolving a delta whose result has children of its own,
index-pack adds the result to work_head, accounts its data in
base_cache_used, and calls prune_base_data(). It then immediately frees
that same data.

This bypasses the existing delta base cache policy and can force later
descendants to reconstruct the queued base again. Let the existing
delta_base_cache_limit pruning policy decide whether to keep or evict
the data instead.

This does not add a new cache or increase the cache limit. The object
data is already accounted in base_cache_used before prune_base_data()
runs, and the existing pruning and base cleanup paths still release it.

On a quiet Ubuntu 24.04 VM with 16 vCPUs, 32 GiB RAM, and local SSD,
direct index-pack timings on single-pack Linux fixtures improved as
follows:

  linux blobless: 69.17s -> 57.98s (16.2% faster), RSS flat
  linux full:     280.72s -> 236.32s (15.8% faster), RSS +1.9%

Five-repeat medians on public repositories also improved:

  git.git:  12.31s -> 10.70s (13.1% faster)
  libgit2:   3.35s ->  2.88s (14.0% faster)
  redis:     6.52s ->  5.64s (13.5% faster)
  cpython:  33.02s -> 31.44s (4.8% faster)

The standard p5302 perf test on a smaller git.git fixture was neutral:

  5302.9 index-pack default threads:
    11.21(38.07+1.33) -> 11.16(37.90+1.31), -0.4%

t/t5302-pack-index.sh passed, and GitGitGadget's linux-leaks CI also
exercised that test under SANITIZE=leak.

Signed-off-by: Arijit Banerjee <arijit@effectiveailabs.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index cf0bd8280dca83e9da845e4010d9d0c4d0e7d5d4..027c64b522ebb26e88581f5f3266f963d1e586f9 100644 (file)
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -1212,7 +1212,6 @@ static void *threaded_second_pass(void *data)
                        list_add(&child->list, &work_head);
                        base_cache_used += child->size;
                        prune_base_data(NULL);
-                       free_base_data(child);
                } else if (child) {
                        /*
                         * This child does not have its own children. It may be