t/perf: add pack-objects filter and path-walk benchmark

Add p5315-pack-objects-filter.sh to measure the performance of
'git pack-objects --revs --all' under different filter and traversal
combinations:

 * no filter (baseline)
 * --filter=blob:none (blobless)
 * --filter=sparse:oid=<oid> (cone-mode sparse)

Each filter scenario is tested both with and without --path-walk,
producing paired measurements that show the impact of the path-walk
traversal for each filter type as we integrate the --path-walk feature
with different --filter options. It currently has no integration so
falls back to the standard revision walk. Thus, there are no significant
differences in the current results other than a full repack (and even
then, the --path-walk feature is not incredibly different for the
default Git repository):

Test                                             HEAD
-----------------------------------------------------
5315.2: repack (no filter)                      27.91
5315.3: repack size (no filter)                250.7M
5315.4: repack (no filter, --path-walk)         34.92
5315.5: repack size (no filter, --path-walk)   220.0M
5315.6: repack (blob:none)                      13.63
5315.7: repack size (blob:none)                137.6M
5315.8: repack (blob:none, --path-walk)         13.48
5315.9: repack size (blob:none, --path-walk)   137.7M
5315.10: repack (sparse:oid)                    72.67
5315.11: repack size (sparse:oid)              187.4M
5315.12: repack (sparse:oid, --path-walk)       72.47
5315.13: repack size (sparse:oid, --path-walk) 187.4M

The sparse filter definition is built automatically by sampling
depth-2 directories from the test repository, making the test work
on any repo passed via GIT_PERF_LARGE_REPO. For repos that lack
depth-2 directories, a single top-level directory is used; for flat
repos, the sparse tests are skipped via prerequisite.

Signed-off-by: Derrick Stolee <stolee@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

diff --git a/t/perf/p5315-pack-objects-filter.sh b/t/perf/p5315-pack-objects-filter.sh
new file mode 100755 (executable)
index 0000000..445ff25
--- /dev/null
+++ b/t/perf/p5315-pack-objects-filter.sh
@@ -0,0 +1,129 @@
+#!/bin/sh
+
+test_description='Tests pack-objects performance with filters and --path-walk'
+. ./perf-lib.sh
+
+test_perf_large_repo
+
+test_expect_success 'setup filter inputs' '
+       # Sample a few depth-2 directories from the test repo to build
+       # a cone-mode sparse-checkout definition.  The sampling picks
+       # directories at evenly-spaced positions so the choice is stable
+       # and scales to repos of any shape.
+
+       git ls-tree -d HEAD >top-entries &&
+       grep "^040000" top-entries |
+               awk "{print \$4;}" >top-dirs &&
+       top_nr=$(wc -l <top-dirs) &&
+
+       while read tdir
+       do
+               git ls-tree -d --format="$tdir/%(path)" "HEAD:$tdir" || return 1
+       done <top-dirs >depth2-dirs &&
+
+       d2_nr=$(wc -l <depth2-dirs) &&
+
+       if test "$d2_nr" -ge 2
+       then
+               # Pick two directories from evenly-spaced positions.
+               first=$(sed -n "1p" depth2-dirs) &&
+               mid=$(sed -n "$((d2_nr / 2 + 1))p" depth2-dirs) &&
+
+               p1=$(dirname "$first") &&
+               p2=$(dirname "$mid") &&
+
+               # Build cone-mode sparse-checkout patterns.
+               {
+                       echo "/*" &&
+                       echo "!/*/" &&
+                       echo "/$p1/" &&
+                       echo "!/$p1/*/" &&
+                       if test "$p1" != "$p2"
+                       then
+                               echo "/$p2/" &&
+                               echo "!/$p2/*/"
+                       fi &&
+                       echo "/$first/" &&
+                       if test "$first" != "$mid"
+                       then
+                               echo "/$mid/"
+                       fi
+               } >sparse-patterns &&
+
+               git hash-object -w sparse-patterns >sparse-oid &&
+               echo "Sparse cone: $first $mid" &&
+               cat sparse-patterns &&
+               test_set_prereq SPARSE_OID
+       elif test "$top_nr" -ge 1
+       then
+               # Fallback: use a single top-level directory.
+               first=$(sed -n "1p" top-dirs) &&
+               {
+                       echo "/*" &&
+                       echo "!/*/" &&
+                       echo "/$first/"
+               } >sparse-patterns &&
+
+               git hash-object -w sparse-patterns >sparse-oid &&
+               echo "Sparse cone: $first" &&
+               cat sparse-patterns &&
+               test_set_prereq SPARSE_OID
+       fi
+'
+
+test_perf 'repack (no filter)' '
+       git pack-objects --stdout --no-reuse-delta --revs --all </dev/null >pk
+'
+
+test_size 'repack size (no filter)' '
+       test_file_size pk
+'
+
+test_perf 'repack (no filter, --path-walk)' '
+       git pack-objects --stdout --no-reuse-delta --revs --all --path-walk </dev/null >pk
+'
+
+test_size 'repack size (no filter, --path-walk)' '
+       test_file_size pk
+'
+
+test_perf 'repack (blob:none)' '
+       git pack-objects --stdout --no-reuse-delta --revs --all --filter=blob:none </dev/null >pk
+'
+
+test_size 'repack size (blob:none)' '
+       test_file_size pk
+'
+
+test_perf 'repack (blob:none, --path-walk)' '
+       git pack-objects --stdout --no-reuse-delta --revs --all --path-walk \
+               --filter=blob:none </dev/null >pk
+'
+
+test_size 'repack size (blob:none, --path-walk)' '
+       test_file_size pk
+'
+
+test_perf 'repack (sparse:oid)' \
+       --prereq SPARSE_OID '
+       git pack-objects --stdout --no-reuse-delta --revs --all \
+               --filter=sparse:oid=$(cat sparse-oid) </dev/null >pk
+'
+
+test_size 'repack size (sparse:oid)' \
+       --prereq SPARSE_OID '
+       test_file_size pk
+'
+
+test_perf 'repack (sparse:oid, --path-walk)' \
+       --prereq SPARSE_OID '
+       git pack-objects --stdout --no-reuse-delta --revs --all --path-walk \
+               --filter=sparse:oid=$(cat sparse-oid) </dev/null >pk
+'
+
+test_size 'repack size (sparse:oid, --path-walk)' \
+       --prereq SPARSE_OID '
+       test_file_size pk
+'
+
+test_done