unpack-objects: use the bulk-checkin infrastructure

The unpack-objects functionality is used by fetch, push, and fast-import
to turn the transfered data into object database entries when there are
fewer objects than the 'unpacklimit' setting.

By enabling an odb-transaction when unpacking objects, we can take advantage
of batched fsyncs.

Here are some performance numbers to justify batch mode for
unpack-objects, collected on a WSL2 Ubuntu VM.

Fsync Mode | Time for 90 objects (ms)
-------------------------------------
       Off | 170
  On,fsync | 760
  On,batch | 230

Note that the default unpackLimit is 100 objects, so there's a 3x
benefit in the worst case. The non-batch mode fsync scales linearly
with the number of objects, so there are significant benefits even with
smaller numbers of objects.

Signed-off-by: Neeraj Singh <neerajsi@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

diff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c
index 4a9466295ba10de5752cca92fa08b1a28461777b..c76a812eb0b1396ca59edf68b230fc096c88cb8f 100644 (file)
--- a/builtin/unpack-objects.c
+++ b/builtin/unpack-objects.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "bulk-checkin.h"
 #include "config.h"
 #include "object-store.h"
 #include "object.h"
@@ -503,10 +504,12 @@ static void unpack_all(void)
        if (!quiet)
                progress = start_progress(_("Unpacking objects"), nr_objects);
        CALLOC_ARRAY(obj_list, nr_objects);
+       begin_odb_transaction();
        for (i = 0; i < nr_objects; i++) {
                unpack_one(i);
                display_progress(progress, i + 1);
        }
+       end_odb_transaction();
        stop_progress(&progress);
 
        if (delta_list)