]>
Commit | Line | Data |
---|---|---|
5f40520f JH |
1 | git-pack-objects(1) |
2 | =================== | |
5f40520f JH |
3 | |
4 | NAME | |
5 | ---- | |
7bd7f280 | 6 | git-pack-objects - Create a packed archive of objects |
5f40520f JH |
7 | |
8 | ||
9 | SYNOPSIS | |
10 | -------- | |
ca5381d4 | 11 | [verse] |
4f366275 NP |
12 | 'git pack-objects' [-q | --progress | --all-progress] [--all-progress-implied] |
13 | [--no-reuse-delta] [--delta-base-offset] [--non-empty] | |
62b4698e | 14 | [--local] [--incremental] [--window=<n>] [--depth=<n>] |
ed7e5fc3 | 15 | [--revs [--unpacked | --all]] [--keep-pack=<pack-name>] |
b7573536 | 16 | [--cruft] [--cruft-expiration=<time>] |
49cbad0e JNA |
17 | [--stdout [--filter=<filter-spec>] | <base-name>] |
18 | [--shallow] [--keep-true-parents] [--[no-]sparse] < <object-list> | |
5f40520f JH |
19 | |
20 | ||
21 | DESCRIPTION | |
22 | ----------- | |
4a4becfb JT |
23 | Reads list of objects from the standard input, and writes either one or |
24 | more packed archives with the specified base-name to disk, or a packed | |
25 | archive to the standard output. | |
5f40520f | 26 | |
738820a9 SB |
27 | A packed archive is an efficient way to transfer a set of objects |
28 | between two repositories as well as an access efficient archival | |
29 | format. In a packed archive, an object is either stored as a | |
30 | compressed whole or as a difference from some other object. | |
31 | The latter is often called a delta. | |
32 | ||
33 | The packed archive format (.pack) is designed to be self-contained | |
34 | so that it can be unpacked without any further information. Therefore, | |
35 | each object that a delta depends upon must be present within the pack. | |
36 | ||
37 | A pack index file (.idx) is generated for fast, random access to the | |
38 | objects in the pack. Placing both the index file (.idx) and the packed | |
39 | archive (.pack) in the pack/ subdirectory of $GIT_OBJECT_DIRECTORY (or | |
1e6ab5de | 40 | any of the directories on $GIT_ALTERNATE_OBJECT_DIRECTORIES) |
2de9b711 | 41 | enables Git to read from the pack archive. |
1e6ab5de | 42 | |
0b444cdb | 43 | The 'git unpack-objects' command can read the packed archive and |
5f40520f JH |
44 | expand the objects contained in the pack into "one-file |
45 | one-object" format; this is typically done by the smart-pull | |
46 | commands when a pack is created on-the-fly for efficient network | |
47 | transport by their peers. | |
48 | ||
5f40520f JH |
49 | |
50 | OPTIONS | |
51 | ------- | |
52 | base-name:: | |
4a4becfb | 53 | Write into pairs of files (.pack and .idx), using |
5f40520f | 54 | <base-name> to determine the name of the created file. |
4a4becfb | 55 | When this option is used, the two files in a pair are written in |
d5fa1f1a | 56 | <base-name>-<SHA-1>.{pack,idx} files. <SHA-1> is a hash |
40a4f5a7 | 57 | based on the pack content and is written to the standard |
5f40520f JH |
58 | output of the command. |
59 | ||
60 | --stdout:: | |
89438677 | 61 | Write the pack contents (what would have been written to |
5f40520f JH |
62 | .pack file) out to the standard output. |
63 | ||
4321134c JH |
64 | --revs:: |
65 | Read the revision arguments from the standard input, instead of | |
66 | individual object names. The revision arguments are processed | |
0b444cdb | 67 | the same way as 'git rev-list' with the `--objects` flag |
4321134c JH |
68 | uses its `commit` arguments to build the list of objects it |
69 | outputs. The objects on the resulting list are packed. | |
b790e0f6 NTND |
70 | Besides revisions, `--not` or `--shallow <SHA-1>` lines are |
71 | also accepted. | |
4321134c JH |
72 | |
73 | --unpacked:: | |
74 | This implies `--revs`. When processing the list of | |
75 | revision arguments read from the standard input, limit | |
76 | the objects packed to those that are not already packed. | |
77 | ||
78 | --all:: | |
79 | This implies `--revs`. In addition to the list of | |
80 | revision arguments read from the standard input, pretend | |
cc1b8d8b | 81 | as if all refs under `refs/` are specified to be |
4321134c JH |
82 | included. |
83 | ||
f0a24aa5 SP |
84 | --include-tag:: |
85 | Include unasked-for annotated tags if the object they | |
86 | reference was included in the resulting packfile. This | |
2de9b711 | 87 | can be useful to send new tags to native Git clients. |
f0a24aa5 | 88 | |
339bce27 TB |
89 | --stdin-packs:: |
90 | Read the basenames of packfiles (e.g., `pack-1234abcd.pack`) | |
91 | from the standard input, instead of object names or revision | |
92 | arguments. The resulting pack contains all objects listed in the | |
93 | included packs (those not beginning with `^`), excluding any | |
94 | objects listed in the excluded packs (beginning with `^`). | |
95 | + | |
96 | Incompatible with `--revs`, or options that imply `--revs` (such as | |
97 | `--all`), with the exception of `--unpacked`, which is compatible. | |
98 | ||
b7573536 TB |
99 | --cruft:: |
100 | Packs unreachable objects into a separate "cruft" pack, denoted | |
101 | by the existence of a `.mtimes` file. Typically used by `git | |
102 | repack --cruft`. Callers provide a list of pack names and | |
103 | indicate which packs will remain in the repository, along with | |
104 | which packs will be deleted (indicated by the `-` prefix). The | |
105 | contents of the cruft pack are all objects not contained in the | |
106 | surviving packs which have not exceeded the grace period (see | |
107 | `--cruft-expiration` below), or which have exceeded the grace | |
108 | period, but are reachable from an other object which hasn't. | |
109 | + | |
110 | When the input lists a pack containing all reachable objects (and lists | |
111 | all other packs as pending deletion), the corresponding cruft pack will | |
112 | contain all unreachable objects (with mtime newer than the | |
113 | `--cruft-expiration`) along with any unreachable objects whose mtime is | |
114 | older than the `--cruft-expiration`, but are reachable from an | |
115 | unreachable object whose mtime is newer than the `--cruft-expiration`). | |
116 | + | |
117 | Incompatible with `--unpack-unreachable`, `--keep-unreachable`, | |
118 | `--pack-loose-unreachable`, `--stdin-packs`, as well as any other | |
61568efa | 119 | options which imply `--revs`. |
b7573536 TB |
120 | |
121 | --cruft-expiration=<approxidate>:: | |
122 | If specified, objects are eliminated from the cruft pack if they | |
123 | have an mtime older than `<approxidate>`. If unspecified (and | |
124 | given `--cruft`), then no objects are eliminated. | |
125 | ||
62b4698e ŠN |
126 | --window=<n>:: |
127 | --depth=<n>:: | |
3df19671 | 128 | These two options affect how the objects contained in |
5f40520f JH |
129 | the pack are stored using delta compression. The |
130 | objects are first internally sorted by type, size and | |
131 | optionally names and compared against the other objects | |
132 | within --window to see if using delta compression saves | |
133 | space. --depth limits the maximum delta depth; making | |
134 | it too deep affects the performance on the unpacker | |
135 | side, because delta data needs to be applied that many | |
136 | times to get to the necessary object. | |
b5c0cbd8 NTND |
137 | + |
138 | The default value for --window is 10 and --depth is 50. The maximum | |
139 | depth is 4095. | |
5f40520f | 140 | |
62b4698e | 141 | --window-memory=<n>:: |
e93b15cd BD |
142 | This option provides an additional limit on top of `--window`; |
143 | the window size will dynamically scale down so as to not take | |
62b4698e | 144 | up more than '<n>' bytes in memory. This is useful in |
e93b15cd BD |
145 | repositories with a mix of large and small objects to not run |
146 | out of memory with a large window, but still be able to take | |
147 | advantage of the large window for the smaller objects. The | |
148 | size can be suffixed with "k", "m", or "g". | |
954176c1 MS |
149 | `--window-memory=0` makes memory usage unlimited. The default |
150 | is taken from the `pack.windowMemory` configuration variable. | |
e93b15cd | 151 | |
62b4698e | 152 | --max-pack-size=<n>:: |
4a4becfb JT |
153 | In unusual scenarios, you may not be able to create files |
154 | larger than a certain size on your filesystem, and this option | |
155 | can be used to tell the command to split the output packfile | |
156 | into multiple independent packfiles, each not larger than the | |
157 | given size. The size can be suffixed with | |
07cf0f24 | 158 | "k", "m", or "g". The minimum size allowed is limited to 1 MiB. |
2b84b5a8 | 159 | The default is unlimited, unless the config variable |
6fb9195f JK |
160 | `pack.packSizeLimit` is set. Note that this option may result in |
161 | a larger and slower repository; see the discussion in | |
162 | `pack.packSizeLimit`. | |
6b94b1a0 | 163 | |
e96fb9b8 BC |
164 | --honor-pack-keep:: |
165 | This flag causes an object already in a local pack that | |
0353a0c4 | 166 | has a .keep file to be ignored, even if it would have |
18879bc5 | 167 | otherwise been packed. |
e96fb9b8 | 168 | |
ed7e5fc3 NTND |
169 | --keep-pack=<pack-name>:: |
170 | This flag causes an object already in the given pack to be | |
171 | ignored, even if it would have otherwise been | |
24966cd9 | 172 | packed. `<pack-name>` is the pack file name without |
ed7e5fc3 NTND |
173 | leading directory (e.g. `pack-123.pack`). The option could be |
174 | specified multiple times to keep multiple packs. | |
175 | ||
5f40520f | 176 | --incremental:: |
21da4262 | 177 | This flag causes an object already in a pack to be ignored |
3909f14f | 178 | even if it would have otherwise been packed. |
5f40520f | 179 | |
12ea5bea | 180 | --local:: |
21da4262 | 181 | This flag causes an object that is borrowed from an alternate |
3909f14f JH |
182 | object store to be ignored even if it would have otherwise been |
183 | packed. | |
5f40520f | 184 | |
63ae26f8 NW |
185 | --non-empty:: |
186 | Only create a packed archive if it would contain at | |
187 | least one object. | |
188 | ||
231f240b NP |
189 | --progress:: |
190 | Progress status is reported on the standard error stream | |
191 | by default when it is attached to a terminal, unless -q | |
192 | is specified. This flag forces progress status even if | |
193 | the standard error stream is not directed to a terminal. | |
194 | ||
195 | --all-progress:: | |
196 | When --stdout is specified then progress report is | |
4f366275 | 197 | displayed during the object count and compression phases |
231f240b NP |
198 | but inhibited during the write-out phase. The reason is |
199 | that in some cases the output stream is directly linked | |
200 | to another command which may wish to display progress | |
201 | status of its own as it processes incoming pack data. | |
202 | This flag is like --progress except that it forces progress | |
203 | report for the write-out phase as well even if --stdout is | |
204 | used. | |
205 | ||
4f366275 NP |
206 | --all-progress-implied:: |
207 | This is used to imply --all-progress whenever progress display | |
208 | is activated. Unlike --all-progress this flag doesn't actually | |
209 | force any progress display by itself. | |
210 | ||
ca5381d4 JH |
211 | -q:: |
212 | This flag makes the command not to report its progress | |
213 | on the standard error stream. | |
214 | ||
215 | --no-reuse-delta:: | |
216 | When creating a packed archive in a repository that | |
217 | has existing packs, the command reuses existing deltas. | |
218 | This sometimes results in a slightly suboptimal pack. | |
219 | This flag tells the command not to reuse existing deltas | |
220 | but compute them from scratch. | |
221 | ||
fa736f72 NP |
222 | --no-reuse-object:: |
223 | This flag tells the command not to reuse existing object data at all, | |
224 | including non deltified object, forcing recompression of everything. | |
960ccca6 | 225 | This implies --no-reuse-delta. Useful only in the obscure case where |
fa736f72 NP |
226 | wholesale enforcement of a different compression level on the |
227 | packed data is desired. | |
228 | ||
62b4698e | 229 | --compression=<n>:: |
960ccca6 DH |
230 | Specifies compression level for newly-compressed data in the |
231 | generated pack. If not specified, pack compression level is | |
232 | determined first by pack.compression, then by core.compression, | |
233 | and defaults to -1, the zlib default, if neither is set. | |
483bc4f0 | 234 | Add --no-reuse-object if you want to force a uniform compression |
05cc2ffc | 235 | level on all data no matter the source. |
960ccca6 | 236 | |
de3a8641 DS |
237 | --[no-]sparse:: |
238 | Toggle the "sparse" algorithm to determine which objects to include in | |
4f6d26b1 DS |
239 | the pack, when combined with the "--revs" option. This algorithm |
240 | only walks trees that appear in paths that introduce new objects. | |
241 | This can have significant performance benefits when computing | |
242 | a pack to send a small change. However, it is possible that extra | |
243 | objects are added to the pack-file if the included commits contain | |
de3a8641 DS |
244 | certain types of direct renames. If this option is not included, |
245 | it defaults to the value of `pack.useSparse`, which is true unless | |
246 | otherwise specified. | |
4f6d26b1 | 247 | |
738820a9 SB |
248 | --thin:: |
249 | Create a "thin" pack by omitting the common objects between a | |
250 | sender and a receiver in order to reduce network transfer. This | |
251 | option only makes sense in conjunction with --stdout. | |
252 | + | |
253 | Note: A thin pack violates the packed archive format by omitting | |
2de9b711 | 254 | required objects and is thus unusable by Git without making it |
738820a9 SB |
255 | self-contained. Use `git index-pack --fix-thin` |
256 | (see linkgit:git-index-pack[1]) to restore the self-contained property. | |
257 | ||
2dacf26d | 258 | --shallow:: |
259 | Optimize a pack that will be provided to a client with a shallow | |
1c262bb7 | 260 | repository. This option, combined with --thin, can result in a |
2dacf26d | 261 | smaller pack at the cost of speed. |
262 | ||
63fba759 | 263 | --delta-base-offset:: |
2f8ee02c SB |
264 | A packed archive can express the base object of a delta as |
265 | either a 20-byte object name or as an offset in the | |
2de9b711 | 266 | stream, but ancient versions of Git don't understand the |
0b444cdb | 267 | latter. By default, 'git pack-objects' only uses the |
63fba759 JH |
268 | former format for better compatibility. This option |
269 | allows the command to use the latter format for | |
270 | compactness. Depending on the average delta chain | |
271 | length, this option typically shrinks the resulting | |
272 | packfile by 3-5 per-cent. | |
c14f3727 JH |
273 | + |
274 | Note: Porcelain commands such as `git gc` (see linkgit:git-gc[1]), | |
275 | `git repack` (see linkgit:git-repack[1]) pass this option by default | |
2de9b711 | 276 | in modern Git when they put objects in your repository into pack files. |
c14f3727 | 277 | So does `git bundle` (see linkgit:git-bundle[1]) when it creates a bundle. |
63fba759 | 278 | |
367f4a43 NP |
279 | --threads=<n>:: |
280 | Specifies the number of threads to spawn when searching for best | |
281 | delta matches. This requires that pack-objects be compiled with | |
282 | pthreads otherwise this option is ignored with a warning. | |
283 | This is meant to reduce packing time on multiprocessor machines. | |
284 | The required amount of memory for the delta search window is | |
285 | however multiplied by the number of threads. | |
2de9b711 | 286 | Specifying 0 will cause Git to auto-detect the number of CPU's |
833e3df1 | 287 | and set the number of threads accordingly. |
367f4a43 | 288 | |
be18c1fe NP |
289 | --index-version=<version>[,<offset>]:: |
290 | This is intended to be used by the test suite only. It allows | |
291 | to force the version for the generated pack index, and to force | |
292 | 64-bit index entries on objects located above the given offset. | |
293 | ||
7f3140cd JS |
294 | --keep-true-parents:: |
295 | With this option, parents that are hidden by grafts are packed | |
296 | nevertheless. | |
297 | ||
9535ce73 | 298 | --filter=<filter-spec>:: |
6cfcabfb CC |
299 | Omits certain objects (usually blobs) from the resulting |
300 | packfile. See linkgit:git-rev-list[1] for valid | |
9535ce73 JH |
301 | `<filter-spec>` forms. |
302 | ||
4875c979 JH |
303 | --no-filter:: |
304 | Turns off any previous `--filter=` argument. | |
305 | ||
9535ce73 JH |
306 | --missing=<missing-action>:: |
307 | A debug option to help with future "partial clone" development. | |
308 | This option specifies how missing objects are handled. | |
309 | + | |
310 | The form '--missing=error' requests that pack-objects stop with an error if | |
ee47243d JT |
311 | a missing object is encountered. If the repository is a partial clone, an |
312 | attempt to fetch missing objects will be made before declaring them missing. | |
313 | This is the default action. | |
9535ce73 JH |
314 | + |
315 | The form '--missing=allow-any' will allow object traversal to continue | |
ee47243d JT |
316 | if a missing object is encountered. No fetch of a missing object will occur. |
317 | Missing objects will silently be omitted from the results. | |
0c16cd49 JT |
318 | + |
319 | The form '--missing=allow-promisor' is like 'allow-any', but will only | |
320 | allow object traversal to continue for EXPECTED promisor missing objects. | |
ee47243d JT |
321 | No fetch of a missing object will occur. An unexpected missing object will |
322 | raise an error. | |
0c16cd49 JT |
323 | |
324 | --exclude-promisor-objects:: | |
325 | Omit objects that are known to be in the promisor remote. (This | |
326 | option has the purpose of operating only on locally created objects, | |
327 | so that when we repack, we still maintain a distinction between | |
328 | locally created objects [without .promisor] and objects from the | |
329 | promisor remote [with .promisor].) This is used with partial clone. | |
9535ce73 | 330 | |
58bd77b6 NTND |
331 | --keep-unreachable:: |
332 | Objects unreachable from the refs in packs named with | |
333 | --unpacked= option are added to the resulting pack, in | |
334 | addition to the reachable objects that are not in packs marked | |
335 | with *.keep files. This implies `--revs`. | |
336 | ||
337 | --pack-loose-unreachable:: | |
338 | Pack unreachable loose objects (and their loose counterparts | |
339 | removed). This implies `--revs`. | |
340 | ||
341 | --unpack-unreachable:: | |
342 | Keep unreachable objects in loose form. This implies `--revs`. | |
343 | ||
28b8a730 JK |
344 | --delta-islands:: |
345 | Restrict delta matches based on "islands". See DELTA ISLANDS | |
346 | below. | |
347 | ||
348 | ||
349 | DELTA ISLANDS | |
350 | ------------- | |
351 | ||
352 | When possible, `pack-objects` tries to reuse existing on-disk deltas to | |
353 | avoid having to search for new ones on the fly. This is an important | |
354 | optimization for serving fetches, because it means the server can avoid | |
355 | inflating most objects at all and just send the bytes directly from | |
356 | disk. This optimization can't work when an object is stored as a delta | |
357 | against a base which the receiver does not have (and which we are not | |
358 | already sending). In that case the server "breaks" the delta and has to | |
359 | find a new one, which has a high CPU cost. Therefore it's important for | |
360 | performance that the set of objects in on-disk delta relationships match | |
361 | what a client would fetch. | |
362 | ||
363 | In a normal repository, this tends to work automatically. The objects | |
364 | are mostly reachable from the branches and tags, and that's what clients | |
365 | fetch. Any deltas we find on the server are likely to be between objects | |
366 | the client has or will have. | |
367 | ||
368 | But in some repository setups, you may have several related but separate | |
369 | groups of ref tips, with clients tending to fetch those groups | |
370 | independently. For example, imagine that you are hosting several "forks" | |
371 | of a repository in a single shared object store, and letting clients | |
372 | view them as separate repositories through `GIT_NAMESPACE` or separate | |
373 | repos using the alternates mechanism. A naive repack may find that the | |
374 | optimal delta for an object is against a base that is only found in | |
375 | another fork. But when a client fetches, they will not have the base | |
376 | object, and we'll have to find a new delta on the fly. | |
377 | ||
378 | A similar situation may exist if you have many refs outside of | |
379 | `refs/heads/` and `refs/tags/` that point to related objects (e.g., | |
380 | `refs/pull` or `refs/changes` used by some hosting providers). By | |
381 | default, clients fetch only heads and tags, and deltas against objects | |
382 | found only in those other groups cannot be sent as-is. | |
383 | ||
384 | Delta islands solve this problem by allowing you to group your refs into | |
385 | distinct "islands". Pack-objects computes which objects are reachable | |
386 | from which islands, and refuses to make a delta from an object `A` | |
387 | against a base which is not present in all of `A`'s islands. This | |
388 | results in slightly larger packs (because we miss some delta | |
389 | opportunities), but guarantees that a fetch of one island will not have | |
390 | to recompute deltas on the fly due to crossing island boundaries. | |
391 | ||
392 | When repacking with delta islands the delta window tends to get | |
393 | clogged with candidates that are forbidden by the config. Repacking | |
394 | with a big --window helps (and doesn't take as long as it otherwise | |
395 | might because we can reject some object pairs based on islands before | |
396 | doing any computation on the content). | |
397 | ||
398 | Islands are configured via the `pack.island` option, which can be | |
399 | specified multiple times. Each value is a left-anchored regular | |
400 | expressions matching refnames. For example: | |
401 | ||
402 | ------------------------------------------- | |
403 | [pack] | |
404 | island = refs/heads/ | |
405 | island = refs/tags/ | |
406 | ------------------------------------------- | |
407 | ||
408 | puts heads and tags into an island (whose name is the empty string; see | |
409 | below for more on naming). Any refs which do not match those regular | |
410 | expressions (e.g., `refs/pull/123`) is not in any island. Any object | |
411 | which is reachable only from `refs/pull/` (but not heads or tags) is | |
412 | therefore not a candidate to be used as a base for `refs/heads/`. | |
413 | ||
414 | Refs are grouped into islands based on their "names", and two regexes | |
415 | that produce the same name are considered to be in the same | |
416 | island. The names are computed from the regexes by concatenating any | |
417 | capture groups from the regex, with a '-' dash in between. (And if | |
418 | there are no capture groups, then the name is the empty string, as in | |
419 | the above example.) This allows you to create arbitrary numbers of | |
420 | islands. Only up to 14 such capture groups are supported though. | |
421 | ||
422 | For example, imagine you store the refs for each fork in | |
423 | `refs/virtual/ID`, where `ID` is a numeric identifier. You might then | |
424 | configure: | |
425 | ||
426 | ------------------------------------------- | |
427 | [pack] | |
428 | island = refs/virtual/([0-9]+)/heads/ | |
429 | island = refs/virtual/([0-9]+)/tags/ | |
430 | island = refs/virtual/([0-9]+)/(pull)/ | |
431 | ------------------------------------------- | |
432 | ||
433 | That puts the heads and tags for each fork in their own island (named | |
434 | "1234" or similar), and the pull refs for each go into their own | |
435 | "1234-pull". | |
436 | ||
437 | Note that we pick a single island for each regex to go into, using "last | |
438 | one wins" ordering (which allows repo-specific config to take precedence | |
439 | over user-wide config, and so forth). | |
440 | ||
3a837b58 CW |
441 | |
442 | CONFIGURATION | |
443 | ------------- | |
444 | ||
445 | Various configuration variables affect packing, see | |
446 | linkgit:git-config[1] (search for "pack" and "delta"). | |
447 | ||
448 | Notably, delta compression is not used on objects larger than the | |
449 | `core.bigFileThreshold` configuration variable and on files with the | |
450 | attribute `delta` set to false. | |
451 | ||
56ae8df5 | 452 | SEE ALSO |
e31bb3bb | 453 | -------- |
5162e697 DM |
454 | linkgit:git-rev-list[1] |
455 | linkgit:git-repack[1] | |
456 | linkgit:git-prune-packed[1] | |
e31bb3bb | 457 | |
5f40520f JH |
458 | GIT |
459 | --- | |
9e1f0a85 | 460 | Part of the linkgit:git[1] suite |