]>
Commit | Line | Data |
---|---|---|
5f40520f JH |
1 | git-pack-objects(1) |
2 | =================== | |
5f40520f JH |
3 | |
4 | NAME | |
5 | ---- | |
7bd7f280 | 6 | git-pack-objects - Create a packed archive of objects |
5f40520f JH |
7 | |
8 | ||
9 | SYNOPSIS | |
10 | -------- | |
ca5381d4 | 11 | [verse] |
4f366275 NP |
12 | 'git pack-objects' [-q | --progress | --all-progress] [--all-progress-implied] |
13 | [--no-reuse-delta] [--delta-base-offset] [--non-empty] | |
62b4698e | 14 | [--local] [--incremental] [--window=<n>] [--depth=<n>] |
ed7e5fc3 | 15 | [--revs [--unpacked | --all]] [--keep-pack=<pack-name>] |
9535ce73 | 16 | [--stdout [--filter=<filter-spec>] | base-name] |
4f6d26b1 | 17 | [--shallow] [--keep-true-parents] [--sparse] < object-list |
5f40520f JH |
18 | |
19 | ||
20 | DESCRIPTION | |
21 | ----------- | |
4a4becfb JT |
22 | Reads list of objects from the standard input, and writes either one or |
23 | more packed archives with the specified base-name to disk, or a packed | |
24 | archive to the standard output. | |
5f40520f | 25 | |
738820a9 SB |
26 | A packed archive is an efficient way to transfer a set of objects |
27 | between two repositories as well as an access efficient archival | |
28 | format. In a packed archive, an object is either stored as a | |
29 | compressed whole or as a difference from some other object. | |
30 | The latter is often called a delta. | |
31 | ||
32 | The packed archive format (.pack) is designed to be self-contained | |
33 | so that it can be unpacked without any further information. Therefore, | |
34 | each object that a delta depends upon must be present within the pack. | |
35 | ||
36 | A pack index file (.idx) is generated for fast, random access to the | |
37 | objects in the pack. Placing both the index file (.idx) and the packed | |
38 | archive (.pack) in the pack/ subdirectory of $GIT_OBJECT_DIRECTORY (or | |
1e6ab5de | 39 | any of the directories on $GIT_ALTERNATE_OBJECT_DIRECTORIES) |
2de9b711 | 40 | enables Git to read from the pack archive. |
1e6ab5de | 41 | |
0b444cdb | 42 | The 'git unpack-objects' command can read the packed archive and |
5f40520f JH |
43 | expand the objects contained in the pack into "one-file |
44 | one-object" format; this is typically done by the smart-pull | |
45 | commands when a pack is created on-the-fly for efficient network | |
46 | transport by their peers. | |
47 | ||
5f40520f JH |
48 | |
49 | OPTIONS | |
50 | ------- | |
51 | base-name:: | |
4a4becfb | 52 | Write into pairs of files (.pack and .idx), using |
5f40520f | 53 | <base-name> to determine the name of the created file. |
4a4becfb | 54 | When this option is used, the two files in a pair are written in |
d5fa1f1a | 55 | <base-name>-<SHA-1>.{pack,idx} files. <SHA-1> is a hash |
40a4f5a7 | 56 | based on the pack content and is written to the standard |
5f40520f JH |
57 | output of the command. |
58 | ||
59 | --stdout:: | |
89438677 | 60 | Write the pack contents (what would have been written to |
5f40520f JH |
61 | .pack file) out to the standard output. |
62 | ||
4321134c JH |
63 | --revs:: |
64 | Read the revision arguments from the standard input, instead of | |
65 | individual object names. The revision arguments are processed | |
0b444cdb | 66 | the same way as 'git rev-list' with the `--objects` flag |
4321134c JH |
67 | uses its `commit` arguments to build the list of objects it |
68 | outputs. The objects on the resulting list are packed. | |
b790e0f6 NTND |
69 | Besides revisions, `--not` or `--shallow <SHA-1>` lines are |
70 | also accepted. | |
4321134c JH |
71 | |
72 | --unpacked:: | |
73 | This implies `--revs`. When processing the list of | |
74 | revision arguments read from the standard input, limit | |
75 | the objects packed to those that are not already packed. | |
76 | ||
77 | --all:: | |
78 | This implies `--revs`. In addition to the list of | |
79 | revision arguments read from the standard input, pretend | |
cc1b8d8b | 80 | as if all refs under `refs/` are specified to be |
4321134c JH |
81 | included. |
82 | ||
f0a24aa5 SP |
83 | --include-tag:: |
84 | Include unasked-for annotated tags if the object they | |
85 | reference was included in the resulting packfile. This | |
2de9b711 | 86 | can be useful to send new tags to native Git clients. |
f0a24aa5 | 87 | |
62b4698e ŠN |
88 | --window=<n>:: |
89 | --depth=<n>:: | |
3df19671 | 90 | These two options affect how the objects contained in |
5f40520f JH |
91 | the pack are stored using delta compression. The |
92 | objects are first internally sorted by type, size and | |
93 | optionally names and compared against the other objects | |
94 | within --window to see if using delta compression saves | |
95 | space. --depth limits the maximum delta depth; making | |
96 | it too deep affects the performance on the unpacker | |
97 | side, because delta data needs to be applied that many | |
98 | times to get to the necessary object. | |
b5c0cbd8 NTND |
99 | + |
100 | The default value for --window is 10 and --depth is 50. The maximum | |
101 | depth is 4095. | |
5f40520f | 102 | |
62b4698e | 103 | --window-memory=<n>:: |
e93b15cd BD |
104 | This option provides an additional limit on top of `--window`; |
105 | the window size will dynamically scale down so as to not take | |
62b4698e | 106 | up more than '<n>' bytes in memory. This is useful in |
e93b15cd BD |
107 | repositories with a mix of large and small objects to not run |
108 | out of memory with a large window, but still be able to take | |
109 | advantage of the large window for the smaller objects. The | |
110 | size can be suffixed with "k", "m", or "g". | |
954176c1 MS |
111 | `--window-memory=0` makes memory usage unlimited. The default |
112 | is taken from the `pack.windowMemory` configuration variable. | |
e93b15cd | 113 | |
62b4698e | 114 | --max-pack-size=<n>:: |
4a4becfb JT |
115 | In unusual scenarios, you may not be able to create files |
116 | larger than a certain size on your filesystem, and this option | |
117 | can be used to tell the command to split the output packfile | |
118 | into multiple independent packfiles, each not larger than the | |
119 | given size. The size can be suffixed with | |
07cf0f24 | 120 | "k", "m", or "g". The minimum size allowed is limited to 1 MiB. |
4a4becfb | 121 | This option |
9cea46cd | 122 | prevents the creation of a bitmap index. |
2b84b5a8 JS |
123 | The default is unlimited, unless the config variable |
124 | `pack.packSizeLimit` is set. | |
6b94b1a0 | 125 | |
e96fb9b8 BC |
126 | --honor-pack-keep:: |
127 | This flag causes an object already in a local pack that | |
0353a0c4 | 128 | has a .keep file to be ignored, even if it would have |
18879bc5 | 129 | otherwise been packed. |
e96fb9b8 | 130 | |
ed7e5fc3 NTND |
131 | --keep-pack=<pack-name>:: |
132 | This flag causes an object already in the given pack to be | |
133 | ignored, even if it would have otherwise been | |
24966cd9 | 134 | packed. `<pack-name>` is the pack file name without |
ed7e5fc3 NTND |
135 | leading directory (e.g. `pack-123.pack`). The option could be |
136 | specified multiple times to keep multiple packs. | |
137 | ||
5f40520f | 138 | --incremental:: |
21da4262 | 139 | This flag causes an object already in a pack to be ignored |
3909f14f | 140 | even if it would have otherwise been packed. |
5f40520f | 141 | |
12ea5bea | 142 | --local:: |
21da4262 | 143 | This flag causes an object that is borrowed from an alternate |
3909f14f JH |
144 | object store to be ignored even if it would have otherwise been |
145 | packed. | |
5f40520f | 146 | |
63ae26f8 NW |
147 | --non-empty:: |
148 | Only create a packed archive if it would contain at | |
149 | least one object. | |
150 | ||
231f240b NP |
151 | --progress:: |
152 | Progress status is reported on the standard error stream | |
153 | by default when it is attached to a terminal, unless -q | |
154 | is specified. This flag forces progress status even if | |
155 | the standard error stream is not directed to a terminal. | |
156 | ||
157 | --all-progress:: | |
158 | When --stdout is specified then progress report is | |
4f366275 | 159 | displayed during the object count and compression phases |
231f240b NP |
160 | but inhibited during the write-out phase. The reason is |
161 | that in some cases the output stream is directly linked | |
162 | to another command which may wish to display progress | |
163 | status of its own as it processes incoming pack data. | |
164 | This flag is like --progress except that it forces progress | |
165 | report for the write-out phase as well even if --stdout is | |
166 | used. | |
167 | ||
4f366275 NP |
168 | --all-progress-implied:: |
169 | This is used to imply --all-progress whenever progress display | |
170 | is activated. Unlike --all-progress this flag doesn't actually | |
171 | force any progress display by itself. | |
172 | ||
ca5381d4 JH |
173 | -q:: |
174 | This flag makes the command not to report its progress | |
175 | on the standard error stream. | |
176 | ||
177 | --no-reuse-delta:: | |
178 | When creating a packed archive in a repository that | |
179 | has existing packs, the command reuses existing deltas. | |
180 | This sometimes results in a slightly suboptimal pack. | |
181 | This flag tells the command not to reuse existing deltas | |
182 | but compute them from scratch. | |
183 | ||
fa736f72 NP |
184 | --no-reuse-object:: |
185 | This flag tells the command not to reuse existing object data at all, | |
186 | including non deltified object, forcing recompression of everything. | |
960ccca6 | 187 | This implies --no-reuse-delta. Useful only in the obscure case where |
fa736f72 NP |
188 | wholesale enforcement of a different compression level on the |
189 | packed data is desired. | |
190 | ||
62b4698e | 191 | --compression=<n>:: |
960ccca6 DH |
192 | Specifies compression level for newly-compressed data in the |
193 | generated pack. If not specified, pack compression level is | |
194 | determined first by pack.compression, then by core.compression, | |
195 | and defaults to -1, the zlib default, if neither is set. | |
483bc4f0 | 196 | Add --no-reuse-object if you want to force a uniform compression |
05cc2ffc | 197 | level on all data no matter the source. |
960ccca6 | 198 | |
4f6d26b1 DS |
199 | --sparse:: |
200 | Use the "sparse" algorithm to determine which objects to include in | |
201 | the pack, when combined with the "--revs" option. This algorithm | |
202 | only walks trees that appear in paths that introduce new objects. | |
203 | This can have significant performance benefits when computing | |
204 | a pack to send a small change. However, it is possible that extra | |
205 | objects are added to the pack-file if the included commits contain | |
206 | certain types of direct renames. | |
207 | ||
738820a9 SB |
208 | --thin:: |
209 | Create a "thin" pack by omitting the common objects between a | |
210 | sender and a receiver in order to reduce network transfer. This | |
211 | option only makes sense in conjunction with --stdout. | |
212 | + | |
213 | Note: A thin pack violates the packed archive format by omitting | |
2de9b711 | 214 | required objects and is thus unusable by Git without making it |
738820a9 SB |
215 | self-contained. Use `git index-pack --fix-thin` |
216 | (see linkgit:git-index-pack[1]) to restore the self-contained property. | |
217 | ||
2dacf26d | 218 | --shallow:: |
219 | Optimize a pack that will be provided to a client with a shallow | |
1c262bb7 | 220 | repository. This option, combined with --thin, can result in a |
2dacf26d | 221 | smaller pack at the cost of speed. |
222 | ||
63fba759 | 223 | --delta-base-offset:: |
2f8ee02c SB |
224 | A packed archive can express the base object of a delta as |
225 | either a 20-byte object name or as an offset in the | |
2de9b711 | 226 | stream, but ancient versions of Git don't understand the |
0b444cdb | 227 | latter. By default, 'git pack-objects' only uses the |
63fba759 JH |
228 | former format for better compatibility. This option |
229 | allows the command to use the latter format for | |
230 | compactness. Depending on the average delta chain | |
231 | length, this option typically shrinks the resulting | |
232 | packfile by 3-5 per-cent. | |
c14f3727 JH |
233 | + |
234 | Note: Porcelain commands such as `git gc` (see linkgit:git-gc[1]), | |
235 | `git repack` (see linkgit:git-repack[1]) pass this option by default | |
2de9b711 | 236 | in modern Git when they put objects in your repository into pack files. |
c14f3727 | 237 | So does `git bundle` (see linkgit:git-bundle[1]) when it creates a bundle. |
63fba759 | 238 | |
367f4a43 NP |
239 | --threads=<n>:: |
240 | Specifies the number of threads to spawn when searching for best | |
241 | delta matches. This requires that pack-objects be compiled with | |
242 | pthreads otherwise this option is ignored with a warning. | |
243 | This is meant to reduce packing time on multiprocessor machines. | |
244 | The required amount of memory for the delta search window is | |
245 | however multiplied by the number of threads. | |
2de9b711 | 246 | Specifying 0 will cause Git to auto-detect the number of CPU's |
833e3df1 | 247 | and set the number of threads accordingly. |
367f4a43 | 248 | |
be18c1fe NP |
249 | --index-version=<version>[,<offset>]:: |
250 | This is intended to be used by the test suite only. It allows | |
251 | to force the version for the generated pack index, and to force | |
252 | 64-bit index entries on objects located above the given offset. | |
253 | ||
7f3140cd JS |
254 | --keep-true-parents:: |
255 | With this option, parents that are hidden by grafts are packed | |
256 | nevertheless. | |
257 | ||
9535ce73 JH |
258 | --filter=<filter-spec>:: |
259 | Requires `--stdout`. Omits certain objects (usually blobs) from | |
260 | the resulting packfile. See linkgit:git-rev-list[1] for valid | |
261 | `<filter-spec>` forms. | |
262 | ||
4875c979 JH |
263 | --no-filter:: |
264 | Turns off any previous `--filter=` argument. | |
265 | ||
9535ce73 JH |
266 | --missing=<missing-action>:: |
267 | A debug option to help with future "partial clone" development. | |
268 | This option specifies how missing objects are handled. | |
269 | + | |
270 | The form '--missing=error' requests that pack-objects stop with an error if | |
271 | a missing object is encountered. This is the default action. | |
272 | + | |
273 | The form '--missing=allow-any' will allow object traversal to continue | |
274 | if a missing object is encountered. Missing objects will silently be | |
275 | omitted from the results. | |
0c16cd49 JT |
276 | + |
277 | The form '--missing=allow-promisor' is like 'allow-any', but will only | |
278 | allow object traversal to continue for EXPECTED promisor missing objects. | |
279 | Unexpected missing object will raise an error. | |
280 | ||
281 | --exclude-promisor-objects:: | |
282 | Omit objects that are known to be in the promisor remote. (This | |
283 | option has the purpose of operating only on locally created objects, | |
284 | so that when we repack, we still maintain a distinction between | |
285 | locally created objects [without .promisor] and objects from the | |
286 | promisor remote [with .promisor].) This is used with partial clone. | |
9535ce73 | 287 | |
58bd77b6 NTND |
288 | --keep-unreachable:: |
289 | Objects unreachable from the refs in packs named with | |
290 | --unpacked= option are added to the resulting pack, in | |
291 | addition to the reachable objects that are not in packs marked | |
292 | with *.keep files. This implies `--revs`. | |
293 | ||
294 | --pack-loose-unreachable:: | |
295 | Pack unreachable loose objects (and their loose counterparts | |
296 | removed). This implies `--revs`. | |
297 | ||
298 | --unpack-unreachable:: | |
299 | Keep unreachable objects in loose form. This implies `--revs`. | |
300 | ||
28b8a730 JK |
301 | --delta-islands:: |
302 | Restrict delta matches based on "islands". See DELTA ISLANDS | |
303 | below. | |
304 | ||
305 | ||
306 | DELTA ISLANDS | |
307 | ------------- | |
308 | ||
309 | When possible, `pack-objects` tries to reuse existing on-disk deltas to | |
310 | avoid having to search for new ones on the fly. This is an important | |
311 | optimization for serving fetches, because it means the server can avoid | |
312 | inflating most objects at all and just send the bytes directly from | |
313 | disk. This optimization can't work when an object is stored as a delta | |
314 | against a base which the receiver does not have (and which we are not | |
315 | already sending). In that case the server "breaks" the delta and has to | |
316 | find a new one, which has a high CPU cost. Therefore it's important for | |
317 | performance that the set of objects in on-disk delta relationships match | |
318 | what a client would fetch. | |
319 | ||
320 | In a normal repository, this tends to work automatically. The objects | |
321 | are mostly reachable from the branches and tags, and that's what clients | |
322 | fetch. Any deltas we find on the server are likely to be between objects | |
323 | the client has or will have. | |
324 | ||
325 | But in some repository setups, you may have several related but separate | |
326 | groups of ref tips, with clients tending to fetch those groups | |
327 | independently. For example, imagine that you are hosting several "forks" | |
328 | of a repository in a single shared object store, and letting clients | |
329 | view them as separate repositories through `GIT_NAMESPACE` or separate | |
330 | repos using the alternates mechanism. A naive repack may find that the | |
331 | optimal delta for an object is against a base that is only found in | |
332 | another fork. But when a client fetches, they will not have the base | |
333 | object, and we'll have to find a new delta on the fly. | |
334 | ||
335 | A similar situation may exist if you have many refs outside of | |
336 | `refs/heads/` and `refs/tags/` that point to related objects (e.g., | |
337 | `refs/pull` or `refs/changes` used by some hosting providers). By | |
338 | default, clients fetch only heads and tags, and deltas against objects | |
339 | found only in those other groups cannot be sent as-is. | |
340 | ||
341 | Delta islands solve this problem by allowing you to group your refs into | |
342 | distinct "islands". Pack-objects computes which objects are reachable | |
343 | from which islands, and refuses to make a delta from an object `A` | |
344 | against a base which is not present in all of `A`'s islands. This | |
345 | results in slightly larger packs (because we miss some delta | |
346 | opportunities), but guarantees that a fetch of one island will not have | |
347 | to recompute deltas on the fly due to crossing island boundaries. | |
348 | ||
349 | When repacking with delta islands the delta window tends to get | |
350 | clogged with candidates that are forbidden by the config. Repacking | |
351 | with a big --window helps (and doesn't take as long as it otherwise | |
352 | might because we can reject some object pairs based on islands before | |
353 | doing any computation on the content). | |
354 | ||
355 | Islands are configured via the `pack.island` option, which can be | |
356 | specified multiple times. Each value is a left-anchored regular | |
357 | expressions matching refnames. For example: | |
358 | ||
359 | ------------------------------------------- | |
360 | [pack] | |
361 | island = refs/heads/ | |
362 | island = refs/tags/ | |
363 | ------------------------------------------- | |
364 | ||
365 | puts heads and tags into an island (whose name is the empty string; see | |
366 | below for more on naming). Any refs which do not match those regular | |
367 | expressions (e.g., `refs/pull/123`) is not in any island. Any object | |
368 | which is reachable only from `refs/pull/` (but not heads or tags) is | |
369 | therefore not a candidate to be used as a base for `refs/heads/`. | |
370 | ||
371 | Refs are grouped into islands based on their "names", and two regexes | |
372 | that produce the same name are considered to be in the same | |
373 | island. The names are computed from the regexes by concatenating any | |
374 | capture groups from the regex, with a '-' dash in between. (And if | |
375 | there are no capture groups, then the name is the empty string, as in | |
376 | the above example.) This allows you to create arbitrary numbers of | |
377 | islands. Only up to 14 such capture groups are supported though. | |
378 | ||
379 | For example, imagine you store the refs for each fork in | |
380 | `refs/virtual/ID`, where `ID` is a numeric identifier. You might then | |
381 | configure: | |
382 | ||
383 | ------------------------------------------- | |
384 | [pack] | |
385 | island = refs/virtual/([0-9]+)/heads/ | |
386 | island = refs/virtual/([0-9]+)/tags/ | |
387 | island = refs/virtual/([0-9]+)/(pull)/ | |
388 | ------------------------------------------- | |
389 | ||
390 | That puts the heads and tags for each fork in their own island (named | |
391 | "1234" or similar), and the pull refs for each go into their own | |
392 | "1234-pull". | |
393 | ||
394 | Note that we pick a single island for each regex to go into, using "last | |
395 | one wins" ordering (which allows repo-specific config to take precedence | |
396 | over user-wide config, and so forth). | |
397 | ||
56ae8df5 | 398 | SEE ALSO |
e31bb3bb | 399 | -------- |
5162e697 DM |
400 | linkgit:git-rev-list[1] |
401 | linkgit:git-repack[1] | |
402 | linkgit:git-prune-packed[1] | |
e31bb3bb | 403 | |
5f40520f JH |
404 | GIT |
405 | --- | |
9e1f0a85 | 406 | Part of the linkgit:git[1] suite |