]>
Commit | Line | Data |
---|---|---|
94c0956b DS |
1 | git-sparse-checkout(1) |
2 | ====================== | |
3 | ||
4 | NAME | |
5 | ---- | |
9023535b | 6 | git-sparse-checkout - Reduce your working tree to a subset of tracked files |
94c0956b DS |
7 | |
8 | ||
9 | SYNOPSIS | |
10 | -------- | |
11 | [verse] | |
00408ade | 12 | 'git sparse-checkout' (init | list | set | add | reapply | disable | check-rules) [<options>] |
94c0956b DS |
13 | |
14 | ||
15 | DESCRIPTION | |
16 | ----------- | |
17 | ||
5d4b2933 EN |
18 | This command is used to create sparse checkouts, which change the |
19 | working tree from having all tracked files present to only having a | |
20 | subset of those files. It can also switch which subset of files are | |
21 | present, or undo and go back to having all tracked files present in | |
22 | the working copy. | |
9023535b EN |
23 | |
24 | The subset of files is chosen by providing a list of directories in | |
2d95707a EN |
25 | cone mode (the default), or by providing a list of patterns in |
26 | non-cone mode. | |
9023535b EN |
27 | |
28 | When in a sparse-checkout, other Git commands behave a bit differently. | |
29 | For example, switching branches will not update paths outside the | |
30 | sparse-checkout directories/patterns, and `git commit -a` will not record | |
31 | paths outside the sparse-checkout directories/patterns as deleted. | |
94c0956b DS |
32 | |
33 | THIS COMMAND IS EXPERIMENTAL. ITS BEHAVIOR, AND THE BEHAVIOR OF OTHER | |
34 | COMMANDS IN THE PRESENCE OF SPARSE-CHECKOUTS, WILL LIKELY CHANGE IN | |
35 | THE FUTURE. | |
36 | ||
37 | ||
38 | COMMANDS | |
39 | -------- | |
40 | 'list':: | |
9023535b | 41 | Describe the directories or patterns in the sparse-checkout file. |
94c0956b | 42 | |
ba2f3f58 | 43 | 'set':: |
7316dc5f DS |
44 | Enable the necessary sparse-checkout config settings |
45 | (`core.sparseCheckout`, `core.sparseCheckoutCone`, and | |
46 | `index.sparse`) if they are not already set to the desired values, | |
f69dfef3 EN |
47 | populate the sparse-checkout file from the list of arguments |
48 | following the 'set' subcommand, and update the working directory to | |
49 | match. | |
bab3c359 | 50 | + |
7316dc5f DS |
51 | To ensure that adjusting the sparse-checkout settings within a worktree |
52 | does not alter the sparse-checkout settings in other worktrees, the 'set' | |
53 | subcommand will upgrade your repository config to use worktree-specific | |
54 | config if not already present. The sparsity defined by the arguments to | |
55 | the 'set' subcommand are stored in the worktree-specific sparse-checkout | |
56 | file. See linkgit:git-worktree[1] and the documentation of | |
57 | `extensions.worktreeConfig` in linkgit:git-config[1] for more details. | |
58 | + | |
9023535b EN |
59 | When the `--stdin` option is provided, the directories or patterns are |
60 | read from standard in as a newline-delimited list instead of from the | |
61 | arguments. | |
a402723e | 62 | + |
f69dfef3 EN |
63 | By default, the input list is considered a list of directories, matching |
64 | the output of `git ls-tree -d --name-only`. This includes interpreting | |
65 | pathnames that begin with a double quote (") as C-style quoted strings. | |
66 | Note that all files under the specified directories (at any depth) will | |
67 | be included in the sparse checkout, as well as files that are siblings | |
68 | of either the given directory or any of its ancestors (see 'CONE PATTERN | |
69 | SET' below for more details). In the past, this was not the default, | |
70 | and `--cone` needed to be specified or `core.sparseCheckoutCone` needed | |
71 | to be enabled. | |
9023535b | 72 | + |
71ceb816 | 73 | When `--no-cone` is passed, the input list is considered a list of |
a8defed0 EN |
74 | patterns. This mode has a number of drawbacks, including not working |
75 | with some options like `--sparse-index`. As explained in the | |
76 | "Non-cone Problems" section below, we do not recommend using it. | |
122ba1f7 | 77 | + |
ba2f3f58 EN |
78 | Use the `--[no-]sparse-index` option to use a sparse index (the |
79 | default is to not use it). A sparse index reduces the size of the | |
80 | index to be more closely aligned with your sparse-checkout | |
81 | definition. This can have significant performance advantages for | |
82 | commands such as `git status` or `git add`. This feature is still | |
83 | experimental. Some commands might be slower with a sparse index until | |
84 | they are properly integrated with the feature. | |
122ba1f7 DS |
85 | + |
86 | **WARNING:** Using a sparse index requires modifying the index in a way | |
87 | that is not completely understood by external tools. If you have trouble | |
88 | with this compatibility, then run `git sparse-checkout init --no-sparse-index` | |
89 | to rewrite your index to not be sparse. Older versions of Git will not | |
90 | understand the sparse directory entries index extension and may fail to | |
91 | interact with your repository until it is disabled. | |
94c0956b | 92 | |
2631dc87 | 93 | 'add':: |
9023535b EN |
94 | Update the sparse-checkout file to include additional directories |
95 | (in cone mode) or patterns (in non-cone mode). By default, these | |
96 | directories or patterns are read from the command-line arguments, | |
97 | but they can be read from stdin using the `--stdin` option. | |
2631dc87 | 98 | |
e2643353 | 99 | 'reapply':: |
5644ca28 EN |
100 | Reapply the sparsity pattern rules to paths in the working tree. |
101 | Commands like merge or rebase can materialize paths to do their | |
102 | work (e.g. in order to show you a conflict), and other | |
103 | sparse-checkout commands might fail to sparsify an individual file | |
104 | (e.g. because it has unstaged changes or conflicts). In such | |
105 | cases, it can make sense to run `git sparse-checkout reapply` later | |
106 | after cleaning up affected paths (e.g. resolving conflicts, undoing | |
107 | or committing changes, etc.). | |
ba2f3f58 EN |
108 | + |
109 | The `reapply` command can also take `--[no-]cone` and `--[no-]sparse-index` | |
110 | flags, with the same meaning as the flags from the `set` command, in order | |
111 | to change which sparsity mode you are using without needing to also respecify | |
112 | all sparsity paths. | |
5644ca28 | 113 | |
72918c1a | 114 | 'disable':: |
99dfa6f9 | 115 | Disable the `core.sparseCheckout` config setting, and restore the |
ba2f3f58 EN |
116 | working directory to include all files. |
117 | ||
118 | 'init':: | |
119 | Deprecated command that behaves like `set` with no specified paths. | |
120 | May be removed in the future. | |
121 | + | |
122 | Historically, `set` did not handle all the necessary config settings, | |
123 | which meant that both `init` and `set` had to be called. Invoking | |
124 | both meant the `init` step would first remove nearly all tracked files | |
125 | (and in cone mode, ignored files too), then the `set` step would add | |
126 | many of the tracked files (but not ignored files) back. In addition | |
127 | to the lost files, the performance and UI of this combination was | |
128 | poor. | |
129 | + | |
130 | Also, historically, `init` would not actually initialize the | |
131 | sparse-checkout file if it already existed. This meant it was | |
132 | possible to return to a sparse-checkout without remembering which | |
133 | paths to pass to a subsequent 'set' or 'add' command. However, | |
134 | `--cone` and `--sparse-index` options would not be remembered across | |
135 | the disable command, so the easy restore of calling a plain `init` | |
136 | decreased in utility. | |
72918c1a | 137 | |
00408ade WS |
138 | 'check-rules':: |
139 | Check whether sparsity rules match one or more paths. | |
140 | + | |
141 | By default `check-rules` reads a list of paths from stdin and outputs only | |
142 | the ones that match the current sparsity rules. The input is expected to consist | |
143 | of one path per line, matching the output of `git ls-tree --name-only` including | |
144 | that pathnames that begin with a double quote (") are interpreted as C-style | |
145 | quoted strings. | |
146 | + | |
147 | When called with the `--rules-file <file>` flag the input files are matched | |
148 | against the sparse checkout rules found in `<file>` instead of the current ones. | |
149 | The rules in the files are expected to be in the same form as accepted by `git | |
150 | sparse-checkout set --stdin` (in particular, they must be newline-delimited). | |
151 | + | |
152 | By default, the rules passed to the `--rules-file` option are interpreted as | |
153 | cone mode directories. To pass non-cone mode patterns with `--rules-file`, | |
154 | combine the option with the `--no-cone` option. | |
155 | + | |
156 | When called with the `-z` flag, the format of the paths input on stdin as well | |
157 | as the output paths are \0 terminated and not quoted. Note that this does not | |
158 | apply to the format of the rules passed with the `--rules-file` option. | |
159 | ||
160 | ||
5d295dc3 EN |
161 | EXAMPLES |
162 | -------- | |
163 | `git sparse-checkout set MY/DIR1 SUB/DIR2`:: | |
164 | ||
165 | Change to a sparse checkout with all files (at any depth) under | |
166 | MY/DIR1/ and SUB/DIR2/ present in the working copy (plus all | |
167 | files immediately under MY/ and SUB/ and the toplevel | |
168 | directory). If already in a sparse checkout, change which files | |
169 | are present in the working copy to this new selection. Note | |
170 | that this command will also delete all ignored files in any | |
171 | directory that no longer has either tracked or | |
172 | non-ignored-untracked files present. | |
173 | ||
174 | `git sparse-checkout disable`:: | |
175 | ||
176 | Repopulate the working directory with all files, disabling sparse | |
177 | checkouts. | |
178 | ||
179 | `git sparse-checkout add SOME/DIR/ECTORY`:: | |
180 | ||
181 | Add all files under SOME/DIR/ECTORY/ (at any depth) to the | |
182 | sparse checkout, as well as all files immediately under | |
183 | SOME/DIR/ and immediately under SOME/. Must already be in a | |
184 | sparse checkout before using this command. | |
185 | ||
186 | `git sparse-checkout reapply`:: | |
187 | ||
188 | It is possible for commands to update the working tree in a | |
189 | way that does not respect the selected sparsity directories. | |
190 | This can come from tools external to Git writing files, or | |
191 | even affect Git commands because of either special cases (such | |
192 | as hitting conflicts when merging/rebasing), or because some | |
193 | commands didn't fully support sparse checkouts (e.g. the old | |
194 | `recursive` merge backend had only limited support). This | |
195 | command reapplies the existing sparse directory specifications | |
196 | to make the working directory match. | |
197 | ||
0d86f59a EN |
198 | INTERNALS -- SPARSE CHECKOUT |
199 | ---------------------------- | |
94c0956b | 200 | |
9023535b EN |
201 | "Sparse checkout" allows populating the working directory sparsely. It |
202 | uses the skip-worktree bit (see linkgit:git-update-index[1]) to tell Git | |
203 | whether a file in the working directory is worth looking at. If the | |
204 | skip-worktree bit is set, and the file is not present in the working tree, | |
205 | then its absence is ignored. Git will avoid populating the contents of | |
206 | those files, which makes a sparse checkout helpful when working in a | |
207 | repository with many files, but only a few are important to the current | |
208 | user. | |
94c0956b DS |
209 | |
210 | The `$GIT_DIR/info/sparse-checkout` file is used to define the | |
211 | skip-worktree reference bitmap. When Git updates the working | |
212 | directory, it updates the skip-worktree bits in the index based | |
213 | on this file. The files matching the patterns in the file will | |
214 | appear in the working directory, and the rest will not. | |
215 | ||
a8defed0 EN |
216 | INTERNALS -- NON-CONE PROBLEMS |
217 | ------------------------------ | |
218 | ||
219 | The `$GIT_DIR/info/sparse-checkout` file populated by the `set` and | |
220 | `add` subcommands is defined to be a bunch of patterns (one per line) | |
221 | using the same syntax as `.gitignore` files. In cone mode, these | |
222 | patterns are restricted to matching directories (and users only ever | |
223 | need supply or see directory names), while in non-cone mode any | |
224 | gitignore-style pattern is permitted. Using the full gitignore-style | |
225 | patterns in non-cone mode has a number of shortcomings: | |
226 | ||
227 | * Fundamentally, it makes various worktree-updating processes (pull, | |
228 | merge, rebase, switch, reset, checkout, etc.) require O(N*M) pattern | |
229 | matches, where N is the number of patterns and M is the number of | |
230 | paths in the index. This scales poorly. | |
231 | ||
232 | * Avoiding the scaling issue has to be done via limiting the number | |
233 | of patterns via specifying leading directory name or glob. | |
234 | ||
235 | * Passing globs on the command line is error-prone as users may | |
236 | forget to quote the glob, causing the shell to expand it into all | |
237 | matching files and pass them all individually along to | |
238 | sparse-checkout set/add. While this could also be a problem with | |
239 | e.g. "git grep -- *.c", mistakes with grep/log/status appear in | |
240 | the immediate output. With sparse-checkout, the mistake gets | |
241 | recorded at the time the sparse-checkout command is run and might | |
242 | not be problematic until the user later switches branches or rebases | |
243 | or merges, thus putting a delay between the user's error and when | |
244 | they have a chance to catch/notice it. | |
245 | ||
246 | * Related to the previous item, sparse-checkout has an 'add' | |
247 | subcommand but no 'remove' subcommand. Even if a 'remove' | |
248 | subcommand were added, undoing an accidental unquoted glob runs | |
249 | the risk of "removing too much", as it may remove entries that had | |
250 | been included before the accidental add. | |
251 | ||
252 | * Non-cone mode uses gitignore-style patterns to select what to | |
253 | *include* (with the exception of negated patterns), while | |
254 | .gitignore files use gitignore-style patterns to select what to | |
255 | *exclude* (with the exception of negated patterns). The | |
256 | documentation on gitignore-style patterns usually does not talk in | |
257 | terms of matching or non-matching, but on what the user wants to | |
258 | "exclude". This can cause confusion for users trying to learn how | |
259 | to specify sparse-checkout patterns to get their desired behavior. | |
260 | ||
261 | * Every other git subcommand that wants to provide "special path | |
262 | pattern matching" of some sort uses pathspecs, but non-cone mode | |
263 | for sparse-checkout uses gitignore patterns, which feels | |
264 | inconsistent. | |
265 | ||
266 | * It has edge cases where the "right" behavior is unclear. Two examples: | |
267 | ||
268 | First, two users are in a subdirectory, and the first runs | |
269 | git sparse-checkout set '/toplevel-dir/*.c' | |
270 | while the second runs | |
271 | git sparse-checkout set relative-dir | |
272 | Should those arguments be transliterated into | |
273 | current/subdirectory/toplevel-dir/*.c | |
274 | and | |
275 | current/subdirectory/relative-dir | |
276 | before inserting into the sparse-checkout file? The user who typed | |
277 | the first command is probably aware that arguments to set/add are | |
278 | supposed to be patterns in non-cone mode, and probably would not be | |
279 | happy with such a transliteration. However, many gitignore-style | |
280 | patterns are just paths, which might be what the user who typed the | |
281 | second command was thinking, and they'd be upset if their argument | |
282 | wasn't transliterated. | |
283 | ||
284 | Second, what should bash-completion complete on for set/add commands | |
285 | for non-cone users? If it suggests paths, is it exacerbating the | |
286 | problem above? Also, if it suggests paths, what if the user has a | |
287 | file or directory that begins with either a '!' or '#' or has a '*', | |
288 | '\', '?', '[', or ']' in its name? And if it suggests paths, will | |
548afb0d | 289 | it complete "/pro" to "/proc" (in the root filesystem) rather than to |
a8defed0 EN |
290 | "/progress.txt" in the current directory? (Note that users are |
291 | likely to want to start paths with a leading '/' in non-cone mode, | |
292 | for the same reason that .gitignore files often have one.) | |
293 | Completing on files or directories might give nasty surprises in | |
294 | all these cases. | |
295 | ||
296 | * The excessive flexibility made other extensions essentially | |
297 | impractical. `--sparse-index` is likely impossible in non-cone | |
298 | mode; even if it is somehow feasible, it would have been far more | |
299 | work to implement and may have been too slow in practice. Some | |
300 | ideas for adding coupling between partial clones and sparse | |
301 | checkouts are only practical with a more restricted set of paths | |
302 | as well. | |
303 | ||
304 | For all these reasons, non-cone mode is deprecated. Please switch to | |
305 | using cone mode. | |
306 | ||
879321eb | 307 | |
72fa58ef EN |
308 | INTERNALS -- CONE MODE HANDLING |
309 | ------------------------------- | |
310 | ||
311 | The "cone mode", which is the default, lets you specify only what | |
312 | directories to include. For any directory specified, all paths below | |
313 | that directory will be included, and any paths immediately under | |
314 | leading directories (including the toplevel directory) will also be | |
315 | included. Thus, if you specified the directory | |
316 | Documentation/technical/ | |
317 | then your sparse checkout would contain: | |
318 | ||
319 | * all files in the toplevel-directory | |
320 | * all files immediately under Documentation/ | |
321 | * all files at any depth under Documentation/technical/ | |
322 | ||
323 | Also, in cone mode, even if no directories are specified, then the | |
324 | files in the toplevel directory will be included. | |
325 | ||
326 | When changing the sparse-checkout patterns in cone mode, Git will inspect each | |
327 | tracked directory that is not within the sparse-checkout cone to see if it | |
328 | contains any untracked files. If all of those files are ignored due to the | |
329 | `.gitignore` patterns, then the directory will be deleted. If any of the | |
330 | untracked files within that directory is not ignored, then no deletions will | |
331 | occur within that directory and a warning message will appear. If these files | |
332 | are important, then reset your sparse-checkout definition so they are included, | |
333 | use `git add` and `git commit` to store them, then remove any remaining files | |
334 | manually to ensure Git can behave optimally. | |
335 | ||
336 | See also the "Internals -- Cone Pattern Set" section to learn how the | |
337 | directories are transformed under the hood into a subset of the | |
338 | Full Pattern Set of sparse-checkout. | |
339 | ||
340 | ||
341 | INTERNALS -- FULL PATTERN SET | |
0d86f59a | 342 | ----------------------------- |
879321eb DS |
343 | |
344 | The full pattern set allows for arbitrary pattern matches and complicated | |
345 | inclusion/exclusion rules. These can result in O(N*M) pattern matches when | |
346 | updating the index, where N is the number of patterns and M is the number | |
347 | of paths in the index. To combat this performance issue, a more restricted | |
7aa9ef2f | 348 | pattern set is allowed when `core.sparseCheckoutCone` is enabled. |
879321eb | 349 | |
72fa58ef EN |
350 | The sparse-checkout file uses the same syntax as `.gitignore` files; |
351 | see linkgit:gitignore[5] for details. Here, though, the patterns are | |
352 | usually being used to select which files to include rather than which | |
353 | files to exclude. (However, it can get a bit confusing since | |
354 | gitignore-style patterns have negations defined by patterns which | |
355 | begin with a '!', so you can also select files to _not_ include.) | |
356 | ||
357 | For example, to select everything, and then to remove the file | |
358 | `unwanted` (so that every file will appear in your working tree except | |
359 | the file named `unwanted`): | |
360 | ||
361 | git sparse-checkout set --no-cone '/*' '!unwanted' | |
362 | ||
363 | These patterns are just placed into the | |
364 | `$GIT_DIR/info/sparse-checkout` as-is, so the contents of that file | |
365 | at this point would be | |
366 | ||
367 | ---------------- | |
368 | /* | |
369 | !unwanted | |
370 | ---------------- | |
371 | ||
372 | See also the "Sparse Checkout" section of linkgit:git-read-tree[1] to | |
373 | learn more about the gitignore-style patterns used in sparse | |
374 | checkouts. | |
375 | ||
376 | ||
377 | INTERNALS -- CONE PATTERN SET | |
378 | ----------------------------- | |
379 | ||
380 | In cone mode, only directories are accepted, but they are translated into | |
381 | the same gitignore-style patterns used in the full pattern set. We refer | |
382 | to the particular patterns used in those mode as being of one of two types: | |
879321eb DS |
383 | |
384 | 1. *Recursive:* All paths inside a directory are included. | |
385 | ||
386 | 2. *Parent:* All files immediately inside a directory are included. | |
387 | ||
72fa58ef EN |
388 | Since cone mode always includes files at the toplevel, when running |
389 | `git sparse-checkout set` with no directories specified, the toplevel | |
390 | directory is added as a parent pattern. At this point, the | |
391 | sparse-checkout file contains the following patterns: | |
879321eb DS |
392 | |
393 | ---------------- | |
394 | /* | |
395 | !/*/ | |
396 | ---------------- | |
397 | ||
72fa58ef EN |
398 | This says "include everything immediately under the toplevel |
399 | directory, but nothing at any level below that." | |
d2e65f4c | 400 | |
72fa58ef EN |
401 | When in cone mode, the `git sparse-checkout set` subcommand takes a |
402 | list of directories. The command `git sparse-checkout set A/B/C` sets | |
403 | the directory `A/B/C` as a recursive pattern, the directories `A` and | |
404 | `A/B` are added as parent patterns. The resulting sparse-checkout file | |
405 | is now | |
879321eb DS |
406 | |
407 | ---------------- | |
408 | /* | |
409 | !/*/ | |
410 | /A/ | |
411 | !/A/*/ | |
412 | /A/B/ | |
413 | !/A/B/*/ | |
414 | /A/B/C/ | |
415 | ---------------- | |
416 | ||
417 | Here, order matters, so the negative patterns are overridden by the positive | |
418 | patterns that appear lower in the file. | |
419 | ||
2d95707a EN |
420 | Unless `core.sparseCheckoutCone` is explicitly set to `false`, Git will |
421 | parse the sparse-checkout file expecting patterns of these types. Git will | |
422 | warn if the patterns do not match. If the patterns do match the expected | |
423 | format, then Git will use faster hash-based algorithms to compute inclusion | |
72fa58ef EN |
424 | in the sparse-checkout. If they do not match, git will behave as though |
425 | `core.sparseCheckoutCone` was false, regardless of its setting. | |
879321eb | 426 | |
72fa58ef EN |
427 | In the cone mode case, despite the fact that full patterns are written |
428 | to the $GIT_DIR/info/sparse-checkout file, the `git sparse-checkout | |
429 | list` subcommand will list the directories that define the recursive | |
430 | patterns. For the example sparse-checkout file above, the output is as | |
431 | follows: | |
de11951b DS |
432 | |
433 | -------------------------- | |
434 | $ git sparse-checkout list | |
435 | A/B/C | |
436 | -------------------------- | |
437 | ||
190a65f9 DS |
438 | If `core.ignoreCase=true`, then the pattern-matching algorithm will use a |
439 | case-insensitive check. This corrects for case mismatched filenames in the | |
440 | 'git sparse-checkout set' command to reflect the expected cone in the working | |
441 | directory. | |
442 | ||
0d86f59a EN |
443 | |
444 | INTERNALS -- SUBMODULES | |
445 | ----------------------- | |
4fd683b6 | 446 | |
e7d7c732 EN |
447 | If your repository contains one or more submodules, then submodules |
448 | are populated based on interactions with the `git submodule` command. | |
449 | Specifically, `git submodule init -- <path>` will ensure the submodule | |
450 | at `<path>` is present, while `git submodule deinit [-f] -- <path>` | |
451 | will remove the files for the submodule at `<path>` (including any | |
452 | untracked files, uncommitted changes, and unpushed history). Similar | |
453 | to how sparse-checkout removes files from the working tree but still | |
454 | leaves entries in the index, deinitialized submodules are removed from | |
455 | the working directory but still have an entry in the index. | |
456 | ||
457 | Since submodules may have unpushed changes or untracked files, | |
458 | removing them could result in data loss. Thus, changing sparse | |
459 | inclusion/exclusion rules will not cause an already checked out | |
460 | submodule to be removed from the working copy. Said another way, just | |
461 | as `checkout` will not cause submodules to be automatically removed or | |
462 | initialized even when switching between branches that remove or add | |
463 | submodules, using `sparse-checkout` to reduce or expand the scope of | |
464 | "interesting" files will not cause submodules to be automatically | |
465 | deinitialized or initialized either. | |
466 | ||
467 | Further, the above facts mean that there are multiple reasons that | |
468 | "tracked" files might not be present in the working copy: sparsity | |
469 | pattern application from sparse-checkout, and submodule initialization | |
470 | state. Thus, commands like `git grep` that work on tracked files in | |
471 | the working copy may return results that are limited by either or both | |
472 | of these restrictions. | |
4fd683b6 DS |
473 | |
474 | ||
94c0956b DS |
475 | SEE ALSO |
476 | -------- | |
477 | ||
478 | linkgit:git-read-tree[1] | |
479 | linkgit:gitignore[5] | |
480 | ||
481 | GIT | |
482 | --- | |
483 | Part of the linkgit:git[1] suite |