]>
Commit | Line | Data |
---|---|---|
2057d750 DS |
1 | git-maintenance(1) |
2 | ================== | |
3 | ||
4 | NAME | |
5 | ---- | |
6 | git-maintenance - Run tasks to optimize Git repository data | |
7 | ||
8 | ||
9 | SYNOPSIS | |
10 | -------- | |
11 | [verse] | |
12 | 'git maintenance' run [<options>] | |
f4976ef7 | 13 | 'git maintenance' start [--scheduler=<scheduler>] |
1ebe6b02 | 14 | 'git maintenance' (stop|register|unregister) [<options>] |
2057d750 DS |
15 | |
16 | ||
17 | DESCRIPTION | |
18 | ----------- | |
19 | Run tasks to optimize Git repository data, speeding up other Git commands | |
20 | and reducing storage requirements for the repository. | |
21 | ||
22 | Git commands that add repository data, such as `git add` or `git fetch`, | |
23 | are optimized for a responsive user experience. These commands do not take | |
24 | time to optimize the Git data, since such optimizations scale with the full | |
25 | size of the repository while these user commands each perform a relatively | |
26 | small action. | |
27 | ||
28 | The `git maintenance` command provides flexibility for how to optimize the | |
29 | Git repository. | |
30 | ||
31 | SUBCOMMANDS | |
32 | ----------- | |
33 | ||
f4976ef7 DS |
34 | run:: |
35 | Run one or more maintenance tasks. If one or more `--task` options | |
36 | are specified, then those tasks are run in that order. Otherwise, | |
37 | the tasks are determined by which `maintenance.<task>.enabled` | |
38 | config options are true. By default, only `maintenance.gc.enabled` | |
39 | is true. | |
40 | ||
41 | start:: | |
42 | Start running maintenance on the current repository. This performs | |
43 | the same config updates as the `register` subcommand, then updates | |
44 | the background scheduler to run `git maintenance run --scheduled` | |
45 | on an hourly basis. | |
46 | ||
47 | stop:: | |
48 | Halt the background maintenance schedule. The current repository | |
49 | is not removed from the list of maintained repositories, in case | |
50 | the background maintenance is restarted later. | |
51 | ||
0c18b700 | 52 | register:: |
1f80129d RP |
53 | Initialize Git config values so any scheduled maintenance will start |
54 | running on this repository. This adds the repository to the | |
55 | `maintenance.repo` config variable in the current user's global config, | |
56 | or the config specified by --config-file option, and enables some | |
57 | recommended configuration values for `maintenance.<task>.schedule`. The | |
58 | tasks that are enabled are safe for running in the background without | |
59 | disrupting foreground processes. | |
61f7a383 | 60 | + |
7efc3782 | 61 | The `register` subcommand will also set the `maintenance.strategy` config |
61f7a383 DS |
62 | value to `incremental`, if this value is not previously set. The |
63 | `incremental` strategy uses the following schedule for each maintenance | |
64 | task: | |
65 | + | |
66 | -- | |
67 | * `gc`: disabled. | |
68 | * `commit-graph`: hourly. | |
69 | * `prefetch`: hourly. | |
70 | * `loose-objects`: daily. | |
71 | * `incremental-repack`: daily. | |
72 | -- | |
73 | + | |
74 | `git maintenance register` will also disable foreground maintenance by | |
75 | setting `maintenance.auto = false` in the current repository. This config | |
76 | setting will remain after a `git maintenance unregister` command. | |
0c18b700 | 77 | |
0c18b700 DS |
78 | unregister:: |
79 | Remove the current repository from background maintenance. This | |
80 | only removes the repository from the configured list. It does not | |
81 | stop the background maintenance processes from running. | |
1ebe6b02 DS |
82 | + |
83 | The `unregister` subcommand will report an error if the current repository | |
84 | is not already registered. Use the `--force` option to return success even | |
85 | when the current repository is not registered. | |
0c18b700 | 86 | |
2057d750 DS |
87 | TASKS |
88 | ----- | |
89 | ||
663b2b1b DS |
90 | commit-graph:: |
91 | The `commit-graph` job updates the `commit-graph` files incrementally, | |
92 | then verifies that the written data is correct. The incremental | |
93 | write is safe to run alongside concurrent Git processes since it | |
94 | will not expire `.graph` files that were in the previous | |
95 | `commit-graph-chain` file. They will be deleted by a later run based | |
96 | on the expiration delay. | |
97 | ||
28cb5e66 DS |
98 | prefetch:: |
99 | The `prefetch` task updates the object directory with the latest | |
100 | objects from all registered remotes. For each remote, a `git fetch` | |
cfd781ea DS |
101 | command is run. The configured refspec is modified to place all |
102 | requested refs within `refs/prefetch/`. Also, tags are not updated. | |
28cb5e66 DS |
103 | + |
104 | This is done to avoid disrupting the remote-tracking branches. The end users | |
cf6cac20 EN |
105 | expect these refs to stay unmoved unless they initiate a fetch. However, |
106 | with the prefetch task, the objects necessary to complete a later real fetch | |
107 | would already be obtained, making the real fetch faster. In the ideal case, | |
83fcadd6 | 108 | it will just become an update to a bunch of remote-tracking branches without |
28cb5e66 DS |
109 | any object transfer. |
110 | ||
2057d750 DS |
111 | gc:: |
112 | Clean up unnecessary files and optimize the local repository. "GC" | |
113 | stands for "garbage collection," but this task performs many | |
114 | smaller tasks. This task can be expensive for large repositories, | |
115 | as it repacks all Git objects into a single pack-file. It can also | |
116 | be disruptive in some situations, as it deletes stale data. See | |
117 | linkgit:git-gc[1] for more details on garbage collection in Git. | |
118 | ||
252cfb7c DS |
119 | loose-objects:: |
120 | The `loose-objects` job cleans up loose objects and places them into | |
121 | pack-files. In order to prevent race conditions with concurrent Git | |
122 | commands, it follows a two-step process. First, it deletes any loose | |
123 | objects that already exist in a pack-file; concurrent Git processes | |
124 | will examine the pack-file for the object data instead of the loose | |
125 | object. Second, it creates a new pack-file (starting with "loose-") | |
126 | containing a batch of loose objects. The batch size is limited to 50 | |
127 | thousand objects to prevent the job from taking too long on a | |
128 | repository with many loose objects. The `gc` task writes unreachable | |
129 | objects as loose objects to be cleaned up by a later step only if | |
130 | they are not re-added to a pack-file; for this reason it is not | |
131 | advisable to enable both the `loose-objects` and `gc` tasks at the | |
132 | same time. | |
133 | ||
52fe41ff DS |
134 | incremental-repack:: |
135 | The `incremental-repack` job repacks the object directory | |
136 | using the `multi-pack-index` feature. In order to prevent race | |
137 | conditions with concurrent Git commands, it follows a two-step | |
138 | process. First, it calls `git multi-pack-index expire` to delete | |
139 | pack-files unreferenced by the `multi-pack-index` file. Second, it | |
140 | calls `git multi-pack-index repack` to select several small | |
141 | pack-files and repack them into a bigger one, and then update the | |
142 | `multi-pack-index` entries that refer to the small pack-files to | |
143 | refer to the new pack-file. This prepares those small pack-files | |
144 | for deletion upon the next run of `git multi-pack-index expire`. | |
145 | The selection of the small pack-files is such that the expected | |
146 | size of the big pack-file is at least the batch size; see the | |
147 | `--batch-size` option for the `repack` subcommand in | |
148 | linkgit:git-multi-pack-index[1]. The default batch-size is zero, | |
149 | which is a special case that attempts to repack all pack-files | |
150 | into a single pack-file. | |
151 | ||
41abfe15 DS |
152 | pack-refs:: |
153 | The `pack-refs` task collects the loose reference files and | |
154 | collects them into a single file. This speeds up operations that | |
155 | need to iterate across many references. See linkgit:git-pack-refs[1] | |
156 | for more information. | |
157 | ||
2057d750 DS |
158 | OPTIONS |
159 | ------- | |
160 | --auto:: | |
161 | When combined with the `run` subcommand, run maintenance tasks | |
162 | only if certain thresholds are met. For example, the `gc` task | |
163 | runs when the number of loose objects exceeds the number stored | |
164 | in the `gc.auto` config setting, or when the number of pack-files | |
b08ff1fe DS |
165 | exceeds the `gc.autoPackLimit` config setting. Not compatible with |
166 | the `--schedule` option. | |
167 | ||
168 | --schedule:: | |
169 | When combined with the `run` subcommand, run maintenance tasks | |
170 | only if certain time conditions are met, as specified by the | |
171 | `maintenance.<task>.schedule` config value for each `<task>`. | |
172 | This config value specifies a number of seconds since the last | |
173 | time that task ran, according to the `maintenance.<task>.lastRun` | |
174 | config value. The tasks that are tested are those provided by | |
175 | the `--task=<task>` option(s) or those with | |
176 | `maintenance.<task>.enabled` set to true. | |
2057d750 | 177 | |
3ddaad0e DS |
178 | --quiet:: |
179 | Do not report progress or other information over `stderr`. | |
180 | ||
090511bc DS |
181 | --task=<task>:: |
182 | If this option is specified one or more times, then only run the | |
65d655b5 DS |
183 | specified tasks in the specified order. If no `--task=<task>` |
184 | arguments are specified, then only the tasks with | |
185 | `maintenance.<task>.enabled` configured as `true` are considered. | |
186 | See the 'TASKS' section for the list of accepted `<task>` values. | |
090511bc | 187 | |
b681b191 | 188 | --scheduler=auto|crontab|systemd-timer|launchctl|schtasks:: |
eba1ba9d LH |
189 | When combined with the `start` subcommand, specify the scheduler |
190 | for running the hourly, daily and weekly executions of | |
191 | `git maintenance run`. | |
b681b191 LH |
192 | Possible values for `<scheduler>` are `auto`, `crontab` |
193 | (POSIX), `systemd-timer` (Linux), `launchctl` (macOS), and | |
194 | `schtasks` (Windows). When `auto` is specified, the | |
195 | appropriate platform-specific scheduler is used; on Linux, | |
196 | `systemd-timer` is used if available, otherwise | |
197 | `crontab`. Default is `auto`. | |
eba1ba9d | 198 | |
0016b618 DS |
199 | |
200 | TROUBLESHOOTING | |
201 | --------------- | |
202 | The `git maintenance` command is designed to simplify the repository | |
203 | maintenance patterns while minimizing user wait time during Git commands. | |
204 | A variety of configuration options are available to allow customizing this | |
205 | process. The default maintenance options focus on operations that complete | |
206 | quickly, even on large repositories. | |
207 | ||
208 | Users may find some cases where scheduled maintenance tasks do not run as | |
209 | frequently as intended. Each `git maintenance run` command takes a lock on | |
210 | the repository's object database, and this prevents other concurrent | |
211 | `git maintenance run` commands from running on the same repository. Without | |
212 | this safeguard, competing processes could leave the repository in an | |
213 | unpredictable state. | |
214 | ||
215 | The background maintenance schedule runs `git maintenance run` processes | |
216 | on an hourly basis. Each run executes the "hourly" tasks. At midnight, | |
217 | that process also executes the "daily" tasks. At midnight on the first day | |
218 | of the week, that process also executes the "weekly" tasks. A single | |
219 | process iterates over each registered repository, performing the scheduled | |
220 | tasks for that frequency. Depending on the number of registered | |
221 | repositories and their sizes, this process may take longer than an hour. | |
222 | In this case, multiple `git maintenance run` commands may run on the same | |
223 | repository at the same time, colliding on the object database lock. This | |
224 | results in one of the two tasks not running. | |
225 | ||
226 | If you find that some maintenance windows are taking longer than one hour | |
227 | to complete, then consider reducing the complexity of your maintenance | |
228 | tasks. For example, the `gc` task is much slower than the | |
229 | `incremental-repack` task. However, this comes at a cost of a slightly | |
230 | larger object database. Consider moving more expensive tasks to be run | |
231 | less frequently. | |
232 | ||
233 | Expert users may consider scheduling their own maintenance tasks using a | |
234 | different schedule than is available through `git maintenance start` and | |
235 | Git configuration options. These users should be aware of the object | |
236 | database lock and how concurrent `git maintenance run` commands behave. | |
237 | Further, the `git gc` command should not be combined with | |
238 | `git maintenance run` commands. `git gc` modifies the object database | |
239 | but does not take the lock in the same way as `git maintenance run`. If | |
240 | possible, use `git maintenance run --task=gc` instead of `git gc`. | |
241 | ||
16c56909 DS |
242 | The following sections describe the mechanisms put in place to run |
243 | background maintenance by `git maintenance start` and how to customize | |
244 | them. | |
245 | ||
246 | BACKGROUND MAINTENANCE ON POSIX SYSTEMS | |
247 | --------------------------------------- | |
248 | ||
249 | The standard mechanism for scheduling background tasks on POSIX systems | |
250 | is cron(8). This tool executes commands based on a given schedule. The | |
251 | current list of user-scheduled tasks can be found by running `crontab -l`. | |
252 | The schedule written by `git maintenance start` is similar to this: | |
253 | ||
254 | ----------------------------------------------------------------------- | |
255 | # BEGIN GIT MAINTENANCE SCHEDULE | |
256 | # The following schedule was created by Git | |
257 | # Any edits made in this region might be | |
258 | # replaced in the future by a Git command. | |
259 | ||
260 | 0 1-23 * * * "/<path>/git" --exec-path="/<path>" for-each-repo --config=maintenance.repo maintenance run --schedule=hourly | |
261 | 0 0 * * 1-6 "/<path>/git" --exec-path="/<path>" for-each-repo --config=maintenance.repo maintenance run --schedule=daily | |
262 | 0 0 * * 0 "/<path>/git" --exec-path="/<path>" for-each-repo --config=maintenance.repo maintenance run --schedule=weekly | |
263 | ||
264 | # END GIT MAINTENANCE SCHEDULE | |
265 | ----------------------------------------------------------------------- | |
266 | ||
267 | The comments are used as a region to mark the schedule as written by Git. | |
268 | Any modifications within this region will be completely deleted by | |
269 | `git maintenance stop` or overwritten by `git maintenance start`. | |
270 | ||
271 | The `crontab` entry specifies the full path of the `git` executable to | |
272 | ensure that the executed `git` command is the same one with which | |
273 | `git maintenance start` was issued independent of `PATH`. If the same user | |
274 | runs `git maintenance start` with multiple Git executables, then only the | |
275 | latest executable is used. | |
276 | ||
277 | These commands use `git for-each-repo --config=maintenance.repo` to run | |
278 | `git maintenance run --schedule=<frequency>` on each repository listed in | |
279 | the multi-valued `maintenance.repo` config option. These are typically | |
280 | loaded from the user-specific global config. The `git maintenance` process | |
281 | then determines which maintenance tasks are configured to run on each | |
282 | repository with each `<frequency>` using the `maintenance.<task>.schedule` | |
283 | config options. These values are loaded from the global or repository | |
284 | config values. | |
285 | ||
286 | If the config values are insufficient to achieve your desired background | |
287 | maintenance schedule, then you can create your own schedule. If you run | |
288 | `crontab -e`, then an editor will load with your user-specific `cron` | |
289 | schedule. In that editor, you can add your own schedule lines. You could | |
290 | start by adapting the default schedule listed earlier, or you could read | |
291 | the crontab(5) documentation for advanced scheduling techniques. Please | |
292 | do use the full path and `--exec-path` techniques from the default | |
293 | schedule to ensure you are executing the correct binaries in your | |
294 | schedule. | |
295 | ||
0016b618 | 296 | |
b681b191 LH |
297 | BACKGROUND MAINTENANCE ON LINUX SYSTEMD SYSTEMS |
298 | ----------------------------------------------- | |
299 | ||
300 | While Linux supports `cron`, depending on the distribution, `cron` may | |
301 | be an optional package not necessarily installed. On modern Linux | |
302 | distributions, systemd timers are superseding it. | |
303 | ||
304 | If user systemd timers are available, they will be used as a replacement | |
305 | of `cron`. | |
306 | ||
307 | In this case, `git maintenance start` will create user systemd timer units | |
308 | and start the timers. The current list of user-scheduled tasks can be found | |
309 | by running `systemctl --user list-timers`. The timers written by `git | |
310 | maintenance start` are similar to this: | |
311 | ||
312 | ----------------------------------------------------------------------- | |
313 | $ systemctl --user list-timers | |
314 | NEXT LEFT LAST PASSED UNIT ACTIVATES | |
315 | Thu 2021-04-29 19:00:00 CEST 42min left Thu 2021-04-29 18:00:11 CEST 17min ago git-maintenance@hourly.timer git-maintenance@hourly.service | |
316 | Fri 2021-04-30 00:00:00 CEST 5h 42min left Thu 2021-04-29 00:00:11 CEST 18h ago git-maintenance@daily.timer git-maintenance@daily.service | |
317 | Mon 2021-05-03 00:00:00 CEST 3 days left Mon 2021-04-26 00:00:11 CEST 3 days ago git-maintenance@weekly.timer git-maintenance@weekly.service | |
318 | ----------------------------------------------------------------------- | |
319 | ||
320 | One timer is registered for each `--schedule=<frequency>` option. | |
321 | ||
322 | The definition of the systemd units can be inspected in the following files: | |
323 | ||
324 | ----------------------------------------------------------------------- | |
325 | ~/.config/systemd/user/git-maintenance@.timer | |
326 | ~/.config/systemd/user/git-maintenance@.service | |
327 | ~/.config/systemd/user/timers.target.wants/git-maintenance@hourly.timer | |
328 | ~/.config/systemd/user/timers.target.wants/git-maintenance@daily.timer | |
329 | ~/.config/systemd/user/timers.target.wants/git-maintenance@weekly.timer | |
330 | ----------------------------------------------------------------------- | |
331 | ||
332 | `git maintenance start` will overwrite these files and start the timer | |
333 | again with `systemctl --user`, so any customization should be done by | |
334 | creating a drop-in file, i.e. a `.conf` suffixed file in the | |
335 | `~/.config/systemd/user/git-maintenance@.service.d` directory. | |
336 | ||
337 | `git maintenance stop` will stop the user systemd timers and delete | |
338 | the above mentioned files. | |
339 | ||
340 | For more details, see `systemd.timer(5)`. | |
341 | ||
342 | ||
2afe7e35 DS |
343 | BACKGROUND MAINTENANCE ON MACOS SYSTEMS |
344 | --------------------------------------- | |
345 | ||
346 | While macOS technically supports `cron`, using `crontab -e` requires | |
347 | elevated privileges and the executed process does not have a full user | |
348 | context. Without a full user context, Git and its credential helpers | |
349 | cannot access stored credentials, so some maintenance tasks are not | |
350 | functional. | |
351 | ||
352 | Instead, `git maintenance start` interacts with the `launchctl` tool, | |
353 | which is the recommended way to schedule timed jobs in macOS. Scheduling | |
354 | maintenance through `git maintenance (start|stop)` requires some | |
355 | `launchctl` features available only in macOS 10.11 or later. | |
356 | ||
357 | Your user-specific scheduled tasks are stored as XML-formatted `.plist` | |
358 | files in `~/Library/LaunchAgents/`. You can see the currently-registered | |
359 | tasks using the following command: | |
360 | ||
361 | ----------------------------------------------------------------------- | |
362 | $ ls ~/Library/LaunchAgents/org.git-scm.git* | |
363 | org.git-scm.git.daily.plist | |
364 | org.git-scm.git.hourly.plist | |
365 | org.git-scm.git.weekly.plist | |
366 | ----------------------------------------------------------------------- | |
367 | ||
368 | One task is registered for each `--schedule=<frequency>` option. To | |
369 | inspect how the XML format describes each schedule, open one of these | |
370 | `.plist` files in an editor and inspect the `<array>` element following | |
371 | the `<key>StartCalendarInterval</key>` element. | |
372 | ||
373 | `git maintenance start` will overwrite these files and register the | |
374 | tasks again with `launchctl`, so any customizations should be done by | |
375 | creating your own `.plist` files with distinct names. Similarly, the | |
376 | `git maintenance stop` command will unregister the tasks with `launchctl` | |
377 | and delete the `.plist` files. | |
378 | ||
379 | To create more advanced customizations to your background tasks, see | |
380 | launchctl.plist(5) for more information. | |
381 | ||
382 | ||
3797a0a7 DS |
383 | BACKGROUND MAINTENANCE ON WINDOWS SYSTEMS |
384 | ----------------------------------------- | |
385 | ||
386 | Windows does not support `cron` and instead has its own system for | |
387 | scheduling background tasks. The `git maintenance start` command uses | |
388 | the `schtasks` command to submit tasks to this system. You can inspect | |
389 | all background tasks using the Task Scheduler application. The tasks | |
390 | added by Git have names of the form `Git Maintenance (<frequency>)`. | |
391 | The Task Scheduler GUI has ways to inspect these tasks, but you can also | |
392 | export the tasks to XML files and view the details there. | |
393 | ||
394 | Note that since Git is a console application, these background tasks | |
395 | create a console window visible to the current user. This can be changed | |
396 | manually by selecting the "Run whether user is logged in or not" option | |
397 | in Task Scheduler. This change requires a password input, which is why | |
398 | `git maintenance start` does not select it by default. | |
399 | ||
400 | If you want to customize the background tasks, please rename the tasks | |
401 | so future calls to `git maintenance (start|stop)` do not overwrite your | |
402 | custom tasks. | |
403 | ||
16f6b0d1 ÆAB |
404 | CONFIGURATION |
405 | ------------- | |
406 | ||
407 | include::includes/cmd-config-section-all.txt[] | |
408 | ||
409 | include::config/maintenance.txt[] | |
410 | ||
0016b618 | 411 | |
2057d750 DS |
412 | GIT |
413 | --- | |
414 | Part of the linkgit:git[1] suite |