]>
Commit | Line | Data |
---|---|---|
c401b33c JS |
1 | git-filter-branch(1) |
2 | ==================== | |
3 | ||
4 | NAME | |
5 | ---- | |
6 | git-filter-branch - Rewrite branches | |
7 | ||
8 | SYNOPSIS | |
9 | -------- | |
10 | [verse] | |
b1889c36 | 11 | 'git filter-branch' [--env-filter <command>] [--tree-filter <command>] |
c401b33c JS |
12 | [--index-filter <command>] [--parent-filter <command>] |
13 | [--msg-filter <command>] [--commit-filter <command>] | |
14 | [--tag-name-filter <command>] [--subdirectory-filter <directory>] | |
e3679ab4 | 15 | [--prune-empty] |
5433235d | 16 | [--original <namespace>] [-d <directory>] [-f | --force] |
8afa4210 | 17 | [--] [<rev-list options>...] |
c401b33c JS |
18 | |
19 | DESCRIPTION | |
20 | ----------- | |
08203668 JS |
21 | Lets you rewrite git revision history by rewriting the branches mentioned |
22 | in the <rev-list options>, applying custom filters on each revision. | |
c401b33c JS |
23 | Those filters can modify each tree (e.g. removing a file or running |
24 | a perl rewrite on all files) or information about each commit. | |
25 | Otherwise, all information (including original commit times or merge | |
26 | information) will be preserved. | |
27 | ||
08203668 | 28 | The command will only rewrite the _positive_ refs mentioned in the |
bf7c9021 | 29 | command line (e.g. if you pass 'a..b', only 'b' will be rewritten). |
08203668 JS |
30 | If you specify no filters, the commits will be recommitted without any |
31 | changes, which would normally have no effect. Nevertheless, this may be | |
32 | useful in the future for compensating for some git bugs or such, | |
33 | therefore such a usage is permitted. | |
c401b33c | 34 | |
0dc310e8 PC |
35 | *NOTE*: This command honors `.git/info/grafts` and `.git/refs/replace/`. |
36 | If you have any grafts or replacement refs defined, running this command | |
37 | will make them permanent. | |
c6d8f763 | 38 | |
73616fd3 | 39 | *WARNING*! The rewritten history will have different object names for all |
c401b33c JS |
40 | the objects and will not converge with the original branch. You will not |
41 | be able to easily push and distribute the rewritten branch on top of the | |
42 | original branch. Please do not use this command if you do not know the | |
43 | full implications, and avoid using it anyway, if a simple single commit | |
97c33c65 TR |
44 | would suffice to fix your problem. (See the "RECOVERING FROM UPSTREAM |
45 | REBASE" section in linkgit:git-rebase[1] for further information about | |
46 | rewriting published history.) | |
c401b33c | 47 | |
dfd05e38 JS |
48 | Always verify that the rewritten version is correct: The original refs, |
49 | if different from the rewritten ones, will be stored in the namespace | |
50 | 'refs/original/'. | |
c401b33c | 51 | |
bf7c9021 | 52 | Note that since this operation is very I/O expensive, it might |
08203668 JS |
53 | be a good idea to redirect the temporary directory off-disk with the |
54 | '-d' option, e.g. on tmpfs. Reportedly the speedup is very noticeable. | |
c401b33c JS |
55 | |
56 | ||
57 | Filters | |
58 | ~~~~~~~ | |
59 | ||
60 | The filters are applied in the order as listed below. The <command> | |
bf7c9021 RW |
61 | argument is always evaluated in the shell context using the 'eval' command |
62 | (with the notable exception of the commit filter, for technical reasons). | |
c401b33c JS |
63 | Prior to that, the $GIT_COMMIT environment variable will be set to contain |
64 | the id of the commit being rewritten. Also, GIT_AUTHOR_NAME, | |
65 | GIT_AUTHOR_EMAIL, GIT_AUTHOR_DATE, GIT_COMMITTER_NAME, GIT_COMMITTER_EMAIL, | |
bf7c9021 RW |
66 | and GIT_COMMITTER_DATE are set according to the current commit. The values |
67 | of these variables after the filters have run, are used for the new commit. | |
68 | If any evaluation of <command> returns a non-zero exit status, the whole | |
69 | operation will be aborted. | |
c401b33c JS |
70 | |
71 | A 'map' function is available that takes an "original sha1 id" argument | |
72 | and outputs a "rewritten sha1 id" if the commit has been already | |
32c37c12 JS |
73 | rewritten, and "original sha1 id" otherwise; the 'map' function can |
74 | return several ids on separate lines if your commit filter emitted | |
75 | multiple commits. | |
c401b33c JS |
76 | |
77 | ||
78 | OPTIONS | |
79 | ------- | |
80 | ||
81 | --env-filter <command>:: | |
bf7c9021 RW |
82 | This filter may be used if you only need to modify the environment |
83 | in which the commit will be performed. Specifically, you might | |
84 | want to rewrite the author/committer name/email/time environment | |
831a8b84 | 85 | variables (see linkgit:git-commit-tree[1] for details). Do not forget |
c401b33c JS |
86 | to re-export the variables. |
87 | ||
88 | --tree-filter <command>:: | |
89 | This is the filter for rewriting the tree and its contents. | |
90 | The argument is evaluated in shell with the working | |
91 | directory set to the root of the checked out tree. The new tree | |
92 | is then used as-is (new files are auto-added, disappeared files | |
93 | are auto-removed - neither .gitignore files nor any other ignore | |
73616fd3 | 94 | rules *HAVE ANY EFFECT*!). |
c401b33c JS |
95 | |
96 | --index-filter <command>:: | |
97 | This is the filter for rewriting the index. It is similar to the | |
98 | tree filter but does not check out the tree, which makes it much | |
3bc427e0 TR |
99 | faster. Frequently used with `git rm \--cached |
100 | \--ignore-unmatch ...`, see EXAMPLES below. For hairy | |
101 | cases, see linkgit:git-update-index[1]. | |
c401b33c JS |
102 | |
103 | --parent-filter <command>:: | |
104 | This is the filter for rewriting the commit's parent list. | |
105 | It will receive the parent string on stdin and shall output | |
106 | the new parent string on stdout. The parent string is in | |
483bc4f0 | 107 | the format described in linkgit:git-commit-tree[1]: empty for |
c401b33c JS |
108 | the initial commit, "-p parent" for a normal commit and |
109 | "-p parent1 -p parent2 -p parent3 ..." for a merge commit. | |
110 | ||
111 | --msg-filter <command>:: | |
112 | This is the filter for rewriting the commit messages. | |
113 | The argument is evaluated in the shell with the original | |
114 | commit message on standard input; its standard output is | |
115 | used as the new commit message. | |
116 | ||
117 | --commit-filter <command>:: | |
118 | This is the filter for performing the commit. | |
119 | If this filter is specified, it will be called instead of the | |
0b444cdb | 120 | 'git commit-tree' command, with arguments of the form |
0adda936 | 121 | "<TREE_ID> [(-p <PARENT_COMMIT_ID>)...]" and the log message on |
c401b33c JS |
122 | stdin. The commit id is expected on stdout. |
123 | + | |
124 | As a special extension, the commit filter may emit multiple | |
c5833f6e | 125 | commit ids; in that case, the rewritten children of the original commit will |
c401b33c | 126 | have all of them as parents. |
f95eef15 JS |
127 | + |
128 | You can use the 'map' convenience function in this filter, and other | |
129 | convenience functions, too. For example, calling 'skip_commit "$@"' | |
130 | will leave out the current commit (but not its changes! If you want | |
0b444cdb | 131 | that, use 'git rebase' instead). |
d3240d93 | 132 | + |
ca768288 TR |
133 | You can also use the `git_commit_non_empty_tree "$@"` instead of |
134 | `git commit-tree "$@"` if you don't wish to keep commits with a single parent | |
d3240d93 | 135 | and that makes no change to the tree. |
c401b33c JS |
136 | |
137 | --tag-name-filter <command>:: | |
138 | This is the filter for rewriting tag names. When passed, | |
139 | it will be called for every tag ref that points to a rewritten | |
140 | object (or to a tag object which points to a rewritten object). | |
141 | The original tag name is passed via standard input, and the new | |
142 | tag name is expected on standard output. | |
143 | + | |
144 | The original tags are not deleted, but can be overwritten; | |
5876b8ee | 145 | use "--tag-name-filter cat" to simply update the tags. In this |
c401b33c JS |
146 | case, be very careful and make sure you have the old tags |
147 | backed up in case the conversion has run afoul. | |
148 | + | |
1bf6551e BC |
149 | Nearly proper rewriting of tag objects is supported. If the tag has |
150 | a message attached, a new tag object will be created with the same message, | |
151 | author, and timestamp. If the tag has a signature attached, the | |
152 | signature will be stripped. It is by definition impossible to preserve | |
153 | signatures. The reason this is "nearly" proper, is because ideally if | |
154 | the tag did not change (points to the same object, has the same name, etc.) | |
155 | it should retain any signature. That is not the case, signatures will always | |
156 | be removed, buyer beware. There is also no support for changing the | |
157 | author or timestamp (or the tag message for that matter). Tags which point | |
158 | to other tags will be rewritten to point to the underlying commit. | |
c401b33c JS |
159 | |
160 | --subdirectory-filter <directory>:: | |
73616fd3 JS |
161 | Only look at the history which touches the given subdirectory. |
162 | The result will contain that directory (and only that) as its | |
7ec344d8 | 163 | project root. Implies <<Remap_to_ancestor>>. |
c401b33c | 164 | |
d3240d93 PH |
165 | --prune-empty:: |
166 | Some kind of filters will generate empty commits, that left the tree | |
167 | untouched. This switch allow git-filter-branch to ignore such | |
168 | commits. Though, this switch only applies for commits that have one | |
169 | and only one parent, it will hence keep merges points. Also, this | |
170 | option is not compatible with the use of '--commit-filter'. Though you | |
171 | just need to use the function 'git_commit_non_empty_tree "$@"' instead | |
ca768288 | 172 | of the `git commit-tree "$@"` idiom in your commit filter to make that |
d3240d93 PH |
173 | happen. |
174 | ||
5433235d GB |
175 | --original <namespace>:: |
176 | Use this option to set the namespace where the original commits | |
177 | will be stored. The default value is 'refs/original'. | |
178 | ||
c401b33c JS |
179 | -d <directory>:: |
180 | Use this option to set the path to the temporary directory used for | |
181 | rewriting. When applying a tree filter, the command needs to | |
bf7c9021 | 182 | temporarily check out the tree to some directory, which may consume |
c401b33c JS |
183 | considerable space in case of large projects. By default it |
184 | does this in the '.git-rewrite/' directory but you can override | |
185 | that choice by this parameter. | |
186 | ||
3240240f SB |
187 | -f:: |
188 | --force:: | |
0b444cdb | 189 | 'git filter-branch' refuses to start with an existing temporary |
dfd05e38 JS |
190 | directory or when there are already refs starting with |
191 | 'refs/original/', unless forced. | |
192 | ||
f448e24e | 193 | <rev-list options>...:: |
0b444cdb | 194 | Arguments for 'git rev-list'. All positive refs included by |
8afa4210 TR |
195 | these options are rewritten. You may also specify options |
196 | such as '--all', but you must use '--' to separate them from | |
7ec344d8 CH |
197 | the 'git filter-branch' options. Implies <<Remap_to_ancestor>>. |
198 | ||
199 | ||
200 | [[Remap_to_ancestor]] | |
201 | Remap to ancestor | |
202 | ~~~~~~~~~~~~~~~~~ | |
203 | ||
204 | By using linkgit:rev-list[1] arguments, e.g., path limiters, you can limit the | |
205 | set of revisions which get rewritten. However, positive refs on the command | |
206 | line are distinguished: we don't let them be excluded by such limiters. For | |
207 | this purpose, they are instead rewritten to point at the nearest ancestor that | |
208 | was not excluded. | |
c401b33c JS |
209 | |
210 | ||
211 | Examples | |
212 | -------- | |
213 | ||
214 | Suppose you want to remove a file (containing confidential information | |
215 | or copyright violation) from all commits: | |
216 | ||
217 | ------------------------------------------------------- | |
dfd05e38 | 218 | git filter-branch --tree-filter 'rm filename' HEAD |
c401b33c JS |
219 | ------------------------------------------------------- |
220 | ||
e4d594c6 JL |
221 | However, if the file is absent from the tree of some commit, |
222 | a simple `rm filename` will fail for that tree and commit. | |
223 | Thus you may instead want to use `rm -f filename` as the script. | |
224 | ||
0b444cdb | 225 | Using `\--index-filter` with 'git rm' yields a significantly faster |
3bc427e0 TR |
226 | version. Like with using `rm filename`, `git rm --cached filename` |
227 | will fail if the file is absent from the tree of a commit. If you | |
228 | want to "completely forget" a file, it does not matter when it entered | |
229 | history, so we also add `\--ignore-unmatch`: | |
c401b33c | 230 | |
dfd05e38 | 231 | -------------------------------------------------------------------------- |
3bc427e0 | 232 | git filter-branch --index-filter 'git rm --cached --ignore-unmatch filename' HEAD |
dfd05e38 | 233 | -------------------------------------------------------------------------- |
c401b33c | 234 | |
8ef44519 | 235 | Now, you will get the rewritten history saved in HEAD. |
c401b33c | 236 | |
8afa4210 TR |
237 | To rewrite the repository to look as if `foodir/` had been its project |
238 | root, and discard all other history: | |
239 | ||
240 | ------------------------------------------------------- | |
241 | git filter-branch --subdirectory-filter foodir -- --all | |
242 | ------------------------------------------------------- | |
243 | ||
244 | Thus you can, e.g., turn a library subdirectory into a repository of | |
245 | its own. Note the `\--` that separates 'filter-branch' options from | |
246 | revision options, and the `\--all` to rewrite all branches and tags. | |
247 | ||
32c37c12 JS |
248 | To set a commit (which typically is at the tip of another |
249 | history) to be the parent of the current initial commit, in | |
250 | order to paste the other history behind the current history: | |
c401b33c | 251 | |
dfd05e38 JS |
252 | ------------------------------------------------------------------- |
253 | git filter-branch --parent-filter 'sed "s/^\$/-p <graft-id>/"' HEAD | |
254 | ------------------------------------------------------------------- | |
c401b33c | 255 | |
08203668 JS |
256 | (if the parent string is empty - which happens when we are dealing with |
257 | the initial commit - add graftcommit as a parent). Note that this assumes | |
c401b33c JS |
258 | history with a single root (that is, no merge without common ancestors |
259 | happened). If this is not the case, use: | |
260 | ||
dfd05e38 | 261 | -------------------------------------------------------------------------- |
c401b33c | 262 | git filter-branch --parent-filter \ |
41e86a37 | 263 | 'test $GIT_COMMIT = <commit-id> && echo "-p <graft-id>" || cat' HEAD |
dfd05e38 | 264 | -------------------------------------------------------------------------- |
c401b33c | 265 | |
32c37c12 JS |
266 | or even simpler: |
267 | ||
268 | ----------------------------------------------- | |
269 | echo "$commit-id $graft-id" >> .git/info/grafts | |
dfd05e38 | 270 | git filter-branch $graft-id..HEAD |
32c37c12 JS |
271 | ----------------------------------------------- |
272 | ||
c401b33c JS |
273 | To remove commits authored by "Darl McBribe" from the history: |
274 | ||
275 | ------------------------------------------------------------------------------ | |
276 | git filter-branch --commit-filter ' | |
277 | if [ "$GIT_AUTHOR_NAME" = "Darl McBribe" ]; | |
278 | then | |
f95eef15 | 279 | skip_commit "$@"; |
c401b33c JS |
280 | else |
281 | git commit-tree "$@"; | |
dfd05e38 | 282 | fi' HEAD |
c401b33c JS |
283 | ------------------------------------------------------------------------------ |
284 | ||
8451c565 | 285 | The function 'skip_commit' is defined as follows: |
f95eef15 JS |
286 | |
287 | -------------------------- | |
288 | skip_commit() | |
289 | { | |
290 | shift; | |
291 | while [ -n "$1" ]; | |
292 | do | |
293 | shift; | |
294 | map "$1"; | |
295 | shift; | |
296 | done; | |
297 | } | |
298 | -------------------------- | |
299 | ||
c401b33c JS |
300 | The shift magic first throws away the tree id and then the -p |
301 | parameters. Note that this handles merges properly! In case Darl | |
302 | committed a merge between P1 and P2, it will be propagated properly | |
303 | and all children of the merge will become merge commits with P1,P2 | |
304 | as their parents instead of the merge commit. | |
305 | ||
a1748890 | 306 | You can rewrite the commit log messages using `--msg-filter`. For |
0b444cdb | 307 | example, 'git svn-id' strings in a repository created by 'git svn' can |
ed10d9aa MV |
308 | be removed this way: |
309 | ||
310 | ------------------------------------------------------- | |
a1748890 | 311 | git filter-branch --msg-filter ' |
ed10d9aa MV |
312 | sed -e "/^git-svn-id:/d" |
313 | ' | |
314 | ------------------------------------------------------- | |
f95eef15 | 315 | |
c401b33c JS |
316 | To restrict rewriting to only part of the history, specify a revision |
317 | range in addition to the new branch name. The new branch name will | |
0b444cdb | 318 | point to the top-most revision that a 'git rev-list' of this range |
c401b33c JS |
319 | will print. |
320 | ||
b8f42332 JS |
321 | If you need to add 'Acked-by' lines to, say, the last 10 commits (none |
322 | of which is a merge), use this command: | |
323 | ||
324 | -------------------------------------------------------- | |
325 | git filter-branch --msg-filter ' | |
326 | cat && | |
327 | echo "Acked-by: Bugs Bunny <bunny@bugzilla.org>" | |
328 | ' HEAD~10..HEAD | |
329 | -------------------------------------------------------- | |
330 | ||
08203668 JS |
331 | *NOTE* the changes introduced by the commits, and which are not reverted |
332 | by subsequent commits, will still be in the rewritten branch. If you want | |
c401b33c | 333 | to throw out _changes_ together with the commits, you should use the |
0b444cdb | 334 | interactive mode of 'git rebase'. |
c401b33c | 335 | |
08203668 | 336 | |
c401b33c JS |
337 | Consider this history: |
338 | ||
339 | ------------------ | |
340 | D--E--F--G--H | |
341 | / / | |
342 | A--B-----C | |
343 | ------------------ | |
344 | ||
345 | To rewrite only commits D,E,F,G,H, but leave A, B and C alone, use: | |
346 | ||
347 | -------------------------------- | |
dfd05e38 | 348 | git filter-branch ... C..H |
c401b33c JS |
349 | -------------------------------- |
350 | ||
351 | To rewrite commits E,F,G,H, use one of these: | |
352 | ||
353 | ---------------------------------------- | |
dfd05e38 JS |
354 | git filter-branch ... C..H --not D |
355 | git filter-branch ... D..H --not C | |
c401b33c JS |
356 | ---------------------------------------- |
357 | ||
358 | To move the whole tree into a subdirectory, or remove it from there: | |
359 | ||
360 | --------------------------------------------------------------- | |
361 | git filter-branch --index-filter \ | |
d2d66f15 | 362 | 'git ls-files -s | sed "s-\t\"*-&newsubdir/-" | |
c401b33c JS |
363 | GIT_INDEX_FILE=$GIT_INDEX_FILE.new \ |
364 | git update-index --index-info && | |
6cb0186a | 365 | mv "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE"' HEAD |
c401b33c JS |
366 | --------------------------------------------------------------- |
367 | ||
368 | ||
d0268de6 TR |
369 | |
370 | Checklist for Shrinking a Repository | |
371 | ------------------------------------ | |
372 | ||
373 | git-filter-branch is often used to get rid of a subset of files, | |
374 | usually with some combination of `\--index-filter` and | |
375 | `\--subdirectory-filter`. People expect the resulting repository to | |
376 | be smaller than the original, but you need a few more steps to | |
377 | actually make it smaller, because git tries hard not to lose your | |
378 | objects until you tell it to. First make sure that: | |
379 | ||
380 | * You really removed all variants of a filename, if a blob was moved | |
381 | over its lifetime. `git log \--name-only \--follow \--all \-- | |
382 | filename` can help you find renames. | |
383 | ||
384 | * You really filtered all refs: use `\--tag-name-filter cat \-- | |
385 | \--all` when calling git-filter-branch. | |
386 | ||
387 | Then there are two ways to get a smaller repository. A safer way is | |
388 | to clone, that keeps your original intact. | |
389 | ||
390 | * Clone it with `git clone +++file:///path/to/repo+++`. The clone | |
391 | will not have the removed objects. See linkgit:git-clone[1]. (Note | |
392 | that cloning with a plain path just hardlinks everything!) | |
393 | ||
394 | If you really don't want to clone it, for whatever reasons, check the | |
395 | following points instead (in this order). This is a very destructive | |
396 | approach, so *make a backup* or go back to cloning it. You have been | |
397 | warned. | |
398 | ||
399 | * Remove the original refs backed up by git-filter-branch: say `git | |
400 | for-each-ref \--format="%(refname)" refs/original/ | xargs -n 1 git | |
401 | update-ref -d`. | |
402 | ||
403 | * Expire all reflogs with `git reflog expire \--expire=now \--all`. | |
404 | ||
405 | * Garbage collect all unreferenced objects with `git gc \--prune=now` | |
406 | (or if your git-gc is not new enough to support arguments to | |
407 | `\--prune`, use `git repack -ad; git prune` instead). | |
408 | ||
c401b33c JS |
409 | GIT |
410 | --- | |
9e1f0a85 | 411 | Part of the linkgit:git[1] suite |