In order to make sure the cloned repository is good, we run "rev-list
--objects --not --all $new_refs" on the repository. This is expensive
on large repositories. This patch attempts to mitigate the impact in
this special case.
In the "good" clone case, we only have one pack. If all of the
following are met, we can be sure that all objects reachable from the
new refs exist, which is the intention of running "rev-list ...":
- all refs point to an object in the pack
- there are no dangling pointers in any object in the pack
- no objects in the pack point to objects outside the pack
The second and third checks can be done with the help of index-pack as
a slight variation of --strict check (which introduces a new condition
for the shortcut: pack transfer must be used and the number of objects
large enough to call index-pack). The first is checked in
check_everything_connected after we get an "ok" from index-pack.
"index-pack + new checks" is still faster than the current "index-pack
+ rev-list", which is the whole point of this patch. If any of the
conditions fail, we fall back to the good old but expensive "rev-list
..". In that case it's even more expensive because we have to pay for
the new checks in index-pack. But that should only happen when the
other side is either buggy or malicious.
Cloning linux-2.6 over file://
before after
real 3m25.693s 2m53.050s
user 5m2.037s 4m42.396s
sys 0m13.750s 0m16.574s
A more realistic test with ssh:// over wireless
before after
real 11m26.629s 10m4.213s
user 5m43.196s 5m19.444s
sys 0m35.812s 0m37.630s
This shortcut is not applied to shallow clones, partly because shallow
clones should have no more objects than a usual fetch and the cost of
rev-list is acceptable, partly to avoid dealing with corner cases when
grafting is involved.
This shortcut does not apply to unpack-objects code path either
because the number of objects must be small in order to trigger that
code path.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch-pack: prepare updated shallow file before fetching the pack
index-pack --strict looks up and follows parent commits. If shallow
information is not ready by the time index-pack is run, index-pack may
be led to non-existent objects. Make fetch-pack save shallow file to
disk before invoking index-pack.
git learns new global option --shallow-file to pass on the alternate
shallow file path. Undocumented (and not even support --shallow-file=
syntax) because it's unlikely to be used again elsewhere.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
clone: let the user know when check_everything_connected is run
check_everything_connected could take a long time, especially in the
clone case where the whole DAG is traversed. The user deserves to know
what's going on.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano [Thu, 9 May 2013 20:32:54 +0000 (13:32 -0700)]
Sync with v1.8.2.3
* maint:
Git 1.8.2.3
t5004: avoid using tar for checking emptiness of archive
t5004: ignore pax global header file
mergetools/kdiff3: do not use --auto when diffing
transport-helper: trivial style cleanup
Junio C Hamano [Thu, 9 May 2013 20:27:49 +0000 (13:27 -0700)]
cherry-pick: picking a tag that resolves to a commit is OK
Earlier, 21246dbb9e0a (cherry-pick: make sure all input objects are
commits, 2013-04-11) tried to catch an unlikely "git cherry-pick $blob"
as an error, but broke a more important use case to cherry-pick a
tag that points at a commit.
René Scharfe [Thu, 9 May 2013 13:13:47 +0000 (15:13 +0200)]
t5004: avoid using tar for checking emptiness of archive
Test 2 of t5004 checks if a supposedly empty tar archive really
contains no files. 24676f02 (t5004: fix issue with empty archive test
and bsdtar) removed our commit hash to make it work with bsdtar, but
the test still fails on NetBSD and OpenBSD, which use their own tar
that considers a tar file containing only NULs as broken.
Here's what the different archivers do when asked to create a tar
file without entries:
So NetBSD's native tar creates an empty file, while GNU tar and bsdtar
both give us 10KB of NULs -- just like git archive with an empty tree.
Now let's see how the archivers handle these two kinds of empty tar
files:
$ tar tf zero.tar; echo $?
tar: Unexpected EOF on archive file
1
$ gtar tf zero.tar; echo $?
gtar: This does not look like a tar archive
gtar: Exiting with failure status due to previous errors
2
$ bsdtar tf zero.tar; echo $?
0
$ tar tf tenk.tar; echo $?
tar: Cannot identify format. Searching...
tar: End of archive volume 1 reached
tar: Sorry, unable to determine archive format.
1
$ gtar tf tenk.tar; echo $?
0
$ bsdtar tf tenk.tar; echo $?
0
NetBSD's tar complains about both, bsdtar happily accepts any of them
and GNU tar doesn't like zero-length archive files. So the safest
course of action is to stay with our block-of-NULs format which is
compatible with GNU tar and bsdtar, as we can't make NetBSD's native
tar happy anyway.
We can simplify our test, however, by taking tar out of the picture.
Instead of extracting the archive and checking for the non-presence of
files, check if the file has a size of 10KB and contains only NULs.
This makes t5004 pass on NetBSD and OpenBSD.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx> Signed-off-by: Junio C Hamano <gitster@pobox.com>
René Scharfe [Thu, 9 May 2013 13:10:48 +0000 (15:10 +0200)]
t5004: ignore pax global header file
Versions of tar that don't know pax headers -- like the ones in NetBSD 6
and OpenBSD 5.2 -- extract them as regular files. Explicitly ignore the
file created for our global header when checking the list of extracted
files, as this is normal and harmless fall-back behaviour. This fixes
test 3 of t5004 on these platforms.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx> Signed-off-by: Junio C Hamano <gitster@pobox.com>
David Aguilar [Thu, 9 May 2013 09:13:28 +0000 (02:13 -0700)]
mergetools/kdiff3: do not use --auto when diffing
The `kdiff3 --auto` help message is, "No GUI if all conflicts are auto-
solvable." This flag was carried over from the original mergetool
commands. diff_cmd() is for two-way comparisons only so remove the
superfluous flag.
Signed-off-by: David Aguilar <davvid@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
It's possible that the previous tip goes away, we should not assume it's
always present. Fortunately we are only using it to calculate the
progress to display to the user, so only that needs to be fixed.
Also, add a test that triggers this issue.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-merge-tree causes a null pointer dereference when a directory
entry exists in only one or two of the three trees being compared with
no corresponding entry in the other tree(s).
When this happens, we want to handle the entry as a directory and not
attempt to mark it as a file merge. Do this by setting the entries bit
in the directory mask when the entry is missing or when it is a
directory, only performing the file comparison when we know that a file
entry exists.
Reported-by: Andreas Jacobsen <andreas@andreasjacobsen.com> Signed-off-by: John Keeping <john@keeping.me.uk> Tested-by: Andreas Jacobsen <andreas@andreasjacobsen.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano [Tue, 7 May 2013 05:16:26 +0000 (22:16 -0700)]
Merge branch 'fc/remote-bzr'
* fc/remote-bzr:
remote-bzr: avoid bad refs
remote-bzr: convert all unicode keys to str
remote-bzr: access branches only when needed
remote-bzr: delay peer branch usage
remote-bzr: iterate revisions properly
remote-bzr: improve progress reporting
remote-bzr: add option to specify branches
remote-bzr: add custom method to find branches
remote-bzr: improve author sanitazion
remote-bzr: add support for shared repo
remote-bzr: fix branch names
remote-bzr: add support for bzr repos
remote-bzr: use branch variable when appropriate
remote-bzr: fix partially pushed merge
remote-bzr: fixes for branch diverge
remote-bzr: add support to push merges
remote-bzr: always try to update the worktree
remote-bzr: fix order of locking in CustomTree
remote-bzr: delay blob fetching until the very end
remote-bzr: cleanup CustomTree
Versions of fast-export before v1.8.2 throws a bad 'reset' commands
because of a behavior in transport-helper that is not even needed.
We should ignore them, otherwise we will treat them as branches and
fail.
This was fixed in v1.8.2, but some people use this script in older
versions of git.
Also, check if the ref was a tag, and skip it for now.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
If there's already a remote-helper tracking ref, we can fetch the
SHA-1 to report proper push messages (as opposed to always reporting
[new branch]).
The remote-helper currently can specify the old SHA-1 to avoid this
problem, but there's no point in forcing all remote-helpers to be aware
of git commit ids; they should be able to be agnostic of them.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano [Fri, 3 May 2013 22:12:38 +0000 (15:12 -0700)]
Merge branch 'tr/remote-tighten-commandline-parsing' into maint
* tr/remote-tighten-commandline-parsing:
remote: 'show' and 'prune' can take more than one remote
remote: check for superfluous arguments in 'git remote add'
remote: add a test for extra arguments, according to docs
* rs/pp-user-info-without-extra-allocation:
pretty: remove intermediate strbufs from pp_user_info()
pretty: simplify output line length calculation in pp_user_info()
pretty: simplify input line length calculation in pp_user_info()
* tr/remote-tighten-commandline-parsing:
remote: 'show' and 'prune' can take more than one remote
remote: check for superfluous arguments in 'git remote add'
remote: add a test for extra arguments, according to docs
Bazaar doesn't seem to be tested for multiple usage of branches, so
resources seem to be leaked all over. Let's try to minimize this by
accessing the Branch objects only when needed.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
This way we don't need to store the list of all the revisions, which
doesn't seem to be very memory efficient with bazaar's design, for
whatever reason.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
If part of the merge was already pushed, we don't have the blob_marks
available, however, the commits are already stored in bazaar, so we can
use the revision_tree to fetch the contents.
We want to do this only when there's no other option.
There's no easy way to test this.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
If the branches diverge we want to reset the pointer to where the remote
actually is. Since we can access remote branches just as easily as local
ones, let's do so.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
In order to do that, we need to store the marks of every file, so that
they can be fetched when needed. Unfortunately we can't tell bazaar that
nothing changed, we need to send the data so that it can figure it out
by itself.
And since it will be requesting a bunch of information by the file_id,
it's better to have a helper dict (rev_files), so that we can fetch it
quickly.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Thomas Rast [Tue, 30 Apr 2013 12:53:06 +0000 (14:53 +0200)]
unpack_entry: avoid freeing objects in base cache
In the !delta_data error path of unpack_entry(), we run free(base).
This became a window for use-after-free() in abe601b (sha1_file:
remove recursion in unpack_entry, 2013-03-27), as follows:
Before abe601b, we got the 'base' from cache_or_unpack_entry(..., 0);
keep_cache=0 tells it to also remove that entry. So the 'base' is at
this point not cached, and freeing it in the error path is the right
thing.
After abe601b, the structure changed: we use a three-phase approach
where phase 1 finds the innermost base or a base that is already in
the cache. In phase 3 we therefore know that all bases we unpack are
not part of the delta cache yet. (Observe that we pop from the cache
in phase 1, so this is also true for the very first base.) So we make
no further attempts to look up the bases in the cache, and just call
add_delta_base_cache() on every base object we have assembled.
But the !delta_data error path remained unchanged, and now calls
free() on a base that has already been entered in the cache. This
means that there is a use-after-free if we later use the same base
again.
So remove that free(); we are still going to use that data.
Reported-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Thomas Rast <trast@inf.ethz.ch> Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-completion.bash: add remote.pushdefault to config list
224c2171 (remote.c: introduce remote.pushdefault, 2013-04-02)
introduced the remote.pushdefault configuration variable, but forgot
to teach git-completion.bash about it. Fix this.
Signed-off-by: Ramkumar Ramachandra <artagnon@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-completion.bash: add branch.*.pushremote to config list
9f765ce (remote.c: introduce branch.<name>.pushremote, 2013-04-02)
introduced the configuration variable branch.*.pushremote, but forgot
to teach git-completion.bash about it. Fix this.
Signed-off-by: Ramkumar Ramachandra <artagnon@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano [Mon, 29 Apr 2013 16:57:38 +0000 (09:57 -0700)]
Merge branch 'maint'
* maint:
complete: zsh: use zsh completion for the main cmd
complete: zsh: trivial simplification
git-completion.bash: complete branch.*.rebase as boolean
git-completion.bash: add diff.submodule to config list
git-completion.bash: lexical sorting for diff.statGraphWidth
Felipe Contreras [Sat, 27 Apr 2013 20:34:06 +0000 (15:34 -0500)]
complete: zsh: use zsh completion for the main cmd
So that we can have a nice zsh completion output:
% git <tab>
add -- add file contents to the index
bisect -- find by binary search the change that introduced a bug
branch -- list, create, or delete branches
checkout -- checkout a branch or paths to the working tree
clone -- clone a repository into a new directory
commit -- record changes to the repository
diff -- show changes between commits, commit and working tree, etc
fetch -- download objects and refs from another repository
grep -- print lines matching a pattern
init -- create an empty Git repository or reinitialize an existing one
log -- show commit logs
merge -- join two or more development histories together
mv -- move or rename a file, a directory, or a symlink
pull -- fetch from and merge with another repository or a local branch
push -- update remote refs along with associated objects
rebase -- forward-port local commits to the updated upstream head
reset -- reset current HEAD to the specified state
rm -- remove files from the working tree and from the index
show -- show various types of objects
status -- show the working tree status
tag -- create, list, delete or verify a tag object signed with GPG
And other niceties, like 'git --git-dir=<tab>' showing only directories.
For the rest, the bash completion stuff is still used.
Also, add my copyright, since this more than a thin wrapper.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Felipe Contreras [Sat, 27 Apr 2013 20:34:05 +0000 (15:34 -0500)]
complete: zsh: trivial simplification
There should be no functional changes.
The only reason I wrapped this code around a sub-function is because zsh
did the same in it's bashcompinit script in order to declare the special
variable 'words' as hidden, but only in this context.
There's no need for that any more since we access __git_main directly,
so 'words' is not modified, so there's no need for the sub-function.
In zsh mode the array indexes are different though.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-completion.bash: complete branch.*.rebase as boolean
6fac1b83 (completion: add missing config variables, 2009-06-29) added
"rebase" to the list of completions for "branch.*.*", but forgot to
specify completions for the values that this configuration variable
can take (namely "false" and "true"). Fix this.
Signed-off-by: Ramkumar Ramachandra <artagnon@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-completion.bash: add diff.submodule to config list
c47ef57 (diff: introduce diff.submodule configuration variable,
2012-11-13) introduced the diff.submodule configuration variable, but
forgot to teach git-completion.bash about it. Fix this.
Signed-off-by: Ramkumar Ramachandra <artagnon@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-completion.bash: lexical sorting for diff.statGraphWidth
df44483a (diff --stat: add config option to limit graph width,
2012-03-01) added the option diff.startGraphWidth to the list of
configuration variables in git-completion.bash, but failed to notice
that the list is sorted alphabetically. Move it to its rightful place
in the list.
Signed-off-by: Ramkumar Ramachandra <artagnon@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Git.pm: call tempfile from File::Temp as a regular function
We call File::Temp's "tempfile" function as a class method, but it was
never designed to be called this way. Older versions seemed to
tolerate it, but as of File::Temp 0.23, it blows up like this:
$ git svn fetch
'tempfile' can't be called as a method at .../Git.pm line 1117.
Fix it by calling it as a regular function, just inside the File::Temp
namespace.
Signed-off-by: H. Merijn Brand <h.m.brand@xs4all.nl> Helped-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
John Keeping [Fri, 26 Apr 2013 17:58:24 +0000 (18:58 +0100)]
t/Makefile: remove smoke test targets
Commit d24fbca (Remove Git's support for smoke testing - 2011-12-23)
removed the smoke test support from the test suite but it was
re-added by commit 342e9ef (Introduce a performance testing
framework - 2012-02-17). This appears to be the result of a
mis-rebase, since re-adding the smoke testing infrastructure does
not relate to the subject of that commit.
The current 'smoke' target is broken since the 'harness' script it
uses no longer exists, so just reapply this section of commit d24fbca
and remove all of the smoke testing section in the makefile.
Signed-off-by: John Keeping <john@keeping.me.uk> Acked-by: Thomas Rast <trast@inf.ethz.ch> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Ramsay Jones [Sat, 27 Apr 2013 19:43:36 +0000 (20:43 +0100)]
pretty: Fix bug in truncation support for %>, %< and %><
Some systems experience failures in t4205-*.sh (tests 18-20, 27)
which all relate to the use of truncation with the %< padding
placeholder. This capability was added in the commit a7f01c6b
("pretty: support truncating in %>, %< and %><", 19-04-2013).
The truncation support was implemented with the assistance of a
new strbuf function (strbuf_utf8_replace). This function contains
the following code:
Unfortunately, this code is subject to unspecified behaviour. In
particular, the order of evaluation of the argument expressions
(along with the associated side effects) is not specified by the
C standard. Note that the second argument expression is a call to
strbuf_detach() which, as a side effect, sets the 'len' and 'alloc'
fields of the sb_dst argument to zero. Depending on the order of
evaluation of the argument expressions to the strbuf_attach call,
this can lead to assigning an empty string to 'sb_src'.
In order to remove the undesired behaviour, we replace the above
line of code with:
Ramsay Jones [Sat, 27 Apr 2013 18:39:04 +0000 (19:39 +0100)]
clone: Make the 'junk_mode' symbol a file static
Sparse issues an "'junk_mode' not declared. Should it be static?"
warning. In order to suppress the warning, since this symbol does
not need more than file visibility, we simply add the static
modifier to its declaration.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
John Keeping [Sat, 27 Apr 2013 13:40:33 +0000 (14:40 +0100)]
merge-tree: fix typo in "both changed identically"
Commit aacecc3 (merge-tree: don't print entries that match "local" -
2013-04-07) had a typo causing the "same in both" check to be incorrect
and check if both the base and "their" versions are removed instead of
checking that both the "our" and "their" versions are removed. Fix
this.
Reported-by: René Scharfe <rene.scharfe@lsrfire.ath.cx> Test-written-by: René Scharfe <rene.scharfe@lsrfire.ath.cx> Signed-off-by: John Keeping <john@keeping.me.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano [Fri, 26 Apr 2013 22:28:39 +0000 (15:28 -0700)]
Merge branch 'rr/shortlog-doc'
Update documentation for "log" and "shortlog".
* rr/shortlog-doc:
builtin/shortlog.c: make usage string consistent with log
builtin/log.c: make usage string consistent with doc
git-shortlog.txt: make SYNOPSIS match log, update OPTIONS
git-log.txt: rewrite note on why "--" may be required
git-log.txt: generalize <since>..<until>
git-log.txt: order OPTIONS properly; move <since>..<until>
revisions.txt: clarify the .. and ... syntax
git-shortlog.txt: remove (-h|--help) from OPTIONS
Junio C Hamano [Fri, 26 Apr 2013 22:28:23 +0000 (15:28 -0700)]
Merge branch 'jn/glossary-revision'
The wording for "revision" in the glossary wanted to say it refers
to "commit (noun) as a concept" but it was badly phrased.
This may need further updates to hint that in contexts where it is
clear, the word may refer to an object name, not necessarily a
commit. But the patch as-is is already an improvement.
* jn/glossary-revision:
glossary: a revision is just a commit
Felipe Contreras [Fri, 26 Apr 2013 21:12:36 +0000 (16:12 -0500)]
remote-bzr: store converted URL
Bazaar might convert the URL to something more appropriate, like an
absolute path. Lets store that instead of the original URL, which won't
work from a different working directory if it's relative.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Felipe Contreras [Fri, 26 Apr 2013 21:12:33 +0000 (16:12 -0500)]
remote-bzr: fix bad state issue
Carried from remote-hg.
The problem reportedly happened after doing a push that fails, the abort
causes the state of remote-hg to go bad, this happens because
remote-hg's marks are not stored, but 'git fast-export' marks are.
Ensure that the marks are _always_ stored.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano [Fri, 26 Apr 2013 22:19:03 +0000 (15:19 -0700)]
Merge branch 'fc/remote-hg'
* fc/remote-hg:
remote-hg: strip extra newline
remote-hg: use marks instead of inlined files
remote-hg: small performance improvement
remote-hg: allow refs with spaces
remote-hg: don't update bookmarks unnecessarily
remote-hg: add support for schemes extension
remote-hg: improve email sanitation
remote-hg: add custom local tag write code
remote-hg: write tags in the appropriate branch
remote-hg: custom method to write tags
remote-hg: add support for tag objects
remote-hg: add branch_tip() helper
remote-hg: properly mark branches up-to-date
remote-hg: use python urlparse
remote-hg: safer bookmark pushing
remote-helpers: avoid has_key
Junio C Hamano [Fri, 26 Apr 2013 20:44:35 +0000 (13:44 -0700)]
git add: avoid "-u/-A without pathspec" warning on stat-dirty paths
In preparation for Git 2.0, "git add -u/-A" without pathspec checks
all the working tree (not limited to the current directory) and
issues a warning when it finds any path that we might add in Git
2.0, because that would mean the users' fingers need to be trained
to explicitly say "." if they want to keep the current behaviour.
However, the check was incomplete, because "git add" usually does
not refresh the index, considers a path that is stat-dirty but has
contents that is otherwise up-to-date in the index as "we might
add", and relies on that it is a no-op to add the same thing again
via the add_file_to_index() API (which also knows not to say "added"
in verbose mode when this happens). We do not want to trigger the
warning for a path that is outside the current directory is merely
stat-dirty, as it won't be added in Git 2.0, either.
Signed-off-by: Junio C Hamano <gitster@pobox.com> Acked-by: Jonathan Nieder <jrnieder@gmail.com>
projects / thirdparty / git.git / log
