Jeff King [Thu, 16 Apr 2015 09:03:26 +0000 (05:03 -0400)]
read_packed_refs: avoid double-checking sane refs
Prior to d0f810f (refs.c: allow listing and deleting badly
named refs, 2014-09-03), read_packed_refs would barf on any
malformed refnames by virtue of calling create_ref_entry
with the "check" parameter set to 1. That commit loosened
our reading so that we call check_refname_format ourselves
and just set a REF_BAD_NAME flag.
We then call create_ref_entry with the check parameter set
to 0. That function learned to do an extra safety check even
when the check parameter is 0, so that we don't load any
dangerous refnames (like "../../../etc/passwd"). This is
implemented by calling refname_is_safe() in
create_ref_entry().
However, we can observe that refname_is_safe() can only be
true if check_refname_format() also failed. So in the common
case of a sanely named ref, we perform _both_ checks, even
though we know that the latter will never trigger. This has
a noticeable performance impact when the packed-refs file is
large.
Let's drop the refname_is_safe check from create_ref_entry(),
and make it the responsibility of the caller. Of the three
callers that pass a check parameter of "0", two will have
just called check_refname_format(), and can check the
refname-safety only when it fails. The third case,
pack_if_possible_fn, is copying from an existing ref entry,
which must have previously passed our safety check.
With this patch, running "git rev-parse refs/heads/does-not-exist"
on a repo with a large (1.6GB) packed-refs file went from:
real 0m6.768s
user 0m6.340s
sys 0m0.432s
to:
real 0m5.703s
user 0m5.276s
sys 0m0.432s
for a wall-clock speedup of 15%.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Thu, 16 Apr 2015 09:01:38 +0000 (05:01 -0400)]
strbuf_getwholeline: use getdelim if it is available
We spend a lot of time in strbuf_getwholeline in a tight
loop reading characters from a stdio handle into a buffer.
The libc getdelim() function can do this for us with less
overhead. It's in POSIX.1-2008, and was a GNU extension
before that. Therefore we can't rely on it, but can fall
back to the existing getc loop when it is not available.
The HAVE_GETDELIM knob is turned on automatically for Linux,
where we have glibc. We don't need to set any new
feature-test macros, because we already define _GNU_SOURCE.
Other systems that implement getdelim may need to other
macros (probably _POSIX_C_SOURCE >= 200809L), but we can
address that along with setting the Makefile knob after
testing the feature on those systems.
Running "git rev-parse refs/heads/does-not-exist" on a repo
with an extremely large (1.6GB) packed-refs file went from
(best-of-5):
real 0m8.601s
user 0m8.084s
sys 0m0.524s
to:
real 0m6.768s
user 0m6.340s
sys 0m0.432s
for a wall-clock speedup of 21%.
Based on a patch from Rasmus Villemoes <rv@rasmusvillemoes.dk>.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Thu, 16 Apr 2015 08:58:54 +0000 (04:58 -0400)]
strbuf_getwholeline: avoid calling strbuf_grow
As with the recent speedup to strbuf_addch, we can avoid
calling strbuf_grow() in a tight loop of single-character
adds by instead checking strbuf_avail.
Note that we would instead call strbuf_addch directly here,
but it does more work than necessary: it will NUL-terminate
the result for each character read. Instead, in this loop we
read the characters one by one and then add the terminator
manually at the end.
Running "git rev-parse refs/heads/does-not-exist" on a repo
with an extremely large (1.6GB) packed-refs file went from
(best-of-5):
real 0m10.948s
user 0m10.548s
sys 0m0.412s
to:
real 0m8.601s
user 0m8.084s
sys 0m0.524s
for a wall-clock speedup of 21%.
Helped-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Thu, 16 Apr 2015 08:53:56 +0000 (04:53 -0400)]
strbuf_addch: avoid calling strbuf_grow
We mark strbuf_addch as inline, because we expect it may be
called from a tight loop. However, the first thing it does
is call the non-inline strbuf_grow(), which can handle
arbitrary-sized growth. Since we know that we only need a
single character, we can use the inline strbuf_avail() to
quickly check whether we need to grow at all.
Our check is redundant when we do call strbuf_grow(), but
that's OK. The common case is that we avoid calling it at
all, and we have made that case faster.
Jeff King [Thu, 16 Apr 2015 08:51:18 +0000 (04:51 -0400)]
config: use getc_unlocked when reading from file
We read config files character-by-character from a stdio
handle using fgetc(). This incurs significant locking
overhead, even though we know that only one thread can
possibly access the handle. We can speed this up by taking
the lock ourselves, and then using getc_unlocked to read
each character.
for a savings of 39%. Most config files are not this big,
but the savings should be proportional to the size of the
file (i.e., we always save 39%, just of a much smaller
number).
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Thu, 16 Apr 2015 08:49:06 +0000 (04:49 -0400)]
strbuf_getwholeline: use getc_unlocked
strbuf_getwholeline calls getc in a tight loop. On modern
libc implementations, the stdio code locks the handle for
every operation, which means we are paying a significant
overhead. We can get around this by locking the handle for
the whole loop and using the unlocked variant.
Running "git rev-parse refs/heads/does-not-exist" on a repo
with an extremely large (1.6GB) packed-refs file went from:
real 0m18.900s
user 0m18.472s
sys 0m0.448s
to:
real 0m10.953s
user 0m10.384s
sys 0m0.580s
for a wall-clock speedup of 42%. All times are best-of-3,
and done on a glibc 2.19 system.
Note that we call into strbuf_grow while holding the lock.
It's possible for that function to call other stdio
functions (e.g., printing to stderr when dying due to malloc
error); however, the POSIX.1-2001 definition of flockfile
makes it clear that the locks are per-handle, so we are fine
unless somebody else tries to read from our same handle.
This doesn't ever happen in the current code, and is
unlikely to be added in the future (we would have to do
something exotic like add a die_routine that tried to read
from stdin).
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Thu, 16 Apr 2015 08:48:45 +0000 (04:48 -0400)]
git-compat-util: add fallbacks for unlocked stdio
POSIX.1-2001 specifies some functions for optimizing the
locking out of tight getc() loops. Not all systems are
POSIX, though, and even not all POSIX systems are required
to implement these functions. We can check for the
feature-test macro to see if they are available, and if not,
provide a noop implementation.
There's no Makefile knob here, because we should just detect
this automatically. If there are very bizarre systems, we
may need to add one, but it's not clear yet in which
direction:
1. If a system defines _POSIX_THREAD_SAFE_FUNCTIONS but
these functions are missing or broken, we would want a
knob to manually turn them off.
2. If a system has these functions but does not define
_POSIX_THREAD_SAFE_FUNCTIONS, we would want a knob to
manually turn them on.
We can add such a knob when we find a real-world system that
matches this.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Thu, 16 Apr 2015 08:48:23 +0000 (04:48 -0400)]
strbuf_getwholeline: use getc macro
strbuf_getwholeline calls fgetc in a tight loop. Using the
getc form, which can be implemented as a macro, should be
faster (and we do not care about it evaluating our argument
twice, as we just have a plain variable).
On my glibc system, running "git rev-parse
refs/heads/does-not-exist" on a file with an extremely large
(1.6GB) packed-refs file went from (best of 3 runs):
real 0m19.383s
user 0m18.876s
sys 0m0.528s
to:
real 0m18.900s
user 0m18.472s
sys 0m0.448s
for a wall-clock speedup of 2.5%.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano [Thu, 26 Mar 2015 18:57:13 +0000 (11:57 -0700)]
Merge branch 'jk/test-chain-lint'
People often forget to chain the commands in their test together
with &&, leaving a failure from an earlier command in the test go
unnoticed. The new GIT_TEST_CHAIN_LINT mechanism allows you to
catch such a mistake more easily.
* jk/test-chain-lint: (36 commits)
t9001: drop save_confirm helper
t0020: use test_* helpers instead of hand-rolled messages
t: simplify loop exit-code status variables
t: fix some trivial cases of ignored exit codes in loops
t7701: fix ignored exit code inside loop
t3305: fix ignored exit code inside loop
t0020: fix ignored exit code inside loops
perf-lib: fix ignored exit code inside loop
t6039: fix broken && chain
t9158, t9161: fix broken &&-chain in git-svn tests
t9104: fix test for following larger parents
t4104: drop hand-rolled error reporting
t0005: fix broken &&-chains
t7004: fix embedded single-quotes
t0050: appease --chain-lint
t9001: use test_when_finished
t4117: use modern test_* helpers
t6034: use modern test_* helpers
t1301: use modern test_* helpers
t0020: use modern test_* helpers
...
Junio C Hamano [Wed, 25 Mar 2015 19:54:27 +0000 (12:54 -0700)]
Merge branch 'jk/run-command-capture'
The run-command interface was easy to abuse and make a pipe for us
to read from the process, wait for the process to finish and then
attempt to read its output, which is a pattern that lead to a
deadlock. Fix such uses by introducing a helper to do this
correctly (i.e. we need to read first and then wait the process to
finish) and also add code to prevent such abuse in the run-command
helper.
* jk/run-command-capture:
run-command: forbid using run_command with piped output
trailer: use capture_command
submodule: use capture_command
wt-status: use capture_command
run-command: introduce capture_command helper
wt_status: fix signedness mismatch in strbuf_read call
wt-status: don't flush before running "submodule status"
Junio C Hamano [Wed, 25 Mar 2015 19:54:26 +0000 (12:54 -0700)]
Merge branch 'jk/prune-with-corrupt-refs'
"git prune" used to largely ignore broken refs when deciding which
objects are still being used, which could spread an existing small
damage and make it a larger one.
* jk/prune-with-corrupt-refs:
refs.c: drop curate_packed_refs
repack: turn on "ref paranoia" when doing a destructive repack
prune: turn on ref_paranoia flag
refs: introduce a "ref paranoia" flag
t5312: test object deletion code paths in a corrupted repository
The split-index mode introduced at v2.3.0-rc0~41 was broken in the
codepath to protect us against a broken reimplementation of Git
that writes an invalid index with duplicated index entries, etc.
* tg/fix-check-order-with-split-index:
read-cache: fix reading of split index
Junio C Hamano [Wed, 25 Mar 2015 19:54:25 +0000 (12:54 -0700)]
Merge branch 'jk/fetch-pack'
"git fetch" that fetches a commit using the allow-tip-sha1-in-want
extension could have failed to fetch all the requested refs.
* jk/fetch-pack:
fetch-pack: remove dead assignment to ref->new_sha1
fetch_refs_via_pack: free extra copy of refs
filter_ref: make a copy of extra "sought" entries
filter_ref: avoid overwriting ref->old_sha1 with garbage
Junio C Hamano [Wed, 25 Mar 2015 19:54:24 +0000 (12:54 -0700)]
Merge branch 'jk/cleanup-failed-clone'
An failure early in the "git clone" that started creating the
working tree and repository could have resulted in some directories
and files left without getting cleaned up.
* jk/cleanup-failed-clone:
clone: drop period from end of die_errno message
clone: initialize atexit cleanup handler earlier
Junio C Hamano [Wed, 25 Mar 2015 19:54:22 +0000 (12:54 -0700)]
Merge branch 'dj/log-graph-with-no-walk'
"git log --graph --no-walk A B..." is a otcnflicting request that
asks nonsense; no-walk tells us show discrete points in the
history, while graph asks to draw connections between these
discrete points. Forbid the combination.
* dj/log-graph-with-no-walk:
revision: forbid combining --graph and --no-walk
Junio C Hamano [Wed, 25 Mar 2015 19:54:21 +0000 (12:54 -0700)]
Merge branch 'kd/rev-list-bisect-first-parent'
"git rev-list --bisect --first-parent" does not work (yet) and can
even cause SEGV; forbid it. "git log --bisect --first-parent"
would not be useful until "git bisect --first-parent" materializes,
so it is also forbidden for now.
* kd/rev-list-bisect-first-parent:
rev-list: refuse --first-parent combined with --bisect
Jeff King [Wed, 25 Mar 2015 05:32:20 +0000 (01:32 -0400)]
t9001: drop save_confirm helper
The idea of this helper is that we want to save the current
value of a config variable and then restore it again after
the test completes. However, there's no point in actually
saving the value; it should always be restored to the string
"never" (which you can confirm by instrumenting
save_confirm to print the value it finds).
Let's just replace it with a single test_when_finished call.
Suggested-by: SZEDER Gábor <szeder@ira.uka.de> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Wed, 25 Mar 2015 05:31:41 +0000 (01:31 -0400)]
t0020: use test_* helpers instead of hand-rolled messages
These tests are not wrong, but it is much shorter and more
idiomatic to say "verbose" or "test_must_fail" rather than
printing our own messages on failure. Likewise, there is no
need to say "happy" at the end of a test; the test suite
takes care of that.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Wed, 25 Mar 2015 05:30:17 +0000 (01:30 -0400)]
t: simplify loop exit-code status variables
Since shell loops may drop the exit code of failed commands
inside the loop, some tests try to keep track of the status
by setting a variable. This can end up cumbersome and hard
to read; it is much simpler to just exit directly from the
loop using "return 1" (since each case is either in a helper
function or inside a test snippet).
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Wed, 25 Mar 2015 05:29:52 +0000 (01:29 -0400)]
t: fix some trivial cases of ignored exit codes in loops
These are all cases where we do a setup step of the form:
for i in $foo; do
set_up $i || break
done &&
more_setup
would not notice a failure in set_up (because break always
returns a 0 exit code). These are just setup steps that we
do not expect to fail, but it does not hurt to be defensive.
Most can be fixed by converting the "break" to a "return 1"
(since we eval our tests inside a function for just this
purpose). A few of the loops are inside subshells, so we can
use just "exit 1" to break out of the subshell. And a few
can actually be made shorter by just unrolling the loop.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Wed, 25 Mar 2015 05:29:10 +0000 (01:29 -0400)]
t7701: fix ignored exit code inside loop
When checking a list of file mtimes, we use a loop and break
out early from the loop if any entry does not match.
However, the exit code of a loop exited via break is always
0, meaning that the test will fail to notice we had a
mismatch. Since the loop is inside a function, we can fix
this by doing an early "return 1".
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Wed, 25 Mar 2015 05:28:57 +0000 (01:28 -0400)]
t3305: fix ignored exit code inside loop
When we test deleting notes, we run "git notes remove" in a
loop. However, the exit value of the loop will only reflect
the final note we process. We should break out of the loop
with a failing exit code as soon as we see a problem.
Note that we can call "exit 1" here without explicitly
creating a subshell, because the while loop on the
right-hand side of a pipe executes in its own implicit
subshell.
Note also that the "break" above does not suffer the same
problem; it is meant to exit the loop early at a certain
number of iterations. We can bump it into the conditional of
the loop to make this more obvious.
Signed-off-by: Jeff King <peff@peff.net> Acked-by: Johan Herland <johan@herland.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Wed, 25 Mar 2015 05:28:44 +0000 (01:28 -0400)]
t0020: fix ignored exit code inside loops
A loop like:
for f in one two; do
something $f ||
break
done
will correctly break out of the loop when we see a failure
of one item, but the resulting exit code will always be
zero. We can fix that by putting the loop into a function or
subshell, but in this case it is simpler still to just
unroll the loop. We do add a helper function, which
hopefully makes the end result even more readable (in
addition to being shorter).
Reported-by: SZEDER Gábor <szeder@ira.uka.de> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Wed, 25 Mar 2015 05:25:55 +0000 (01:25 -0400)]
perf-lib: fix ignored exit code inside loop
When copying the test repository, we try to detect whether
the copy succeeded. However, most of the heavy lifting is
done inside a for loop, where our "break" will lose the exit
code of the failing "cp". We can take advantage of the fact
that we are in a subshell, and just "exit 1" to break out
with a code.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano [Tue, 24 Mar 2015 23:10:37 +0000 (16:10 -0700)]
Merge branch 'master' of git://ozlabs.org/~paulus/gitk
* 'master' of git://ozlabs.org/~paulus/gitk:
gitk: Update .po files
gitk: l10n: Add Catalan translation
gitk: Fix typo in Russian translation
gitk: Remove tcl-format flag from a message that shouldn't have it
gitk: Pass --invert-grep option down to "git log"
gitk: Synchronize config file writes
gitk: Report errors in saving config file
gitk: Only write changed configuration variables
gitk: Enable mouse horizontal scrolling in diff pane
gitk: Default wrcomcmd to use --pretty=email
Junio C Hamano [Tue, 24 Mar 2015 21:12:10 +0000 (14:12 -0700)]
report_path_error(): move to dir.c
The expected call sequence is for the caller to use match_pathspec()
repeatedly on a set of pathspecs, accumulating the "hits" in a
separate array, and then call this function to diagnose a pathspec
that never matched anything, as that can indicate a typo from the
command line, e.g. "git commit Maekfile".
Many builtin commands use this function from builtin/ls-files.c,
which is not a very healthy arrangement. ls-files might have been
the first command to feel the need for such a helper, but the need
is shared by everybody who uses the "match and then report" pattern.
Move it to dir.c where match_pathspec() is defined.
At the first look, a user may think the default version is "23". Even
with UNIX background, there's no reference anywhere close that may
indicate this is glob or regex.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
The code that reads from the ctags file in the completion script
(in contrib/) did not spell ${param/pattern/string} substitution
correctly, which happened to work with bash but not with zsh.
* js/completion-ctags-pattern-substitution-fix:
contrib/completion: escape the forward slash in __git_match_ctag
Junio C Hamano [Mon, 23 Mar 2015 18:28:13 +0000 (11:28 -0700)]
Merge branch 'jk/push-config'
Restructure "git push" codepath to make it easier to add new
configuration bits and then add push.followTags configuration that
turns --follow-tags option on by default.
* jk/push-config:
push: allow --follow-tags to be set by config push.followTags
cmd_push: pass "flags" pointer to config callback
cmd_push: set "atomic" bit directly
git_push_config: drop cargo-culted wt_status pointer
Junio C Hamano [Mon, 23 Mar 2015 18:28:10 +0000 (11:28 -0700)]
Merge branch 'jk/test-annoyances'
Test fixes.
* jk/test-annoyances:
t5551: make EXPENSIVE test cheaper
t5541: move run_with_cmdline_limit to test-lib.sh
t: pass GIT_TRACE through Apache
t: redirect stderr GIT_TRACE to descriptor 4
t: translate SIGINT to an exit
Junio C Hamano [Mon, 23 Mar 2015 18:23:28 +0000 (11:23 -0700)]
Merge branch 'jc/decorate-leaky-separator-color' into maint
"git log --decorate" did not reset colors correctly around the
branch names.
* jc/decorate-leaky-separator-color:
log --decorate: do not leak "commit" color into the next item
Documentation/config.txt: simplify boolean description in the syntax section
Documentation/config.txt: describe 'color' value type in the "Values" section
Documentation/config.txt: have a separate "Values" section
Documentation/config.txt: describe the structure first and then meaning
Documentation/config.txt: explain multi-valued variables once
Documentation/config.txt: avoid unnecessary negation
Junio C Hamano [Mon, 23 Mar 2015 18:23:22 +0000 (11:23 -0700)]
Merge branch 'km/imap-send-libcurl-options' into maint
"git imap-send" learned to optionally talk with an IMAP server via
libcURL; because there is no other option when Git is built with
NO_OPENSSL option, use that codepath by default under such
configuration.
* km/imap-send-libcurl-options:
imap-send: use cURL automatically when NO_OPENSSL defined
Junio C Hamano [Mon, 23 Mar 2015 18:23:17 +0000 (11:23 -0700)]
Merge branch 'es/rebase-i-count-todo' into maint
"git rebase -i" recently started to include the number of
commits in the insn sheet to be processed, but on a platform
that prepends leading whitespaces to "wc -l" output, the numbers
are shown with extra whitespaces that aren't necessary.
Junio C Hamano [Mon, 23 Mar 2015 18:23:12 +0000 (11:23 -0700)]
Merge branch 'tb/connect-ipv6-parse-fix' into maint
We did not parse username followed by literal IPv6 address in SSH
transport URLs, e.g. ssh://user@[2001:db8::1]:22/repo.git
correctly.
* tb/connect-ipv6-parse-fix:
t5500: show user name and host in diag-url
t5601: add more test cases for IPV6
connect.c: allow ssh://user@[2001:db8::1]/repo.git
Jeff King [Mon, 23 Mar 2015 03:54:05 +0000 (23:54 -0400)]
run-command: forbid using run_command with piped output
Because run_command both spawns and wait()s for the command
before returning control to the caller, any reads from the
pipes we open must necessarily happen after wait() returns.
This can lead to deadlock, as the child process may block
on writing to us while we are blocked waiting for it to
exit.
Worse, it only happens when the child fills the pipe
buffer, which means that the problem may come and go
depending on the platform and the size of the output
produced by the child.
Let's detect and flag this dangerous construct so that we
can catch potential bugs early in the test suite rather than
having them happen in the field.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Mon, 23 Mar 2015 03:54:00 +0000 (23:54 -0400)]
trailer: use capture_command
When we read from a trailer.*.command sub-program, the
current code uses run_command followed by a pipe read, which
can result in deadlock (though in practice you would have to
have a large trailer for this to be a problem). The current
code also leaks the file descriptor for the pipe to the
sub-command.
Instead, let's use capture_command, which makes this simpler
(and we can get rid of our custom helper).
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Mon, 23 Mar 2015 03:53:56 +0000 (23:53 -0400)]
submodule: use capture_command
In is_submodule_commit_present, we call run_command followed
by a pipe read, which is prone to deadlock. It is unlikely
to happen in this case, as rev-list should never produce
more than a single line of output, but it does not hurt to
avoid an anti-pattern (and using the helper simplifies the
setup and cleanup).
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Mon, 23 Mar 2015 03:53:52 +0000 (23:53 -0400)]
wt-status: use capture_command
When we spawn "git submodule status" to read its output, we
use run_command() followed by strbuf_read() read from the
pipe. This can deadlock if the subprocess output is larger
than the system pipe buffer.
Furthermore, if start_command() fails, we'll try to read
from a bogus descriptor (probably "-1" or a descriptor we
just closed, but it is a bad idea for us to make assumptions
about how start_command implements its error handling). And
if start_command succeeds, we leak the file descriptor for
the pipe to the child.
All of these can be solved by using the capture_command
helper.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
But that is not quite right either. We are examining cmd.out
and running finish_command whether start_command succeeded
or not, which is wrong. Moreover, these snippets do not do
any error handling. If our read() fails, we must make sure
to still call finish_command (to reap the child process).
And both snippets failed to close the cmd.out descriptor,
which they must do (provided start_command succeeded).
Let's introduce a run-command helper that can make this a
bit simpler for callers to get right.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
SZEDER Gábor [Sun, 22 Mar 2015 12:03:11 +0000 (13:03 +0100)]
completion: use __gitcomp_nl() for completing refs
We do that almost everywhere, because it's faster for large number of
refs, see a31e62629 (completion: optimize refs completion, 2011-10-15).
These were the last two places where we still used __gitcomp() for
completing refs.
Signed-off-by: SZEDER Gábor <szeder@ira.uka.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Sun, 22 Mar 2015 10:00:32 +0000 (06:00 -0400)]
wt_status: fix signedness mismatch in strbuf_read call
We call strbuf_read(), and want to know whether we got any
output. To do so, we assign the result to a size_t, and
check whether it is non-zero.
But strbuf_read returns a signed ssize_t. If it encounters
an error, it will return -1, and we'll end up treating this
the same as if we had gotten output. Instead, we can just
check whether our buffer has anything in it (which is what
we care about anyway, and is the same thing since we know
the buffer was empty to begin with).
Note that the "len" variable actually has two roles in this
function. Now that we've eliminated the first, we can push the
declaration closer to the point of use for the second one.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Sun, 22 Mar 2015 10:00:01 +0000 (06:00 -0400)]
wt-status: don't flush before running "submodule status"
This is a holdover from the original implementation in ac8d5af (builtin-status: submodule summary support,
2008-04-12), which just had the sub-process output to our
descriptor; we had to make sure we had flushed any data that
we produced before it started writing.
Since 3ba7407 (submodule summary: ignore --for-status
option, 2013-09-06), however, we pipe the sub-process output
back to ourselves. So there's no longer any need to flush
(it does not hurt, but it may leave readers wondering why we
do it).
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Thomas Gummerer [Fri, 20 Mar 2015 21:43:14 +0000 (22:43 +0100)]
read-cache: fix reading of split index
The split index extension uses ewah bitmaps to mark index entries as
deleted, instead of removing them from the index directly. This can
result in an on-disk index, in which entries of stage #0 and higher
stages appear, which are removed later when the index bases are merged.
15999d0 read_index_from(): catch out of order entries when reading an
index file introduces a check which checks if the entries are in order
after each index entry is read in do_read_index. This check may however
fail when a split index is read.
Fix this by moving checking the index after we know there is no split
index or after the split index bases are successfully merged instead.
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano [Fri, 20 Mar 2015 20:50:51 +0000 (13:50 -0700)]
Merge branch 'jc/decorate-leaky-separator-color'
"git log --decorate" did not reset colors correctly around the
branch names.
* jc/decorate-leaky-separator-color:
log --decorate: do not leak "commit" color into the next item
Documentation/config.txt: simplify boolean description in the syntax section
Documentation/config.txt: describe 'color' value type in the "Values" section
Documentation/config.txt: have a separate "Values" section
Documentation/config.txt: describe the structure first and then meaning
Documentation/config.txt: explain multi-valued variables once
Documentation/config.txt: avoid unnecessary negation
Junio C Hamano [Fri, 20 Mar 2015 20:11:50 +0000 (13:11 -0700)]
Merge branch 'km/imap-send-libcurl-options'
"git imap-send" learned to optionally talk with an IMAP server via
libcURL; because there is no other option when Git is built with
NO_OPENSSL option, use that codepath by default under such
configuration.
* km/imap-send-libcurl-options:
imap-send: use cURL automatically when NO_OPENSSL defined
Junio C Hamano [Fri, 20 Mar 2015 20:11:48 +0000 (13:11 -0700)]
Merge branch 'km/bsd-shells'
Portability fixes and workarounds for shell scripts have been added
to help BSD-derived systems.
* km/bsd-shells:
t5528: do not fail with FreeBSD shell
help.c: use SHELL_PATH instead of hard-coded "/bin/sh"
git-compat-util.h: move SHELL_PATH default into header
git-instaweb: use @SHELL_PATH@ instead of /bin/sh
git-instaweb: allow running in a working tree subdirectory
Junio C Hamano [Fri, 20 Mar 2015 20:11:46 +0000 (13:11 -0700)]
Merge branch 'mg/detached-head-report'
"git branch" on a detached HEAD always said "(detached from xyz)",
even when "git status" would report "detached at xyz". The HEAD is
actually at xyz and haven't been moved since it was detached in
such a case, but the user cannot read what the current value of
HEAD is when "detached from" is used.
* mg/detached-head-report:
branch: name detached HEAD analogous to status
wt-status: refactor detached HEAD analysis
Jeff King [Fri, 20 Mar 2015 18:43:17 +0000 (14:43 -0400)]
refs.c: drop curate_packed_refs
When we delete a ref, we have to rewrite the entire
packed-refs file. We take this opportunity to "curate" the
packed-refs file and drop any entries that are crufty or
broken.
Dropping broken entries (e.g., with bogus names, or ones
that point to missing objects) is actively a bad idea, as it
means that we lose any notion that the data was there in the
first place. Aside from the general hackiness that we might
lose any information about ref "foo" while deleting an
unrelated ref "bar", this may seriously hamper any attempts
by the user at recovering from the corruption in "foo".
They will lose the sha1 and name of "foo"; the exact pointer
may still be useful even if they recover missing objects
from a different copy of the repository. But worse, once the
ref is gone, there is no trace of the corruption. A
follow-up "git prune" may delete objects, even though it
would otherwise bail when seeing corruption.
We could just drop the "broken" bits from
curate_packed_refs, and continue to drop the "crufty" bits:
refs whose loose counterpart exists in the filesystem. This
is not wrong to do, and it does have the advantage that we
may write out a slightly smaller packed-refs file. But it
has two disadvantages:
1. It is a potential source of races or mistakes with
respect to these refs that are otherwise unrelated to
the operation. To my knowledge, there aren't any active
problems in this area, but it seems like an unnecessary
risk.
2. We have to spend time looking up the matching loose
refs for every item in the packed-refs file. If you
have a large number of packed refs that do not change,
that outweighs the benefit from writing out a smaller
packed-refs file (it doesn't get smaller, and you do a
bunch of directory traversal to find that out).
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Fri, 20 Mar 2015 18:43:13 +0000 (14:43 -0400)]
repack: turn on "ref paranoia" when doing a destructive repack
If we are repacking with "-ad", we will drop any unreachable
objects. Likewise, using "-Ad --unpack-unreachable=<time>"
will drop any old, unreachable objects. In these cases, we
want to make sure the reachability we compute with "--all"
is complete. We can do this by passing GIT_REF_PARANOIA=1 in
the environment to pack-objects.
Note that "-Ad" is safe already, because it only loosens
unreachable objects. It is up to "git prune" to avoid
deleting them.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Fri, 20 Mar 2015 18:43:09 +0000 (14:43 -0400)]
prune: turn on ref_paranoia flag
Prune should know about broken objects at the tips of refs,
so that we can feed them to our traversal rather than
ignoring them. It's better for us to abort the operation on
the broken object than it is to start deleting objects with
an incomplete view of the reachability namespace.
Note that for missing objects, aborting is the best we can
do. For a badly-named ref, we technically could use its sha1
as a reachability tip. However, the iteration code just
feeds us a null sha1, so there would be a reasonable amount
of code involved to pass down our wishes. It's not really
worth trying to do better, because this is a case that
should happen extremely rarely, and the message we provide:
fatal: unable to parse object: refs/heads/bogus:name
is probably enough to point the user in the right direction.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Fri, 20 Mar 2015 18:43:06 +0000 (14:43 -0400)]
refs: introduce a "ref paranoia" flag
Most operations that iterate over refs are happy to ignore
broken cruft. However, some operations should be performed
with knowledge of these broken refs, because it is better
for the operation to choke on a missing object than it is to
silently pretend that the ref did not exist (e.g., if we are
computing the set of reachable tips in order to prune
objects).
These processes could just call for_each_rawref, except that
ref iteration is often hidden behind other interfaces. For
instance, for a destructive "repack -ad", we would have to
inform "pack-objects" that we are destructive, and then it
would in turn have to tell the revision code that our
"--all" should include broken refs.
It's much simpler to just set a global for "dangerous"
operations that includes broken refs in all iterations.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Fri, 20 Mar 2015 18:43:02 +0000 (14:43 -0400)]
t5312: test object deletion code paths in a corrupted repository
When we are doing a destructive operation like "git prune",
we want to be extra careful that the set of reachable tips
we compute is valid. If there is any corruption or oddity,
we are better off aborting the operation and letting the
user figure things out rather than plowing ahead and
possibly deleting some data that cannot be recovered.
The tests here include:
1. Pruning objects mentioned only be refs with invalid
names. This used to abort prior to d0f810f (refs.c:
allow listing and deleting badly named refs,
2014-09-03), but since then we silently ignore the tip.
Likewise, we test repacking that can drop objects
(either "-ad", which drops anything unreachable,
or "-Ad --unpack-unreachable=<time>", which tries to
optimize out a loose object write that would be
directly pruned).
2. Pruning objects when some refs point to missing
objects. We don't know whether any dangling objects
would have been reachable from the missing objects. We
are better to keep them around, as they are better than
nothing for helping the user recover history.
3. Packed refs that point to missing objects can sometimes
be dropped. By itself, this is more of an annoyance
(you do not have the object anyway; even if you can
recover it from elsewhere, all you are losing is a
placeholder for your state at the time of corruption).
But coupled with (2), if we drop the ref and then go
on to prune, we may lose unrecoverable objects.
Note that we use test_might_fail for some of the operations.
In some cases, it would be appropriate to abort the
operation, and in others, it might be acceptable to continue
but taking the information into account. The tests don't
care either way, and check only for data loss.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Thomas Gummerer [Fri, 20 Mar 2015 18:20:30 +0000 (19:20 +0100)]
t1700: make test pass with index-v4
The different index versions have different sha-1 checksums. Those
checksums are checked in t1700, which makes it fail when the test suite
is run with TEST_GIT_INDEX_VERSION=4. Fix it.
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Michael J Gruber [Fri, 20 Mar 2015 14:32:55 +0000 (15:32 +0100)]
t9158, t9161: fix broken &&-chain in git-svn tests
All of these cases are moderate since they would most probably not
lead to missed failing tests; either they would fail otherwise, or
fail a rm in test_when_finished only.
Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Michael J Gruber [Fri, 20 Mar 2015 14:32:56 +0000 (15:32 +0100)]
t9104: fix test for following larger parents
This test is special for several reasons:
It ends with a "true" statement, which should be a no-op.
It is not because the &&-chain is broken right before it.
Also, looking at what the test intended to test according to 7f578c5 (git-svn: --follow-parent now works on sub-directories of larger
branches, 2007-01-24)
it is not clear how it would achieve that with the given steps.
Amend the test to include the second svn id to be tested for, and
change the tested refs to the ones which are to be expected, and which
make the test pass.
Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Fri, 20 Mar 2015 10:13:36 +0000 (06:13 -0400)]
t4104: drop hand-rolled error reporting
This use of "||" fools --chain-lint into thinking the
&&-chain is broken (and indeed, it is somewhat broken; a
failure of update-index in these tests would show the patch
file, even if we never got to the part of the test where we
fed the patch to git-apply).
The extra blocks were there to include more debugging
output, but it hardly seems worth it; the user should know
which command failed (because git-apply will produce error
messages) and can look in the trash directory themselves.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Fri, 20 Mar 2015 10:13:29 +0000 (06:13 -0400)]
t7004: fix embedded single-quotes
This test uses single quotes inside the single-quoted test
snippet, which effectively makes the contents unquoted.
Since they don't need quoted anyway, this isn't a problem,
but let's switch them to double-quotes to make it more
obviously correct.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Fri, 20 Mar 2015 10:13:25 +0000 (06:13 -0400)]
t0050: appease --chain-lint
Some of the symlink tests check an either-or case using the
"||". This is not wrong, but fools --chain-lint into
thinking the &&-chain is broken (in fact, there is no &&
chain here).
We can solve this by wrapping the "||" inside a {} block.
This is a bit more verbose, but this construct is rare, and
the {} block helps call attention to it.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Fri, 20 Mar 2015 10:13:22 +0000 (06:13 -0400)]
t9001: use test_when_finished
The confirmation tests in t9001 all save the value of
sendemail.confirm, do something to it, then restore it at
the end, in a way that breaks the &&-chain (they are not
wrong, because they save the $? value, but it fools
--chain-lint).
Instead, they can all use test_when_finished, and we can
even make the code simpler by factoring out the shared
lines.
Note that we can _almost_ use test_config here, except that:
1. We do not restore the config with test_unconfig, but by
setting it back to some prior value.
2. We are not always setting a config variable. Sometimes
the change to be undone is unsetting it entirely.
We could teach test_config to handle these cases, but it's
not worth the complexity for a single call-site.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Fri, 20 Mar 2015 10:13:18 +0000 (06:13 -0400)]
t4117: use modern test_* helpers
We can use test_must_fail and test_path_* to avoid some
hand-rolled if statements. This makes the code shorter, and
makes it more obvious when we are breaking the &&-chain.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Fri, 20 Mar 2015 10:13:15 +0000 (06:13 -0400)]
t6034: use modern test_* helpers
These say roughly the same thing as the hand-rolled
messages. We do lose the "merge did not complete" debug
message, but merge and write-tree are prefectly capable of
writing useful error messages when they fail.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Fri, 20 Mar 2015 10:13:08 +0000 (06:13 -0400)]
t0020: use modern test_* helpers
This test contains a lot of hand-rolled messages to show
when the test fails. We can omit most of these by using
"verbose" and "test_must_fail". A few of them are for
update-index, but we can assume it produces reasonable error
messages when it fails.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Fri, 20 Mar 2015 10:13:05 +0000 (06:13 -0400)]
t6030: use modern test_* helpers
We can get rid of a lot of hand-rolled error messages by
using test_must_fail and test_expect_code. The existing code
was careful to use "|| return 1" when breaking the
&&-chain, but it did fool --chain-lint; the new code is more
idiomatic.
We also add some uses of test_when_finished, which is less
cryptic and more robust than putting code at the end of a
test. In two cases we run "git bisect reset" from a
subshell, which is a problem for test_when_finished (it
would not run). However, in both of these cases, we are
performing the tests in one-off sub-repos, so we do not need
to clean up at all (and in fact it is nicer not to if the
user wants to inspect the trash directory after a failure).
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Fri, 20 Mar 2015 10:13:01 +0000 (06:13 -0400)]
t9502: fix &&-chain breakage
This script misses a trivial &&-chain in one of its tests,
but it also has a weird reverse: it includes an &&-chain
outside of any test_expect block! This "cat" should never
fail, but if it did, we would not notice, as it would cause
us to skip the follow-on test entirely (which does not
appear intentional; there are many later tests which rely on
this cat).
Let's instead move the setup into its own test_expect_success
block, which is the standard practice nowadays.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Fri, 20 Mar 2015 10:12:55 +0000 (06:12 -0400)]
t7201: fix &&-chain breakage
One of these breakages is in setup, but one is more severe
and may miss a real test failure. These are pulled out from
the rest, though, because we also clean up a few other
anachronisms. The most interesting is the use of this
here-doc construct:
(cat >... <<EOF
...
EOF
) &&
It looks like an attempt to make the &&-chaining more
natural by letting it come at the end of the here-doc. But
the extra sub-shell is so non-idiomatic (plus the lack of
"<<-") that it ends up confusing.
Since these are just using a single line, we can accomplish
the same thing with a single printf (which also makes the
use of tab more obvious than the verbatim whitespace).
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
projects / thirdparty / git.git / log
