Eric Wong [Wed, 29 Oct 2014 19:55:02 +0000 (19:55 +0000)]
git-svn: prepare SVN::Ra config pieces once
Memoizing these initialization functions saves some memory for
long fetches which require scanning many unwanted revisions
before any wanted revisions happen.
Eric Wong [Mon, 27 Oct 2014 01:39:39 +0000 (01:39 +0000)]
git-svn: disable _rev_list memoization
This memoization appears unneeded as the check_cherry_pick2 cache is
in front of it does enough.
With this change applied, importing from local svn+ssh and http copies
of the R repo[1] takes only 2:00 (2 hours) on my system and the git-svn
process never uses more than 60MB RSS on my x86-64 GNU/Linux system[2].
This 60M measurement is only for the git-svn Perl process itself and
does not include memory used by git subprocesses accessing large packs
(subprocess memory usage _is_ measured by my time(1) tool).
Before this change, an import took longer (2:20) on svn+ssh:// but
git-svn used around 240MB during the imports. Worse yet, git-svn
ballooned to over 400M when writing out the cache to the filesystem.
I also tried removing memoization for `has_no_changes', too, but a
local copy of the R repository(*) was not close to finishing within
10 hours on my system.
[1] http://svn.r-project.org/R
[2] file:// repos causes libsvn to use more memory internally
Signed-off-by: Eric Wong <normalperson@yhbt.net> Cc: Hin-Tak Leung <htl10@users.sourceforge.net>
Eric Wong [Fri, 24 Oct 2014 22:53:52 +0000 (22:53 +0000)]
git-svn: reload RA every log-window-size
Despite attempting to use local memory pools everywhere we can,
(including our call to SVN::Ra::do_update and all subsequent reporter
calls), there does not appear to be a way to force the Git::SVN::Fetcher
callbacks to use a pool other than the per-SVN::Ra pool.
Git::SVN::Fetcher ends up using the main RA pool which grows
monotonically in size for the lifetime of the RA object.
Thus the only way to free that memory appears to be to destroy and
recreate the RA connection for at every --log-window-size interval.
This reduces memory usage over the course of fetching 10K revisions
using a test repository created with the script at the end of this
commit message.
As reported by time(1) on my x86-64 system:
before: 54024k
after: 28680k
Unfortunately, there remains some yet-to-be-tracked-down slow memory
growth which would be evident as the `nr' parameter increases in
the repository generation script:
-----------------------------8<------------------------------
set -e
tmp=$(mktemp -d svntestrepo-XXXXXXXX)
svnadmin create "$tmp"
repo=file://"$(cd $tmp && pwd)"
svn co "$repo" "$tmp/wd"
cd "$tmp/wd"
if ! test -f a
then
> a
svn add a
svn commit -m 'A'
fi
nr=10000
while test $nr -gt 0
do
echo $nr > a
svn commit -q -m A
nr=$((nr - 1))
done
echo "repository created in $repo"
-----------------------------8<------------------------------
Advertise that the svn-remote.<name>.pushurl config key allows specifying
the commit URL for the entire SVN repository in the documentation of the
git svn dcommit command.
Signed-off-by: Sveinung Kvilhaugsvik <sveinung84@users.sourceforge.net> Signed-off-by: Eric Wong <normalperson@yhbt.net>
Eric Wong [Tue, 21 Oct 2014 06:23:22 +0000 (06:23 +0000)]
git-svn: remove mergeinfo rev caching
This should further reduce memory usage from the new mergeinfo
speedups without hurting performance too much, assuming
reasonable latency to the SVN server.
Cc: Hin-Tak Leung <htl10@users.sourceforge.net> Suggested-by: Jakob Stoklund Olesen <stoklund@2pi.dk> Signed-off-by: Eric Wong <normalperson@yhbt.net>
Eric Wong [Mon, 20 Oct 2014 01:02:53 +0000 (01:02 +0000)]
git-svn: cache only mergeinfo revisions
This should reduce excessive memory usage from the new mergeinfo
caches without hurting performance too much, assuming reasonable
latency to the SVN server.
Cc: Hin-Tak Leung <htl10@users.sourceforge.net> Suggested-by: Jakob Stoklund Olesen <stoklund@2pi.dk> Signed-off-by: Eric Wong <normalperson@yhbt.net>
Eric Wong [Sun, 19 Oct 2014 04:08:31 +0000 (04:08 +0000)]
git-svn: reduce check_cherry_pick cache overhead
We do not need to store entire lists of commits, only the
number of incomplete and the first commit for reference.
This reduces the amount of data we need to store in memory
and on disk stores.
git-svn: only look at the root path for svn:mergeinfo
Subversion can put mergeinfo on any sub-directory to track cherry-picks.
Since cherry-picks are not represented explicitly in git, git-svn should
just ignore it.
Signed-off-by: Jakob Stoklund Olesen <stoklund@2pi.dk> Signed-off-by: Eric Wong <normalperson@yhbt.net>
git-svn: only look at the new parts of svn:mergeinfo
In a Subversion repository where many feature branches are merged into a
trunk, the svn:mergeinfo property can grow very large. This severely
slows down git-svn's make_log_entry() because it is checking all
mergeinfo entries every time the property changes.
In most cases, the additions to svn:mergeinfo since the last commit are
pretty small, and there is nothing to gain by checking merges that were
already checked for the last commit in the branch.
Add a mergeinfo_changes() function which computes the set of interesting
changes to svn:mergeinfo since the last commit. Filter out merged
branches whose ranges haven't changed, and remove a common prefix of
ranges from other merged branches.
This speeds up "git svn fetch" by several orders of magnitude on a large
repository where thousands of feature branches have been merged.
Signed-off-by: Jakob Stoklund Olesen <stoklund@2pi.dk> Signed-off-by: Eric Wong <normalperson@yhbt.net>
Junio C Hamano [Fri, 24 Oct 2014 21:59:10 +0000 (14:59 -0700)]
Merge branch 'eb/no-pthreads'
Allow us build with NO_PTHREADS=NoThanks compilation option.
* eb/no-pthreads:
Handle atexit list internaly for unthreaded builds
pack-objects: set number of threads before checking and warning
index-pack: fix compilation with NO_PTHREADS
Junio C Hamano [Tue, 21 Oct 2014 20:28:42 +0000 (13:28 -0700)]
Merge branch 'da/mergetool-temporary-directory'
Allow a temporary directory specified to be used while running "git
mergetool" backend.
* da/mergetool-temporary-directory:
t7610-mergetool: add test cases for mergetool.writeToTemp
mergetool: add an option for writing to a temporary directory
Junio C Hamano [Tue, 21 Oct 2014 20:28:37 +0000 (13:28 -0700)]
Merge branch 'da/mergetool-tool-help'
Allow "git mergetool --help" to run outside a Git repository.
* da/mergetool-tool-help:
difftool: don't assume that default sh is sane
mergetool: don't require a work tree for --tool-help
git-sh-setup: move GIT_DIR initialization into a function
mergetool: use more conservative temporary filenames
test-lib-functions: adjust style to match CodingGuidelines
t7610-mergetool: prefer test_config over git config
Junio C Hamano [Tue, 21 Oct 2014 20:28:19 +0000 (13:28 -0700)]
Merge branch 'da/mergetool-temporary-filename'
Tweak the names of the three throw-away files "git mergetool" comes
up with to feed the merge tool backend, so that a file with a
single dot in its name in the original (e.g. "hello.c") will have
only one dot in these variants (e.g. "hello_BASE_4321.c").
* da/mergetool-temporary-filename:
mergetool: use more conservative temporary filenames
Junio C Hamano [Tue, 21 Oct 2014 20:28:14 +0000 (13:28 -0700)]
Merge branch 'da/mergetool-tests'
The clean-up of this test script was long overdue and is a very
welcome change.
* da/mergetool-tests:
test-lib-functions: adjust style to match CodingGuidelines
t7610-mergetool: use test_config to isolate tests
t7610-mergetool: add missing && and remove commented-out code
t7610-mergetool: use tabs instead of a mix of tabs and spaces
Junio C Hamano [Tue, 21 Oct 2014 20:28:10 +0000 (13:28 -0700)]
Merge branch 'rs/ref-transaction'
The API to update refs have been restructured to allow introducing
a true transactional updates later. We would even allow storing
refs in backends other than the traditional filesystem-based one.
* rs/ref-transaction: (25 commits)
ref_transaction_commit: bail out on failure to remove a ref
lockfile: remove unable_to_lock_error
refs.c: do not permit err == NULL
remote rm/prune: print a message when writing packed-refs fails
for-each-ref: skip and warn about broken ref names
refs.c: allow listing and deleting badly named refs
test: put tests for handling of bad ref names in one place
packed-ref cache: forbid dot-components in refnames
branch -d: simplify by using RESOLVE_REF_READING
branch -d: avoid repeated symref resolution
reflog test: test interaction with detached HEAD
refs.c: change resolve_ref_unsafe reading argument to be a flags field
refs.c: make write_ref_sha1 static
fetch.c: change s_update_ref to use a ref transaction
refs.c: ref_transaction_commit: distinguish name conflicts from other errors
refs.c: pass a list of names to skip to is_refname_available
refs.c: call lock_ref_sha1_basic directly from commit
refs.c: refuse to lock badly named refs in lock_ref_sha1_basic
rename_ref: don't ask read_ref_full where the ref came from
refs.c: pass the ref log message to _create/delete/update instead of _commit
...
Stefan Beller [Tue, 21 Oct 2014 01:50:45 +0000 (18:50 -0700)]
.mailmap: add Stefan Bellers corporate mail address
Note that despite the private address being first and primary,
Google owns the copyright on this patch as any other patch I'll be
sending signed off by the sbeller@google.com address.
Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano [Mon, 20 Oct 2014 19:25:30 +0000 (12:25 -0700)]
Merge branch 'cc/interpret-trailers'
A new filter to programatically edit the tail end of the commit log
messages.
* cc/interpret-trailers:
Documentation: add documentation for 'git interpret-trailers'
trailer: add tests for commands in config file
trailer: execute command from 'trailer.<name>.command'
trailer: add tests for "git interpret-trailers"
trailer: add interpret-trailers command
trailer: put all the processing together and print
trailer: parse trailers from file or stdin
trailer: process command line trailer arguments
trailer: read and process config information
trailer: process trailers from input message and arguments
trailer: add data structures and basic functions
Junio C Hamano [Mon, 20 Oct 2014 19:23:40 +0000 (12:23 -0700)]
Merge branch 'jk/test-shell-trace'
Test scripts were taught to notice "-x" option to show shell trace,
as if the tests were run under "sh -x".
* jk/test-shell-trace:
test-lib.sh: support -x option for shell-tracing
t5304: use helper to report failure of "test foo = bar"
t5304: use test_path_is_* instead of "test -f"
Junio C Hamano [Mon, 20 Oct 2014 19:04:46 +0000 (12:04 -0700)]
Revert "archive: honor tar.umask even for pax headers"
This reverts commit 10f343ea814f5c18a0913997904ee11cd9b7da24, whose
output is no longer bit-for-bit equivalent from the older versions
of Git, which the infrastructure to (pretend to) upload tarballs
kernel.org uses depends on.
core.filemode is set automatically when a repo is created.
But when a repo is exported via CIFS or cygwin is mixed with Git for Windows
or Eclipse core.filemode may better be set manually to false.
Update and improve the documentation
Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
W. Trevor King [Fri, 17 Oct 2014 21:39:00 +0000 (14:39 -0700)]
t1304: Set LOGNAME even if USER is unset or null
Avoid:
# ./t1304-default-acl.sh
ok 1 - checking for a working acl setup
ok 2 - Setup test repo
not ok 3 - Objects creation does not break ACLs with restrictive umask
#
# # SHA1 for empty blob
# check_perms_and_acl .git/objects/e6/9de29bb2d1d6434b8b29ae775ad8c2e48c5391
#
not ok 4 - git gc does not break ACLs with restrictive umask
#
# git gc &&
# check_perms_and_acl .git/objects/pack/*.pack
#
# failed 2 among 4 test(s)
1..4
on systems where USER isn't set. It's usually set by the login
process, but it isn't set when launching some Docker images. For
example:
$ docker run --rm debian env
HOME=/
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=b2dfdfe797ed
'id -u -n' has been in POSIX from Issue 2 through 2013 [1], so I don't
expect compatibility issues.
Etienne Buira [Sat, 18 Oct 2014 12:31:15 +0000 (14:31 +0200)]
Handle atexit list internaly for unthreaded builds
Wrap atexit()s calls on unthreaded builds to handle callback list
internally.
This is needed because on unthreaded builds, asyncs inherits parent's
atexit() list, that gets run as soon as the async exit()s (and again at
the end of async's parent process). That led to remove temporary files
too early.
Also remove a by-atexit-callback guard against this kind of issue in
clone.c, as this patch makes it redundant.
Fixes test 5537 (temporary shallow file vanished before unpack-objects
could open it)
BTW remove an unused variable in shallow.c.
Helped-by: Duy Nguyen <pclouds@gmail.com> Helped-by: Andreas Schwab <schwab@linux-m68k.org> Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Etienne Buira <etienne.buira@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
René Scharfe [Sun, 19 Oct 2014 11:14:20 +0000 (13:14 +0200)]
use env_array member of struct child_process
Convert users of struct child_process to using the managed env_array for
specifying environment variables instead of supplying an array on the
stack or bringing their own argv_array. This shortens and simplifies
the code and ensures automatically that the allocated memory is freed
after use.
Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
René Scharfe [Sun, 19 Oct 2014 11:13:55 +0000 (13:13 +0200)]
run-command: add env_array, an optional argv_array for env
Similar to args, add a struct argv_array member to struct child_process
that simplifies specifying the environment for children. It is freed
automatically by finish_command() or if start_command() encounters an
error.
Suggested-by: Jeff King <peff@peff.net> Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Fri, 17 Oct 2014 01:11:43 +0000 (21:11 -0400)]
pack-objects: turn off bitmaps when we split packs
If a pack.packSizeLimit is set, we may split the pack data
across multiple packfiles. This means we cannot generate
.bitmap files, as they require that all of the reachable
objects are in the same pack. We check that condition when
we are generating the list of objects to pack (and disable
bitmaps if we are not packing everything), but we forgot to
update it when we notice that we needed to split (which
doesn't happen until the actual write phase).
The resulting bitmaps are quite bogus (they mention entries
that do not exist in the pack!) and can cause a fetch or
push to send insufficient objects.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano [Thu, 16 Oct 2014 21:16:42 +0000 (14:16 -0700)]
Merge branch 'po/everyday-doc'
"git help everyday" to show the Everyday Git document.
* po/everyday-doc:
doc: add 'everyday' to 'git help'
doc: Makefile regularise OBSOLETE_HTML list building
doc: modernise everyday.txt wording and format in man page style
Roland Mas [Thu, 16 Oct 2014 06:54:47 +0000 (08:54 +0200)]
gitweb: use start_form, not startform that was removed in CGI.pm 4.04
CGI.pm 4.04 removed the startform method, which had previously been
deprecated in favour of start_form. Changes file for CGI.pm says:
4.04 2014-09-04
[ REMOVED / DEPRECATIONS ]
- startform and endform methods removed (previously deprecated,
you should be using the start_form and end_form methods)
Signed-off-by: Roland Mas <lolando@debian.org> Reviewed-by: Jakub Narębski <jnareb@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
David Aguilar [Fri, 10 Oct 2014 08:19:47 +0000 (01:19 -0700)]
mergetool: use more conservative temporary filenames
Avoid filenames with multiple dots so that overly-picky tools do
not misinterpret their extension.
Previously, foo/bar.ext in the worktree would result in e.g.
./foo/bar.ext.BASE.1234.ext
This can be improved by having only a single .ext and using
underscore instead of dot so that the extension cannot be
misinterpreted. The resulting path becomes:
./foo/bar_BASE_1234.ext
Suggested-by: Sergio Ferrero <sferrero@ensoftcorp.com> Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: David Aguilar <davvid@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
David Aguilar [Wed, 15 Oct 2014 08:35:21 +0000 (01:35 -0700)]
test-lib-functions: adjust style to match CodingGuidelines
Prefer "test" over "[ ]" for conditionals.
Prefer "$()" over backticks for command substitutions.
Avoid control structures on a single line with semicolons.
Signed-off-by: David Aguilar <davvid@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
David Aguilar [Thu, 16 Oct 2014 04:45:14 +0000 (21:45 -0700)]
mergetools/meld: make usage of `--output` configurable and more robust
Older versions of meld listed --output in `meld --help`.
Newer versions only mention `meld [OPTIONS...]`.
Improve the checks to catch these newer versions.
Add a `mergetool.meld.hasOutput` configuration to allow
overriding the heuristic.
Reported-by: Andrey Novoseltsev <novoselt@gmail.com> Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: David Aguilar <davvid@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Charles Bailey [Sat, 11 Oct 2014 08:39:38 +0000 (01:39 -0700)]
difftool: don't assume that default sh is sane
git-difftool used to create a command list script containing $( ... )
and explicitly calls "sh -c" with this list.
Instead, allow mergetool --tool-help to take a mode parameter and call
mergetool directly to invoke the show_tool_help function. This mode
parameter is intented for use solely by difftool.
Signed-off-by: Charles Bailey <cbailey32@bloomberg.net> Helped-by: John Keeping <john@keeping.me.uk> Signed-off-by: David Aguilar <davvid@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Charles Bailey [Sat, 11 Oct 2014 08:39:37 +0000 (01:39 -0700)]
mergetool: don't require a work tree for --tool-help
Signed-off-by: Charles Bailey <cbailey32@bloomberg.net> Signed-off-by: David Aguilar <davvid@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
David Aguilar [Fri, 10 Oct 2014 08:19:47 +0000 (01:19 -0700)]
mergetool: use more conservative temporary filenames
Avoid filenames with multiple dots so that overly-picky tools do
not misinterpret their extension.
Previously, foo/bar.ext in the worktree would result in e.g.
./foo/bar.ext.BASE.1234.ext
This can be improved by having only a single .ext and using
underscore instead of dot so that the extension cannot be
misinterpreted. The resulting path becomes:
./foo/bar_BASE_1234.ext
Suggested-by: Sergio Ferrero <sferrero@ensoftcorp.com> Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: David Aguilar <davvid@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
David Aguilar [Wed, 15 Oct 2014 08:35:21 +0000 (01:35 -0700)]
test-lib-functions: adjust style to match CodingGuidelines
Prefer "test" over "[ ]" for conditionals.
Prefer "$()" over backticks for command substitutions.
Avoid control structures on a single line with semicolons.
Signed-off-by: David Aguilar <davvid@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jonathan Nieder [Fri, 29 Aug 2014 00:01:35 +0000 (17:01 -0700)]
ref_transaction_commit: bail out on failure to remove a ref
When removal of a loose or packed ref fails, bail out instead of
trying to finish the transaction. This way, a single error message
can be printed (instead of multiple messages being concatenated by
mistake) and the operator can try to solve the underlying problem
before there is a chance to muck things up even more.
In particular, when git fails to remove a ref, git goes on to try to
delete the reflog. Exiting early lets us keep the reflog.
When git succeeds in deleting a ref A and fails to remove a ref B, it
goes on to try to delete both reflogs. It would be better to just
remove the reflog for A, but that would be a more invasive change.
Failing early means we keep both reflogs, which puts the operator in a
good position to understand the problem and recover.
A long term goal is to avoid these problems altogether and roll back
the transaction on failure. That kind of transactionality will have
to wait for a later series (the plan for which is to make all
destructive work happen in a single update of the packed-refs file).
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Reviewed-by: Ronnie Sahlberg <sahlberg@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jonathan Nieder [Thu, 28 Aug 2014 23:41:34 +0000 (16:41 -0700)]
lockfile: remove unable_to_lock_error
The former caller uses unable_to_lock_message now.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Reviewed-by: Ronnie Sahlberg <sahlberg@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jonathan Nieder [Thu, 28 Aug 2014 23:42:37 +0000 (16:42 -0700)]
refs.c: do not permit err == NULL
Some functions that take a strbuf argument to append an error treat
!err as an indication that the message should be suppressed (e.g.,
ref_update_reject_duplicates). Others write the message to stderr on
!err (e.g., repack_without_refs). Others crash (e.g.,
ref_transaction_update).
Some of these behaviors are for historical reasons and others were
accidents. Luckily no callers pass err == NULL any more. Simplify
by consistently requiring the strbuf argument.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Reviewed-by: Ronnie Sahlberg <sahlberg@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Ronnie Sahlberg [Thu, 11 Sep 2014 15:42:57 +0000 (08:42 -0700)]
remote rm/prune: print a message when writing packed-refs fails
Until v2.1.0-rc0~22^2~11 (refs.c: add an err argument to
repack_without_refs, 2014-06-20), repack_without_refs forgot to
provide an error message when commit_packed_refs fails. Even today,
it only provides a message for callers that pass a non-NULL err
parameter. Internal callers in refs.c pass non-NULL err but
"git remote" does not.
That means that "git remote rm" and "git remote prune" can fail
without printing a message about why. Fix them by passing in a
non-NULL err parameter and printing the returned message.
This is the last caller to a ref handling function passing err ==
NULL. A later patch can drop support for err == NULL, avoiding such
problems in the future.
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Ronnie Sahlberg [Fri, 5 Sep 2014 21:35:17 +0000 (14:35 -0700)]
for-each-ref: skip and warn about broken ref names
Print a warning message for any bad ref names we find in the repo and
skip them so callers don't have to deal with parsing them.
It might be useful in the future to have a flag where we would not
skip these refs for those callers that want to and are prepared (for
example by using a --format argument with %0 as a delimiter after the
ref name).
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Users cannot recover from a badly named ref without manually finding
and deleting the loose ref file or appropriate line in packed-refs.
Making that easier will make it easier to tweak the ref naming rules
in the future, for example to forbid shell metacharacters like '`'
and '"', without putting people in a state that is hard to get out of.
So allow "branch --list" to show these refs and allow "branch -d/-D"
and "update-ref -d" to delete them. Other commands (for example to
rename refs) will continue to not handle these refs but can be changed
in later patches.
Details:
In resolving functions, refuse to resolve refs that don't pass the
git-check-ref-format(1) check unless the new RESOLVE_REF_ALLOW_BAD_NAME
flag is passed. Even with RESOLVE_REF_ALLOW_BAD_NAME, refuse to
resolve refs that escape the refs/ directory and do not match the
pattern [A-Z_]* (think "HEAD" and "MERGE_HEAD").
In locking functions, refuse to act on badly named refs unless they
are being deleted and either are in the refs/ directory or match [A-Z_]*.
Just like other invalid refs, flag resolved, badly named refs with the
REF_ISBROKEN flag, treat them as resolving to null_sha1, and skip them
in all iteration functions except for for_each_rawref.
Flag badly named refs (but not symrefs pointing to badly named refs)
with a REF_BAD_NAME flag to make it easier for future callers to
notice and handle them specially. For example, in a later patch
for-each-ref will use this flag to detect refs whose names can confuse
callers parsing for-each-ref output.
In the transaction API, refuse to create or update badly named refs,
but allow deleting them (unless they try to escape refs/ and don't match
[A-Z_]*).
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Ronnie Sahlberg [Thu, 25 Sep 2014 22:02:39 +0000 (15:02 -0700)]
test: put tests for handling of bad ref names in one place
There's no straightforward way to grep for all tests dealing with
invalid refs. Put them in a single test script so it is easy to see
what functionality has not been exercised with bad ref names yet.
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jonathan Nieder [Fri, 26 Sep 2014 19:22:22 +0000 (12:22 -0700)]
packed-ref cache: forbid dot-components in refnames
Since v1.7.9-rc1~10^2 (write_head_info(): handle "extra refs" locally,
2012-01-06), this trick to keep track of ".have" refs that are only
valid on the wire and not on the filesystem is not needed any more.
Simplify by removing support for the REFNAME_DOT_COMPONENT flag.
This means we'll be slightly stricter with invalid refs found in a
packed-refs file or during clone. read_loose_refs() already checks
for and skips refnames with .components so it is not affected.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Reviewed-by: Ronnie Sahlberg <sahlberg@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Ronnie Sahlberg [Thu, 11 Sep 2014 17:34:36 +0000 (10:34 -0700)]
branch -d: simplify by using RESOLVE_REF_READING
When "git branch -d" reads the branch it is about to delete, it used
to avoid passing the RESOLVE_REF_READING ('treat missing ref as
error') flag because a symref pointing to a nonexistent ref would show
up as missing instead of as something that could be deleted. To check
if a ref is actually missing, we then check
- is it a symref?
- if not, did it resolve to null_sha1?
Now we pass RESOLVE_REF_NO_RECURSE and the correct information is
returned for a symref even when it points to a missing ref. Simplify
by relying on RESOLVE_REF_READING.
No functional change intended.
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Worse, "git update-ref --no-deref -d" doesn't work for such repairs
either:
$ git update-ref -d refs/heads/nonsense
error: unable to resolve reference refs/heads/nonsense: Too many levels of symbolic links
Fix both by teaching resolve_ref_unsafe a new RESOLVE_REF_NO_RECURSE
flag and passing it when appropriate.
Callers can still read the value of a symref (for example to print a
message about it) with that flag set --- resolve_ref_unsafe will
resolve one level of symrefs and stop there.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Reviewed-by: Ronnie Sahlberg <sahlberg@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano [Sat, 13 Sep 2014 17:52:25 +0000 (10:52 -0700)]
reflog test: test interaction with detached HEAD
A proposed patch produced broken HEAD reflog entries when checking out
anything other than a branch. The testsuite still passed, so it took
a few days for the bug to be noticed.
Add tests checking the content of the reflog after detaching and
reattaching HEAD so we don't have to rely on manual testing to catch
such problems in the future.
[jn: using 'log -g --format=%H' instead of parsing --oneline output,
resetting state in each test so they can be safely reordered or
skipped]
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Reviewed-by: Ronnie Sahlberg <sahlberg@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Ronnie Sahlberg [Tue, 15 Jul 2014 19:59:36 +0000 (12:59 -0700)]
refs.c: change resolve_ref_unsafe reading argument to be a flags field
resolve_ref_unsafe takes a boolean argument for reading (a nonexistent ref
resolves successfully for writing but not for reading). Change this to be
a flags field instead, and pass the new constant RESOLVE_REF_READING when
we want this behaviour.
While at it, swap two of the arguments in the function to put output
arguments at the end. As a nice side effect, this ensures that we can
catch callers that were unaware of the new API so they can be audited.
Give the wrapper functions resolve_refdup and read_ref_full the same
treatment for consistency.
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Ronnie Sahlberg [Mon, 28 Apr 2014 22:36:58 +0000 (15:36 -0700)]
refs.c: make write_ref_sha1 static
No external users call write_ref_sha1 any more so let's declare it static.
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Ronnie Sahlberg [Mon, 28 Apr 2014 20:49:07 +0000 (13:49 -0700)]
fetch.c: change s_update_ref to use a ref transaction
Change s_update_ref to use a ref transaction for the ref update.
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Ronnie Sahlberg [Fri, 16 May 2014 21:14:38 +0000 (14:14 -0700)]
refs.c: ref_transaction_commit: distinguish name conflicts from other errors
In _commit, ENOTDIR can happen in the call to lock_ref_sha1_basic, either
when we lstat the new refname or if the name checking function reports that
the same type of conflict happened. In both cases, it means that we can not
create the new ref due to a name conflict.
Start defining specific return codes for _commit. TRANSACTION_NAME_CONFLICT
refers to a failure to create a ref due to a name conflict with another ref.
TRANSACTION_GENERIC_ERROR is for all other errors.
When "git fetch" is creating refs, name conflicts differ from other errors in
that they are likely to be resolved by running "git remote prune <remote>".
"git fetch" currently inspects errno to decide whether to give that advice.
Once it switches to the transaction API, it can check for
TRANSACTION_NAME_CONFLICT instead.
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Ronnie Sahlberg [Thu, 1 May 2014 18:16:07 +0000 (11:16 -0700)]
refs.c: pass a list of names to skip to is_refname_available
Change is_refname_available to take a list of strings to exclude when
checking for conflicts instead of just one single name. We can already
exclude a single name for the sake of renames. This generalizes that support.
ref_transaction_commit already tracks a set of refs that are being deleted
in an array. This array is then used to exclude refs from being written to
the packed-refs file. At some stage we will want to change this array to a
struct string_list and then we can pass it to is_refname_available via the
call to lock_ref_sha1_basic. That will allow us to perform transactions
that perform multiple renames as long as there are no conflicts within the
starting or ending state.
For example, that would allow a single transaction that contains two
renames that are both individually conflicting:
m -> n/n
n -> m/m
No functional change intended yet.
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Ronnie Sahlberg [Thu, 1 May 2014 17:43:39 +0000 (10:43 -0700)]
refs.c: call lock_ref_sha1_basic directly from commit
Skip using the lock_any_ref_for_update wrapper and call lock_ref_sha1_basic
directly from the commit function.
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com> Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Ronnie Sahlberg [Thu, 1 May 2014 17:40:10 +0000 (10:40 -0700)]
refs.c: refuse to lock badly named refs in lock_ref_sha1_basic
Move the check for check_refname_format from lock_any_ref_for_update to
lock_ref_sha1_basic. At some later stage we will get rid of
lock_any_ref_for_update completely. This has no visible impact to callers
except for the inability to lock badly named refs, which is not possible
today already for other reasons.(*)
Keep lock_any_ref_for_update as a no-op wrapper. It is the public facing
version of this interface and keeping it as a separate function will make
it easier to experiment with the internal lock_ref_sha1_basic signature.
(*) For example, if lock_ref_sha1_basic checks the refname format and
refuses to lock badly named refs, it will not be possible to delete
such refs because the first step of deletion is to lock the ref. We
currently already fail in that case because these refs are not recognized
to exist:
This has been broken for a while. Later patches in the series will start
repairing the handling of badly named refs.
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com> Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Ronnie Sahlberg [Wed, 30 Apr 2014 19:41:04 +0000 (12:41 -0700)]
rename_ref: don't ask read_ref_full where the ref came from
We call read_ref_full with a pointer to flags from rename_ref but since
we never actually use the returned flags we can just pass NULL here instead.
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com> Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Ronnie Sahlberg [Wed, 30 Apr 2014 19:22:42 +0000 (12:22 -0700)]
refs.c: pass the ref log message to _create/delete/update instead of _commit
Change the ref transaction API so that we pass the reflog message to the
create/delete/update functions instead of to ref_transaction_commit.
This allows different reflog messages for each ref update in a multi-ref
transaction.
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Ronnie Sahlberg [Thu, 15 May 2014 15:25:23 +0000 (08:25 -0700)]
refs.c: add an err argument to delete_ref_loose
Add an err argument to delete_ref_loose so that we can pass a descriptive
error string back to the caller. Pass the err argument from transaction
commit to this function so that transaction users will have a nice error
string if the transaction failed due to delete_ref_loose.
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Ronnie Sahlberg [Wed, 16 Jul 2014 18:20:36 +0000 (11:20 -0700)]
wrapper.c: add a new function unlink_or_msg
This behaves like unlink_or_warn except that on failure it writes the message
to its 'err' argument, which the caller can display in an appropriate way or
ignore.
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com> Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Ronnie Sahlberg [Thu, 2 Oct 2014 14:59:02 +0000 (07:59 -0700)]
refs.c: lock_ref_sha1_basic is used for all refs
lock_ref_sha1_basic is used to lock refs that sit directly in the .git
dir such as HEAD and MERGE_HEAD in addition to the more ordinary refs
under "refs/". Remove the note claiming otherwise.
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Ronnie Sahlberg [Wed, 16 Jul 2014 18:01:18 +0000 (11:01 -0700)]
wrapper.c: remove/unlink_or_warn: simplify, treat ENOENT as success
Simplify the function warn_if_unremovable slightly. Additionally, change
behaviour slightly. If we failed to remove the object because the object
does not exist, we can still return success back to the caller since none of
the callers depend on "fail if the file did not exist".
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jonathan Nieder [Wed, 10 Sep 2014 21:01:46 +0000 (14:01 -0700)]
mv test: recreate mod/ directory instead of relying on stale copy
The tests for 'git mv moves a submodule' functionality often run
commands like
git mv sub mod/sub
to move a submodule into a subdirectory. Just like plain /bin/mv,
this is supposed to succeed if the mod/ parent directory exists
and fail if it doesn't exist.
Usually these tests mkdir the parent directory beforehand, but some
instead rely on it being left behind by previous tests.
More precisely, when 'git reset --hard' tries to move to a state where
mod/sub is not present any more, it would perform the following
operations:
rmdir("mod/sub")
rmdir("mod")
The first fails with ENOENT because the test script removed mod/sub
with "rm -rf" already, so 'reset --hard' doesn't bother to move on to
the second, and the mod/ directory is kept around.
Better to explicitly remove and re-create the mod/ directory so later
tests don't have to depend on the directory left behind by the earlier
ones at all (making it easier to rearrange or skip some tests in the
file or to tweak 'reset --hard' behavior without breaking unrelated
tests).
Noticed while testing a patch that fixes the reset --hard behavior
described above.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Reviewed-by: Ronnie Sahlberg <sahlberg@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
John Szakmeister [Tue, 14 Oct 2014 10:49:45 +0000 (06:49 -0400)]
completion: silence "fatal: Not a git repository" error
It is possible that a user is trying to run a git command and fail
to realize that they are not in a git repository or working tree.
When trying to complete an operation, __git_refs would fall to a
degenerate case and attempt to use "git for-each-ref", which would
emit the error.
Hide this error message coming from "git for-each-ref".
Signed-off-by: John Szakmeister <john@szakmeister.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
projects / thirdparty / git.git / log
