fetch: add --update-shallow to accept refs that update .git/shallow
The same steps are done as in when --update-shallow is not given. The
only difference is we now add all shallow commits in "ours" and
"theirs" to .git/shallow (aka "step 8").
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
upload-pack: make sure deepening preserves shallow roots
When "fetch --depth=N" where N exceeds the longest chain of history in
the source repo, usually we just send an "unshallow" line to the
client so full history is obtained.
When the source repo is shallow we need to make sure to "unshallow"
the current shallow point _and_ "shallow" again when the commit
reaches its shallow bottom in the source repo.
This should fix both cases: large <N> and --unshallow.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
This patch just put together pieces from the 8 steps patch. We stop at
step 7 and reject refs that require new shallow commits.
Note that, by rejecting refs that require new shallow commits, we
leave dangling objects in the repo, which become "object islands" by
the next "git fetch" of the same source.
If the first fetch our "ours" set is zero and we do practically
nothing at step 7, "ours" is full at the next fetch and we may need to
walk through commits for reachability test. Room for improvement.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Cloning from a shallow repository does not follow the "8 steps for new
.git/shallow" because if it does we need to get through step 6 for all
refs. That means commit walking down to the bottom.
Instead the rule to create .git/shallow is simpler and, more
importantly, cheap: if a shallow commit is found in the pack, it's
probably used (i.e. reachable from some refs), so we add it. Others
are dropped.
One may notice this method seems flawed by the word "probably". A
shallow commit may not be reachable from any refs at all if it's
attached to an object island (a group of objects that are not
reachable by any refs).
If that object island is not complete, a new fetch request may send
more objects to connect it to some ref. At that time, because we
incorrectly installed the shallow commit in this island, the user will
not see anything after that commit (fsck is still ok). This is not
desired.
Given that object islands are rare (C Git never sends such islands for
security reasons) and do not really harm the repository integrity, a
tradeoff is made to surprise the user occasionally but work faster
everyday.
A new option --strict could be added later that follows exactly the 8
steps. "git prune" can also learn to remove dangling objects _and_ the
shallow commits that are attached to them from .git/shallow.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
shallow.c: the 8 steps to select new commits for .git/shallow
Suppose a fetch or push is requested between two shallow repositories
(with no history deepening or shortening). A pack that contains
necessary objects is transferred over together with .git/shallow of
the sender. The receiver has to determine whether it needs to update
.git/shallow if new refs needs new shallow comits.
The rule here is avoid updating .git/shallow by default. But we don't
want to waste the received pack. If the pack contains two refs, one
needs new shallow commits installed in .git/shallow and one does not,
we keep the latter and reject/warn about the former.
Even if .git/shallow update is allowed, we only add shallow commits
strictly necessary for the former ref (remember the sender can send
more shallow commits than necessary) and pay attention not to
accidentally cut the receiver history short (no history shortening is
asked for)
So the steps to figure out what ref need what new shallow commits are:
1. Split the sender shallow commit list into "ours" and "theirs" list
by has_sha1_file. Those that exist in current repo in "ours", the
remaining in "theirs".
2. Check the receiver .git/shallow, remove from "ours" the ones that
also exist in .git/shallow.
3. Fetch the new pack. Either install or unpack it.
4. Do has_sha1_file on "theirs" list again. Drop the ones that fail
has_sha1_file. Obviously the new pack does not need them.
5. If the pack is kept, remove from "ours" the ones that do not exist
in the new pack.
6. Walk the new refs to answer the question "what shallow commits,
both ours and theirs, are required in .git/shallow in order to add
this ref?". Shallow commits not associated to any refs are removed
from their respective list.
7. (*) Check reachability (from the current refs) of all remaining
commits in "ours". Those reachable are removed. We do not want to
cut any part of our (reachable) history. We only check up
commits. True reachability test is done by
check_everything_connected() at the end as usual.
8. Combine the final "ours" and "theirs" and add them all to
.git/shallow. Install new refs. The case where some hook rejects
some refs on a push is explained in more detail in the push
patches.
Of these steps, #6 and #7 are expensive. Both require walking through
some commits, or in the worst case all commits. And we rather avoid
them in at least common case, where the transferred pack does not
contain any shallow commits that the sender advertises. Let's look at
each scenario:
1) the sender has longer history than the receiver
All shallow commits from the sender will be put into "theirs" list
at step 1 because none of them exists in current repo. In the
common case, "theirs" becomes empty at step 4 and exit early.
2) the sender has shorter history than the receiver
All shallow commits from the sender are likely in "ours" list at
step 1. In the common case, if the new pack is kept, we could empty
"ours" and exit early at step 5.
If the pack is not kept, we hit the expensive step 6 then exit
after "ours" is emptied. There'll be only a handful of objects to
walk in fast-forward case. If it's forced update, we may need to
walk to the bottom.
3) the sender has same .git/shallow as the receiver
This is similar to case 2 except that "ours" should be emptied at
step 2 and exit early.
A fetch after "clone --depth=X" is case 1. A fetch after "clone" (from
a shallow repo) is case 3. Luckily they're cheap for the common case.
A push from "clone --depth=X" falls into case 2, which is expensive.
Some more work may be done at the sender/client side to avoid more
work on the server side: if the transferred pack does not contain any
shallow commits, send-pack should not send any shallow commits to the
receive-pack, effectively turning it into a normal push and avoid all
steps.
This patch implements all steps except #3, already handled by
fetch-pack and receive-pack, #6 and #7, which has their own patch due
to their size.
(*) in previous versions step 7 was put before step 3. I reorder it so
that the common case that keeps the pack does not need to walk
commits at all. In future if we implement faster commit
reachability check (maybe with the help of pack bitmaps or commit
cache), step 7 could become cheap and be moved up before 6 again.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
connect.c: teach get_remote_heads to parse "shallow" lines
No callers pass a non-empty pointer as shallow_points at this
stage. As a result, all clients still refuse to talk to shallow
repository on the other end.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
make the sender advertise shallow commits to the receiver
If either receive-pack or upload-pack is called on a shallow
repository, shallow commits (*) will be sent after the ref
advertisement (but before the packet flush), so that the receiver has
the full "shape" of the sender's commit graph. This will be needed for
the receiver to update its .git/shallow if necessary.
This breaks the protocol for all clients trying to push to a shallow
repo, or fetch from one. Which is basically the same end result as
today's "is_repository_shallow() && die()" in receive-pack and
upload-pack. New clients will be made aware of shallow upstream and
can make use of this information.
The sender must send all shallow commits that are sent in the
following pack. It may send more shallow commits than necessary.
upload-pack for example may choose to advertise no shallow commits if
it knows in advance that the pack it's going to send contains no
shallow commits. But upload-pack is the server, so we choose the
cheaper way, send full .git/shallow and let the client deal with it.
Smart HTTP is not affected by this patch. Shallow support on
smart-http comes later separately.
(*) A shallow commit is a commit that terminates the revision
walker. It is usually put in .git/shallow in order to keep the
revision walker from going out of bound because there is no
guarantee that objects behind this commit is available.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
clone: prevent --reference to a shallow repository
If we borrow objects from another repository, we should also pay
attention to their $GIT_DIR/shallow (and even info/grafts). But
current alternates code does not.
Reject alternate repos that are shallow because we do not do it
right. In future the alternate code may be updated to check
$GIT_DIR/shallow properly so that this restriction could be lifted.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
send-pack: forbid pushing from a shallow repository
send-pack can send a pack with loose ends to the server. receive-pack
before 6d4bb38 (fetch: verify we have everything we need before
updating our ref - 2011-09-01) does not detect this and keeps the pack
anyway, which corrupts the repository, at least from fsck point of
view.
send-pack will learn to safely push from a shallow repository later.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Richard Hansen [Mon, 18 Nov 2013 04:12:42 +0000 (23:12 -0500)]
remote-hg: don't decode UTF-8 paths into Unicode objects
The internal mercurial API expects ordinary 8-bit string objects, not
Unicode string objects. With this change, the test-hg.sh unit tests
pass again.
Signed-off-by: Richard Hansen <rhansen@bbn.com> Reviewed-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Richard Hansen [Mon, 18 Nov 2013 04:12:42 +0000 (23:12 -0500)]
remote-hg: don't decode UTF-8 paths into Unicode objects
The internal mercurial API expects ordinary 8-bit string objects, not
Unicode string objects. With this change, the test-hg.sh unit tests
pass again.
Signed-off-by: Richard Hansen <rhansen@bbn.com> Reviewed-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Torstein Hegge [Thu, 14 Nov 2013 18:18:01 +0000 (19:18 +0100)]
branch: fix --verbose output column alignment
Commit f2e0873 (branch: report invalid tracking branch as gone) removed
an early return from fill_tracking_info() in the path taken when 'git
branch -v' lists a branch in sync with its upstream. This resulted in an
unconditionally added space in front of the subject line:
Junio C Hamano [Mon, 18 Nov 2013 18:15:45 +0000 (10:15 -0800)]
Revert "upload-pack: send non-HEAD symbolic refs"
This reverts commit 5e7dcad771cb873e278a0571b46910d7c32e2f6c; there
may be unbounded number of symbolic refs in the repository, but the
capability header line in the on-wire protocol has a rather low
length limit.
Junio C Hamano [Fri, 8 Nov 2013 20:01:13 +0000 (12:01 -0800)]
Merge branch 'ap/remote-hg-unquote-cquote' into maint
A fast-import stream expresses a pathname with funny characters by
quoting them in C style; remote-hg remote helper (in contrib/)
forgot to unquote such a path.
* ap/remote-hg-unquote-cquote:
remote-hg: unquote C-style paths when exporting
Junio C Hamano [Fri, 8 Nov 2013 19:38:00 +0000 (11:38 -0800)]
Merge branch 'jc/upload-pack-send-symref' into maint
One long-standing flaw in the pack transfer protocol used by "git
clone" was that there was no way to tell the other end which branch
"HEAD" points at, and the receiving end needed to guess. A new
capability has been defined in the pack protocol to convey this
information so that cloning from a repository with more than one
branches pointing at the same commit where the HEAD is at now
reliably sets the initial branch in the resulting repository.
* jc/upload-pack-send-symref:
t5570: Update for clone-progress-to-stderr branch
t5570: Update for symref capability
clone: test the new HEAD detection logic
connect: annotate refs with their symref information in get_remote_head()
connect.c: make parse_feature_value() static
upload-pack: send non-HEAD symbolic refs
upload-pack: send symbolic ref information as capability
upload-pack.c: do not pass confusing cb_data to mark_our_ref()
t5505: fix "set-head --auto with ambiguous HEAD" test
Junio C Hamano [Fri, 8 Nov 2013 19:37:25 +0000 (11:37 -0800)]
Merge branch 'jk/http-auth-redirects' into maint
We did not handle cases where http transport gets redirected during
the authorization request (e.g. from http:// to https://).
* jk/http-auth-redirects:
http.c: Spell the null pointer as NULL
remote-curl: rewrite base url from info/refs redirects
remote-curl: store url as a strbuf
remote-curl: make refs_url a strbuf
http: update base URLs when we see redirects
http: provide effective url to callers
http: hoist credential request out of handle_curl_result
http: refactor options to http_get_*
http_request: factor out curlinfo_strbuf
http_get_file: style fixes
Junio C Hamano [Thu, 7 Nov 2013 22:36:59 +0000 (14:36 -0800)]
Merge branch 'mm/checkout-auto-track-fix' into maint
"git checkout topic", when there is not yet a local "topic" branch
but there is a unique remote-tracking branch for a remote "topic"
branch, pretended as if "git checkout -t -b topic remote/$r/topic"
(for that unique remote $r) was run. This hack however was not
implemented for "git checkout topic --".
* mm/checkout-auto-track-fix:
checkout: proper error message on 'git checkout foo bar --'
checkout: allow dwim for branch creation for "git checkout $branch --"
Junio C Hamano [Mon, 4 Nov 2013 22:58:19 +0000 (14:58 -0800)]
Merge branch 'bw/solaris-sed-tr-test-portability'
* bw/solaris-sed-tr-test-portability:
t4015: simplify sed command that is not even seen by sed
Avoid difference in tr semantics between System V and BSD
Change sed i\ usage to something Solaris' sed can handle
Junio C Hamano [Fri, 1 Nov 2013 14:38:54 +0000 (07:38 -0700)]
Merge branch 'js/tests-windows-port-fix'
* js/tests-windows-port-fix:
tests: undo special treatment of CRLF for Windows
Windows: a test_cmp that is agnostic to random LF <> CRLF conversions
t5300-pack-object: do not compare binary data using test_cmp
Junio C Hamano [Thu, 31 Oct 2013 20:43:35 +0000 (13:43 -0700)]
doc/pull: clarify the illustrations
The second illustration that shows the history after "git pull"
spelled the remote-tracking branch with "remotes/" prefix, which
is not necessary. Drop it.
To match the assumption that a remote-tracking branch is used to
keep track of the advancement of the master at the origin, update
the first illustration that shows the history before "git pull"
to show the distinction between the master currently at origin and
the stale origin/master remote-tracking branch.
Noticed-by: Felipe Contreras <felipe.contreras@gmail.com> Helped-by: Max Horn <max@quendi.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Kacper Kornet [Thu, 31 Oct 2013 11:51:32 +0000 (12:51 +0100)]
Fix '\%o' for printf from coreutils
The printf utility provided by coreutils when interpreting '\%o' format
does not recognize %o as formatting directive. For example
printf '\%o 0 returns \%o and warning: ignoring excess arguments,
starting with ‘0’, which results in failed tests in
t5309-pack-delta-cycles.sh. In most shells the test ends with success as
the printf is a builtin utility.
Fix it by using '\\%o' which is interpreted consistently in all versions
of printf.
Signed-off-by: Kacper Kornet <draenog@pld-linux.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
xdg-open is a tool similar to git-web--browse. It opens a file or URL in the
user's preferred application. It could probably be made default at least on
Linux with a graphical environment.
Signed-off-by: Rüdiger Sonderfeld <ruediger@c-plusplus.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Johannes Sixt [Sat, 26 Oct 2013 18:48:46 +0000 (20:48 +0200)]
t3200: do not open a HTML manual page when DEFAULT_MAN_FORMAT is html
We have the build configuration option DEFAULT_MAN_FORMAT to choose a
format different from man pages to be used by 'git help' when no format
is requested explicitly. Since 65db0443 (Set the default help format to
html for msys builds, 2013-06-04) we use html on Windows by default.
There is one test in t3200-branch.sh that invokes a help page. The
intent of the redirections applied to the command invocation is to avoid
that the man page viewer interferes with the automated test. But when
the default format is not "man", this does not have the intended effect,
and the HTML manual page is opened during the test run. Request "man"
format explicitly to keep the test silent.
Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
projects / thirdparty / git.git / log
