Junio C Hamano [Tue, 20 Jun 2023 22:53:10 +0000 (15:53 -0700)]
Merge branch 'ps/fetch-cleanups'
Code clean-up.
* ps/fetch-cleanups:
fetch: use `fetch_config` to store "submodule.fetchJobs" value
fetch: use `fetch_config` to store "fetch.parallel" value
fetch: use `fetch_config` to store "fetch.recurseSubmodules" value
fetch: use `fetch_config` to store "fetch.showForcedUpdates" value
fetch: use `fetch_config` to store "fetch.pruneTags" value
fetch: use `fetch_config` to store "fetch.prune" value
fetch: pass through `fetch_config` directly
fetch: drop unneeded NULL-check for `remote_ref`
fetch: drop unused DISPLAY_FORMAT_UNKNOWN enum value
Junio C Hamano [Tue, 13 Jun 2023 19:29:46 +0000 (12:29 -0700)]
Merge branch 'zh/ls-files-format-atoms'
Some atoms that can be used in "--format=<format>" for "git ls-tree"
were not supported by "git ls-files", even though they were relevant
in the context of the latter.
* zh/ls-files-format-atoms:
ls-files: align format atoms with ls-tree
Junio C Hamano [Tue, 13 Jun 2023 19:29:45 +0000 (12:29 -0700)]
Merge branch 'jc/pack-ref-exclude-include'
"git pack-refs" learns "--include" and "--exclude" to tweak the ref
hierarchy to be packed using pattern matching.
* jc/pack-ref-exclude-include:
pack-refs: teach pack-refs --include option
pack-refs: teach --exclude option to exclude refs from being packed
docs: clarify git-pack-refs --all will pack all refs
Junio C Hamano [Tue, 13 Jun 2023 19:29:45 +0000 (12:29 -0700)]
Merge branch 'sa/doc-ls-remote'
Doc update.
* sa/doc-ls-remote:
ls-remote doc: document the output format
ls-remote doc: explain what each example does
ls-remote doc: show peeled tags in examples
ls-remote doc: remove redundant --tags example
show-branch doc: say <ref>, not <reference>
show-ref doc: update for internal consistency
Junio C Hamano [Tue, 13 Jun 2023 19:29:44 +0000 (12:29 -0700)]
Merge branch 'jc/diff-s-with-other-options'
The "-s" (silent, squelch) option of the "diff" family of commands
did not interact with other options that specify the output format
well. This has been cleaned up so that it will clear all the
formatting options given before.
* jc/diff-s-with-other-options:
diff: fix interaction between the "-s" option and other options
Junio C Hamano [Tue, 13 Jun 2023 19:29:44 +0000 (12:29 -0700)]
Merge branch 'kh/keep-tag-editmsg-upon-failure'
"git tag" learned to leave the "$GIT_DIR/TAG_EDITMSG" file when the
command failed, so that the user can salvage what they typed.
* kh/keep-tag-editmsg-upon-failure:
tag: keep the message file in case ref transaction fails
t/t7004-tag: add regression test for successful tag creation
doc: tag: document `TAG_EDITMSG`
Jeff King [Thu, 1 Jun 2023 18:10:40 +0000 (14:10 -0400)]
ci: drop linux-clang job
Since the linux-asan-ubsan job runs using clang under Linux, there is
not much point in running a separate clang job. Any errors that a normal
clang compile-and-test cycle would find are likely to be a subset of
what the sanitizer job will find. Since this job takes ~14 minutes to
run in CI, this shaves off some of our CPU load (though it does not
affect end-to-end runtime, since it's typically run in parallel and is
not the longest job).
Technically this provides us with slightly less signal for a given run,
since you won't immediately know if a failure in the sanitizer job is
from using clang or from the sanitizers themselves. But it's generally
obvious from the logs, and anyway your next step would be to fix the
probvlem and re-run CI, since we expect all of these jobs to pass
normally.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Thu, 1 Jun 2023 18:09:47 +0000 (14:09 -0400)]
ci: run ASan/UBSan in a single job
When we started running sanitizers in CI via 1c0962c0c4 (ci: add address
and undefined sanitizer tasks, 2022-10-20), we ran them as two separate
CI jobs, since as that commit notes, the combination "seems to take
forever".
And indeed, it does with gcc. However, since the previous commit
switched to using clang, the situation is different, and we can save
some CPU by using a single job for both. Comparing before/after CI runs,
this saved about 14 minutes (the single combined job took 54m, versus
44m plus 24m for ASan and UBSan jobs, respectively). That's wall-clock
and not CPU, but since our jobs are mostly CPU-bound, the two should be
closely proportional.
This does increase the end-to-end time of a CI run, though, since before
this patch the two jobs could run in parallel, and the sanitizer job is
our longest single job. It also means that we won't get a separate
result for "this passed with UBSan but not with ASan" or vice versa).
But as 1c0962c0c4 noted, that is not a very useful signal in practice.
Below are some more detailed timings of gcc vs clang that I measured by
running the test suite on my local workstation. Each measurement counts
only the time to run the test suite with each compiler (not the compile
time itself). We'll focus on the wall-clock times for simplicity, though
the CPU times follow roughly similar trends.
Here's a run with CC=gcc as a baseline:
real 1m12.931s
user 9m30.566s
sys 8m9.538s
Running with SANITIZE=address increases the time by a factor of ~4.7x:
real 5m40.352s
user 49m37.044s
sys 36m42.950s
Running with SANITIZE=undefined increases the time by a factor of ~1.7x:
real 2m5.956s
user 12m42.847s
sys 19m27.067s
So let's call that 6.4 time units to run them separately (where a unit
is the time it takes to run the test suite with no sanitizers). As a
simplistic model, we might imagine that running them together would take
5.4 units (we save 1 unit because we are no longer running the test
suite twice, but just paying the sanitizer overhead on top of a single
run).
But that's not what happens. Running with SANITIZE=address,undefined
results in a factor of 9.3x:
real 11m9.817s
user 77m31.284s
sys 96m40.454s
So not only did we not get faster when doing them together, we actually
spent 1.5x as much CPU as doing them separately! And while those
wall-clock numbers might not look too terrible, keep in mind that this
is on an unloaded 8-core machine. In the CI environment, wall-clock
times will be much closer to CPU times. So not only are we wasting CPU,
but we risk hitting timeouts.
Now let's try the same thing with clang. Here's our no-sanitizer
baseline run, which is almost identical to the gcc one (which is quite
convenient, because we can keep using the same "time units" to get an
apples-to-apples comparison):
real 1m11.844s
user 9m28.313s
sys 8m8.240s
And now again with SANITIZE=address, we get a 5x factor (so slightly
worse than gcc's 4.7x, though I wouldn't read too much into it; there is
a fair bit of run-to-run noise):
real 6m7.662s
user 49m24.330s
sys 44m13.846s
And with SANITIZE=undefined, we are at 1.5x, slightly outperforming gcc
(though again, that's probably mostly noise):
real 1m50.028s
user 11m0.973s
sys 16m42.731s
So running them separately, our total cost is 6.5x. But if we combine
them in a single run (SANITIZE=address,undefined), we get:
real 6m51.804s
user 52m32.049s
sys 51m46.711s
which is a factor of 5.7x. That's along the lines we'd hoped for!
Running them together saves us almost a whole time unit. And that's not
counting any time spent outside the test suite itself (starting the job,
setting up the environment, compiling) that we're no longer duplicating
by having two jobs.
So clang behaves like we'd hope: the overhead to run the sanitizers is
additive as you add more sanitizers. Whereas gcc's numbers seem very
close to multiplicative, almost as if the sanitizers were enforcing
their overheads on each other (though that is purely a guess on what is
going on; ultimately what matters to us is the amount of time it takes).
And that roughly matches the CI improvement I saw. A "time unit" there
is more like 12 minutes, and the observed time savings was 14 minutes
(with the extra presumably coming from avoiding duplicated setup, etc).
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Thu, 1 Jun 2023 18:03:51 +0000 (14:03 -0400)]
ci: use clang for ASan/UBSan checks
Both gcc and clang support the "address" and "undefined" sanitizers.
However, they may produce different results. We've seen at least two
real world cases where gcc missed a UBSan problem but clang found it:
1. Clang's UBSan (using clang 14.0.6) found a string index that was
subtracted to "-1", causing an out-of-bounds read (curiously this
didn't trigger ASan, but that may be because the string was in the
argv memory, not stack or heap). Using gcc (version 12.2.0) didn't
find the same problem.
Original thread:
https://lore.kernel.org/git/20230519005447.GA2955320@coredump.intra.peff.net/
2. Clang's UBSan (using clang 4.0.1) complained about pointer
arithmetic with NULL, but gcc at the time did not. This was in
2017, and modern gcc does seem to find the issue, though.
Original thread:
https://lore.kernel.org/git/32a8b949-638a-1784-7fba-948ae32206fc@web.de/
Since we don't otherwise have a particular preference for one compiler
over the other for this test, let's switch to the one that we think may
be more thorough.
Note that it's entirely possible that the two are simply _different_,
and we are trading off problems that gcc would find that clang wouldn't.
However, my subjective and anecdotal experience has been that clang's
sanitizer support is a bit more mature (e.g., I recall other oddities
around leak-checking where clang performed more sensibly).
Obviously running both and cross-checking the results would give us the
best coverage, but that's very expensive to run (and these are already
some of our most expensive CI jobs). So let's use clang as our best
guess, and we can re-evaluate if we get more data points.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Todd Zullinger [Thu, 25 May 2023 03:10:24 +0000 (23:10 -0400)]
t/lib-gpg: fix ssh-keygen -Y check-novalidate with openssh-9.0
OpenSSH-9.0 requires a namespace option with `-Y check-novalidate`.
This was added in openssh-portable commit a0b5816f8 (upstream:
ssh-keygen -Y check-novalidate requires namespace or SEGV, 2022-03-18).
The -n option was documented as a required option since check-novalidate
was added in openssh-portable 8aa2aa3cd (upstream: Allow testing
signature syntax and validity without verifying, 2019-09-16).
Signed-off-by: Todd Zullinger <tmz@pobox.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
ZheNing Hu [Tue, 23 May 2023 09:00:18 +0000 (09:00 +0000)]
ls-files: align format atoms with ls-tree
"git ls-files --format" can be used to format the output of
multiple file entries in the index, while "git ls-tree --format"
can be used to format the contents of a tree object. However,
the current set of %(objecttype), "(objectsize)", and
"%(objectsize:padded)" atoms supported by "git ls-files --format"
is a subset of what is available in "git ls-tree --format".
Users sometimes need to establish a unified view between the index
and tree, which can help with comparison or conversion between the two.
Therefore, this patch adds the missing atoms to "git ls-files --format".
"%(objecttype)" can be used to retrieve the object type corresponding
to a file in the index, "(objectsize)" can be used to retrieve the
object size corresponding to a file in the index, and "%(objectsize:padded)"
is the same as "(objectsize)", except with padded format.
Signed-off-by: ZheNing Hu <adlternative@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jiang Xin [Sat, 20 May 2023 00:44:08 +0000 (08:44 +0800)]
Merge branch 'master' of github.com:git/git
* 'master' of github.com:git/git:
A few more topics after 2.41-rc1
Git 2.41-rc1
t/lib-httpd: make CGIPassAuth support conditional
t9001: mark the script as no longer leak checker clean
send-email: clear the $message_id after validation
upload-pack: advertise capabilities when cloning empty repos
A bit more before -rc1
imap-send: include strbuf.h
run-command.c: fix missing include under `NO_PTHREADS`
test: do not negate test_path_is_* to assert absense
t2021: do not negate test_path_is_dir
tests: do not negate test_path_exists
doc/git-config: add unit for http.lowSpeedLimit
rebase -r: fix the total number shown in the progress
rebase --update-refs: fix loops
attr: teach "--attr-source=<tree>" global option to "git"
Junio C Hamano [Fri, 19 May 2023 20:35:56 +0000 (05:35 +0900)]
Merge branch 'jc/do-not-negate-test-helpers'
Small fixes.
* jc/do-not-negate-test-helpers:
test: do not negate test_path_is_* to assert absense
t2021: do not negate test_path_is_dir
tests: do not negate test_path_exists
Jeff King [Fri, 19 May 2023 00:05:43 +0000 (20:05 -0400)]
format-patch: free elements of rev.ref_message_ids list
When we are showing multiple patches with format-patch, we have to
repeatedly overwrite the rev.message_id field. We take care to avoid
leaking the old value by either freeing it, or adding it to
ref_message_ids, a string list of ids to reference in subsequent
messages.
But unfortunately we do leak the value via that string list. We try
to clear the string list, courtesy of 89f45cf4eb (format-patch: don't
leak "extra_headers" or "ref_message_ids", 2022-04-13). But since it was
initialized as "nodup", the string list doesn't realize it owns the
strings, and it leaks them.
We have two options here:
1. Continue to init with "nodup", but then tweak the value of
ref_message_ids.strdup_strings just before clearing.
2. Init with "dup", but use "append_nodup" when transferring ownership
of strings to the list. Clearing just works.
I picked the second here, as I think it calls attention to the tricky
part (transferring ownership via the nodup call).
There's one other related fix we have to do, though. We also insert the
result of clean_message_id() into the list. This _sometimes_ allocates
and sometimes does not, depending on whether we have to remove cruft
from the end of the string. Let's teach it to consistently return an
allocated string, so that the caller knows it must be freed.
There's no new test here, as the leak can already be seen in t4014.44 (as
well as others in that script). We can't mark all of t4014 as leak-free,
though, as there are other unrelated leaks that it triggers.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
We started unconditionally testing with CGIPassAuth directive but
it is unavailable in older Apache that ships with CentOS 7 that has
about a year of shelf-life still left. The test has conditionally
been disabled when running with an ancient Apache. This was a fix
for a recent regression caught before the release, so no need to
mention it in the release notes.
* jk/http-test-cgipassauth-unavailable-in-older-apache:
t/lib-httpd: make CGIPassAuth support conditional
The server side of "git clone" now advertises the necessary hints
to clients to help them to clone from an empty repository and learn
object hash algorithm and the (unborn) branch pointed at by HEAD,
even over the older v0/v1 protocol.
* bc/clone-empty-repo-via-protocol-v0:
upload-pack: advertise capabilities when cloning empty repos
Junio C Hamano [Fri, 19 May 2023 16:27:06 +0000 (09:27 -0700)]
Merge branch 'jc/send-email-pre-process-fix'
When "git send-email" that uses the validate hook is fed a message
without and then with Message-ID, it failed to auto-assign a unique
Message-ID to the former and instead reused the Message-ID from the
latter, which has been corrected. This was a fix for a recent
regression caught before the release, so no need to mention it in
the release notes.
* jc/send-email-pre-process-fix:
t9001: mark the script as no longer leak checker clean
send-email: clear the $message_id after validation
Sean Allred [Fri, 19 May 2023 04:17:54 +0000 (04:17 +0000)]
ls-remote doc: document the output format
While well-established, the output format of ls-remote was not actually
documented. This patch adds an OUTPUT section to the documentation
following the format of git-show-ref.txt (which has similar semantics).
Add a basic example immediately after this to solidify the 'normal'
output format.
Signed-off-by: Sean Allred <allred.sean@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Sean Allred [Fri, 19 May 2023 04:17:53 +0000 (04:17 +0000)]
ls-remote doc: explain what each example does
While it's good to have several examples to solidify the output pattern
and generally demonstrate how to use the command, most other EXAMPLES
sections (e.g., git-show-branch.txt, git-remote.txt) additionally
describe the problem/situation to which the example is applicable.
Follow this example in the ls-remote documentation.
Signed-off-by: Sean Allred <allred.sean@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Sean Allred [Fri, 19 May 2023 04:17:52 +0000 (04:17 +0000)]
ls-remote doc: show peeled tags in examples
Without `--refs`, this command will show peeled tags. Make this clearer
in the examples to further mitigate the possibility of surprises in
consuming scripts.
Signed-off-by: Sean Allred <allred.sean@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Sean Allred [Fri, 19 May 2023 04:17:51 +0000 (04:17 +0000)]
ls-remote doc: remove redundant --tags example
The --tags option is already demonstrated in the later example that
lists version-patterned tags. As it doesn't appear to add anything to
the documentation, it ought to be removed to keep the documentation
easier to read.
Signed-off-by: Sean Allred <allred.sean@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano [Fri, 19 May 2023 04:17:50 +0000 (04:17 +0000)]
show-branch doc: say <ref>, not <reference>
The glossary defines 'ref' as the official name of the thing,
and the output from "git grep -e '<ref' Documentation/" shows
that most everybody uses <ref>, not <reference>. In addition,
the page already says <ref> in its SYNOPSIS section for the
command when it is used in the mode to follow the reflogs.
Strictly speaking, many references of these should be updated to
<commit> after adding an explanation on how these <commit>s are
discovered (i.e. we take <rev>, <glob>, or <ref> and starting from
these commits, follow their ancestry or reflog entries to list
commits), but that would be a lot bigger change I would rather not
to do in this patch, whose primary purpose is to make the existing
documentation more consistent.
Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Sean Allred <allred.sean@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Sean Allred [Fri, 19 May 2023 04:17:49 +0000 (04:17 +0000)]
show-ref doc: update for internal consistency
- Use inline-code syntax for options where appropriate.
- Use code blocks to clarify output format.
- Use 'OID' (for 'object ID') instead of 'SHA-1' as we support
different hashing algorithms these days.
Signed-off-by: Sean Allred <allred.sean@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Fri, 19 May 2023 00:03:25 +0000 (20:03 -0400)]
format-patch: free rev.message_id when exiting
We may allocate a message-id string via gen_message_id(), but we never
free it, causing a small leak. This can be demonstrated by running t9001
with a leak-checking build. The offending test is the one touched by 3ece9bf0f9 (send-email: clear the $message_id after validation,
2023-05-17), but the leak is much older than that. The test was simply
unlucky enough to trigger the leaking code path for the first time.
We can fix this by freeing the string at the end of the function. We can
also re-mark the test script as leak-free, effectively reverting 20bd08aefb (t9001: mark the script as no longer leak checker clean,
2023-05-17).
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Thu, 18 May 2023 19:21:02 +0000 (15:21 -0400)]
t/lib-httpd: make CGIPassAuth support conditional
Commit 988aad99b4 (t5563: add tests for basic and anoymous HTTP access,
2023-02-27) added tests that require Apache to support the CGIPassAuth
directive, which was added in Apache 2.4.13. This is fairly old (~8
years), but recent enough that we still encounter it in the wild (e.g.,
RHEL/CentOS 7, which is not EOL until June 2024).
We can live with skipping the new tests on such a platform. But
unfortunately, since the directive is used unconditionally in our
apache.conf, it means the web server fails to start entirely, and we
cannot run other HTTP tests at all (e.g., the basic ones in t5551).
We can fix that by making the config conditional, and only triggering it
for t5563. That solves the problem for t5551 (which then ignores the
directive entirely). For t5563, we'd see apache complain in start_httpd;
with the default setting of GIT_TEST_HTTPD, we'd then skip the whole
script.
But that leaves one small problem: people may set GIT_TEST_HTTPD=1
explicitly, which instructs the tests to fail (rather than skip) when we
can't start the webserver (to avoid accidentally missing some tests).
This could be worked around by having the user manually set
GIT_SKIP_TESTS on a platform with an older Apache. But we can be a bit
friendlier by doing the version check ourselves and setting an
appropriate prereq. We'll use the (lack of) prereq to then skip the rest
of t5563. In theory we could use the prereq to skip individual tests, but
in practice this whole script depends on it.
Reported-by: Todd Zullinger <tmz@pobox.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Shuqi Liang [Thu, 18 May 2023 15:44:54 +0000 (11:44 -0400)]
diff-tree: integrate with sparse index
The index is read in 'cmd_diff_tree' at two points:
1. The first index read was added in fd66bcc31ff (diff-tree: read the
index so attribute checks work in bare repositories, 2017-12-06) to deal
with reading '.gitattributes' content. 77efbb366ab (attr: be careful
about sparse directories, 2021-09-08) established that, in a sparse
index, we do _not_ try to load a '.gitattributes' file from within a
sparse directory.
2. The second index access point is involved in rename detection,
specifically when reading from stdin.This was initially added in f0c6b2a2fd9 ([PATCH] Optimize diff-tree -[CM]--stdin, 2005-05-27), where
'setup' was set to 'DIFF_SETUP_USE_SIZE_CACHE |DIFF_SETUP_USE_CACHE'.
That assignment was later modified to drop the'DIFF_SETUP_USE_CACHE' in ff7fe37b053 (diff.c: move read_index() code back to the caller,
2018-08-13).However, 'DIFF_SETUP_USE_SIZE_CACHE' seems to be unused as
of 6e0b8ed6d35 (diff.c: do not use a separate "size cache"., 2007-05-07)
and nothing about 'detect_rename' otherwise indicates index usage.
Hence we can just set the requires-full-index to false for "diff-tree".
Add tests that verify that 'git diff-tree' behaves correctly when the
sparse index is enabled and test to ensure the index is not expanded.
The `p2000` tests demonstrate a ~98% execution time reduction for
'git diff-tree' using a sparse index:
Test before after
-----------------------------------------------------------------------
2000.94: git diff-tree HEAD (full-v3) 0.05 0.04 -20.0%
2000.95: git diff-tree HEAD (full-v4) 0.06 0.05 -16.7%
2000.96: git diff-tree HEAD (sparse-v3) 0.59 0.01 -98.3%
2000.97: git diff-tree HEAD (sparse-v4) 0.61 0.01 -98.4%
2000.98: git diff-tree HEAD -- f2/f4/a (full-v3) 0.05 0.05 +0.0%
2000.99: git diff-tree HEAD -- f2/f4/a (full-v4) 0.05 0.04 -20.0%
2000.100: git diff-tree HEAD -- f2/f4/a (sparse-v3) 0.58 0.01 -98.3%
2000.101: git diff-tree HEAD -- f2/f4/a (sparse-v4) 0.55 0.01 -98.2%
Helped-by: Victoria Dye <vdye@github.com> Signed-off-by: Shuqi Liang <cheskaqiqi@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano [Wed, 17 May 2023 21:10:39 +0000 (14:10 -0700)]
t9001: mark the script as no longer leak checker clean
The test uses "format-patch --thread" which is known to leak the
generated message ID list.
Plugging these leaks involves straightening out the memory ownership
rules around rev_info.message_id and rev_info.ref_message_ids, and
is beyond the scope of send-email fix, so for now mark the test as
leaky to unblock the topic before the release.
Junio C Hamano [Wed, 17 May 2023 21:10:39 +0000 (14:10 -0700)]
send-email: clear the $message_id after validation
Recently git-send-email started parsing the same message twice, once
to validate _all_ the message before sending even the first one, and
then after the validation hook is happy and each message gets sent,
to read the contents to find out where to send to etc.
Unfortunately, the effect of reading the messages for validation
lingered even after the validation is done. Namely $message_id gets
assigned if exists in the input files but the variable is global,
and it is not cleared before pre_process_file runs. This causes
reading a message without a message-id followed by reading a message
with a message-id to misbehave---the sub reports as if the message
had the same id as the previously written one.
Clear the variable before starting to read the headers in
pre_process_file.
Tested-by: Douglas Anderson <dianders@chromium.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
brian m. carlson [Wed, 17 May 2023 19:24:43 +0000 (19:24 +0000)]
upload-pack: advertise capabilities when cloning empty repos
When cloning an empty repository, protocol versions 0 and 1 currently
offer nothing but the header and flush packets for the /info/refs
endpoint. This means that no capabilities are provided, so the client
side doesn't know what capabilities are present.
However, this does pose a problem when working with SHA-256
repositories, since we use the capabilities to know the remote side's
object format (hash algorithm). As of 8b214c2e9d ("clone: propagate
object-format when cloning from void", 2023-04-05), this has been fixed
for protocol v2, since there we always read the hash algorithm from the
remote.
Fortunately, the push version of the protocol already indicates a clue
for how to solve this. When the /info/refs endpoint is accessed for a
push and the remote is empty, we include a dummy "capabilities^{}" ref
pointing to the all-zeros object ID. The protocol documentation already
indicates this should _always_ be sent, even for fetches and clones, so
let's just do that, which means we'll properly announce the hash
algorithm as part of the capabilities. This just works with the
existing code because we share the same ref code for fetches and clones,
and libgit2, JGit, and dulwich do as well.
There is one minor issue to fix, though. If we called send_ref with
namespaces, we would return NULL with the capabilities entry, which
would cause a crash. Instead, let's refactor out a function to print
just the ref itself without stripping the namespace and use it for our
special capabilities entry.
Add several sets of tests for HTTP as well as for local clones. The
behavior can be slightly different for HTTP versus a local or SSH clone
because of the stateless-rpc functionality, so it's worth testing both.
Signed-off-by: brian m. carlson <bk2204@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch: use `fetch_config` to store "submodule.fetchJobs" value
Move the parsed "submodule.fetchJobs" config value into the
`fetch_config` structure. This reduces our reliance on global variables
and further unifies the way we parse the configuration in git-fetch(1).
Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch: use `fetch_config` to store "fetch.parallel" value
Move the parsed "fetch.parallel" config value into the `fetch_config`
structure. This reduces our reliance on global variables and further
unifies the way we parse the configuration in git-fetch(1).
Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch: use `fetch_config` to store "fetch.recurseSubmodules" value
Move the parsed "fetch.recurseSubmodules" config value into the
`fetch_config` structure. This reduces our reliance on global variables
and further unifies the way we parse the configuration in git-fetch(1).
Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch: use `fetch_config` to store "fetch.showForcedUpdates" value
Move the parsed "fetch.showForcedUpdaets" config value into the
`fetch_config` structure. This reduces our reliance on global variables
and further unifies the way we parse the configuration in git-fetch(1).
Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch: use `fetch_config` to store "fetch.pruneTags" value
Move the parsed "fetch.pruneTags" config value into the `fetch_config`
structure. This reduces our reliance on global variables and further
unifies the way we parse the configuration in git-fetch(1).
Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch: use `fetch_config` to store "fetch.prune" value
Move the parsed "fetch.prune" config value into the `fetch_config`
structure. This reduces our reliance on global variables and further
unifies the way we parse the configuration in git-fetch(1).
Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
The `fetch_config` structure currently only has a single member, which
is the `display_format`. We're about extend it to contain all parsed
config values and will thus need it available in most of the code.
Prepare for this change by passing through the `fetch_config` directly
instead of only passing its single member.
Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Drop the `NULL` check for `remote_ref` in `update_local_ref()`. The
function only has a single caller, and that caller always passes in a
non-`NULL` value.
This fixes a false positive in Coverity.
Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch: drop unused DISPLAY_FORMAT_UNKNOWN enum value
With 50957937f9 (fetch: introduce `display_format` enum, 2023-05-10), a
new enumeration was introduced to determine the display format that is
to be used by git-fetch(1). The `DISPLAY_FORMAT_UNKNOWN` value isn't
ever used though, and neither do we rely on the explicit `0` value for
initialization anywhere.
Remove the enum value.
Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Christian Hesse [Wed, 17 May 2023 07:06:32 +0000 (09:06 +0200)]
imap-send: include strbuf.h
We make liberal use of the strbuf API functions and types, but the
inclusion of <strbuf.h> comes indirectly by including <http.h>,
which does not happen if you build with NO_CURL.
Signed-off-by: Christian Hesse <mail@eworm.de> Helped-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Taylor Blau [Tue, 16 May 2023 21:33:43 +0000 (17:33 -0400)]
run-command.c: fix missing include under `NO_PTHREADS`
Git 2.41-rc0 fails to compile run-command.c with `NO_PTHREADS` defined,
i.e.
$ make NO_PTHREADS=1 run-command.o
as `ALLOC_GROW()` macro is used in the `atexit()` emulation but the
macro definition is not available.
This bisects to 36bf195890 (alloc.h: move ALLOC_GROW() functions from
cache.h, 2023-02-24), which replaced includes of <cache.h> with
<alloc.h>, which is the new home of `ALLOC_GROW()` (and
`ALLOC_GROW_BY()`).
We can see that run-command.c is the only one that try to use these
macros without including <alloc.h>.
The "compat/" file only talks about the macro in the comment,
and is not broken.
We could fix this by conditionally including "alloc.h" when
`NO_PTHREADS` is defined. But we have relatively few examples of
conditional includes throughout the tree[^1].
Instead, include "alloc.h" unconditionally in run-command.c to allow it
to successfully compile even when NO_PTHREADS is defined.
Reported-by: Randall S. Becker <randall.becker@nexbridge.ca> Co-authored-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
tag: keep the message file in case ref transaction fails
The ref transaction can fail after the user has written their tag
message. In particular, if there exists a tag `foo/bar` and `git tag -a
foo` is said then the command will only fail once it tries to write
`refs/tags/foo`, which is after the file has been unlinked.
Hold on to the message file for a little longer so that it is not
unlinked before the fatal error occurs.
Signed-off-by: Kristoffer Haugsbakk <code@khaugsbakk.name> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Document `TAG_EDITMSG` which we have told the user about on unsuccessful
command invocations since commit 3927bbe9a4 (tag: delete TAG_EDITMSG
only on successful tag, 2008-12-06).
Introduce this documentation since we are going to add tests for the
lifetime of this file in the case of command failure and success.
Use the documentation for `COMMIT_EDITMSG` from `git-commit.txt` as a
template since these two files share the same purpose.[1]
“ This matches the behavior of COMMIT_EDITMSG, which stays around
in case of error.
Suggested-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Kristoffer Haugsbakk <code@khaugsbakk.name> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano [Tue, 16 May 2023 02:26:46 +0000 (19:26 -0700)]
test: do not negate test_path_is_* to assert absense
These tests use "! test_path_is_dir" or "! test_path_is_file" to
ensure that the path is not recursively checked out or "submodule
update" did not touch the working tree.
Use "test_path_is_missing" to assert that the path does not exist,
instead of negating test_path_is_* helpers; they are designed to be
loud in wrong occasions. Besides, negating "test_path_is_dir" would
mean we would be happy if a file exists there, which is not the case
for these tests.
Junio C Hamano [Tue, 16 May 2023 02:26:45 +0000 (19:26 -0700)]
t2021: do not negate test_path_is_dir
In this test, a path (some_dir) that is originally a directory is to
be removed and then to be replaced with a file of the same name.
The expectation is that the path becomes a file at the end.
However, "! test_path_is_dir some_dir" is used to catch a breakage
that leaves the path as a directory.
But as with all the "test_path_is..." helpers, this use of the
helper makes it loud when the expectation (i.e. it is a directory)
is met, and otherwise is silent when it is not---this does not help
debugging.
Be more explicit and state that we expect the path to become a file.
Junio C Hamano [Tue, 16 May 2023 02:26:44 +0000 (19:26 -0700)]
tests: do not negate test_path_exists
As a way to assert the path 'foo' is missing, "! test_path_exists
foo" is a poor way to do so, as the helper is designed to complain
when 'foo' is missing, but the intention of the author who used
negated form was to make sure it does not exist. This does not
help debugging the tests.
Use test_path_is_missing instead, which is a more appropriate helper.
Jiang Xin [Tue, 16 May 2023 02:19:48 +0000 (10:19 +0800)]
Merge tag 'v2.41.0-rc0'
Git 2.41-rc0
* tag 'v2.41.0-rc0': (508 commits)
Git 2.41-rc0
t5583: fix shebang line
merge-tree: load default git config
fetch: introduce machine-parseable "porcelain" output format
fetch: move option related variables into main function
fetch: lift up parsing of "fetch.output" config variable
fetch: introduce `display_format` enum
fetch: refactor calculation of the display table width
fetch: print left-hand side when fetching HEAD:foo
fetch: add a test to exercise invalid output formats
fetch: split out tests for output format
fetch: fix `--no-recurse-submodules` with multi-remote fetches
The eighteenth batch
The seventeenth batch
diff-files: integrate with sparse index
t1092: add tests for `git diff-files`
test: rev-parse-upstream: add missing cmp
t: drop "verbose" helper function
t7001: use "ls-files --format" instead of "cut"
t7001: avoid git on upstream of pipe
...
Junio C Hamano [Mon, 15 May 2023 20:59:07 +0000 (13:59 -0700)]
Merge branch 'ps/fetch-output-format'
"git fetch" learned the "--porcelain" option that emits what it did
in a machine-parseable format.
* ps/fetch-output-format:
fetch: introduce machine-parseable "porcelain" output format
fetch: move option related variables into main function
fetch: lift up parsing of "fetch.output" config variable
fetch: introduce `display_format` enum
fetch: refactor calculation of the display table width
fetch: print left-hand side when fetching HEAD:foo
fetch: add a test to exercise invalid output formats
fetch: split out tests for output format
fetch: fix `--no-recurse-submodules` with multi-remote fetches
The "--stdin" option of "git name-rev" has been replaced with
the "--annotate-stdin" option more than a year ago. We stop
advertising it in the "git name-rev -h" output.
* jc/name-rev-deprecate-stdin-further:
name-rev: make --stdin hidden
projects / thirdparty / git.git / log
