grep tests: move binary pattern tests into their own file
Move the tests for "-f <file>" where "<file>" contains a NUL byte
pattern into their own file. I added most of these tests in 966be95549 ("grep: add tests to fix blind spots with \0 patterns",
2017-05-20).
Whether a regex engine supports matching binary content is very
different from whether it matches binary patterns. Since 2f8952250a ("regex: add regexec_buf() that can work on a non
NUL-terminated string", 2016-09-21) we've required REG_STARTEND of our
regex engines so we can match binary content, but only the PCRE v2
engine can sensibly match binary patterns.
Since 9eceddeec6 ("Use kwset in grep", 2011-08-21) we've been punting
patterns containing NUL-byte and considering them fixed, except in
cases where "--ignore-case" is provided and they're non-ASCII, see 5c1ebcca4d ("grep/icase: avoid kwsset on literal non-ascii strings",
2016-06-25). Subsequent commits will change this behavior.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Move the "grep binary" test case added in aca20dd558 ("grep: add test
script for binary file handling", 2010-05-22) so that it lives
alongside the rest of the "grep" tests in t781*. This would have left
a gap in the t/700* namespace, so move a "filter-branch" test down,
leaving the "t7010-setup.sh" test as the next one after that.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
grep: inline the return value of a function call used only once
Since e944d9d932 ("grep: rewrite an if/else condition to avoid
duplicate expression", 2016-06-25) the "ascii_only" variable has only
been used once in compile_regexp(), let's just inline it there.
This makes the code easier to read, and might make it marginally
faster depending on compiler optimizations.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
t4210: skip more command-line encoding tests on MinGW
In 5212f91deb ("t4210: skip command-line encoding tests on mingw",
2014-07-17) the positive tests in this file were skipped. That left
the negative tests that don't produce a match.
An upcoming change to migrate the "fixed" backend of grep to PCRE v2
will cause these "log" commands to produce an error instead on
MinGW. This is because the command-line on that platform implicitly
has its encoding changed before being passed to git. See [1].
grep: don't use PCRE2?_UTF8 with "log --encoding=<non-utf8>"
Fix a bug introduced in 18547aacf5 ("grep/pcre: support utf-8",
2016-06-25) that was missed due to a blindspot in our tests, as
discussed in the previous commit. I then blindly copied the same bug
in 94da9193a6 ("grep: add support for PCRE v2", 2017-06-01) when
adding the PCRE v2 code.
We should not tell PCRE that we're processing UTF-8 just because we're
dealing with non-ASCII. In the case of e.g. "log --encoding=<...>"
under is_utf8_locale() the haystack might be in ISO-8859-1, and the
needle might be in a non-UTF-8 encoding.
Maybe we should be more strict here and die earlier? Should we also be
converting the needle to the encoding in question, and failing if it's
not a string that's valid in that encoding? Maybe.
But for now matching this as non-UTF8 at least has some hope of
producing sensible results, since we know that our default heuristic
of assuming the text to be matched is in the user locale encoding
isn't true when we've explicitly encoded it to be in a different
encoding.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
log tests: test regex backends in "--encode=<enc>" tests
Improve the tests added in 04deccda11 ("log: re-encode commit messages
before grepping", 2013-02-11) to test the regex backends. Those tests
never worked as advertised, due to the is_fixed() optimization in
grep.c (which was in place at the time), and the needle in the tests
being a fixed string.
We'd thus always use the "fixed" backend during the tests, which would
use the kwset() backend. This backend liberally accepts any garbage
input, so invalid encodings would be silently accepted.
In a follow-up commit we'll fix this bug, this test just demonstrates
the existing issue.
In practice this issue happened on Windows, see [1], but due to the
structure of the existing tests & how liberal the kwset code is about
garbage we missed this.
Cover this blind spot by testing all our regex engines. The PCRE
backend will spot these invalid encodings. It's possible that this
test breaks the "basic" and "extended" backends on some systems that
are more anal than glibc about the encoding of locale issues with
POSIX functions that I can remember, but PCRE is more careful about
the validation.
Junio C Hamano [Fri, 21 Jun 2019 18:24:11 +0000 (11:24 -0700)]
Merge branch 'an/ignore-doc-update'
The description about slashes in gitignore patterns (used to
indicate things like "anchored to this level only" and "only
matches directories") has been revamped.
* an/ignore-doc-update:
gitignore.txt: make slash-rules more readable
Junio C Hamano [Fri, 21 Jun 2019 18:24:09 +0000 (11:24 -0700)]
Merge branch 'md/list-objects-filter-memfix'
The filter_data used in the list-objects-filter (which manages a
lazily sparse clone repository) did not use the dynamic array API
correctly---'nr' is supposed to point at one past the last element
of the array in use. This has been corrected.
* md/list-objects-filter-memfix:
list-objects-filter: correct usage of ALLOC_GROW
"git fetch" into a lazy clone forgot to fetch base objects that are
necessary to complete delta in a thin packfile, which has been
corrected.
* jt/partial-clone-missing-ref-delta-base:
t5616: cover case of client having delta base
t5616: use correct flag to check object is missing
index-pack: prefetch missing REF_DELTA bases
t5616: refactor packfile replacement
Junio C Hamano [Mon, 17 Jun 2019 17:15:20 +0000 (10:15 -0700)]
Merge branch 'xl/record-partial-clone-origin'
When creating a partial clone, the object filtering criteria is
recorded for the origin of the clone, but this incorrectly used a
hardcoded name "origin" to name that remote; it has been corrected
to honor the "--origin <name>" option.
* xl/record-partial-clone-origin:
clone: respect user supplied origin name when setting up partial clone
Junio C Hamano [Mon, 17 Jun 2019 17:15:20 +0000 (10:15 -0700)]
Merge branch 'pb/request-pull-verify-remote-ref'
"git request-pull" learned to warn when the ref we ask them to pull
from in the local repository and in the published repository are
different.
* pb/request-pull-verify-remote-ref:
request-pull: warn if the remote object is not the same as the local one
request-pull: quote regex metacharacters in local ref
Junio C Hamano [Mon, 17 Jun 2019 17:15:19 +0000 (10:15 -0700)]
Merge branch 'mm/p4-unshelve-windows-fix'
The command line to invoke a "git cat-file" command from inside
"git p4" was not properly quoted to protect a caret and running a
broken command on Windows, which has been corrected.
* mm/p4-unshelve-windows-fix:
p4 unshelve: fix "Not a valid object name HEAD0" on Windows
Junio C Hamano [Mon, 17 Jun 2019 17:15:18 +0000 (10:15 -0700)]
Merge branch 'es/first-contrib-tutorial'
A new tutorial targetting specifically aspiring git-core
developers.
* es/first-contrib-tutorial:
doc: add some nit fixes to MyFirstContribution
documentation: add anchors to MyFirstContribution
documentation: add tutorial for first contribution
Junio C Hamano [Mon, 17 Jun 2019 17:15:17 +0000 (10:15 -0700)]
Merge branch 'ba/clone-remote-submodules'
"git clone --recurse-submodules" learned to set up the submodules
to ignore commit object names recorded in the superproject gitlink
and instead use the commits that happen to be at the tip of the
remote-tracking branches from the get-go, by passing the new
"--remote-submodules" option.
* ba/clone-remote-submodules:
clone: add `--remote-submodules` flag
"git merge --squash" is designed to update the working tree and the
index without creating the commit, and this cannot be countermanded
by adding the "--commit" option; the command now refuses to work
when both options are given.
* vv/merge-squash-with-explicit-commit:
merge: refuse --commit with --squash
"git bundle verify" needs to see if prerequisite objects exist in
the receiving repository, but the command did not check if we are
in a repository upfront, which has been corrected.
* js/bundle-verify-require-object-store:
bundle verify: error out if called without an object database
Junio C Hamano [Mon, 17 Jun 2019 17:15:15 +0000 (10:15 -0700)]
Merge branch 'jk/am-i-resolved-fix'
"git am -i --resolved" segfaulted after trying to see a commit as
if it were a tree, which has been corrected.
* jk/am-i-resolved-fix:
am: fix --interactive HEAD tree resolution
am: drop tty requirement for --interactive
am: read interactive input from stdin
am: simplify prompt response handling
Junio C Hamano [Mon, 17 Jun 2019 17:15:15 +0000 (10:15 -0700)]
Merge branch 'jk/HEAD-symref-in-xfer-namespaces'
The server side support for "git fetch" used to show incorrect
value for the HEAD symbolic ref when the namespace feature is in
use, which has been corrected.
* jk/HEAD-symref-in-xfer-namespaces:
upload-pack: strip namespace from symref data
A "merge -c" instruction during "git rebase --rebase-merges" should
give the user a chance to edit the log message, even when there is
otherwise no need to create a new merge and replace the existing
one (i.e. fast-forward instead), but did not. Which has been
corrected.
* pw/rebase-edit-message-for-replayed-merge:
rebase -r: always reword merge -c
Junio C Hamano [Thu, 13 Jun 2019 20:19:43 +0000 (13:19 -0700)]
Merge branch 'ab/deprecate-R-for-dynpath'
The way of specifying the path to find dynamic libraries at runtime
has been simplified. The old default to pass -R/path/to/dir has been
replaced with the new default to pass -Wl,-rpath,/path/to/dir,
which is the more recent GCC uses. Those who need to build with an
old GCC can still use "CC_LD_DYNPATH=-R"
* ab/deprecate-R-for-dynpath:
Makefile: remove the NO_R_TO_GCC_LINKER flag
Junio C Hamano [Thu, 13 Jun 2019 20:19:43 +0000 (13:19 -0700)]
Merge branch 'mh/import-transport-fd-fix'
The ownership rule for the file descriptor to fast-import remote
backend was mixed up, leading to unrelated file descriptor getting
closed, which has been fixed.
* mh/import-transport-fd-fix:
Use xmmap_gently instead of xmmap in use_pack
dup() the input fd for fast-import used for remote helpers
Junio C Hamano [Thu, 13 Jun 2019 20:19:41 +0000 (13:19 -0700)]
Merge branch 'ab/fail-prereqs-in-test'
Developer support to emulate unsatisfied prerequisites in tests to
ensure that the remainer of the tests still succeeds when tests
with prerequisites are skipped.
* ab/fail-prereqs-in-test:
tests: add a special setup where prerequisites fail
Junio C Hamano [Thu, 13 Jun 2019 20:19:40 +0000 (13:19 -0700)]
Merge branch 'js/rebase-cleanup'
Update supporting parts of "git rebase" to remove code that should
no longer be used.
* js/rebase-cleanup:
rebase: fold git-rebase--common into the -p backend
sequencer: the `am` and `rebase--interactive` scripts are gone
.gitignore: there is no longer a built-in `git-rebase--interactive`
t3400: stop referring to the scripted rebase
Drop unused git-rebase--am.sh
Junio C Hamano [Thu, 13 Jun 2019 20:19:40 +0000 (13:19 -0700)]
Merge branch 'nd/worktree-name-sanitization'
In recent versions of Git, per-worktree refs are exposed in
refs/worktrees/<wtname>/ hierarchy, which means that worktree names
must be a valid refname component. The code now sanitizes the names
given to worktrees, to make sure these refs are well-formed.
Junio C Hamano [Thu, 13 Jun 2019 20:19:39 +0000 (13:19 -0700)]
Merge branch 'en/fast-export-encoding'
The "git fast-export/import" pair has been taught to handle commits
with log messages in encoding other than UTF-8 better.
* en/fast-export-encoding:
fast-export: do automatic reencoding of commit messages only if requested
fast-export: differentiate between explicitly UTF-8 and implicitly UTF-8
fast-export: avoid stripping encoding header if we cannot reencode
fast-import: support 'encoding' commit header
t9350: fix encoding test to actually test reencoding
Junio C Hamano [Thu, 13 Jun 2019 20:19:34 +0000 (13:19 -0700)]
Merge branch 'jk/unused-params-final-batch'
* jk/unused-params-final-batch:
verify-commit: simplify parameters to run_gpg_verify()
show-branch: drop unused parameter from show_independent()
rev-list: drop unused void pointer from finish_commit()
remove_all_fetch_refspecs(): drop unused "remote" parameter
receive-pack: drop unused "commands" from prepare_shallow_update()
pack-objects: drop unused rev_info parameters
name-rev: drop unused parameters from is_better_name()
mktree: drop unused length parameter
wt-status: drop unused status parameter
read-cache: drop unused parameter from threaded load
clone: drop dest parameter from copy_alternates()
submodule: drop unused prefix parameter from some functions
builtin: consistently pass cmd_* prefix to parse_options
cmd_{read,write}_tree: rename "unused" variable that is used
Junio C Hamano [Thu, 13 Jun 2019 20:18:46 +0000 (13:18 -0700)]
Merge branch 'sb/format-patch-base-patch-id-fix'
The "--base" option of "format-patch" computed the patch-ids for
prerequisite patches in an unstable way, which has been updated to
compute in a way that is compatible with "git patch-id --stable".
* sb/format-patch-base-patch-id-fix:
format-patch: make --base patch-id output stable
format-patch: inform user that patch-id generation is unstable
Junio C Hamano [Thu, 13 Jun 2019 20:18:46 +0000 (13:18 -0700)]
Merge branch 'nd/init-relative-template-fix'
A relative pathname given to "git init --template=<path> <repo>"
ought to be relative to the directory "git init" gets invoked in,
but it instead was made relative to the repository, which has been
corrected.
* nd/init-relative-template-fix:
init: make --template path relative to $CWD
Junio C Hamano [Thu, 13 Jun 2019 20:18:46 +0000 (13:18 -0700)]
Merge branch 'ab/send-email-transferencoding-fix'
Since "git send-email" learned to take 'auto' as the value for the
transfer-encoding, it by mistake stopped honoring the values given
to the configuration variables sendemail.transferencoding and/or
sendemail.<ident>.transferencoding. This has been corrected to
(finally) redoing the order of setting the default, reading the
configuration and command line options.
* ab/send-email-transferencoding-fix:
send-email: fix regression in sendemail.identity parsing
send-email: document --no-[to|cc|bcc]
send-email: fix broken transferEncoding tests
send-email: remove cargo-culted multi-patch pattern in tests
send-email: do defaults -> config -> getopt in that order
send-email: rename the @bcclist variable for consistency
send-email: move the read_config() function above getopts
Jonathan Tan [Tue, 11 Jun 2019 21:06:47 +0000 (14:06 -0700)]
t5616: cover case of client having delta base
When fetching into a partial clone, Git first prefetches missing
REF_DELTA bases from the promisor remote. (This feature was introduced
in [1].) But as can be seen in a recent test coverage report [2], the
case in which a REF_DELTA base is already present is not covered by
tests.
Jonathan Tan [Tue, 11 Jun 2019 21:06:46 +0000 (14:06 -0700)]
t5616: use correct flag to check object is missing
If we want to check whether an object is missing, the correct flag to
pass to rev-list is --ignore-missing; --exclude-promisor-objects will
exclude any object that came from the promisor remote, whether it is
present or missing. Use the correct flag.
Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Elijah Newren [Tue, 4 Jun 2019 20:27:50 +0000 (13:27 -0700)]
merge-recursive: restore accidentally dropped setting of path
In commit 8daec1df03de ("merge-recursive: switch from (oid,mode) pairs
to a diff_filespec", 2019-04-05), we actually switched from
(oid,mode,path) triplets to a diff_filespec -- but most callsites in the
patch only needed to worry about oid and mode so the commit message
focused on that. The oversight in the commit message apparently spilled
over to the code as well; one of the dozen or so callsites accidentally
dropped the setting of the path in the conversion. Restore the path
setting in that location.
Also, this pointed out that our testsuite was lacking a good rename/add
test, at least one that involved the need for merge content with the
rename. Add such a test, and since rename/add vs. add/rename could
possibly be important, redo the merge the opposite direction to make
sure we don't have issues with the direction of the merge. These
testcases failed before restoring the setting of path, but with the
paths appropriately set the testcases both pass.
Reported-by: Ben Humphreys <behumphreys@atlassian.com> Based-on-patch-by: SZEDER Gábor <szeder.dev@gmail.com> Tested-by: Ben Humphreys <behumphreys@atlassian.com> Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Matthew DeVore [Tue, 4 Jun 2019 17:57:05 +0000 (10:57 -0700)]
url: do not allow %00 to represent NUL in URLs
There is no reason to allow %00 to terminate a string, so do not allow it.
Otherwise, we end up returning arbitrary content in the string (that which is
after the %00) which is effectively hidden from callers and can escape sanity
checks and validation, and possible be used in tandem with a security
vulnerability to introduce a payload.
Helped-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Matthew DeVore <matvore@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Matthew DeVore [Tue, 4 Jun 2019 17:57:04 +0000 (10:57 -0700)]
url: do not read past end of buffer
url_decode_internal could have been tricked into reading past the length
of the **query buffer if there are fewer than 2 characters after a % (in
a null-terminated string, % would have to be the last character).
Prevent this from happening by checking len before decoding the %
sequence.
Helped-by: René Scharfe <l.s.r@web.de> Signed-off-by: Matthew DeVore <matvore@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jiang Xin [Tue, 4 Jun 2019 00:24:28 +0000 (08:24 +0800)]
Merge branch 'master' of git://git.kernel.org/pub/scm/git/git
* 'master' of git://git.kernel.org/pub/scm/git/git:
Git 2.22-rc3
i18n: fix typos found during l10n for git 2.22.0
RelNotes: minor typo fixes in 2.22.0 draft
list-objects-filter: disable 'sparse:path' filters
Jiang Xin [Sun, 2 Jun 2019 15:11:22 +0000 (23:11 +0800)]
i18n: fix typos found during l10n for git 2.22.0
Fix two typos introduced by the following commits:
+ 31fba9d3b4 (diff-parseopt: convert --[src|dst]-prefix, 2019-03-24)
+ ed8b4132c8 (remote-curl: mark all error messages for translation,
2019-03-05)
Signed-off-by: Jiang Xin <worldhello.net@gmail.com> Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Chris Mayo [Thu, 30 May 2019 18:39:10 +0000 (19:39 +0100)]
send-email: update documentation of required Perl modules
Improve and complete the list of required email related Perl modules,
clarifying which are core Perl modules and remove Net::SMTP::SSL.
git-send-email uses the TLS support in the Net::SMTP core module from
recent versions of Perl. Documenting the minimum version is complex
because of separate numbering for Perl (5.21.5~169), Net:SMTP (2.34)
and libnet (3.01). Version numbers from commit: bfbfc9a953 ("send-email: Net::SMTP::starttls was introduced in v2.34",
2017-05-31).
Users of older Perl versions without Net::SMTP::SSL installed will get a
clear error message.
Signed-off-by: Chris Mayo <aklhfex@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Matthew DeVore [Fri, 31 May 2019 18:46:06 +0000 (11:46 -0700)]
list-objects-filter: correct usage of ALLOC_GROW
In the sparse filter data, array_frame array is used in a way such that
nr is the index of the last element. Fix this so that nr is actually the
number of elements in the array.
The filter_sparse_free function also has an unaddressed TODO to free the
memory associated with the sparse filter data. Address that TODO and fix
the memory leak.
Signed-off-by: Matthew DeVore <matvore@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>