doc: notes: clearly state that --stripspace is the default
Clearly state when which of the regular and negated form of the
option take effect.[1]
Also mention the subtle behavior that occurs when you mix options like
`-m` and `-C`, including a note that it might be fixed in the future.
The topic was brought up on v8 of the `--separator` series.[2][3]
[1]: https://lore.kernel.org/git/xmqqcyct1mtq.fsf@gitster.g/
[2]: https://lore.kernel.org/git/xmqq4jp326oj.fsf@gitster.g/
† 3: v11 was the version that landed
Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Kristoffer Haugsbakk <code@khaugsbakk.name> Signed-off-by: Junio C Hamano <gitster@pobox.com>
doc: notes: remove stripspace discussion from other options
Cleaning up whitespace in metadata is typical porcelain behavior and
this default does not need to be pointed out.[1] Only speak up when
the default `--stripspace` is not used.
Also remove all misleading mentions of comment lines in the process;
see the previous commit.
Also remove the period that trails the parenthetical here.
† 1: See `-F` in git-commit(1) which has nothing to say about whitespace
cleanup. The cleanup discussion is on `--cleanup`.
Signed-off-by: Kristoffer Haugsbakk <code@khaugsbakk.name> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Document this option by copying the bullet list from git-stripspace(1).
A bullet list is cleaner when there are this many points to consider.
We also get a more standardized description of the multiple-blank-lines
behavior. Compare the repeating (git-notes(1)):
empty lines other than a single line between paragraphs
With (git-stripspace(1)):
multiple consecutive empty lines
And:
leading [...] whitespace
With:
empty lines from the beginning
Leading whitespace in the form of spaces (indentation) are not removed.
However, empty lines at the start of the message are removed.
Note that we drop the mentions of comment line handling because they are
wrong; this option does not control how lines which can be recognized as
comment lines are handled. Only interactivity controls that:
• Comment lines are stripped after editing interactively
• Lines which could be recognized as comment lines are left alone when
the message is given non-interactively
So it is misleading to document the comment line behavior on
this option.
Further, the text is wrong:
Lines starting with `#` will be stripped out in non-editor cases
like `-m`, [...]
Comment lines are still indirectly discussed on other options. We will
deal with them in the next commit.
Signed-off-by: Kristoffer Haugsbakk <code@khaugsbakk.name> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano [Tue, 27 May 2025 20:59:10 +0000 (13:59 -0700)]
Merge branch 'js/misc-fixes'
Assorted fixes for issues found with CodeQL.
* js/misc-fixes:
sequencer: stop pretending that an assignment is a condition
bundle-uri: avoid using undefined output of `sscanf()`
commit-graph: avoid using stale stack addresses
trace2: avoid "futile conditional"
Avoid redundant conditions
fetch: avoid unnecessary work when there is no current branch
has_dir_name(): make code more obvious
upload-pack: rename `enum` to reflect the operation
commit-graph: avoid malloc'ing a local variable
fetch: carefully clear local variable's address after use
commit: simplify code
Junio C Hamano [Tue, 27 May 2025 20:59:10 +0000 (13:59 -0700)]
Merge branch 'sj/use-mmap-to-check-packed-refs'
The code path to access the "packed-refs" file while "fsck" is
taught to mmap the file, instead of reading the whole file in the
memory.
* sj/use-mmap-to-check-packed-refs:
packed-backend: mmap large "packed-refs" file during fsck
packed-backend: extract snapshot allocation in `load_contents`
packed-backend: fsck should warn when "packed-refs" file is empty
Junio C Hamano [Tue, 27 May 2025 20:59:09 +0000 (13:59 -0700)]
Merge branch 'ds/sparse-apply-add-p'
"git apply" and "git add -i/-p" code paths no longer unnecessarily
expand sparse-index while working.
* ds/sparse-apply-add-p:
p2000: add performance test for patch-mode commands
reset: integrate sparse index with --patch
git add: make -p/-i aware of sparse index
apply: integrate with the sparse index
Junio C Hamano [Tue, 27 May 2025 20:59:09 +0000 (13:59 -0700)]
Merge branch 'rj/build-tweaks-part2'
Updates to meson-based build procedure.
* rj/build-tweaks-part2:
configure.ac: upgrade to a compilation check for sysinfo
meson.build: correct setting of GIT_EXEC_PATH
meson: correct path to system config/attribute files
meson: correct install location of YAML.pm
meson.build: quote the GITWEBDIR build configuration
Junio C Hamano [Tue, 27 May 2025 20:59:08 +0000 (13:59 -0700)]
Merge branch 'jk/no-funny-object-types'
Support to create a loose object file with unknown object type has
been dropped.
* jk/no-funny-object-types:
object-file: drop support for writing objects with unknown types
hash-object: handle --literally with OPT_NEGBIT
hash-object: merge HASH_* and INDEX_* flags
hash-object: stop allowing unknown types
t: add lib-loose.sh
t/helper: add zlib test-tool
oid_object_info(): drop type_name strbuf
fsck: stop using object_info->type_name strbuf
oid_object_info_convert(): stop using string for object type
cat-file: use type enum instead of buffer for -t option
object-file: drop OBJECT_INFO_ALLOW_UNKNOWN_TYPE flag
cat-file: make --allow-unknown-type a noop
object-file.h: fix typo in variable declaration
Function 'escapeRefName' introduced in 51a7e6dbc9 has never been used.
Despite being dead code, changes in Perl 5.41.4 exposed precedence
warning within its logic, which then caused test failures in t9402 by
logging the warnings to stderr while parsing the code. The affected
tests are t9402.30, t9402.31, t9402.32 and t9402.34.
Remove this unused function to simplify the codebase and stop the
warnings and test failures. Its corresponding unescapeRefName function,
which remains in use, has had its comments updated.
Reported-by: Jitka Plesnikova <jplesnik@redhat.com> Signed-off-by: Ondřej Pohořelský <opohorel@redhat.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jean-Noël Avila [Sun, 25 May 2025 20:27:09 +0000 (20:27 +0000)]
doc: convert git-switch manpage to new synopsis style
- Switch the synopsis to a synopsis block which will automatically
format placeholders in italics and keywords in monospace
- Use _<placeholder>_ instead of <placeholder> in the description
- Use `backticks` for keywords and more complex option
descriptions. The new rendering engine will apply synopsis rules to
these spans.
Signed-off-by: Jean-Noël Avila <jn.avila@free.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jean-Noël Avila [Sun, 25 May 2025 20:27:08 +0000 (20:27 +0000)]
doc: convert git-mergetool options to new synopsis style
- Use _<placeholder>_ instead of <placeholder> in the description
- Use `backticks` for keywords and more complex option
descriptions. The new rendering engine will apply synopsis rules to
these spans.
Signed-off-by: Jean-Noël Avila <jn.avila@free.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jean-Noël Avila [Sun, 25 May 2025 20:27:07 +0000 (20:27 +0000)]
doc: convert git-mergetool manpage to new synopsis style
- Switch the synopsis to a synopsis block which will automatically
format placeholders in italics and keywords in monospace
- Use _<placeholder>_ instead of <placeholder> in the description
- Use `backticks` for keywords and more complex option
descriptions. The new rendering engine will apply synopsis rules to
these spans.
Signed-off-by: Jean-Noël Avila <jn.avila@free.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jean-Noël Avila [Sun, 25 May 2025 20:27:06 +0000 (20:27 +0000)]
doc: switch merge config description to new synopsis format
- Use _<placeholder>_ instead of <placeholder> in the description
- Use `backticks` for keywords and more complex option
descriptions. The new rendering engine will apply synopsis rules to
these spans.
Additionally, a list of option possible values has been reformatted as a
standalone definition list.
Signed-off-by: Jean-Noël Avila <jn.avila@free.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jean-Noël Avila [Sun, 25 May 2025 20:27:05 +0000 (20:27 +0000)]
doc: convert merge strategies to synopsis format
- Switch the synopsis to a synopsis block which will automatically
format placeholders in italics and keywords in monospace
- Use _<placeholder>_ instead of <placeholder> in the description
- Use `backticks` for keywords and more complex option
descriptions. The new rendering engine will apply synopsis rules to
these spans.
Signed-off-by: Jean-Noël Avila <jn.avila@free.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jean-Noël Avila [Sun, 25 May 2025 20:27:03 +0000 (20:27 +0000)]
doc: convert merge options to new synopsis format
- Use _<placeholder>_ instead of <placeholder> in the description
- Use `backticks` for keywords and more complex option
descriptions. The new rendering engine will apply synopsis rules to
these spans.
Signed-off-by: Jean-Noël Avila <jn.avila@free.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jean-Noël Avila [Sun, 25 May 2025 20:27:02 +0000 (20:27 +0000)]
doc: convert git-merge manpage to new style
- Switch the synopsis to a synopsis block which will automatically
format placeholders in italics and keywords in monospace
- Use _<placeholder>_ instead of <placeholder> in the description
- Use `backticks` for keywords and more complex option
descriptions. The new rendering engine will apply synopsis rules to
these spans.
In order to avoid breaking the format on '<<<<<<' and '>>>>>' lines
by applying the synopsis rules to these spans, they are formatted using '+'
signs instead of '`' signs.
Signed-off-by: Jean-Noël Avila <jn.avila@free.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jean-Noël Avila [Sun, 25 May 2025 20:27:01 +0000 (20:27 +0000)]
doc: convert git-checkout manpage to new style
- Switch the synopsis to a synopsis block which will automatically
format placeholders in italics and keywords in monospace
- Use _<placeholder>_ instead of <placeholder> in the description
- Use `backticks` for keywords and more complex option
descriptions. The new rendering engine will apply synopsis rules to
these spans.
Signed-off-by: Jean-Noël Avila <jn.avila@free.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Mark Mentovai [Sat, 24 May 2025 03:40:46 +0000 (23:40 -0400)]
apply: set file mode when --reverse creates a deleted file
Commit 01aff0a (apply: correctly reverse patch's pre- and post-image
mode bits, 2023-12-26) revised reverse_patches() to maintain the desired
property that when only one of patch::old_mode and patch::new_mode is
set, the mode will be carried in old_mode. That property is generally
correct, with one notable exception: when creating a file, only new_mode
will be set. Since reversing a deletion results in a creation, new_mode
must be set in that case.
Omitting handling for this case means that reversing a patch that
removes an executable file will not result in the executable permission
being set on the re-created file. Existing test coverage for file modes
focuses only on mode changes of existing files.
Swap old_mode and new_mode in reverse_patches() for what's represented
in the patch as a file deletion, as it is transformed into a file
creation under reversal. This causes git apply --reverse to set the
executable permission properly when re-creating a deleted executable
file.
Add tests ensuring that git apply sets file modes correctly on file
creation, both in the forward and reverse directions.
Signed-off-by: Mark Mentovai <mark@chromium.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Mark Mentovai [Sat, 24 May 2025 03:40:45 +0000 (23:40 -0400)]
t4129: test that git apply warns for unexpected mode changes
There is no test covering what commit 01aff0a (apply: correctly reverse
patch's pre- and post-image mode bits, 2023-12-26) addressed. Prior to
that commit, git apply was erroneously unaware of a file's expected mode
while reverse-patching a file whose mode was not changing.
Add the missing test coverage to assure that git apply is aware of the
expected mode of a file being patched when the patch does not indicate
that the file's mode is changing. This is achieved by arranging a file
mode so that it doesn't agree with patch being applied, and checking git
apply's output for the warning it's supposed to raise in this situation.
Test in both reverse and normal (forward) directions.
Signed-off-by: Mark Mentovai <mark@chromium.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano [Fri, 23 May 2025 22:34:07 +0000 (15:34 -0700)]
Merge branch 'ds/scalar-no-maintenance'
Two "scalar" subcommands that adds a repository that hasn't been
under "scalar"'s control are taught an option not to enable the
scheduled maintenance on it.
Junio C Hamano [Fri, 23 May 2025 22:34:06 +0000 (15:34 -0700)]
Merge branch 'js/ci-build-win-in-release-mode'
win+Meson CI pipeline, unlike other pipelines for Windows,
used to build artifacts in develper mode, which has been changed to
build them in release mode for consistency.
* js/ci-build-win-in-release-mode:
ci(win+Meson): build in Release mode
bundle-uri: fix arbitrary file writes via parameter injection
We fetch bundle URIs via `download_https_uri_to_file()`. The logic to
fetch those bundles is not handled in-process, but we instead use a
separate git-remote-https(1) process that performs the fetch for us. The
information about which file should be downloaded and where that file
should be put gets communicated via stdin of that process via a "get"
request. This "get" request has the form "get $uri $file\n\n". As may be
obvious to the reader, this will cause git-remote-https(1) to download
the URI "$uri" and put it into "$file".
The fact that we are using plain spaces and newlines as separators for
the request arguments means that we have to be extra careful with the
respective vaules of these arguments:
- If "$uri" contained a space we would interpret this as both URI and
target location.
- If either "$uri" or "$file" contained a newline we would interpret
this as a new command.
But we neither quote the arguments such that any characters with special
meaning would be escaped, nor do we verify that none of these special
characters are contained.
If either the URI or file contains a newline character, we are open to
protocol injection attacks. Likewise, if the URI itself contains a
space, then an attacker-controlled URI can lead to partially-controlled
file writes.
Note that the attacker-controlled URIs do not permit completely
arbitrary file writes, but instead allows an attacker to control the
path in which we will write a temporary (e.g., "tmp_uri_XXXXXX")
file.
The result is twofold:
- By adding a space in "$uri" we can control where exactly a file will
be written to, including out-of-repository writes. The final
location is not completely arbitrary, as the injected string will be
concatenated with the original "$file" path. Furthermore, the name
of the bundle will be "tmp_uri_XXXXXX", further restricting what an
adversary would be able to write.
Also note that is not possible for the URI to contain a newline
because we end up in `credential_from_url_1()` before we try to
issue any requests using that URI. As such, it is not possible to
inject arbitrary commands via the URI.
- By adding a newline to "$file" we can inject arbitrary commands.
This gives us full control over where a specific file will be
written to. Potential attack vectors would be to overwrite hooks,
but if an adversary were to guess where the user's home directory is
located they might also easily write e.g. a "~/.profile" file and
thus cause arbitrary code execution.
This injection can only become possible when the adversary has full
control over the target path where a bundle will be downloaded to.
While this feels unlikely, it is possible to control this path when
users perform a recursive clone with a ".gitmodules" file that is
controlled by the adversary.
Luckily though, the use of bundle URIs is not enabled by default in Git
clients (yet): they have to be enabled by setting the `bundle.heuristic`
config key explicitly. As such, the blast radius of this parameter
injection should overall be quite contained.
Fix the issue by rejecting spaces in the URI and newlines in both the
URI and the file. As explained, it shouldn't be required to also
restrict the use of newlines in the URI, as we would eventually die
anyway in `credential_from_url_1()`. But given that we're only one small
step away from arbitrary code execution, let's rather be safe and
restrict newlines in URIs, as well.
Eventually we should probably refactor the way that Git talks with the
git-remote-https(1) subprocess so that it is less fragile. Until then,
these two restrictions should plug the issue.
Reported-by: David Leadbeater <dgl@dgl.cx> Based-on-patch-by: David Leadbeater <dgl@dgl.cx> Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Justin Tobler [Tue, 20 May 2025 02:26:04 +0000 (21:26 -0500)]
config: quote values containing CR character
When reading the config, values that contain a trailing CRLF are
stripped. If the value itself has a trailing CR, the normal LF that
follows results in the CR being unintentionally stripped. This may lead
to unintended behavior due to the config value written being different
when it gets read.
One such issue involves a repository with a submodule path containing a
trailing CR. When the submodule gets initialized, the submodule is
cloned without being checked out and has "core.worktree" set to the
submodule path. The git-checkout(1) that gets spawned later reads the
"core.worktree" config value, but without the trailing CR, and
consequently attempts to checkout to a different path than intended.
If the repository contains a matching path that is a symlink, it is
possible for the submodule repository to be checked out in arbitrary
locations. This is extra bad when the symlink points to the submodule
hooks directory and the submodule repository contains an executable
"post-checkout" hook. Once the submodule repository checkout completes,
the "post-checkout" hook immediately executes.
To prevent mismatched config state due to misinterpreting a trailing CR,
wrap config values containing CR in double quotes when writing the
entry. This ensures a trailing CR is always separated for an LF and thus
prevented from getting stripped.
Note that this problem cannot be addressed by just quoting each CR with
"\r". The reading side of the config interprets only a few backslash
escapes, and "\r" is not among them. This fix is sufficient though
because it only affects the CR at the end of a line and any literal CR
in the interior is already preserved.
Co-authored-by: David Leadbeater <dgl@dgl.cx> Signed-off-by: Justin Tobler <jltobler@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Johannes Sixt [Tue, 20 May 2025 06:56:09 +0000 (08:56 +0200)]
Merge branch 'js/fix-open-exec'
This addresses CVE-2025-46835, Git GUI can create and overwrite a
user's files:
When a user clones an untrusted repository and is tricked into editing
a file located in a maliciously named directory in the repository, then
Git GUI can create and overwrite files for which the user has write
permission.
Johannes Sixt [Tue, 20 May 2025 06:54:24 +0000 (08:54 +0200)]
Merge branch 'ml/replace-auto-execok'
This addresses CVE-2025-46334, Git GUI malicious command injection on
Windows.
A malicious repository can ship versions of sh.exe or typical textconv
filter programs such as astextplain. Due to the unfortunate design of
Tcl on Windows, the search path when looking for an executable always
includes the current directory. The mentioned programs are invoked when
the user selects "Git Bash" or "Browse Files" from the menu.
Johannes Sixt [Wed, 14 May 2025 17:56:27 +0000 (19:56 +0200)]
Merge branch 'js/fix-open-exec'
This addresses CVE-2025-27613, Gitk can create and truncate a user's
files:
When a user clones an untrusted repository and runs gitk without
additional command arguments, files for which the user has write
permission can be created and truncated. The option "Support per-file
encoding" must have been enabled before in Gitk's Preferences. This
option is disabled by default.
The same happens when "Show origin of this line" is used in the main
window (regardless of whether "Support per-file encoding" is enabled or
not).
Johannes Sixt [Wed, 14 May 2025 16:27:05 +0000 (18:27 +0200)]
Merge branch 'ah/fix-open-with-stdin'
This addresses CVE-2025-27614, Arbitrary command execution with Gitk:
A Git repository can be crafted in such a way that with some social
engineering a user who has cloned the repository can be tricked into
running any script (e.g., Bourne shell, Perl, Python, ...) supplied by
the attacker by invoking `gitk filename`, where `filename` has a
particular structure. The script is run with the privileges of the user.
Johannes Sixt [Sun, 4 May 2025 19:59:19 +0000 (21:59 +0200)]
git-gui: do not mistake command arguments as redirection operators
Tcl 'open' assigns special meaning to its argument when they begin with
redirection, pipe or background operator. There are many calls of the
'open' variant that runs a process which construct arguments that are
taken from the Git repository or are user input. However, when file
names or ref names are taken from the repository, it is possible to
find names that have these special forms. They must not be interpreted
by 'open' lest it redirects input or output, or attempts to build a
pipeline using a command name controlled by the repository.
Use the helper function make_arglist_safe, which identifies such
arguments and prepends "./" to force such a name to be regarded as a
relative file name.
After this change the following 'open' calls that start a process do not
apply the argument processing:
In all cases, the command arguments are constant strings (or begin with
a constant string) that are of a form that would not be affected by the
processing anyway.
Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Johannes Sixt [Sun, 4 May 2025 18:26:11 +0000 (20:26 +0200)]
git-gui: introduce function git_redir for git calls with redirections
Proc git invokes git and collects all output, which is it returns.
We are going to treat command arguments and redirections differently to
avoid passing arguments that look like redirections to the command
accidentally. A few invocations also pass redirection operators as
command arguments deliberately. Rewrite these cases to use a new
function git_redir that takes two lists, one for the regular command
arguments and one for the redirection operations.
Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Johannes Sixt [Sun, 4 May 2025 13:39:03 +0000 (15:39 +0200)]
git-gui: pass redirections as separate argument to git_read
We are going to treat command arguments and redirections differently to
avoid passing arguments that look like redirections to the command
accidentally. To do so, it will be necessary to know which arguments
are intentional redirections. Rewrite direct call sites of git_read
to pass intentional redirections as a second (optional) argument.
git_read defers to safe_open_command, but we cannot make it safe, yet,
because one of the callers of git_read is proc git, which does not yet
know which of its arguments are redirections. This is the topic of the
next commit.
Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Johannes Sixt [Sun, 4 May 2025 13:06:11 +0000 (15:06 +0200)]
git-gui: pass redirections as separate argument to _open_stdout_stderr
We are going to treat command arguments and redirections differently to
avoid passing arguments that look like redirections to the command
accidentally. To do so, it will be necessary to know which arguments
are intentional redirections. Rewrite direct callers of
_open_stdout_stderr to pass intentional redirections as a second
(optional) argument.
Passing arbitrary arguments is not safe right now, but we rename it
to safe_open_command anyway to avoid having to touch the call sites
again later when we make it actually safe.
We cannot make the function safe right away because one caller is
git_read, which does not yet know which of its arguments are
redirections. This is the topic of the next commit.
Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Johannes Sixt [Sat, 3 May 2025 11:24:48 +0000 (13:24 +0200)]
git-gui: convert git_read*, git_write to be non-variadic
We are going to treat command arguments and redirections differently to
avoid passing arguments that look like redirections to the command
accidentally. To do so, it will be necessary to know which arguments
are intentional redirections. As a preparation, convert git_read,
git_read_nice, and git_write to take just a single argument that is
the command in a list. Adjust all call sites accordingly.
In the future, this argument will be the regular command arguments and
a second argument will be the redirection operations.
Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Mark Levedahl [Fri, 11 Apr 2025 14:58:20 +0000 (10:58 -0400)]
git-gui: override exec and open only on Windows
Since aae9560a355d (Work around Tcl's default `PATH` lookup,
2022-11-23), git-gui overrides exec and open on all platforms. But,
this was done in response to Tcl adding elements to $PATH on Windows,
while exec, open, and auto_execok honor $PATH as given on all other
platforms.
Let's do the override only on Windows, restoring others to using their
native exec and open. These honor the sanitized $PATH as that is written
out to env(PATH) in a previous commit. auto_execok is also safe on these
platforms, so can be used for _which.
Signed-off-by: Mark Levedahl <mlevedahl@gmail.com> Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
The previous commits bb5cb23daf75 (gitk: prevent overly long command
lines, 2023-01-24) rewrote a set of the 'open' calls substantially.
These were then later updated by 7dd272eca153 (gitk: escape file paths
before piping to git log, 2023-01-24) and d5d1b91e5327 (gitk: encode
arguments correctly with "open", 2025-03-07). In the preceding merge,
the conversions to a safe_open variant were undone to ensure that the
principal operation of the new 'open' calls is not modified by accident.
Since the 'open' calls now pass a redirection from a Tcl string as
stdin, convert the calls to 'safe_open_command_redirect'.
Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Johannes Sixt [Sat, 3 May 2025 17:21:53 +0000 (19:21 +0200)]
git-gui: use git_read in githook_read
0730a5a3a5e6 ("git-gui - use git-hook, honor core.hooksPath", 2023-09-17)
rewrote githook_read to use `git hook` to run a hook script. The code
that was replaced discovered the hook script file manually and invoked
it using function _open_stdout_stderr. After the rewrite, this function
is still invoked, but it calls into `git` instead of the hook scripts.
Notice though, that we have function git_read that invokes git and
prepares a pipe for the caller to read from. Replace the implementation
of githook_read to be just a wrapper around git_read. This unifies the
way in which the git executable is invoked. git_read ultimately also
calls into _open_stdout_stderr, but it modifies the path to the git
executable before doing so.
Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Mark Levedahl [Fri, 11 Apr 2025 14:47:04 +0000 (10:47 -0400)]
git-gui: sanitize $PATH on all platforms
Since 8f23432b38d9 (windows: ignore empty `PATH` elements, 2022-11-23),
git-gui removes empty elements from $PATH, and a prior commit made this
remove all non-absolute elements from $PATH. But, this happens only on
Windows. Unsafe $PATH elements in $PATH are possible on all platforms.
Let's sanitize $PATH on all platforms to have consistent behavior. If a
user really wants the current repository on $PATH, they can add its
absolute name to $PATH.
Signed-off-by: Mark Levedahl <mlevedahl@gmail.com> Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Johannes Sixt [Sat, 3 May 2025 11:11:21 +0000 (13:11 +0200)]
git-gui: break out a separate function git_read_nice
There are two callers of git_read that request special treatment using
option --nice. Rewrite them to call a new function git_read_nice that
does the special treatment. Now we can remove all option treatment from
git_read.
git_write has the same capability, but there are no callers that
request --nice. Remove the feature without substitution.
This is a preparation for a later change where we want to make git_read
and friends non-variadic. Then it cannot have optional arguments.
Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Mark Levedahl [Fri, 11 Apr 2025 14:08:52 +0000 (10:08 -0400)]
git-gui: assure PATH has only absolute elements.
Since 8f23432b38d9 (windows: ignore empty `PATH` elements, 2022-11-23),
git-gui excises all empty paths from $PATH, but still allows '.' or
other relative paths, which can also allow executing code from the
repository. Let's remove anything except absolute elements. While here,
let's remove duplicated elements, which are very common on Windows:
only the first such item can do anything except waste time repeating a
search.
Signed-off-by: Mark Levedahl <mlevedahl@gmail.com> Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Johannes Sixt [Sat, 3 May 2025 09:52:35 +0000 (11:52 +0200)]
git-gui: remove option --stderr from git_read
Some callers of git_read want to redirect stderr of the invoked command
to stdout. The function offers option --stderr for this purpose.
However, the option only appends 2>@1 to the commands. The callers can
do that themselves. In lib/console.tcl we even have a caller that
already knew implictly what --stderr does behind the scenes.
This is a preparation for a later change where we want to make git_read
non-variadic. Then it cannot have optional leading arguments.
Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Mark Levedahl [Mon, 7 Apr 2025 21:12:56 +0000 (17:12 -0400)]
git-gui: cleanup git-bash menu item
git-gui on Git for Windows creates a menu item to start a git-bash
session for the current repository. This menu-item works as desired when
git-gui is installed in the Git for Windows (g4w) distribution, but
not when run from a different location such as normally done in
development. The reason is that git-bash's location is known to be
'/git-bash' in the Unix pathname space known to MSYS, but this is not
known in the Windows pathname space. Instead, git-gui derives a pathname
for git-bash assuming it is at a known relative location.
If git-gui is run from a different directory than assumed in g4w, the
relative location changes, and git-gui resorts to running a generic bash
login session in a Windows console.
But, the MSYS system underlying Git for Windows includes the 'cygpath'
utility to convert between Unix and Windows pathnames. Let's use this so
git-bash's Windows pathname is determined directly from /git-bash.
Signed-off-by: Mark Levedahl <mlevedahl@gmail.com> Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Johannes Sixt [Sat, 26 Apr 2025 16:46:06 +0000 (18:46 +0200)]
git-gui: sanitize 'exec' arguments: background
As in the previous commits, introduce a function that sanitizes
arguments intended for the process, but runs the process in the
background. Convert 'exec' calls to use this new function.
Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Mark Levedahl [Thu, 3 Apr 2025 04:37:08 +0000 (00:37 -0400)]
git-gui: avoid auto_execok in do_windows_shortcut
git-gui on Windows uses auto_execok to locate git-gui.exe,
which performs the same flawed search as does the builtin exec.
Use _which instead, performing a safe PATH lookup.
Signed-off-by: Mark Levedahl <mlevedahl@gmail.com> Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Johannes Sixt [Mon, 21 Apr 2025 16:14:54 +0000 (18:14 +0200)]
git-gui: sanitize 'exec' arguments: simple cases
Tcl 'exec' assigns special meaning to its argument when they begin with
redirection, pipe or background operator. There are a number of
invocations of 'exec' which construct arguments that are taken from the
Git repository or a user input. However, when file names or ref names
are taken from the repository, it is possible to find names that have
these special forms. They must not be interpreted by 'exec' lest it
redirects input or output, or attempts to build a pipeline using a
command name controlled by the repository.
Introduce a helper function that identifies such arguments and prepends
"./" to force such a name to be regarded as a relative file name.
Convert those 'exec' calls where the arguments can simply be packed
into a list.
Note that most commands containing the word 'exec' route through
console::exec or console::chain, which we will treat in another commit.
Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Mark Levedahl [Wed, 2 Apr 2025 21:37:27 +0000 (17:37 -0400)]
git-gui: avoid auto_execok for git-bash menu item
On Windows, git-gui offers to open a git-bash session for the current
repository from the menu, but uses [auto_execok start] to get the
command to actually run that shell.
The code for auto_execok, in /usr/share/tcl8.6/tcl.init, has 'start' in
the 'shellBuiltins' list for cmd.exe on Windows: as a result,
auto_execok does not actually search for start, meaning this usage is
technically ok with auto_execok now. However, leaving this use of
auto_execok in place will just induce confusion about why a known unsafe
function is being used on Windows. Instead, let's switch to using our
known safe _which function that looks only in $PATH, excluding the
current working directory.
Signed-off-by: Mark Levedahl <mlevedahl@gmail.com> Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Johannes Sixt [Mon, 21 Apr 2025 15:07:10 +0000 (17:07 +0200)]
git-gui: treat file names beginning with "|" as relative paths
The Tcl 'open' function has a very wide interface. It can open files as
well as pipes to external processes. The difference is made only by the
first character of the file name: if it is "|", a process is spawned.
We have a number of calls of Tcl 'open' that take a file name from the
environment in which Git GUI is running. Be prepared that insane values
are injected. In particular, when we intend to open a file, do not take
a file name that happens to begin with "|" as a request to run a process.
Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Mark Levedahl [Fri, 4 Apr 2025 20:55:59 +0000 (16:55 -0400)]
git-gui: remove unused proc is_shellscript
Commit 7d076d56757c (git-gui: handle shell script text filters when
loading for blame, 2011-12-09) added is_shellscript to test if a file
is executable by the shell, used only when searching for textconv
filters. The previous commit rearranged the tests for finding such
filters, and removed the only user of is_shellscript. Remove this
function.
Signed-off-by: Mark Levedahl <mlevedahl@gmail.com> Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Johannes Sixt [Sat, 3 May 2025 11:37:35 +0000 (13:37 +0200)]
git-gui: remove git config --list handling for git < 1.5.3
git-gui uses `git config --null --list` to parse configuration. Git
versions prior to 1.5.3 do not have --null and need different treatment.
Nobody should be using such an old version anymore. (Moreover, since 0730a5a3a, git-gui requires git v2.36 or later). Keep only the code for
modern Git.
Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Johannes Sixt [Sun, 18 May 2025 14:08:06 +0000 (16:08 +0200)]
git-gui: remove special treatment of Windows from open_cmd_pipe
Commit 7d076d56757c (git-gui: handle shell script text filters when
loading for blame, 2011-12-09) added open_cmd_pipe to run text
conversion in support of blame, with special handling for shell
scripts on Windows. To determine whether the command is a shell
script, 'lindex' is used to pick off the first token from the command.
However, cmd is actually a command string taken from .gitconfig
literally and is not necessarily a syntactically correct Tcl list.
Hence, it cannot be processed by 'lindex' and 'lrange' reliably.
Pass the command string to the shell just like on non-Windows
platforms to avoid the potentially incorrect treatment.
A use of 'auto_execok' is removed by this change. This function is
dangerous on Windows, because it searches programs in the current
directory. Delegating the path lookup to the shell is safe, because
/bin/sh and /bin/bash follow POSIX on all platforms, including the
Git for Windows port.
A possible regression is that the old code, given filter command of
'foo', could find 'foo.bat' as a script, and not just bare 'foo', or
'foo.exe'. This rewrite requires explicitly giving the suffix if it is
not .exe.
This part of Git GUI can be exercised using
git gui blame -- some.file
while some.file has a textconv filter configured and has unstaged
modifications.
Helped-by: Mark Levedahl <mlevedahl@gmail.com> Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Mark Levedahl [Fri, 2 May 2025 15:39:55 +0000 (11:39 -0400)]
git-gui: remove HEAD detachment implementation for git < 1.5.3
git-gui provides an implementation to detach HEAD on Git versions prior
to 1.5.3. Nobody should be using such an old version anymore.
(Moreover, since 0730a5a3a, git-gui requires git v2.36 or later).
Keep only the code for modern Git.
Signed-off-by: Mark Levedahl <mlevedahl@gmail.com>
[j6t: message tweaked] Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Mark Levedahl [Sun, 6 Apr 2025 22:20:14 +0000 (18:20 -0400)]
git-gui: use only the configured shell
git-gui has a few places where a bare "sh" is passed to exec, meaning
that the first instance of "sh" on $PATH will be used rather than the
shell configured. This violates expectations that the configured shell
is being used. Let's use [shellpath] everywhere.
Signed-off-by: Mark Levedahl <mlevedahl@gmail.com> Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Mark Levedahl [Wed, 20 Sep 2023 21:56:14 +0000 (17:56 -0400)]
git-gui: remove Tcl 8.4 workaround on 2>@1 redirection
Since b792230 ("git-gui: Show a progress meter for checking out files",
2007-07-08), git-gui includes a workaround for Tcl that does not support
using 2>@1 to redirect stderr to stdout. Tcl added such support in
8.4.7, released in 2004, and this is fully supported in all 8.5
releases.
As git-gui has a hard-coded requirement for Tcl >= 8.5, the workaround
is no longer needed. Delete it.
Signed-off-by: Mark Levedahl <mlevedahl@gmail.com> Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Mark Levedahl [Tue, 1 Apr 2025 15:45:06 +0000 (11:45 -0400)]
git-gui: make _shellpath usable on startup
Since commit d5257fb3c1de (git-gui: handle textconv filter on
Windows and in development, 2010-08-07), git-gui will search for a
usable shell if _shellpath is not configured, and on Windows may
resort to using auto_execok to find 'sh'. While this was intended for
development use, checks are insufficient to assure a proper
configuration when deployed where _shellpath is always set, but might
not give a usable shell.
Let's make this more robust by only searching if _shellpath was not
defined, and then using only our restricted search functions.
Furthermore, we should convert to a Windows path on Windows. Always
check for a valid shell on startup, meaning an absolute path to an
executable, aborting if these conditions are not met.
Signed-off-by: Mark Levedahl <mlevedahl@gmail.com> Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Mark Levedahl [Wed, 2 Apr 2025 15:23:03 +0000 (11:23 -0400)]
git-gui: use [is_Windows], not bad _shellpath
Commit 7d076d56757c (git-gui: handle shell script text filters when
loading for blame, 2011-12-09) added open_cmd_pipe, with special
handling for Windows detected by seeing that _shellpath does not
point to an executable shell. That is bad practice, and is broken by
the next commit that assures _shellpath is valid on all platforms.
Fix this by using [is_Windows] as done for all Windows specific code.
Signed-off-by: Mark Levedahl <mlevedahl@gmail.com> Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Mark Levedahl [Thu, 3 Apr 2025 14:26:21 +0000 (10:26 -0400)]
git-gui: _which, only add .exe suffix if not present
The _which function finds executables on $PATH, and adds .exe on Windows
unless -script was given. However, win32.tcl executes "wscript.exe"
and "cscript.exe", both of which fail as _which adds .exe to both. This
is already fixed in git-gui released by Git for Windows. Do so here.
Signed-off-by: Mark Levedahl <mlevedahl@gmail.com> Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Taylor Blau [Fri, 23 May 2025 21:04:21 +0000 (17:04 -0400)]
Merge branch 'js/fix-open-exec-2.40.0' into js/fix-open-exec
Branch js/fix-open-exec-2.40.0 converts `open` and `exec` calls to call
wrappers that sanitze the command arguments. This side branch updates
three `open` calls that are in conflict with the fix in the preceding
commit. To keep the intended operation of the 'open' calls, this merge
does not try to merge and resolve the conflicts, but ignores the
conversions that are brought in by the side branch, taking "ours" side
of the code in these three cases.
New fixes are the topic of the next commit.
Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
While "exec" uses a normal arguments list which is applied as
command + arguments (and redirections, etc), "open" uses a single
argument which is this command+arguments, where the command and
arguments are a list inside this one argument to "open".
Commit bb5cb23 (gitk: prevent overly long command lines 2023-05-08)
changed several values from individual arguments in that list (hashes
and file names), to a single value which is fed to git via redirection
to its stdin using "open" [1].
However, it didn't ensure correctly that this aggregate value in this
string is interpreted as a single element in this command+args list.
It did just enough so that newlines (which is how these elements are
concatenated) don't split this single list element.
A followup commit at the same patchset: 7dd272e (gitk: escape file
paths before piping to git log 2023-05-08) added a bit more, by
escaping backslahes and spaces at the file names, so that at least
it doesn't break when such file names get used there.
But these are not enough. At the very least tab is missing, and more,
and trying to manually escape every possible thing which can affect
how this string is interpreted in a list is a sub-par approach.
The solution is simply to tell tcl "this is a single list element".
which we can do by aggregating this value completely normally (hashes
and files separated by newlines), and then do [list $value].
So this is what this commit does, for all 3 places where bb5cb23
changed individual elements into an aggregate value.
[1]
That was not a fully accurate description. The accurate version
is that this string originally included two lists: hashes and files.
When used with "open" these lists correctly become the individual
elements of these lists, even if they contain spaces etc, so the
arguments which were used at this "git" commands were correct.
Commit bb5cb23 couldn't use these two lists as-is, because it needed
to process the individual elements in them (one element per line of
the aggregate value), and the issue is that ensuring this aggregate
is indeed interpreted as a single list element was sub-par.
Note: all the (double) quotes before/after the modification are not
required and with zero effect, even for \n. But this commit preserves
the original quoting form intentionally. It can be cleaned up later.
Signed-off-by: Avi Halachmi (:avih) <avihpit@yahoo.com> Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Johannes Sixt [Sun, 23 Mar 2025 21:34:11 +0000 (22:34 +0100)]
gitk: collect construction of blameargs into a single conditional
The command line to invoke 'git blame' for a single line is constructed
using several if-conditionals, each with the same condition
{$from_index new {}}. Merge all of them into a single conditional.
This requires to duplicate significant parts of the command, but it
helps the next change, where we will have to deal with a nested list
structure.
Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Johannes Sixt [Thu, 20 Mar 2025 19:00:57 +0000 (20:00 +0100)]
gitk: sanitize 'open' arguments: simple commands with redirections
As in the previous commits, introduce a function that sanitizes
arguments intended for the process and in addition allows to pass
redirections, which are passed to Tcl's 'open' verbatim.
Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Johannes Sixt [Thu, 20 Mar 2025 18:32:56 +0000 (19:32 +0100)]
gitk: sanitize 'open' arguments: simple commands
Tcl 'open' treats the second argument as a command when it begins
with |. The remainder of the argument is a list comprising the command
and its arguments. It assigns special meaning to these arguments when
they begin with a redirection, pipe or background operator. There are a
number of invocations of 'open' which construct arguments that are
taken from the Git repository or a user input. However, when file names
or ref names are taken from the repository, it is possible to find
names which have these special forms. They must not be interpreted by
'open' lest it redirects input or output, or attempts to build a
pipeline using a command name controlled by the repository.
Introduce a helper function that identifies such arguments and prepends
"./" to force such a name to be regarded as a relative file name.
Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Johannes Sixt [Sat, 29 Mar 2025 16:35:19 +0000 (17:35 +0100)]
gitk: sanitize 'exec' arguments: redirect to process
Convert one 'exec' call that sends output to a process (pipeline).
Fortunately, the command does not contain any variables. For this
reason, just treat it as a "redirection".
Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Johannes Sixt [Sat, 29 Mar 2025 16:21:27 +0000 (17:21 +0100)]
gitk: sanitize 'exec' arguments: redirections and background
Convert 'exec' calls that both redirect output to a file and run the
process in the background. 'safe_exec_redirect' can take both these
"redirections" in the second argument simultaneously.
Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Johannes Sixt [Sat, 29 Mar 2025 16:01:54 +0000 (17:01 +0100)]
gitk: sanitize 'exec' arguments: redirections
As in the previous commits, introduce a function that sanitizes
arguments intended for the process and in addition allows to pass
redirections verbatim, which are interpreted by Tcl's 'exec'.
Redirections can include the background operator '&'.
Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Johannes Sixt [Sat, 29 Mar 2025 15:51:29 +0000 (16:51 +0100)]
gitk: sanitize 'exec' arguments: 'eval exec'
Convert calls of 'exec' where the arguments are already available in
a list and 'eval' is used to unpack the list. Use 'concat' to unite
the arguments into a single list before passing them to 'safe_exec'.
Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Johannes Sixt [Mon, 17 Mar 2025 21:59:27 +0000 (22:59 +0100)]
gitk: sanitize 'exec' arguments: simple cases
Tcl 'exec' assigns special meaning to its argument when they begin with
redirection, pipe or background operator. There are a number of
invocations of 'exec' which construct arguments that are taken from the
Git repository or a user input. However, when file names or ref names
are taken from the repository, it is possible to find names with have
these special forms. They must not be interpreted by 'exec' lest it
redirects input or output, or attempts to build a pipeline using a
command name controlled by the repository.
Introduce a helper function that identifies such arguments and prepends
"./" to force such a name to be regarded as a relative file name.
Convert those 'exec' calls where the arguments can simply be packed
into a list.
Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Johannes Sixt [Mon, 17 Mar 2025 20:39:58 +0000 (21:39 +0100)]
gitk: have callers of diffcmd supply pipe symbol when necessary
Function 'diffcmd' derives which of git diff-files, git diff-index, or
git diff-tree must be invoked depending on the ids provided. It puts
the pipe symbol as the first element of the returned command list.
Note though that of the four callers only two use the command with
Tcl 'open' and need the pipe symbol. The other two callers pass the
command to Tcl 'exec' and must remove the pipe symbol.
Do not include the pipe symbol in the constructed command list, but let
the call sites decide whether to add it or not. Note that Tcl 'open'
inspects only the first character of the command list, which is also
the first character of the first element in the list. For this reason,
it is valid to just tack on the pipe symbol with |$cmd and it is not
necessary to use [concat | $cmd].
Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Johannes Sixt [Mon, 17 Mar 2025 19:36:04 +0000 (20:36 +0100)]
gitk: treat file names beginning with "|" as relative paths
The Tcl 'open' function has a vary wide interface. It can open files as
well as pipes to external processes. The difference is made only by the
first character of the file name: if it is "|", an process is spawned.
We have a number of calls of Tcl 'open' that take a file name from the
environment in which Gitk is running. Be prepared that insane values are
injected. In particular, when we intend to open a file, do not mistake
a file name that happens to begin with "|" as a request to run a process.
Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Taylor Blau <me@ttaylorr.com>
Phillip Wood [Thu, 22 May 2025 15:55:23 +0000 (16:55 +0100)]
midx docs: clarify tie breaking
Clarify what happens when an object exists in more than one pack, but
not in the preferred pack. "git multi-pack-index repack" relies on ties
for objects that are not in the preferred pack being resolved in favor
of the newest pack that contains a copy of the object. If ties were
resolved in favor of the oldest pack as the current documentation
suggests the multi-pack index would not reference any of the objects in
the pack created by "git multi-pack-index repack".
Helped-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Phillip Wood [Thu, 22 May 2025 15:55:22 +0000 (16:55 +0100)]
midx: avoid negative array index
nth_midxed_pack_int_id() returns the index of the pack file in the multi
pack index's list of packfiles that the specified object. The index is
returned as a uint32_t. Storing this in an int will make the index
negative if the most significant bit is set. Fix this by using uint32_t
as the rest of the code does. This is unlikely to be a practical problem
as it requires the multipack index to reference 2^31 packfiles.
Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Phillip Wood [Thu, 22 May 2025 15:55:21 +0000 (16:55 +0100)]
midx repack: avoid potential integer overflow on 64 bit systems
On a 64 bit system the calculation
p->pack_size * pack_info[i].referenced_objects
could overflow. If a pack file contains 2^28 objects with an average
compressed size of 1KB then the pack size will be 2^38B. If all of the
objects are referenced by the multi-pack index the sum above will
overflow. Avoid this by using shifted integer arithmetic and changing
the order of the calculation so that the pack size is divided by the
total number of objects in the pack before multiplying by the number of
objects referenced by the multi-pack index. Using a shift of 14 bits
should give reasonable accuracy while avoiding overflow for pack sizes
less that 1PB.
Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
The calculation to estimated size of the objects in the pack referenced
by the multi-pack-index uses st_mult() to multiply the pack size by the
number of referenced objects before dividing by the total number of
objects in the pack. As size_t is 32 bits on 32 bit systems this
calculation easily overflows. Fix this by using 64bit arithmetic instead.
Also fix a potential overflow when caluculating the total size of the
objects referenced by the multipack index with a batch size larger
than SIZE_MAX / 2. In that case
total_size += estimated_size
can overflow as both total_size and estimated_size can be greater that
SIZE_MAX / 2. This is addressed by using saturating arithmetic for the
addition. Although estimated_size is of type uint64_t by the time we
reach this sum it is bounded by the batch size which is of type size_t
and so casting estimated_size to size_t does not truncate the value.
Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jacob Keller [Wed, 21 May 2025 23:29:17 +0000 (16:29 -0700)]
diff --no-index: support limiting by pathspec
The --no-index option of git-diff enables using the diff machinery from
git while operating outside of a repository. This mode of git diff is
able to compare directories and produce a diff of their contents.
When operating git diff in a repository, git has the notion of
"pathspecs" which can specify which files to compare. In particular,
when using git to diff two trees, you might invoke:
$ git diff-tree -r <treeish1> <treeish2>.
where the treeish could point to a subdirectory of the repository.
When invoked this way, users can limit the selected paths of the tree by
using a pathspec. Either by providing some list of paths to accept, or
by removing paths via a negative refspec.
The git diff --no-index mode does not support pathspecs, and cannot
limit the diff output in this way. Other diff programs such as GNU
difftools have options for excluding paths based on a pattern match.
However, using git diff as a diff replacement has several advantages
over many popular diff tools, including coloring moved lines, rename
detections, and similar.
Teach git diff --no-index how to handle pathspecs to limit the
comparisons. This will only be supported if both provided paths are
directories.
For comparisons where one path isn't a directory, the --no-index mode
already has some DWIM shortcuts implemented in the fixup_paths()
function.
Modify the fixup_paths function to return 1 if both paths are
directories. If this is the case, interpret any extra arguments to git
diff as pathspecs via parse_pathspec.
Use parse_pathspec to load the remaining arguments (if any) to git diff
--no-index as pathspec items. Disable PATHSPEC_ATTR support since we do
not have a repository to do attribute lookup. Disable PATHSPEC_FROMTOP
since we do not have a repository root. All pathspecs are treated as
rooted at the provided comparison paths.
After loading the pathspec data, calculate skip offsets for skipping
past the root portion of the paths. This is required to ensure that
pathspecs start matching from the provided path, rather than matching
from the absolute path. We could instead pass the paths as prefix values
to parse_pathspec. This is slightly problematic because the paths come
from the command line and don't necessarily have the proper trailing
slash. Additionally, that would require parsing pathspecs multiple
times.
Pass the pathspec object and the skip offsets into queue_diff, which
in-turn must pass them along to read_directory_contents.
Modify read_directory_contents to check against the pathspecs when
scanning the directory. Use the skip offset to skip past the initial
root of the path, and only match against portions that are below the
intended directory structure being compared.
The search algorithm for finding paths is recursive with read_dir. To
make pathspec matching work properly, we must set both
DO_MATCH_DIRECTORY and DO_MATCH_LEADING_PATHSPEC.
Without DO_MATCH_DIRECTORY, paths like "a/b/c/d" will not match against
pathspecs like "a/b/c". This is usually achieved by setting the is_dir
parameter of match_pathspec.
Without DO_MATCH_LEADING_PATHSPEC, paths like "a/b/c" would not match
against pathspecs like "a/b/c/d". This is crucial because we recursively
iterate down the directories. We could simply avoid checking pathspecs
at subdirectories, but this would force recursion down directories
which would simply be skipped.
If we always passed DO_MATCH_LEADING_PATHSPEC, then we will
incorrectly match in certain cases such as matching 'a/c' against
':(glob)**/d'. The match logic will see that a matches the leading part
of the **/ and accept this even tho c doesn't match.
To avoid this, use the match_leading_pathspec() variant recently
introduced. This sets both flags when is_dir is set, but leaves them
both cleared when is_dir is 0.
Add test cases and documentation covering the new functionality. Note
for the documentation I opted not to move the placement of '--' which is
sometimes used to disambiguate arguments. The diff --no-index mode
requires exactly 2 arguments determining what to compare. Any additional
arguments are interpreted as pathspecs and must come afterwards. Use of
'--' would not actually disambiguate anything, since there will never be
ambiguity over which arguments represent paths or pathspecs.
Signed-off-by: Jacob Keller <jacob.keller@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jacob Keller [Wed, 21 May 2025 23:29:16 +0000 (16:29 -0700)]
pathspec: add flag to indicate operation without repository
A following change will add support for pathspecs to the git diff
--no-index command. This mode of git diff does not load any repository.
Add a new PATHSPEC_NO_REPOSITORY flag indicating that we're parsing
pathspecs without a repository.
Both PATHSPEC_ATTR and PATHSPEC_FROMTOP require a repository to
function. Thus, verify that both of these are set in magic_mask to
ensure they won't be accepted when PATHSPEC_NO_REPOSITORY is set.
Check PATHSPEC_NO_REPOSITORY when warning about paths outside the
directory tree. When the flag is set, do not look for a git repository
when generating the warning message.
Finally, add a BUG in match_pathspec_item if the istate is NULL but the
pathspec has PATHSPEC_ATTR set. Callers which support PATHSPEC_ATTR
should always pass a valid istate, and callers which don't pass a valid
istate should have set PATHSPEC_ATTR in the magic_mask field to disable
support for attribute-based pathspecs.
Signed-off-by: Jacob Keller <jacob.keller@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jacob Keller [Wed, 21 May 2025 23:29:15 +0000 (16:29 -0700)]
pathspec: add match_leading_pathspec variant
The do_match_pathspec() function has the DO_MATCH_LEADING_PATHSPEC
option to allow pathspecs to match when matching "src" against a
pathspec like "src/path/...". This support is not exposed by
match_pathspec, and the internal flags to do_match_pathspec are not
exposed outside of dir.c
The upcoming support for pathspecs in git diff --no-index need the
LEADING matching behavior when iterating down through a directory with
readdir.
We could try to expose the match_pathspec_with_flags to the public API.
However, DO_MATCH_EXCLUDES really shouldn't be public, and its a bit
weird to only have a few of the flags become public.
Instead, add match_leading_pathspec() as a function which sets both
DO_MATCH_DIRECTORY and DO_MATCH_LEADING_PATHSPEC when is_dir is true.
This will be used in a following change to support pathspec matching in
git diff --no-index.
Signed-off-by: Jacob Keller <jacob.keller@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
projects / thirdparty / git.git / log
