Joel Rosdahl [Tue, 7 Apr 2020 19:54:30 +0000 (21:54 +0200)]
Add an Args class with backward compatibility for the old args API (#575)
The idea of this is to make a proper C++ API while at the same time
implementing most of the old legacy args API. This makes it possible to
refactor call sites gradually instead of in a series of large
refactoring steps.
Given that “struct args* args” is replaced with “Args args”, the
following types of legacy API usage can be kept without source changes:
1. The args_* functions can be used as is.
2. args->argv[i] still works for read access.
Things that aren’t emulated:
1. args->argc (use args.size() instead).
2. args->argv to access the underlying char* array (use
args.to_argv().data() instead).
3. Mutating the args->argv[i] string in place (don’t do that).
In other words, code that uses args_foo(args, ...) and args->argv[i] can
easily be converted to args.bar(...) and args[i] (with std::string
arguments instead of char*) at will and in arbitrarily small steps.
Joel Rosdahl [Sun, 22 Mar 2020 13:41:44 +0000 (14:41 +0100)]
Improve name of tmp file in x_unlink
There is no need to generate a unique temporary name in x_unlink since
we don’t care whether the temporary file is clobbered on NFS. The
previous usage of tmp_string() was not quite correct since the returned
string is supposed to be passed to mkstemp, not used as is.
Joel Rosdahl [Sun, 22 Mar 2020 13:30:23 +0000 (14:30 +0100)]
Disable hard link mode when the output object file is /dev/null
When hard link mode is enabled, ccache ≥3.6 unlinks the output file
before writing to it as a workaround for a bug in Clang (#331). This
unfortunately means that /dev/null will be removed when building as root
(don’t do that, BTW) with hard link mode enabled and /dev/null as the
the output file. A similar problem exists if the dependency file is
/dev/null, regardless of hard link mode.
Fix this by not unlinking the output file if it’s /dev/null and by not
copying files to /dev/null at all. (There is no need to handle other
non-regular output files since /dev/null is the only allowed non-regular
output file.)
Joel Rosdahl [Fri, 6 Mar 2020 18:31:15 +0000 (19:31 +0100)]
Simplify lock breaking
Acquiring a .lock.lock file does not make lock breaking significantly
safer; there is still a race condition. Simplify it by simply deleting
the lockfile when breaking and try to acquire it again.
Joel Rosdahl [Wed, 26 Feb 2020 21:50:12 +0000 (22:50 +0100)]
Rewrite the Windows version of the lockfile routines
Instead of emulating the POSIX version, use the native Windows file API
to create a lockfile. This should mitigate several problems with the old
implementation.
Erik Johansson [Sun, 1 Mar 2020 20:30:06 +0000 (21:30 +0100)]
Read/write optimizations (#551)
* Avoid one extra read call after the final bytes are read
If read returns less then the requested number of bytes, the file is at
EOF. This can be used in all places where a read is done, but it makes extra
impact in read_file where if the buffer is made one byte bigger than needed,
EOF can be detected before a unnecessary memory reallocation is done.
* Use write instead of fwrite to write the result file
This avoids the caching in stdio where a write is split into two (at least on
my system): first a small 4k one and then one with the remaining 60k.
Erik Johansson [Sun, 1 Mar 2020 20:17:49 +0000 (21:17 +0100)]
Avoid duplicate stat calls (#550)
hash_source_code_string will, in read_file, stat the file to get the file
size. But when called from verify_result, the file size is already known so by
using this size the number of stat calls in a normal run can be cut in half.
Erik Johansson [Sun, 23 Feb 2020 20:56:59 +0000 (21:56 +0100)]
Fix four minor cppcheck warnings
- uninitvar:src/ccache.cpp:3422: Uninitialized variable: ctx
- redundantInitialization:src/legacy_util.cpp:1067: Redundant initialization
for 'q'. The initialized value is overwritten before it is read.
- constVariable:src/stats.cpp:429: Variable 'ctx' can be declared with const
- variableScope:src/Util.cpp:404: The scope of the variable 'right' can be
reduced.
Joel Rosdahl [Thu, 20 Feb 2020 20:24:15 +0000 (21:24 +0100)]
Don’t use realpath(3) for normalization when computing relative paths
The current working directory (CWD) can come from two sources: Either
the return value of getcwd(3) (“actual CWD” below) or the environment
variable $PWD (“apparent CWD” below). The former is returned by e.g.
$(CURDIR) in Makefiles and by “pwd -P” and is always in normalized form
(no “.” or “..” parts or extra slashes). The latter is returned by “echo
$PWD” or “pwd” and can potentially be in unnormalized form on some
systems.
The actual CWD and apparent CWD may also differ if there are symlinks in
the path. Absolute paths to files given to ccache can therefore be based
on either of these CWD forms. When computing relative paths under the
base directory the CWD needs be in normalized form for the algorithm to
be reasonably simple.
2df269a3 solved a bug with an unnormalized apparent CWD by using
realpath(3) for normalization. Using realpath also makes the algorithm
correct in the presence of symlinks. It however also means that all
symlinks (both in CWD and in command line arguments) are dereferenced.
The downside of this is that if either of the symlink targets contain
specific names (such as build ID, date, username or similar) then the
relative paths will also contain those specific path names, leading to
cache misses.
Solve this by:
- Performing normalization without using realpath, i.e. without
expanding symlinks.
- Computing a relative path based on normalized CWD and normalized path.
- Checking whether the relative path resolves to the same i-node as the
original path. If it does, use it, otherwise just use the original
path (and take a potential cache miss).
- Doing the above calculation both for the actual and the apparent CWD
and choose the best one.
This solves the problem that PR #491 intended to address in a better
way.
Joel Rosdahl [Wed, 19 Feb 2020 10:00:24 +0000 (11:00 +0100)]
Implement Util::normalize_absolute_path
Normalization here means syntactically removing redundant slashes and
resolving "." and ".." parts. The algorithm does however *not* follow
symlinks, so the result may not actually resolve to `path`.
Joel Rosdahl [Sat, 22 Feb 2020 12:46:30 +0000 (13:46 +0100)]
Use find_executable_in_path to find executable in hash_command_output
This makes the Windows version behave similar to the non-Windows
version: just search in $PATH, not potentially in the path specified by
the “path” configuration setting.
Joel Rosdahl [Sat, 22 Feb 2020 12:12:16 +0000 (13:12 +0100)]
Make failure to run compiler_check command a soft error
No need to make a failure when running the compiler check fatal since
other misconfigured settings can have a similar effect without resulting
in fatal errors.
Joel Rosdahl [Sun, 16 Feb 2020 12:06:24 +0000 (13:06 +0100)]
Improve functions related to CWD
The different functions related to current working directory (CWD) have
become messy during the years:
- gnu_getcwd is a simple wrapper around getcwd(3), thus returning the
actual canonical path.
- get_cwd returns $PWD, falling back to getcwd(3) if $PWD is not sane.
- get_current_working_dir (local function in ccache.cpp) memoizes
x_realpath(get_cwd()) (i.e., getcwd(3) in essence...) in the global
current_working_dir variable. Unit tests may manipulate
current_working_dir.
Improve this by:
- Replacing gnu_getcwd with Util::get_actual_cwd.
- Replacing get_cwd with Util::get_apparent_cwd.
- Removing get_current_working_dir and placing both actual and apparent
CWD in the context object.