Joel Rosdahl [Mon, 5 Sep 2022 18:23:21 +0000 (20:23 +0200)]
refactor: Use memory buffers instead of streams for results
- Result objects now only know and care about the result payload part of
a result cache entry.
- Result object are no longer tightly coupled with the primary storage
implementation.
This is part of a larger refactoring effort with the goal of simplifying
how cache entries are read and processed.
Joel Rosdahl [Tue, 30 Aug 2022 18:34:32 +0000 (20:34 +0200)]
chore: Remove share-hits attribute for secondary storage
[1] added a share-hits attribute for secondary storages so that it's
possible to avoid sharing hits to primary storage for a specific
secondary storage. I believe that nobody needs that level of control --
what one would like is the ability to not use the primary storage at
all. Such a feature will be added in a a future commit, but for now the
share-hits=false functionality is just in the way, so let's remove it.
Joel Rosdahl [Sat, 27 Aug 2022 18:04:26 +0000 (20:04 +0200)]
test: Disable "output file failure" test for unsupported filesystem
The "Failure to write output file" test assumes that the filesystem
supports read-only directories. Improve this by probing this assumption
before running the test.
Joel Rosdahl [Fri, 5 Aug 2022 14:39:29 +0000 (16:39 +0200)]
feat: Improve inode cache robustness
- Only enable the inode cache at compile-time if it's possible to
determine filesystem type.
- Only use the inode cache at run-time if the filesystem type is known
to work with the inode cache instead of refusing just on NFS.
Joel Rosdahl [Wed, 3 Aug 2022 08:04:03 +0000 (10:04 +0200)]
fix: Hash time information on inode cache hit
As mentioned in discussion #1086: If the inode cache is enabled,
hash_source_code_file will on an inode cache hit fail to hash time
information if there are temporal macros in the code. This is because
hash_source_code_string (called from hash_source_code_file via
hash_source_code_file_nocache) on an inode cache miss adds time
information to the hash in case one of those macros were found. However,
on an inode cache hit the return value of hash_source_code_file will be
correctly fetched from the cache, but the hash sum will only be updated
with (the hash of) the include file and not the time information.
The fix is to let the inode cache only cache the effects of hashing the
file and checking for macros, not the hashing of time information since
that's volatile.
After the fix:
- The new do_hash_file function performs file hashing and macro
checking. The inode cache caches this step.
- hash_source_code_file returns a Digest instead of adding data to a
Hash.
- hash_source_code_file calls do_hash_file and then potentially hashes
time information. If there are no temporal macros the returned digest
will be identical to the file hash, otherwise the returned digest will
be of a hash of file content hash + time information.
This also improves hashes that are stored in the direct mode manifest:
Previously they were always the hash of the file content hash but now
they are just the file content hash in the common case when there is no
__DATE__ or __TIMESTAMP__ macro in the file.
Joel Rosdahl [Mon, 1 Aug 2022 12:53:52 +0000 (14:53 +0200)]
fix: Always rewrite dependency file if base_dir is used
[1] added the has_absolute_include_headers variable as a performance
optimization for base_dir/CCACHE_BASEDIR so that the dependency file
only has to be parsed/rewritten when necessary. This is based on the
assumption that if no include file has an absolute path then no rewrite
is needed, but apparently Clang can insert other paths into the
dependency file as well, for instance the asan_blacklist.txt file when
using -fsanitize=address.
Fix this by simply always parsing the dependency file when base_dir is
active.
Joel Rosdahl [Thu, 28 Jul 2022 08:58:05 +0000 (10:58 +0200)]
feat: Improve handling of dependency files
- Cache entries are now shared for different -MT/-MQ options. This is
implemented by (on a cache hit) rewriting the stored dependency file
data to have the correct dependency target before writing it to the
destination. Closes #359.
- An intentional side effect of the above is that the correct dependency
target will be produced even when base_dir/CCACHE_BASEDIR is used -
the dependency target will still be an absolute path. Fixes #1042.
- Buggy support for GCC-specific environment variables
DEPENDENCIES_OUTPUT and SUNPRO_DEPENDENCIES has been removed. When one
of those variables was set, ccache used to store and fetch the content
just as if -MMD or -MD were used. This is however incorrect since GCC
*appends* to the destination file instead of (like -MMD/-MD)
*rewriting* it. Since there is no way for ccache to know what the
compiler appended to the dependency file, we simply can't support it.
Reverts #349.
Joel Rosdahl [Tue, 19 Jul 2022 13:31:34 +0000 (15:31 +0200)]
feat: Don't remove inode cache file on -C/--clear
-C/--clear is tied to the cache directory while the inode cache file is
a temporary file. I think it makes more sense to not consider the inode
cache part of the main cache directory.
Rafael Kitover [Tue, 19 Jul 2022 08:29:27 +0000 (01:29 -0700)]
feat: Download dependencies from the Internet when unavailable (#1112)
Change the behavior of ZSTD_FROM_INTERNET and HIREDIS_FROM_INTERNET to
always be on and only come into affect when the package is not found on
the system by normal means.
Also use the cmake standard module FetchContent for the
download/checksum/unpack logic.
Signed-off-by: Rafael Kitover <rkitover@gmail.com>
Joel Rosdahl [Sun, 17 Jul 2022 19:55:06 +0000 (21:55 +0200)]
fix: Avoid false success for -fcolor-diagnostics probe with GCC
Ccache will produce a false hit in the following scenario, if "cc" is
GCC without being a symlink to "gcc":
1. ccache cc -c example.c # adds a cache entry
2. ccache cc -c example.c -fcolor-diagnostics # unexpectedly succeeds
(Most major Linux distributions install cc as a symlink to gcc, and then
the problem doesn't show up since ccache then is able to guess that the
compiler is GCC.)
This bug was introduced by [1] which tried to make yet another tweak of
the color diagnostics guessing/probing by no longer rejecting
-fcolor-diagnostics for an unknown compiler, based on the assumption
that the probing call (speculatively adding -fdiagnostics-color) is only
made when we have guessed a GCC compiler. Unfortunately, this broke the
fix in [2] when the compiler is unknown.
All of this is inherently brittle and I don't see any good way to make
it better with the current compiler guessing approach and adding options
speculatively. The cure is to start doing proper compiler identification
instead (#958). Until that is done, the only reasonable fix is to avoid
doing anything related to color diagnostics flags when the compiler is
unknown. This means that an unknown compiler from now on won't produce
colors when used via ccache even if the compiler actually is GCC or
Clang.
Joel Rosdahl [Sun, 17 Jul 2022 11:43:22 +0000 (13:43 +0200)]
feat: Add support for GCC -fprofile-abs-path option
Note: -fprofile-abs-path makes the compiler include absolute paths based
on actual CWD in the .gcno file. We therefore need to include the actual
CWD in the hash in order not to get false positive cache hits. To opt
out of this, set "gcno_cwd" sloppiness.
projects / thirdparty / ccache.git / log
