Joel Rosdahl [Wed, 10 Apr 2019 20:22:24 +0000 (22:22 +0200)]
Fix bad calculation of object hash in the depend mode
When the depend mode is enabled, there is a code path where the direct
mode hash state is finalized and then more bytes are fed into it to
create a hash sum for the depend mode. The effect is that the last 0-63
bytes (depending on the number of previously hashed bytes) of the data
used for the depend mode will be ignored, which means that the contents
of the last 0-2 or so header files in the .d file are not accounted for
in the object hash used in the depend mode.
Fix this by making a copy of the direct mode hash state before computing
the hash result in calculate_object_hash().
Include the presence of "-fxxx-prefix-map=" options in the hash (#370)
Recently I landed commit 980e032d14ba ("Allow treating "/dev/null" as
an input file (#365)") which sped up my kernel build a bunch.
Unfortunately I tested my change in isolation on the newest ccache
(AKA I constructed a simple test case and saw that it worked). I only
tested it together with the Linux kernel when backported to an older
version of ccache. A bit later I managed to get the newer version of
ccache fully integrated into my build system and tested my path there.
Then I noticed a problem because the new ccache contained the commit 919af39ccaca ("Support gcc-8 -ffile-prefix-map")
Specifically my compiler doesn't support the
-fmacro-prefix-map=... option and thus my change and the ccache change
to support -fmacro-prefix-map caused caused my kernel build to fail.
It couldn't detect that -fmacro-prefix-map wasn't supported. The
kernel effectively will make these two calls:
Since the newer ccache doesn't use "-fmacro-prefix-map=" when
computing its hash it will return the cached results from the first
call: AKA no compiler error. Now the kernel Makefile will decide that
the "-fmacro-prefix-map=" is supported and will continue to use it,
which is bad.
It seems like an easy fix to all this is to add the _presence_ of the
"-fxxx-prefix-map=" options into the hash. Those that want to use
these options to reuse cache objects still can--they just need to make
sure that none of the invocations of the compiler totally omit the
option.
Joel Rosdahl [Sun, 31 Mar 2019 18:06:05 +0000 (20:06 +0200)]
Ignore unknown keys in configuration files
This way legacy ccache versions can still work with modern ccache
configuration keys in the configuration files. The downside is of course
that errors like typos no longer will be corrected.
Pavol Sakac [Sun, 24 Mar 2019 12:56:24 +0000 (13:56 +0100)]
depend mode: Support multiple versions of dependencies (#363)
Depend mode fix for a case with unchaged common hash (unchanged source file and
other parameters that affect the hash), but with changes in the header file[s]
the source file depends on. This allows for multiple versions of the object
file and dependency file to exist for one common hash. Added unit tests in
test/suites/depend_different_headers.bash to test this behavior.
Joel Rosdahl [Thu, 21 Mar 2019 21:04:22 +0000 (22:04 +0100)]
Fix deletion of tmp_stderr in depend mode
If the compiler produced stderr, a temporary file leaks if depend mode
is enabled. Also, if the compiler does not produce stderr and depend
mode is enable, an unnecessary unlink is attempted.
Fix this by restructuring logic related to deletion of the temporary
file.
dianders [Thu, 21 Mar 2019 19:46:34 +0000 (12:46 -0700)]
Allow treating "/dev/null" as an input file (#365)
One of the slow things for incremental Linux kernel builds is that
during the single-threaded parsing stage of the Makefile the kernel
calls into the C compiler to test which options the compiler supports.
A lot. Specifically there are snippets like this all over the
Makefile:
$(call cc-option,-Oz,-Os)
...which translates into a call to the C compiler:
${CC} ... -Oz -c -x c /dev/null -o .178435.tmp
One of the contributing factors to the overall slowness is that the
input file for this test is "/dev/null". This trips a check in ccache
because "/dev/null" "isn't a plain file".
As far as I understand it it should be totally fine to cache the
result of compiling "/dev/null". It's basically just compiling an
empty file.
On my setup this improves the parsing stage of the kernel Makefile
from 3.25 seconds to 2.0 seconds (so saves 1.25 seconds for each of
build, install, and modules_install for 3.75 seconds total).
Joel Rosdahl [Thu, 24 Jan 2019 20:35:52 +0000 (21:35 +0100)]
Avoid reading outside memory buffer for large debug log messages
When the debug mode is enabled, the vlog function formats a log message
in a stack-allocated buffer using vsnprintf and assumes that the
returned value represents the number of written bytes. This is an
incorrect assumption if the message is larger than the buffer
size (minus one) since the return value tells how many bytes *would*
have been written.
Robert Yang [Tue, 22 Jan 2019 09:04:00 +0000 (17:04 +0800)]
dev.mk.in: Fix file name too long error
The all_cppflags changes path to filename which causes file name too long
error when the path is longer than NAME_MAX (usually 255). Strip srcdir
to fix the problem.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Joel Rosdahl [Sat, 5 Jan 2019 09:56:13 +0000 (10:56 +0100)]
Genererate confitem number fields in confitems.gperf
When merging diverging confitems.gperf changes it is always a hassle to
regenerate the config number sequence fields by hand, and they also
trigger merge conflicts. Those numbers are now instead generated
automatically.
Joel Rosdahl [Fri, 4 Jan 2019 08:54:29 +0000 (09:54 +0100)]
Don’t store pregenerated config lookup tables in the repo
We have had a requirement on gperf for the build-from-repo case for a
while now, so there is no real point in keeping generated gperf lookup
table code in the repository.
Joel Rosdahl [Thu, 3 Jan 2019 13:33:52 +0000 (14:33 +0100)]
Merge pull request #301 from afbjorklund/depend_mode
Add depend mode (avoid the preprocessor).
When using run_second_cpp (which is the default now), ccache executes
the preprocessor just to determine the object hash, and will do so for
every cache miss. While compiling, the same work is done again.
When executing a massively parallel build using ccache and distcc,
the system controlling the build can get a fairly high load because of
all these ccache-only preprocessor executions.
This is based on the work of Geert Kloosterman with remaining items addressed.
jonnyyu [Mon, 10 Dec 2018 19:49:22 +0000 (03:49 +0800)]
Skip '-index-store-path' when building with Xcode (#333)
In Xcode 9 or later, Xcode calls clang with this
new '-index-store-path' option.
The Xcode usually sets IndexStore directory under
a unique build location. This might break the manifest,
especially when cache is shared among multple machines.
jonnyyu [Sun, 9 Dec 2018 18:15:58 +0000 (02:15 +0800)]
Handle several levels of nonexistent directories in make_relative_path (#334)
Currently, ccache supports calculating relative path
for 1 level non-exist path. That is to say,
if the given path does not exist, however if its
parent directory exists, then ccache can calculate
the relative path correctly.
Unfortunately this doesn't fit the needs.
Xcode build system always adds these paths into header search path:
xxxxx/DerivedResources/x86-64
xxxxx/DerivedResources
these paths are build outputs for build rules.
For projects which doesn't use build rule to generate files
these directories do not exist.
So this change refine the logic of make_relative_path
to recursively go up find the nearest existing directory
and use the remaining path as path_suffix.
projects / thirdparty / ccache.git / log
