Joel Rosdahl [Wed, 1 May 2019 12:51:45 +0000 (14:51 +0200)]
Improve fix in #400 to handle more cases
The dependency file name can come from e.g. DEPENDENCIES_OUTPUT as well,
so hash information about a /dev/null .d file after the argument
processing loop instead.
Joel Rosdahl [Wed, 1 May 2019 11:58:18 +0000 (13:58 +0200)]
Bail out on “-MF /dev/null”
This is an alternative fix for #397, based on the observation/assumption
that using “-MF /dev/null” is only ever used as part of a compiler probe
call in combination with “-c /dev/null -o /dev/null”, so there is little
reason to cache the result. The advantage of just bailing out is to
reduce the number of special cases we have to handle.
this is useful for determining the length of the generated argument string
* correctly handle @file syntax on Windows
the @file syntax means that the process reads command arguments from the
specified file. this is commonly used in order to shorten commands which
would otherwise be longer than the maximum length limit: many build systems
do this in all cases to avoid hitting this limit.
when a command exceeds 8192 characters on on Windows, ccache now writes
the parsed/modified arguments to a tmpfile and then runs the command using
that tmpfile with @tmpfile in order to preserve this mechanism and avoid hitting
the length limit
Joel Rosdahl [Mon, 22 Apr 2019 13:22:07 +0000 (15:22 +0200)]
Fix minitrace.c compilation error with GCC 7.3
The error/warning looks like this:
src/minitrace.c: In function ‘mtr_flush’:
src/minitrace.c:256:54: error: ‘%.*s’ directive output may be truncated writing up to 700 bytes into a region of size 252 [-Werror=format-truncation=]
snprintf(arg_buf, ARRAY_SIZE(arg_buf), "\"%s\":\"%.*s\"", raw->arg_name, 700, raw->a_str);
^~~~
In file included from /usr/include/stdio.h:862:0,
from src/minitrace.c:9:
/usr/include/x86_64-linux-gnu/bits/stdio2.h:64:10: note: ‘__builtin___snprintf_chk’ output 6 or more bytes (assuming 706) into a destination of size 256
return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
__bos (__s), __fmt, __va_arg_pack ());
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Joel Rosdahl [Mon, 15 Apr 2019 19:35:38 +0000 (21:35 +0200)]
Touch up NEWS.adoc
* Use “-” for bullets consistently.
* Use “curly quotation marks” instead of ``asciidoctor'' quotation style
for readability, and similar for apostrophes.
Joel Rosdahl [Sat, 13 Apr 2019 20:52:23 +0000 (22:52 +0200)]
Improve handling of debug levels
Fixes #368.
* Remember if we have seen any option on level 3.
* Let “-g0”, “-ggdb0” and similar cancel out any previously seen “-g”
options except “-gsplit-dwarf”. This is based on observations on how
GCC 7.3 behaves.
* Delay acting on seen debug options until after we have processed all
arguments. This way we can avoid e.g. hashing the current directory if
we get “-g3 -g0”.
Joel Rosdahl [Thu, 11 Apr 2019 20:10:43 +0000 (22:10 +0200)]
Rewrite mdfour routines to not modify state when computing the result
This makes the interface much more intuitive, at the minor expense of
having to copy 32-96 extra bytes (unnecessary if the hash state won’t be
fed with more bytes) when fetching the result.
This change is also partly motivated by the code that handles several
-arch options – it calls get_object_name_from_cpp multiple times to
compute a combined hash, but that function also computes the hash result
each time.
Joel Rosdahl [Wed, 10 Apr 2019 20:22:24 +0000 (22:22 +0200)]
Fix bad calculation of object hash in the depend mode
When the depend mode is enabled, there is a code path where the direct
mode hash state is finalized and then more bytes are fed into it to
create a hash sum for the depend mode. The effect is that the last 0-63
bytes (depending on the number of previously hashed bytes) of the data
used for the depend mode will be ignored, which means that the contents
of the last 0-2 or so header files in the .d file are not accounted for
in the object hash used in the depend mode.
Fix this by making a copy of the direct mode hash state before computing
the hash result in calculate_object_hash().
Include the presence of "-fxxx-prefix-map=" options in the hash (#370)
Recently I landed commit 980e032d14ba ("Allow treating "/dev/null" as
an input file (#365)") which sped up my kernel build a bunch.
Unfortunately I tested my change in isolation on the newest ccache
(AKA I constructed a simple test case and saw that it worked). I only
tested it together with the Linux kernel when backported to an older
version of ccache. A bit later I managed to get the newer version of
ccache fully integrated into my build system and tested my path there.
Then I noticed a problem because the new ccache contained the commit 919af39ccaca ("Support gcc-8 -ffile-prefix-map")
Specifically my compiler doesn't support the
-fmacro-prefix-map=... option and thus my change and the ccache change
to support -fmacro-prefix-map caused caused my kernel build to fail.
It couldn't detect that -fmacro-prefix-map wasn't supported. The
kernel effectively will make these two calls:
Since the newer ccache doesn't use "-fmacro-prefix-map=" when
computing its hash it will return the cached results from the first
call: AKA no compiler error. Now the kernel Makefile will decide that
the "-fmacro-prefix-map=" is supported and will continue to use it,
which is bad.
It seems like an easy fix to all this is to add the _presence_ of the
"-fxxx-prefix-map=" options into the hash. Those that want to use
these options to reuse cache objects still can--they just need to make
sure that none of the invocations of the compiler totally omit the
option.
Joel Rosdahl [Sun, 31 Mar 2019 18:06:05 +0000 (20:06 +0200)]
Ignore unknown keys in configuration files
This way legacy ccache versions can still work with modern ccache
configuration keys in the configuration files. The downside is of course
that errors like typos no longer will be corrected.
Pavol Sakac [Sun, 24 Mar 2019 12:56:24 +0000 (13:56 +0100)]
depend mode: Support multiple versions of dependencies (#363)
Depend mode fix for a case with unchaged common hash (unchanged source file and
other parameters that affect the hash), but with changes in the header file[s]
the source file depends on. This allows for multiple versions of the object
file and dependency file to exist for one common hash. Added unit tests in
test/suites/depend_different_headers.bash to test this behavior.
Joel Rosdahl [Thu, 21 Mar 2019 21:04:22 +0000 (22:04 +0100)]
Fix deletion of tmp_stderr in depend mode
If the compiler produced stderr, a temporary file leaks if depend mode
is enabled. Also, if the compiler does not produce stderr and depend
mode is enable, an unnecessary unlink is attempted.
Fix this by restructuring logic related to deletion of the temporary
file.
dianders [Thu, 21 Mar 2019 19:46:34 +0000 (12:46 -0700)]
Allow treating "/dev/null" as an input file (#365)
One of the slow things for incremental Linux kernel builds is that
during the single-threaded parsing stage of the Makefile the kernel
calls into the C compiler to test which options the compiler supports.
A lot. Specifically there are snippets like this all over the
Makefile:
$(call cc-option,-Oz,-Os)
...which translates into a call to the C compiler:
${CC} ... -Oz -c -x c /dev/null -o .178435.tmp
One of the contributing factors to the overall slowness is that the
input file for this test is "/dev/null". This trips a check in ccache
because "/dev/null" "isn't a plain file".
As far as I understand it it should be totally fine to cache the
result of compiling "/dev/null". It's basically just compiling an
empty file.
On my setup this improves the parsing stage of the kernel Makefile
from 3.25 seconds to 2.0 seconds (so saves 1.25 seconds for each of
build, install, and modules_install for 3.75 seconds total).
Joel Rosdahl [Thu, 24 Jan 2019 20:35:52 +0000 (21:35 +0100)]
Avoid reading outside memory buffer for large debug log messages
When the debug mode is enabled, the vlog function formats a log message
in a stack-allocated buffer using vsnprintf and assumes that the
returned value represents the number of written bytes. This is an
incorrect assumption if the message is larger than the buffer
size (minus one) since the return value tells how many bytes *would*
have been written.
projects / thirdparty / ccache.git / log
