The built-in glob implementation does not correctly handle dangling
symlinks. This needs to be fixed by switching to the latest glob
implementation from gnulib but that's a big job: for now avoid the
test if we know it will fail.
Paul Eggert [Tue, 27 Aug 2019 07:46:21 +0000 (00:46 -0700)]
Remove useless code in eval
* src/read.c (eval): Remove useless code. Without this patch,
Oracle Studio 12.6 complains ‘"src/read.c", line 1405: warning:
loop not entered at top’.
Paul Eggert [Tue, 27 Aug 2019 07:40:05 +0000 (00:40 -0700)]
Pacify Oracle Studio 12.6 in init_switches
* src/main.c (init_switches): Use a cast to convert char const *
to char *. Without this patch, the compiler complains
‘"src/main.c", line 2643: warning: assignment type mismatch:
pointer to char "=" pointer to const char’.
Paul Eggert [Tue, 27 Aug 2019 07:37:08 +0000 (00:37 -0700)]
Pacify Oracle Studio 12.6
* src/arscan.c [VMS]: Use ‘#if !defined LBR$_HDRTRUNC’ instead
of ‘#ifndef LBR$_HDRTRUNC’ to pacify Oracle Studio 12.6, which
looks inside unused #if branches for preprocessor syntax errors.
Without this patch it complains ‘"src/arscan.c", line 43: warning:
tokens ignored at end of directive line’.
Eli Zaretskii [Wed, 28 Aug 2019 14:45:49 +0000 (17:45 +0300)]
Fix MS-Windows MinGW build
* src/w32/subproc/sub_proc.c (process_wait_for_multiple_objects):
Fix format specifier for GetLastError's value.
* src/job.c (reap_children): Define the 'remote_status_lose' label
only for Posix platforms, to avoid compiler warning.
* build_w32.bat (LNKOUT): New variable, using forward slashes.
Use forward slashes in calls to :Compile, so that linking with GNU
ld works.
* src/makeint.h [!HAVE_UMASK]: Prototype for 'umask'.
(UMASK, MODE_T): Don't define.
* src/misc.c (get_tmpfile): Don't call UMASK, call umask, to
avoid compilation warning on !HAVE_UMASK platforms.
* src/output.c (output_tmpfd): Likewise.
* src/misc.c (umask) [!HAVE_UMASK]: New no-op function.
* src/config.h.W32.template (__USE_MINGW_ANSI_STDIO)
[__MINGW32__]: Define to 1, to force Make use ANSI-compatible
stdio functions, which also support the non-standard 'Id' and 'Ix'
specifiers.
(HAVE_UMASK) [__MINGW32__]: Define to 1.
* src/arscan.c (ar_member_touch): Type-cast argument of strlen to
avoid compiler warnings.
Paul Smith [Sun, 25 Aug 2019 21:11:10 +0000 (17:11 -0400)]
Update maintainer mode to support debug wait points.
Make a spin() method available when compiled in maintainer mode.
If a file exists in the working directory with a specific name,
make will sleep until the file is deleted.
Ensure that maintainer mode is enabled on Windows, using the same
algorithm we use on POSIX / autoconf systems.
* build_w32.bat: If maintMakefile exists, enable maintainer mode.
* src/main.c (main): Replace Windows-only suspend flag with spin().
* src/makeint.h: A SPIN() macro calls spin() in maintainer mode.
* src/misc.c (spin): If a spin file exists sleep until it's deleted.
[SV 56449] (Windows) Use slow path if '%' appears in the command
* src/job.c (sh_chars_dos): Add '%' as a special character
* src/job.c (construct_command_argv_internal): Check for '%' in quotes
For the windows version of make, a recipe line cannot be executed
diretly by make (without a shell) if a '%' character is present. This
character starts a cmd.exe escape sequence.
Signed-off-by: Christian Eggers <ceggers@arri.de>
Copyright-paperwork-exempt: yes
Mike Haboustak [Fri, 12 Jul 2019 01:08:38 +0000 (21:08 -0400)]
[SV 28456] Don't override $< when no default rule has been defined
The check for matching a file's command to the default rule's command
does not account for null. If no .DEFAULT is defined a rule with no
recipe has it's $< variable set to the value of $@. This breaks second
expansion, particularly when used with pattern rules.
* src/commands.c [set_file_variables]: Check that cmds is set
* tests/scripts/features/se_explicit: Test case
* tests/scripts/features/se_implicit: Test case
Paul Smith [Sun, 19 May 2019 16:16:14 +0000 (12:16 -0400)]
[SV 46013] Allow recursive variable overrides from Makefiles
Ensure that variable overrides are passed to recursive make instances
even if no overrides were provided on the command line.
Fix suggested by Rici Lake <ricilake@gmail.com>
* src/main.c (define_makeflags): Add overrides without respect to the
value of command_variables.
* tests/scripts/features/recursion: Add a test.
Paul Smith [Sun, 19 May 2019 15:25:06 +0000 (11:25 -0400)]
Update developer customizations
* .ccls: Remove -std=c99: it disables __attribute__ causing warnings.
* .dir-locals.el: Force the C coding style to "gnu".
* make-gdb.py: Add GDB macros for pretty-printing GNU make structs.
* scripts/copyright-update: Use GNULIB_SRCDIR to find update-copyright.
Paul Smith [Sun, 19 May 2019 14:59:06 +0000 (10:59 -0400)]
[SV 54549] Don't free used set_lists during merge
When merging the variable_set_lists for two targets it could be
that the "from" set list is a subset of the "to" set list: check
for this situation to avoid freeing used memory.
* src/variable.c (merge_variable_set_lists): Walk the "to" list and
if the "from" list is contained in it, nothing to do.
* tests/scripts/features/se_explicit: Add a test.
Paul Smith [Sun, 12 May 2019 21:01:55 +0000 (17:01 -0400)]
[SV 54740] Ensure .SILENT settings do not leak into sub-makes
Create a new variable run_silent to hold the current instance's global
silence setting, allowing silent_flag to represent only whether the -s
option was provided on the command line.
* src/makeint.h: Change silent_flag variable to run_silent.
* src/job.c: Ditto.
* src/remake.c: Ditto.
* src/file.c: Ditto.
* src/main.c: Add a new global variable run_silent.
(decode_switches): After switches are decoded, initialize run_silent.
* tests/scripts/targets/SILENT: Add a test for recursive behavior.
Kaz Kylheku [Sat, 11 May 2019 22:35:03 +0000 (18:35 -0400)]
[SV 8297] Implement "grouped targets" for explicit rules.
This patch allows "grouped targets" using the &: syntax:
tgt1 tgt2 ... tgtn &: pre1 pre2 ...
recipe
When the &: separator is used (in single or double colon forms), all
the targets are understood to be built by a single invocation of the
recipe. This is accomplished by piggy-backing on the already-existing
pattern rule feature, using the file's "also_make" list.
* NEWS: Add information about grouped targets.
* doc/make.texi (Multiple Targets): Add information on grouped targets.
(Pattern Intro): Refer to the new section to discuss multiple patterns.
* src/main.c (main): Add "grouped-targets" to .FEATURES
* src/read.c (make_word_type): Add new types for &: and &::.
(eval): Recognize the &: and &:: separator and remember when used.
(record_files): Accept an indicator of whether the rule is grouped.
If so, update also_make for each file to depend on the other files.
(get_next_mword): Recognize the &: and &:: word types.
* tests/scripts/features/grouped_targets: New test script.
* AUTHORS: Add Kaz Kylheku
Paul Smith [Sat, 11 May 2019 14:30:26 +0000 (10:30 -0400)]
Add developer customizations
* .ccls: Configure the ccls LSP server
* .dir-locals.el: Reset some parameters for ccls and lsp-mode
* .gitignore: Update for GTAGS and ccls
* maintMakefile: Don't search hidden directories for sources
Paul Smith [Sun, 16 Sep 2018 05:04:10 +0000 (01:04 -0400)]
Change output_write() to writebuf(), add readbuf() helper.
* src/misc.c (writebuf, readbuf): Create helper functions that will
reliably write a buffer to a file descriptor in the face of EINTR
causing short writes, and read from a file descriptor into a buffer
in the face of EINTR causing short reads.
* src/makeint.h: Declare these functions.
* src/output.c: Remove output_write() and replace with writebuf().
(_outputs, out_of_memory): Call writebuf(), not output_write().
* src/arscan.c (ar_scan): Call readbuf() instead of read(2).
(ar_member_touch): Remove duplicate header write, call writebuf()
instead of output_write(), and readbuf() instead of read(2).
Paul Smith [Sat, 15 Sep 2018 18:48:57 +0000 (14:48 -0400)]
* src/main.c (main): Set jobserver permissions before re-execing
Fixes an issue seen in the Linux kernel build system, reported by
Masahiro Yamada <yamada.masahiro@socionext.com>. Fix suggested
on the bug-make mailing list by Mike Shal <marfey@gmail.com>.
* tests/scripts/features/parallelism: Add a test to verify this.
Paul Smith [Sun, 9 Sep 2018 19:58:13 +0000 (15:58 -0400)]
* doc/make.text: [SV 54360] Weaken "obsolete" language in the manual.
There's no need to frighten users away from capabilities which are
perfectly reasonable and definitely not going away, by calling them
obsolete or discouraged in the manual.
Paul Smith [Sun, 9 Sep 2018 19:30:07 +0000 (15:30 -0400)]
[SV 40657] Don't create pattern rules for suffix rules with deps.
* NEWS: Update with a backward-compatibility warning.
* src/rule.c (convert_to_pattern): If a suffix rule has dependencies,
do not create a pattern rule for it. According to the manual suffix
rules with prerequisites are treated as normal targets.
* tests/scrips/features/suffixrules: Create some regression tests for
.SUFFIXES and suffix rules.
Paul Smith [Sat, 4 Aug 2018 23:07:59 +0000 (19:07 -0400)]
[SV 54233] Preserve higher command_state values on also_make targets.
If multiple pattern rules have the same pattern as also-make targets
and we attempt to run them at the same time, we might downgrade the
command state from 'running' to 'deps_running'; this will prevent
that also_make from being considered complete causing make to wait
forever for it to finish.
Ensure that set_command_state never downgrades the state of a target.
* src/file.c (set_command_state): Don't downgrade command_state.
* src/filedef.h (struct file): Document the order prerequisite.
* test/scripts/features/patternrules: Test the behavior.
Ben Hutchings [Sat, 28 Jul 2018 02:31:11 +0000 (10:31 +0800)]
* src/arscan.c (ar_scan): [SV 54395] Allow long names in archives.
Commit bc9d72beb0cb "Resolve issues discovered by static code
analysis." added range checks on archive member name length. However,
on non-AIX systems it also checked BSD-style long names against the
short name limits and and checked the *offset* for GNU-style long
names against the short name limits. This caused valid long names to
be rejected.
* Record the size of the GNU name map and validate offsets against it
* Ensure that the last entry in the name map is null-terminated
* Apply a maximum length of INT_MAX for element sizes
Reported-by: Philipp Wolski <philipp.wolski@kisters.de>
Paul Smith [Sat, 4 Aug 2018 16:20:11 +0000 (12:20 -0400)]
Clean up errors for invalid commands and add regression tests.
* src/function.c (func_shell_base): Use error() instead of recreating
the error output.
* src/job.c (exec_command): Show more standard error messages.
* src/load.c (unload_file): Fix whitespace in the error message.
* tests/scripts/features/errors: Add tests for starting non-
existent commands and new error message formats.
* tests/scripts/features/output-sync: New error message formats.
* tests/scripts/functions/shell: Ditto.
Paul Smith [Sat, 4 Aug 2018 16:18:39 +0000 (12:18 -0400)]
Queue failed fork() (etc.) to be handled like any other failed job.
If we failed to fork() we were essentially exiting make immediately
without respect to ignore flags, etc. On one hand that makes sense
because if you can't fork you're in real trouble, but it doesn't
work so well on systems where we don't fork at all. Instead, treat
a fork error like any other error by delaying the handling until
the next call to reap_children(). Any child with a PID of -1 is
considered to have died before starting so check these first without
waiting for them.
* src/commands.c (fatal_error_signal): Don't kill children that
never started.
* src/function.c (func_shell_base): Handle cleanup properly if the
child doesn't start.
* src/job.c (reap_children): Check for children that died before
starting and handle them without waiting for the PID.
(start_job_command): Free memory when the child doesn't start.
(start_waiting_job): Don't manage children who never started.
(child_execute_job): If the fork fails return PID -1.
* src/vmsjobs.c: Check for children that never started.
* tests/run_make_tests.pl: Parse config.status to get all options.
It happens because in the gl->gl_readdir callback supplied to glob(),
dirent->d_type is set to DT_UNKNOWN, and the glob() implementation
in glibc assumes that such a directory entry *cannot* possibly be a
regular file.
Pass the actual d_type down to glob(); this is the right thing to do
even if glibc is fixed, because it saves an extra stat() syscall for
each dirent.
Paul Smith [Fri, 3 Aug 2018 00:51:11 +0000 (20:51 -0400)]
Enhance the Basic.mk environment to work with Gnulib
* maintMakefile: Omit generated headers from Basic.mk prerequisites.
* Basic.mk.template: Rework commands to use with $(call ...) macros.
* mk/Amiga.mk: Ditto.
* mk/VMS.mk: Ditto.
* mk/Windows32.mk: Ditto, plus P2W to convert POSIX to Windows paths.
* mk/msdosdjgpp.mk: Ditto.
Paul Smith [Sun, 1 Jul 2018 20:13:13 +0000 (16:13 -0400)]
Update regression tests for Windows.
* tests/scripts/features/jobserver: Windows doesn't use pipes
* tests/scripts/functions/shell: Don't test kill -2 on Windows
* tests/scripts/misc/bs-nl: Windows doesn't handle single quotes
* tests/scripts/misc/general3: Ditto.
Paul Smith [Sun, 1 Jul 2018 16:28:25 +0000 (12:28 -0400)]
Convert GNU make to use the gnulib portability library
Move content from glob/* and config/* into standard GNU directory
locations lib/* and m4/*.
Install the gnulib bootstrap script and its configuration file, and
create a bootstrap.bat file for Windows. Update the README.git file
with new requirements and instructions for building from Git.
At this point we only install the alloca, getloadavg, and FDL modules
from gnulib. We keep our old glob/fnmatch implementation since the
gnulib versions require significant amounts of infrastructure which
doesn't exist on Windows yet. Further work is required here.
Due to a problem with gnulib's version of getloadavg, we need to bump
the minimum required version of automake to 1.16.1 unfortunately.
* README.git: Update instructions
* NEWS: Move developer news to a separate section
* configure.ac: Update for use with gnulib modules
* bootstrap: Bootstrap from Git workspace (import from gnulib)
* bootstrap.conf: Bootstrap configuration for GNU make
* bootstrap.bat: Bootstrap from Git workspace for Windows
* gl/modules/make-glob: Support our local fnmatch/glob implementation
* config/acinclude.m4: Move to m4/
* config/dospaths.m4: Move to m4/
* glob/fnmatch.c: Move to lib/
* glob/fnmatch.h.in: Move to lib/
* glob/glob.c: Move to lib/
* glob/glob.h.in: Move to lib/
* Makefile.am: Update for new directories
* build.template: Update for new directories
* build_w32.bat: Update for new directories
* builddos.bat: Update for new directories
* maintMakefile: Update for new directories
* makefile.com: Update for new directories
* mk/Amiga.mk: Update for new directories
* mk/Posix.mk.in: Update for new directories
* mk/VMS.mk: Update for new directories
* mk/Windows32.mk: Update for new directories
* mk/msdosdjgpp.mk: Update for new directories
* po/LINGUAS: One language per line (needed by gnulib)
* INSTALL: Remove (obtained from gnulib)
* src/alloca.c: Remove (obtained from gnulib)
* src/getloadavg.c: Remove (obtained from gnulib)
* po/Makevars: Remove (created by bootstrap)
* config/*: Remove leftover files
* glob/*: Remove leftover files
Paul Smith [Sun, 12 Nov 2017 22:44:38 +0000 (17:44 -0500)]
Remove unsupported build facilities.
Over time the non-standard build and install systems (nmake files,
smake files, Visual Studio project files, etc.) have atrophied and
maintaining them is not worth the effort, for such a simple utility
as make. Remove all the non-standard build tool support and unify
OS-specific build rules under a basic set of (GNU make) makefiles.
Preserve the existing bootstrapping scripts (for POSIX, Windows,
and MS-DOS). Also the existing VMS build scripts are left unchanged:
I don't have enough experience with VMS to venture into this area.
Perhaps one of the VMS maintainers might like to determine whether
conversion would be appropriate.
Rather than create libraries for w32 and glob (non-POSIX), simply
link the object files directly to remove the complexity.
* NEWS: Update with user-facing notes.
* Makefile.am: Clean up to use the latest automake best practices.
Build Windows code directly from the root makefile to avoid recursion.
* README.Amiga, README.DOS.template, README.W32.template: Updated.
* INSTALL: Point readers at the README.git file.
* maintMakefile: Remove obsolete files. Create Basic.mk file.
* Basic.mk.template, mk/*.mk: Create basic GNU make-based makefiles.
* build_w32.bat: Copy Basic.mk to Makefile
* configure.ac: We no longer need AM_PROG_AR.
* dosbuild.bat: Rename to builddos.bat. Incorporate configure.bat.
* Makefile.DOS.template: Remove.
* NMakefile.template, w32/subproc/NMakefile: Remove.
* SMakefile.template, glob/SMakefile, glob/SCOPTIONS, make.lnk: Remove.
* configure.bat, glob/configure.bat: Remove.
* w32/Makefile.am: Remove.
* make_msvc_net2003.sln, make_msvc_net2003.vcproj: Remove.
Paolo Bonzini [Fri, 11 Aug 2017 11:44:33 +0000 (13:44 +0200)]
Do not use STOP_SET for singleton compares.
Compare against '$' directly rather than using MAP_VARIABLE.
This saves ~10% for find_map_function, which is the top hotspot in
QEMU's no-op build. The build is sped up overall by about 1.5%
more (from 11.1s to 10.95s).
* read.c (find_map_function): Do not compare against singleton sets.
Paolo Bonzini [Fri, 11 Aug 2017 11:44:28 +0000 (13:44 +0200)]
Use Jenkins hash.
This is about twice as fast as the current hash, and removes the
need for double hashing (improving locality of reference). The
hash function is based on Bob Jenkins' design, slightly adapted
wherever Make needs to hash NUL-terminated strings. The old hash
function is kept for case-insensitive hashing.
This saves 8.5% on QEMU's no-op build (from 12.87s to 11.78s).
Paolo Bonzini [Fri, 11 Aug 2017 11:44:29 +0000 (13:44 +0200)]
Use strchr for simple case of find_char_unquote.
In most cases, find_char_unquote has a single stopchar. In that
case we can look for it using strchr's optimized implementation.
This saves 3.5% on QEMU's noop build (from 11.78s to 11.37s).
* read.c (find_char_unquote): Rename to find_map_unquote. Replace
with an implementation optimized for the case where the stopchar
is a singleton. Adjust all callers.
Paolo Bonzini [Fri, 11 Aug 2017 11:44:30 +0000 (13:44 +0200)]
Use strchr/memmove in collapse_continuations.
collapse_continuations is already using strchr to speed up the
common case of no backslash-newline sequence, but on modern
processors it is faster to scan the string twice with
strchr+memmove (or strlen+memmove) than to move bytes manually.
Saves about 1.5% on QEMU's no-op build (from 11.37s to 11.23s).
* misc.c (collapse_continuations): Rewrite the scanning of LINE.
Paul Eggert [Sun, 24 Sep 2017 13:12:58 +0000 (09:12 -0400)]
glob: Do not assume glibc glob internals.
It has been proposed that glibc glob start using gl_lstat,
which the API allows it to do. GNU 'make' should not get in
the way of this. See:
https://sourceware.org/ml/libc-alpha/2017-09/msg00409.html
* dir.c (local_lstat): New function, like local_stat.
(dir_setup_glob): Use it to initialize gl_lstat too, as the API
requires.
Paul Smith [Sat, 23 Sep 2017 19:55:37 +0000 (15:55 -0400)]
Compute load from number of running processes.
* job.c (load_too_high): If /proc/loadavg is available, compare the
maximum load against the number of processes currently running. If
we can't parse /proc/loadavg, use the previous algorithm.
Concept and initial patch from Sven C. Dack <sven.c.dack@sky.com>
Paul Smith [Sun, 9 Jul 2017 22:44:17 +0000 (18:44 -0400)]
[SV 51400] Only unblock fatal signals after child invocation
* job.c (unblock_sigs): Only unblock fatal signals not all signals.
(unblock_all_sigs): Unblock all signals not just fatal signals.
(child_execute_job): Call unblock_all_sigs() in child process.
* job.h: Remove unused function definitions.
* remote-cstms.c (start_remote_job): Call unblock_all_sigs() in
child process.
Reported by Koen Van Hoof <koen.van_hoof@nokia.com>