Paul Smith [Sun, 1 Jul 2018 16:28:25 +0000 (12:28 -0400)]
Convert GNU make to use the gnulib portability library
Move content from glob/* and config/* into standard GNU directory
locations lib/* and m4/*.
Install the gnulib bootstrap script and its configuration file, and
create a bootstrap.bat file for Windows. Update the README.git file
with new requirements and instructions for building from Git.
At this point we only install the alloca, getloadavg, and FDL modules
from gnulib. We keep our old glob/fnmatch implementation since the
gnulib versions require significant amounts of infrastructure which
doesn't exist on Windows yet. Further work is required here.
Due to a problem with gnulib's version of getloadavg, we need to bump
the minimum required version of automake to 1.16.1 unfortunately.
* README.git: Update instructions
* NEWS: Move developer news to a separate section
* configure.ac: Update for use with gnulib modules
* bootstrap: Bootstrap from Git workspace (import from gnulib)
* bootstrap.conf: Bootstrap configuration for GNU make
* bootstrap.bat: Bootstrap from Git workspace for Windows
* gl/modules/make-glob: Support our local fnmatch/glob implementation
* config/acinclude.m4: Move to m4/
* config/dospaths.m4: Move to m4/
* glob/fnmatch.c: Move to lib/
* glob/fnmatch.h.in: Move to lib/
* glob/glob.c: Move to lib/
* glob/glob.h.in: Move to lib/
* Makefile.am: Update for new directories
* build.template: Update for new directories
* build_w32.bat: Update for new directories
* builddos.bat: Update for new directories
* maintMakefile: Update for new directories
* makefile.com: Update for new directories
* mk/Amiga.mk: Update for new directories
* mk/Posix.mk.in: Update for new directories
* mk/VMS.mk: Update for new directories
* mk/Windows32.mk: Update for new directories
* mk/msdosdjgpp.mk: Update for new directories
* po/LINGUAS: One language per line (needed by gnulib)
* INSTALL: Remove (obtained from gnulib)
* src/alloca.c: Remove (obtained from gnulib)
* src/getloadavg.c: Remove (obtained from gnulib)
* po/Makevars: Remove (created by bootstrap)
* config/*: Remove leftover files
* glob/*: Remove leftover files
Paul Smith [Sun, 12 Nov 2017 22:44:38 +0000 (17:44 -0500)]
Remove unsupported build facilities.
Over time the non-standard build and install systems (nmake files,
smake files, Visual Studio project files, etc.) have atrophied and
maintaining them is not worth the effort, for such a simple utility
as make. Remove all the non-standard build tool support and unify
OS-specific build rules under a basic set of (GNU make) makefiles.
Preserve the existing bootstrapping scripts (for POSIX, Windows,
and MS-DOS). Also the existing VMS build scripts are left unchanged:
I don't have enough experience with VMS to venture into this area.
Perhaps one of the VMS maintainers might like to determine whether
conversion would be appropriate.
Rather than create libraries for w32 and glob (non-POSIX), simply
link the object files directly to remove the complexity.
* NEWS: Update with user-facing notes.
* Makefile.am: Clean up to use the latest automake best practices.
Build Windows code directly from the root makefile to avoid recursion.
* README.Amiga, README.DOS.template, README.W32.template: Updated.
* INSTALL: Point readers at the README.git file.
* maintMakefile: Remove obsolete files. Create Basic.mk file.
* Basic.mk.template, mk/*.mk: Create basic GNU make-based makefiles.
* build_w32.bat: Copy Basic.mk to Makefile
* configure.ac: We no longer need AM_PROG_AR.
* dosbuild.bat: Rename to builddos.bat. Incorporate configure.bat.
* Makefile.DOS.template: Remove.
* NMakefile.template, w32/subproc/NMakefile: Remove.
* SMakefile.template, glob/SMakefile, glob/SCOPTIONS, make.lnk: Remove.
* configure.bat, glob/configure.bat: Remove.
* w32/Makefile.am: Remove.
* make_msvc_net2003.sln, make_msvc_net2003.vcproj: Remove.
Paolo Bonzini [Fri, 11 Aug 2017 11:44:33 +0000 (13:44 +0200)]
Do not use STOP_SET for singleton compares.
Compare against '$' directly rather than using MAP_VARIABLE.
This saves ~10% for find_map_function, which is the top hotspot in
QEMU's no-op build. The build is sped up overall by about 1.5%
more (from 11.1s to 10.95s).
* read.c (find_map_function): Do not compare against singleton sets.
Paolo Bonzini [Fri, 11 Aug 2017 11:44:28 +0000 (13:44 +0200)]
Use Jenkins hash.
This is about twice as fast as the current hash, and removes the
need for double hashing (improving locality of reference). The
hash function is based on Bob Jenkins' design, slightly adapted
wherever Make needs to hash NUL-terminated strings. The old hash
function is kept for case-insensitive hashing.
This saves 8.5% on QEMU's no-op build (from 12.87s to 11.78s).
Paolo Bonzini [Fri, 11 Aug 2017 11:44:29 +0000 (13:44 +0200)]
Use strchr for simple case of find_char_unquote.
In most cases, find_char_unquote has a single stopchar. In that
case we can look for it using strchr's optimized implementation.
This saves 3.5% on QEMU's noop build (from 11.78s to 11.37s).
* read.c (find_char_unquote): Rename to find_map_unquote. Replace
with an implementation optimized for the case where the stopchar
is a singleton. Adjust all callers.
Paolo Bonzini [Fri, 11 Aug 2017 11:44:30 +0000 (13:44 +0200)]
Use strchr/memmove in collapse_continuations.
collapse_continuations is already using strchr to speed up the
common case of no backslash-newline sequence, but on modern
processors it is faster to scan the string twice with
strchr+memmove (or strlen+memmove) than to move bytes manually.
Saves about 1.5% on QEMU's no-op build (from 11.37s to 11.23s).
* misc.c (collapse_continuations): Rewrite the scanning of LINE.
Paul Eggert [Sun, 24 Sep 2017 13:12:58 +0000 (09:12 -0400)]
glob: Do not assume glibc glob internals.
It has been proposed that glibc glob start using gl_lstat,
which the API allows it to do. GNU 'make' should not get in
the way of this. See:
https://sourceware.org/ml/libc-alpha/2017-09/msg00409.html
* dir.c (local_lstat): New function, like local_stat.
(dir_setup_glob): Use it to initialize gl_lstat too, as the API
requires.
Paul Smith [Sat, 23 Sep 2017 19:55:37 +0000 (15:55 -0400)]
Compute load from number of running processes.
* job.c (load_too_high): If /proc/loadavg is available, compare the
maximum load against the number of processes currently running. If
we can't parse /proc/loadavg, use the previous algorithm.
Concept and initial patch from Sven C. Dack <sven.c.dack@sky.com>
Paul Smith [Sun, 9 Jul 2017 22:44:17 +0000 (18:44 -0400)]
[SV 51400] Only unblock fatal signals after child invocation
* job.c (unblock_sigs): Only unblock fatal signals not all signals.
(unblock_all_sigs): Unblock all signals not just fatal signals.
(child_execute_job): Call unblock_all_sigs() in child process.
* job.h: Remove unused function definitions.
* remote-cstms.c (start_remote_job): Call unblock_all_sigs() in
child process.
Reported by Koen Van Hoof <koen.van_hoof@nokia.com>
Eli Zaretskii [Wed, 7 Jun 2017 05:32:35 +0000 (08:32 +0300)]
Avoid crashes when SHELL=abcde is specified on the command line
* variable.c (do_variable_definition): If $SHELL was not found,
process "SHELL=foo" as any other variable definition. This avoids
segfaults when SHELL=foo is specified on the Make command line.
Reported by Orgad Shaneh <orgads@gmail.com>.
Paul Smith [Sat, 3 Jun 2017 20:20:51 +0000 (16:20 -0400)]
[SV 51159] Use a non-blocking read with pselect to avoid hangs.
* posixos.c (set_blocking): Set blocking on a file descriptor.
(jobserver_setup): Set non-blocking on the jobserver read side.
(jobserver_parse_auth): Ditto.
(jobserver_acquire_all): Set blocking to avoid a busy-wait loop.
(jobserver_acquire): If the non-blocking read() returns without
taking a token then try again.
Paul Smith [Sun, 28 May 2017 04:33:29 +0000 (00:33 -0400)]
[SV 50823] Support filenames containing '$' in MAKEFILE_LIST
* variable.h (enum variable_flavor: Add a new flavor for appended
values that shouldn't be expanded.
* variable.c (do_variable_definition): If given this new flavor,
do not expand the value before appending it.
* read.c (eval_makefile): Use this new flavor for MAKEFILE_LIST
* tests/scripts/variables/MFILE_LIST: Test filenames containing '$'.
Paul Smith [Sun, 28 May 2017 00:07:30 +0000 (20:07 -0400)]
* NEWS: Do not insert a space during '+=' if the value is empty.
* doc/make.texi (Appending): Document this behavior.
* variable.c (do_variable_definition): Only add a space if the variable
value is not empty.
* tests/scripts/variables/flavors: Test this behavior.
Paul Smith [Mon, 2 Jan 2017 20:46:30 +0000 (15:46 -0500)]
[SV 13651] Handle out-of-memory conditions slightly more gracefully.
* makeint.h: Change OUT_OF_MEM() macro to out_of_memory() function.
* output.h, job.h: Move FD_* macros from job.h to output.h.
* output.c (output_write): Write a buffer to an FD directly.
(out_of_memory): Use output_write() to avoid allocating more
memory while writing the error, and call exit() instead of die().
This does mean we can't translate the error string, though.
* misc.c (xmalloc, xcalloc, xrealloc, xstrdup, xstrndup): Call new
out_of_memory() rather than OUT_OF_MEM().
* read.c (parse_file_seq): Ditto.
Paul Smith [Mon, 17 Apr 2017 19:37:57 +0000 (15:37 -0400)]
Add test suite support to Windows
* main.c (main): Sanitize program name detection on Windows.
* makeint.h: 'program' is a const string on all platforms now.
* tests/run_make_tests.bat: Windows bat file to invoke tests
* tests/test_driver.pl: Obtain system-specific error messages.
(get_osname): Compute the $port_type here. Add more $osname checks
for different Windows Perl ports.
(_run_command): Rewrite the timeout capability to work properly
with Windows. Don't use Perl fork/exec; instead use system(1,...)
which allows a more reliable/proper kill operation.
Also, allow options to be given as a list instead of a string, to
allow more complex quoting of command-line arguments.
* tests/run_make_tests.pl (run_make_with_options): Allow options
to be provided as a list in addition to a simple string.
(set_more_defaults): Write sample makefiles and run make on them
instead of trying to run echo and invoking make with -f-, to avoid
relying on shell and echo to get basic configuration values. Also
create a $sh_name variable instead of hard-coding /bin/sh.
* tests/scripts/features/archives: Skip on Windows.
* tests/scripts/features/escape: Use list method for passing options.
* tests/scripts/features/include: Use system-specific error messages.
* tests/scripts/features/output-sync: "Command not found" errors
generate very different / odd output on Windows. This needs to be
addressed but for now disable these tests on Windows.
* tests/scripts/functions/abspath: Disable on Windows.
* tests/scripts/functions/file: Use system-specific error messages.
* tests/scripts/functions/shell: "Command not found" errors generate
very different / odd output on Windows. This needs to be addressed
but for now disable these tests on Windows.
* tests/scripts/misc/close_stdout: Disable on Windows.
* tests/scripts/options/dash-k: Use system-specific error messages.
* tests/scripts/options/dash-l: Disable on Windows.
* tests/scripts/options/eval: Use list method for passing options.
* tests/scripts/options/general: Skip some non-portable tests.
* tests/scripts/targets/ONESHELL: Skip some non-portable tests.
* tests/scripts/targets/POSIX: Skip some non-portable tests.
* tests/scripts/variables/MAKEFILES: Skip some non-portable tests.
* tests/scripts/variables/SHELL: Use a makefile not -f- for testing.
Eli Zaretskii [Wed, 11 Jan 2017 18:43:41 +0000 (20:43 +0200)]
[SV 50021] Avoid infloop on MS-Windows with short scripts
* job.c (reap_children) [WINDOWS32]: Avoid recursive call to
reap_children when the argument passed to
map_windows32_error_to_string is negative or too large.
Paul Smith [Wed, 28 Dec 2016 05:41:38 +0000 (00:41 -0500)]
* main.c (switches): Add -E as an alias for --eval.
* make.1: Document the -E and --eval options.
* doc/make.texi: Document the -E option.
* tests/scripts/options/eval: Test the -E option and MAKEFILES.
* NEWS: Add information about the new option.
Paul Smith [Wed, 28 Dec 2016 05:23:55 +0000 (00:23 -0500)]
* main.c (switches): Add --no-silent to undo -s options.
* make.1: Document the new flag.
* doc/make.texi: Document the new flag. Remove suggestions that the
.SILENT special target is deprecated or should not be used.
* tests/scripts/options/dash-s: Test the -s and --no-silent options.
* NEWS: Add information about the new option.
Paul Smith [Sun, 25 Dec 2016 22:41:50 +0000 (17:41 -0500)]
[SV 40236] Handle included file open failures properly.
* read.c (eval_makefile): Set deps->error if we discovered any
error reading makefiles, and set NONEXISTENT_MTIME so we know
it needs to be rebuilt.
* main.c (main): Clean up management of makefile_mtimes.
* tests/scripts/features/include: Add open failure testcases.
Paul Smith [Fri, 23 Dec 2016 18:33:58 +0000 (13:33 -0500)]
Portability changes for the test suite.
* tests/test_driver.pl: Save error strings for later comparison.
* tests/run_make_tests.pl: Create portable commands for later use.
* tests/*: Use these new variables.
Paul Smith [Thu, 22 Dec 2016 23:47:26 +0000 (18:47 -0500)]
[SV 20513] Un-escaped # are not comments in function invocations
* NEWS: Document the change, as a backward-incompatible change.
* main.c (main): Add 'nocomment' to the .FEATURES variable.
* read.c (remove_comments): Skip variable references during remove.
(find_char_unquote): Fix comments for new STOPMAP support.
* tests/scripts/features/escape: Test new escape syntax.
* tests/scripts/functions/guile: Ditto.
* tests/scripts/functions/shell: Ditto.
Paul Smith [Sun, 25 Sep 2016 23:06:56 +0000 (19:06 -0400)]
Resolve issues discovered by static code analysis.
* maintMakefile: Add a rule to submit code for analysis.
* configure.ac: Check for availability of the umask() function.
* output.c (output_tmpfd, output_tmpfile): Use umask on temp files.
* makeint.h (PATH_VAR): Reserve an extra character for nul bytes.
* function.c (func_error): Initialize buffer to empty string.
* job.c (child_execute_job): Verify validity of fdin.
* main.c (main): Simplify code for makefile updating algorithm.
* arscan.c (ar_scan): Verify member name length before reading.
* read.c (readline): Cast pointer arithmetic to avoid warnings.
* remake.c (update_file): Remove unreachable code.
(name_mtime): Verify symlink name length.
Marc Ullman [Sat, 12 Nov 2016 08:53:28 +0000 (10:53 +0200)]
Support more than 63 jobs on MS-Windows
* job.c (start_waiting_job, load_too_high):
* w32/w32os.c (jobserver_setup, jobserver_acquire): Abstracted out
MAXIMUM_WAIT_OBJECTS. Call process_table_full instead.
* w32/include/sub_proc.h: Update and add prototypes.
* w32/subproc/sub_proc.c (GMAKE_MAXIMUM_WAIT_OBJECTS): New macro.
(process_wait_for_multiple_objects): Drop-in replacement for Windows
API WaitForMultipleOjects.
(process_wait_for_any_private): Replaced MAXIMUM_WAIT_OBJECTS with
GMAKE_MAXIMUM_WAIT_OBJECTS.
(process_table_full): Replacement for process_used_slots.
(process_used_slots): Removed, as no longer needed.
(process_table_usable_size): Returns maximum usable size of process
table.
(process_table_actual_size): Returns actual size of process table.
(process_register): Added assertion.
(process_easy): Abstracted out MAXIMUM_WAIT_OBJECTS.
Eli Zaretskii [Thu, 6 Oct 2016 19:39:14 +0000 (22:39 +0300)]
Avoid compiler warnings with MinGW runtime 3.22.2
* main.c:
* job.c: Include strings.h, to get the prototypes of strcasecmp
and strncasecmp with latest MinGW runtime versions.
* config.h.W32.template (HAVE_STRINGS_H): Define.
Paul Smith [Tue, 31 May 2016 06:56:51 +0000 (02:56 -0400)]
[SV 47995] Ensure forced double-colon rules work with -j.
The fix for SV 44742 had a side-effect that some double-colon targets
were skipped. This happens because the "considered" facility assumed
that all targets would be visited on each walk through the dependency
graph: we used a bit for considered and toggled it on each pass; if
we didn't walk the entire graph on every pass the bit would get out
of sync. The new behavior after SV 44742 might return early without
walking the entire graph. To fix this I changed the considered value
to an integer which is monotonically increasing: it is then never
possible to incorrectly determine that a previous pass through the
graph already considered the current target.
* filedef.h (struct file): make CONSIDERED an unsigned int.
* main.c (main): No longer need to reset CONSIDERED.
* remake.c (update_goal_chain): increment CONSIDERED rather than
inverting it between 0<->1.
(update_file_1): Reset CONSIDERED to 0 so it's re-considered.
(check_dep): Ditto.
* tests/scripts/features/double_colon: Add a regression test.
Joe Crayne [Sat, 21 May 2016 21:26:00 +0000 (17:26 -0400)]
[SV 44742] Fix double-colon rules plus parallel builds.
* remake.c (update_file): Don't update double-colon target status
if we're still building targets.
(ftime_t): Don't propagate timestamps for double-colon targets that
we've not examined yet.
* tests/scripts/features/double_colon: Add parallel build tests.
Paul Smith [Sun, 1 May 2016 23:24:20 +0000 (19:24 -0400)]
Separate the GNU make load ABI from internal types.
Create an internal type "floc" and convert all users to that type.
* gnumake.h (gmk_floc): Remove the offset field from this type.
* loadapi.c (gmk_eval): Convert gmk_floc to internal floc.
Paul Eggert [Sun, 1 May 2016 19:52:58 +0000 (15:52 -0400)]
Fixes for enhanced GCC warnings.
Move function prototypes into header files and out of .c files.
Use void argument lists for functions that accept no args.
Remove unused macros. Make private functions static. Align
types with printf format characters.
Paul Smith [Sun, 1 May 2016 19:34:42 +0000 (15:34 -0400)]
Fix compile issues with Windows and VMS.
* main.c (initialize_stopchar_map): isblank() is not part of C89.
Install bits for space and tab directly.
* makeint.h: Don't define vfork; autoconf handles this for us.
* vmsjobs.c: Rename NEXT_TOKEN to V_NEXT_TOKEN to avoid conflicts.
* dir.c (print_dir_data_base): Visual Studio C doesn't have int64_t.
* w32/subproc/sub_proc.c (process_begin): Missing arg to memset().
* build_w32.bat: Don't use obsolete Visual Studio flags.
Eli Zaretskii [Sun, 24 Apr 2016 06:53:38 +0000 (09:53 +0300)]
Fix the MS-Windows MinGW build
* build_w32.bat (GccCompile): Use -std=gnu99, as some code uses
C99 features ('for' loop initial declarations).
* dir.c (print_dir_data_base) [WINDOWS32]: Cast 'mtime' to
'int64_t', and use %I64d to print it, to avoid compile-time
warning about printing a 'time_t' value, which could be either
a 32-bit or a 64 bit integral type.
Paul Smith [Sun, 10 Apr 2016 21:12:48 +0000 (17:12 -0400)]
[SV 46433] Show recipe line offsets in line number messages.
While displaying line numbers, show the relevant line number inside
the recipe not just the first line of the entire recipe.
Sample changes suggested by Brian Vandenberg <phantall@gmail.com>
* gnumake.h (gmk_floc): Add an 'offset' to track the recipe offset.
* read.c (eval, eval_makefile, eval_buffer): Initialize 'offset'.
(record_files, install_pattern_rule): Ditto.
* job.c (new_job, job_next_command): Update 'offset' based on the
line of the recipe we're expanding or invoking.
(child_error): Add 'offset' when showing the line number.
* function.c (func_shell_base): Ditto.
* output.c (error, fatal): Ditto.
* NEWS: Mention the new ability.
* tests/scripts/features/errors: Check the line number on errors.
* tests/scripts/functions/warning: Check the line number on warnings.
* tests/scripts/features/output-sync,
tests/scripts/features/parallelism, tests/scripts/functions/shell,
tests/scripts/functions/error: Update line numbers.
Paul Smith [Sat, 9 Apr 2016 23:49:27 +0000 (19:49 -0400)]
[SV 102] Don't show unnecessary include file errors.
Delay the generation of error messages for included files until we
are sure that we can't rebuild that included file.
* dep.h (struct dep): Don't reuse "changed"; make a separate field
to keep "flags". Get rid of dontcare and use the flag.
(struct goaldep): Create a new structure for goal prereqs
that tracks an errno value and the floc where the include happened.
Rework the structures to ensure they are supersets as expected.
In maintainer mode with GCC, use inline to get type checking.
* read.c (eval_makefile): Return a struct goaldep for the new
makefile. Ensure errno is set properly to denote a failure.
(read_all_makefiles): Switch to goaldep and check errno.
(eval): Don't show included file errors; instead remember them.
* remake.c (update_goal_chain): Set global variables to the current
goaldep we're building, and the entire chain.
(show_goal_error): Check if the current failure is a consequence
of building an included makefile and if so print an error.
(complain): Call show_goal_error() on rule failure.
* job.c (child_error): Call show_goal_error() on child error.
* main.c (main): Switch from struct dep to goaldep.
* misc.c (free_dep_chain): Not used; make into a macro.
* tests/scripts/features/include: Update and include new tests.
* tests/scripts/options/dash-B, tests/scripts/options/dash-W,
tests/scripts/options/print-directory,
tests/scripts/variables/MAKE_RESTARTS: Update known-good-output.
projects / thirdparty / make.git / log
