Paul Smith [Thu, 12 Nov 2020 22:00:39 +0000 (17:00 -0500)]
[SV 59093] Rewrite filter/filter-out to avoid large stack usage
* src/function.c (func_filter_filterout): Allocate arrays to hold
pattern and word information rather than creating linked lists on
the stack.
* tests/scripts/functions/filter-out: Test large filters.
Paul Smith [Sun, 19 Jul 2020 17:56:23 +0000 (13:56 -0400)]
[SV 58735] Define the order that makefiles are rebuilt.
Ensure that makefiles are rebuilt in the order in which make first
considered them, and document this behavior in the manual.
* NEWS: Add a note about the new behavior
* doc/make.text (How make Processes a Makefile): Document it.
* main.c (main): Inverse the list of makefile goals.
* read.c (read_all_makefiles): Add default makefiles to the list at
the front in reverse order, the same way other makefiles are added.
* tests/scripts/features/include: Add tests to verify rebuild order.
Kevin Buettner [Thu, 23 Apr 2020 21:05:34 +0000 (17:05 -0400)]
[SV 58232] Disable inheritance of jobserver FDs for recursive make
A parent make will invoke a sub-make with close-on-exec disabled for
the jobserver pipe FDs. Force close-on-exec to be to be enabled in
the sub-make so the pipe is not always passed to child jobs.
I have a test case which, when invoked with a suitable -j switch,
will hang if the recipe inherits the jobserver pipe. This test case
was inspired by a real world case in which testing GDB on Fedora
would hang due to some poorly written test GDB cases having been
passed the jobserver file descriptors.
* src/posixos.c (jobserver_parse_auth): Call fd_noinherit() for
jobserver pipe descriptors.
Paul Smith [Wed, 1 Apr 2020 05:13:02 +0000 (01:13 -0400)]
[SV 57674] Use the system default PATH if $PATH is not set
When using execvp() if $PATH is not present in the environment
it will automatically search the system default PATH string. Emulate
this by passing the system default PATH to find_in_given_path() if
we don't find PATH in the environment.
* src/job.c (child_execute_job): Use confstr(_CS_PATH) if PATH is not
found.
Paul Smith [Wed, 1 Apr 2020 05:58:33 +0000 (01:58 -0400)]
tests: Simplify customization of %ENV
Rather than having an %extraENV that is added to the default %ENV
and resetting %ENV _before_ each test, allow the test setup to
modify %ENV directly as needed then reset %ENV _after_ each test.
* tests/test_driver.pl: Remove unused %extraENV.
(resetENV): Don't add in %extraENV.
(_run_command): Reset after we run the command rather than before.
* tests/scripts/features/export: Convert %extraENV to %ENV
* tests/scripts/features/jobserver: Ditto
* tests/scripts/features/parallelism: Ditto
* tests/scripts/features/targetvars: Ditto
* tests/scripts/functions/eval: Ditto
* tests/scripts/functions/foreach: Ditto
* tests/scripts/functions/origin: Ditto
* tests/scripts/misc/general4: Ditto
* tests/scripts/options/dash-e: Ditto
* tests/scripts/targets/POSIX: Ditto
* tests/scripts/variables/GNUMAKEFLAGS: Ditto
* tests/scripts/variables/SHELL: Ditto
Paul Smith [Tue, 31 Mar 2020 02:01:09 +0000 (22:01 -0400)]
[SV 57896] Change directories before checking jobserver auth
We want to process -C options as early as possible, before we might
write informational messages, so that Entering/Leaving messages have
the correct directory.
* src/main.c (main): Move code dealing with changing directories
before parsing of the jobserver auth flag.
* tests/scripts/features/jobserver: Test the order of enter/leave.
Paul Smith [Mon, 30 Mar 2020 18:38:47 +0000 (14:38 -0400)]
[SV 57967] Only set APPEND mode for regular files
APPEND is a permanent mode shared by all users of a file. If we
set it on a tty, pipe, etc. it will stay in effect even after make
exits, which can cause problems.
Patch provided by 0xef967c36@gmail.com
* src/output.c (set_append_mode): Check for a regular file.
Paul Smith [Mon, 30 Mar 2020 18:07:10 +0000 (14:07 -0400)]
Obey order of multiple print/no-print directory options
Previously if --no-print-directory was seen anywhere even once
(environment, command line, etc.) it would always take precedence
over any --print-directory option. Change this so that the last
seen option (which will be the command line, if present there) takes
precedence.
* NEWS: Mark this change in behavior.
* src/makeint.h (print_directory): A new variable to control printing.
* src/output.c (output_dump): Use the new variable.
(output_start): Ditto.
* src/main.c: Add a new variable print_directory. Use -1 for
print_directory_flag so we know of the option was seen or not. Add a
new default_print_directory_flag set to -1 to keep options from being
added.
(switches): Use flag_off for --no-print-directory, rather than a
separate inhibit_print_directory_flag.
(main): If print_directory_flag was set by the user, use that for
print_directory. If not, compute the print_directory value based on
-s, -C, and sub-makes as before.
* tests/scripts/variables/GNUMAKEFLAGS: -w is not added automatically
* tests/scripts/options/print-directory: Add tests for overriding
print-directory options.
Add support for building with Tiny C for MS-Windows
* src/config.h.W32.template (HAVE_DIRECT_H, HAVE_STRCASECMP)
(HAVE_STRNCASECMP, HAVE_UMASK): Add __TINYC__ to MinGW condition.
(BATCH_MODE_ONLY_SHELL): Make this the default for Tiny C.
* build_w32.bat: Support building with Tiny C's tcc compiler.
Paul Smith [Thu, 16 Jan 2020 18:49:07 +0000 (13:49 -0500)]
[SV 40657] Reinstate old behavior for suffix rules with prereqs
POSIX says that suffix rules cannot have prerequisites, but after
making this change we observed a number of makefiles "in the wild"
that were relying on this behavior and failed.
For .POSIX: makefiles, obey POSIX. Otherwise preserve the old
behavior. However, generate a warning so users know this is a
problem. In a future version we will change all behavior to be
POSIX-conforming.
* NEWS: describe the change
* src/rule.c (convert_to_pattern): If posix_pedantic don't make a
pattern rule if prereqs exist. Otherwise show a warning.
* tests/scripts/features/suffixrules: Add tests for the new behavior
including .POSIX vs. non-.POSIX.
Paul Smith [Fri, 3 Jan 2020 23:19:33 +0000 (18:19 -0500)]
Enable compilation with C90 compilers
* configure.ac: Try compiling Guile headers: they don't work with C90.
* maintMakefile: Simplify config checks via target-specific variables.
* src/makeint.h: Use ATTRIBUTE rather than defining __attribute__,
as that causes compile issues with system headers.
(ENUM_BITFIELD): Don't use enum bitfields in ANSI mode.
* src/main.c: Use ATTRIBUTE instead of __attribute__.
* src/job.h: Ditto.
* src/file.c: Don't define variables inside for loops.
* src/rule.c: Ditto.
* src/dep.h (SI): Only use static inline in non-ANSI mode.
Paul Smith [Thu, 2 Jan 2020 10:08:06 +0000 (05:08 -0500)]
Support the .EXTRA_PREREQS special variable
Initial implementation by Christof Warlich <cwarlich@gmx.de>
* NEWS: Announce the new feature.
* doc/make.texi (Other Special Variables): Document .EXTRA_PREREQS.
* src/dep.h (struct dep): New flag to note extra prereq deps.
* src/filedef.h (expand_extra_prereqs): Declare a function to expand
the value of .EXTRA_PREREQS.
* src/file.c (expand_extra_prereqs): Given a struct variable lookup
of .EXTRA_PREREQS, convert it into a list of deps and for each one
make sure it has a struct file and has the new flag set.
(snap_file): A new function invoked by hash_map that will perform
per-file operations: set up second expansion, intermediate, and also
.EXTRA_PREREQS. Manage circular dependencies by ignoring them.
(snap_deps): Defer per-file operations until the end. Look up the
global .EXTRA_PREREQS and pass it along to snap_file for each file.
* src/implicit.c (struct patdeps): Remember the extra prereqs flag.
(pattern_search): Transfer extra prereqs flag settings into the
matched pattern rule.
* src/rule.h (snap_implicit_rules): Rename count_implicit_rules to
snap_implicit_rules since we now do more than count.
* src/rule.c (snap_implicit_rules): As we walk through all the pattern
rules, add in any global .EXTRA_PREREQS to the dep list. Ensure we
take them into account for the max number of prereqs and name length.
* src/main.c (main): Add extra-prereqs to .FEATURES.
Call the renamed snap_implicit_rules.
* tests/scripts/variables/EXTRA_PREREQS: Add tests.
Paul Smith [Fri, 27 Dec 2019 06:27:09 +0000 (01:27 -0500)]
[SV 57022] Avoid posix_spawn which fails asynchronously
Avoid using posix_spawn implementations that fail asynchronously when
the spawned program can't be invoked: this means instead of getting
an error such as "No such file or directory" we get just "Exit 127".
Original implementation of the configure.ac macro provided by
Martin Dorey <martin.dorey@hds.com>
Original implementation of the regression tests provided by
Dmitry Goncharov <dgoncharov@users.sf.net>
* configure.ac: Test whether posix_spawn fails asynchronously. In a
cross-compilation environment, assume that it does not. If we detect
that it does, fall back to fork/exec.
* tests/scripts/features/exec: Add regression tests for different
shebang invocation methods.
Paul Smith [Thu, 26 Dec 2019 23:39:56 +0000 (18:39 -0500)]
[SV 56655] Allow pattern expansion to contain spaces
* src/dep.h: Add a new flag PARSEFS_ONEWORD
* src/read.c (parse_file_seq): If PARSEFS_ONEWORD is given, treat the
entire incoming string as a single pattern.
* src/implicit.c (pattern_search): Pass PARSEFS_ONEWORD when parsing
patterns for wildcards.
* tests/scripts/features/patternrules: Add a new test.
Jouke Witteveen [Sat, 26 Oct 2019 10:24:15 +0000 (12:24 +0200)]
[SV 54161] Fix second expansion of $* for paths
If the stem matches a path containing a directory not just a
filename, make sure the second expansion of $* in the
prerequisites matches $* in the recipe. This requires using
$(*F) when replacing % in the first expansion to preserve the
simple filename.
* src/implicit.c (pattern_search): If lastslash is set prepend
the directory onto the stem. Then use $(*F) when expanding %.
* tests/scripts/features/se_implicit: Add a test case
Jouke Witteveen [Sat, 26 Oct 2019 10:24:29 +0000 (12:24 +0200)]
* src/implicit.c (pattern_search): Set lastslash correctly
If filename contained multiple slashes lastslash is wrongly set to 0.
* configure.ac: Check for the GNU memrchr() extension function.
* src/misc.c (memrchr): Supply memrchr() if not available.
Paul Smith [Tue, 17 Dec 2019 17:29:44 +0000 (12:29 -0500)]
* src/job.c (load_too_high): Disable Linux /proc/loadavg for now
This new feature has a problem: if you provide a load which is
larger than the number of cores then it will always run every job.
Before we can enable it we need to at the least learn how to clamp
this value to the number of cores. To experiment with it, set
PROC_FD_INIT to -2 in job.c to re-enable the feature.
Paul Smith [Mon, 16 Dec 2019 21:18:58 +0000 (16:18 -0500)]
* tests/run_make_tests.pl (subst_make_string): Force use of /
On Windows the path to the helper tool will contain '\': this will
fail if recipes are run with a POSIX shell. Convert '\' to '/'
on Windows. While here, escape any spaces in the path as well.
Paul Smith [Sat, 12 Oct 2019 20:22:01 +0000 (16:22 -0400)]
Rename jhelp.pl to thelp.pl and make it a generic test helper.
* tests/thelp.pl: Rename from tests/jhelp.pl.
(op): Use names instead of options for the operations.
(op): Add new operations for sleep, mkdir, and rm.
(op): Enhance wait to time out
* tests/run_make_tests.pl: Add a new #HELPER# replacement
(subst_make_string): Use fully-qualified path to thelp.pl
* tests/scripts/features/parallelism: Update to use thelp.pl
and the new named operations. Use thelp.pl sleep instead of
system-specific sleep commands.
* tests/scripts/features/output-sync: Update to use thelp.pl
instead of complex shell scripts.
* Makefile.am: Distribute tests/thelp.pl instead of tests/jhelp.pl
Paul Smith [Sat, 5 Oct 2019 18:26:52 +0000 (14:26 -0400)]
tests: Convert %CONFIG_FLAGS to get_config()
* tests/config-flags.pm.W32: Create a predefined Windows file.
* Makefile.am (test_FILES): Add it to the distribution.
* build_w32.bat: Install tests/config-flags.pm if not existing.
* tests/run_make_tests.pl (get_config): Create new function.
* tests/scripts/features/archives: Call get_config() rather than
using %CONFIG_FLAGS directly.
* tests/scripts/features/load: Ditto.
* tests/scripts/features/loadapi: Ditto.
* tests/scripts/functions/wildcard: Ditto.
This reverts commit 6264deece3bb77798240a906ceed79097adbcf48.
Further investigation discovers that the real issue is that
GNU Emacs compile mode doesn't have a matching regex for GNU
make error messages generated when targets fail. I submitted
a patch to GNU Emacs adding a matcher for compile mode.
Paul Smith [Sat, 21 Sep 2019 19:11:21 +0000 (15:11 -0400)]
[SV 56918] Compute job slots properly on failing command
Ensure we properly reduce job_slots_used if a command fails because
it doesn't exist/can't be started.
* src/job.h (struct child): Add a field jobslot to be set when using
a job slot.
* src/job.c (start_waiting_job): Remember if we are using a job slot.
(reap_children): Reduce number of job slots used by jobslot.
Paul Smith [Sun, 15 Sep 2019 19:30:34 +0000 (15:30 -0400)]
Refresh the test suite framework implementation.
Go through both run_make_tests.pl and test_driver.pl and slightly
modernize the Perl and clean up indentation etc. Fix a number of
warnings in the test scripts detected by running with -w.
* tests/test_driver.pl: Move make error string detection out of the
base test driver.
(run_all_tests): Ensure that we always look for tests in the cwd.
* tests/run_make_tests.pl: Use File::Spec for path manipulations.
Correctly use setlocale() when detecting error strings.
Get configuration from the config-flags.pm file not config.status.
* tests/scripts/features/archives: Use new $cwddir variable.
* tests/scripts/features/reinvoke: Add missing semicolon.
* tests/scripts/features/vpath2: Avoid non-existent variable.
* tests/scripts/functions/foreach: Escape variables.
* tests/scripts/misc/bs-nl: Remove non-existing \v escape sequence.
* tests/scripts/misc/general4: Use handy create_file().
* tests/scripts/options/dash-C: Use Cwd/$cwddir.
* tests/scripts/options/dash-I: Use subst_make_string() and #PWD#.
* tests/scripts/options/symlinks: Use File::Spec.
* tests/scripts/targets/DEFAULT: Use create_file and run_make_test.
* tests/scripts/variables/CURDIR: Use run_make_test.
* tests/scripts/variables/automatic: Remove extraneous "\".
* tests/scripts/vms/library: Remove extra "my" and extraneous "\".
Eli Zaretskii [Tue, 10 Sep 2019 16:09:39 +0000 (19:09 +0300)]
Revert "[SV 56449] (Windows) Use slow path if '%' appears in the command"
This reverts commit 38e96eadea6e2cd42a0f414959bac327de9f468a.
Per discussion on Savannah, the modified behavior matches
the way % is handled in batch files, but not in cmd command
lines. And since Make mostly tries to emulate the command
line, the new behavior is unexpected and backward-incompatible.
Paul Smith [Sun, 8 Sep 2019 19:41:20 +0000 (15:41 -0400)]
Rework the creation of build.sh so it's not a template.
* build.sh: Rename from build.template. Get the list of objects
from the Makefile. Move configure-replaced variables ...
* build.cfg.in: to this new .in file.
* configure.ac: Remove special handling of build.sh.in and add
build.cfg as a generated file.
* Makefile.am (EXTRA_DIST): Remove build.sh.in and add build.sh
and build.cfg.in for build.sh.in.
* maintMakefile: Remove handling for build.template. Treat
build.sh as a source file, not a generated file.
* .gitignore: Ignore generated build.cfg file.
Paul Smith [Sat, 7 Sep 2019 02:24:46 +0000 (22:24 -0400)]
[SV 56834] Support local PATH search with posix_spawnp
When using exec we install the child's environment before invoking
execlp(), so commands are found on the child's PATH. posix_spawnp
searches on the parent's PATH, which we don't want.
Import gnulib's findprog-in module and use it to search the child's
PATH, then use posix_spawn() to run it.
Also, posix_spawn() does not fall back to trying sh on ENOEXEC, as
execlp() does, so implement that as well.
* bootstrap.conf: Add the findprog-in gnulib module
* src/job.c: Include findprog.h if we're using posix_spawn.
(start_job_command): Remove the handling of child->cmd_name,
(child_execute_job): and add it here. Look up the command to be
run in the child's path and invoke it if found. If it fails with
ENOEXEC then retry it as an argument to the default shell.
* tests/scripts/misc/general4: Test makefile PATH assignments.
* tests/scripts/features/targetvars: Ditto, for target variables.
Paul Smith [Sat, 7 Sep 2019 20:59:46 +0000 (16:59 -0400)]
* configure.ac: Check whether struct dirent has a d_type field
* src/dir.c (dir_contents_file_exists_p): Use the autoconf macro
HAVE_STRUCT_DIRENT_D_TYPE rather than relying on the GNU libc-
specific _DIRENT_HAVE_D_TYPE.
* lib/glob.c: Set HAVE_D_TYPE if HAVE_STRUCT_DIRENT_D_TYPE.
Paul Smith [Sat, 7 Sep 2019 22:27:50 +0000 (18:27 -0400)]
* src/config.h.W32.template [W32]: Add support for dirent.d_type
* src/w32/include/dirent.h: Add DT_* values for dirent.d_type
(struct dirent): Add d_type
* src/w32/compat/durent.c (readdir): Set dirent.d_type based on
Windows file attributes.
Paul Smith [Sat, 7 Sep 2019 00:56:19 +0000 (20:56 -0400)]
Align child_execute_job among different ports
Create a struct childbase which is the basics of struct child needed
to invoke child_execute_job(), and can be cast back and forth to a
struct child. Ensure all implementations of child_execute_job() take
the same arguments.
* src/job.h (CHILDBASE): Create a macro to hold the base parts.
(struct childbase): A structure containing CHILDBASE.
(struct child): Use CHILDBASE as the initial part of child.
[VMS]: Remove declaration of VMS version of child_execute_job.
* src/job.c (start_job_command): Use new child_execute_job() call.
(child_execute_job) [__EMX__]: Implement new calling signature.
(child_execute_job): Implement new calling signature.
* src/main.c (main) [__EMX__]: Use new child_execute_job() call.
* src/function.c (func_shell_base): Use new child_execute_job() call.
* src/vmsjobs.c (vmsHandleChildTerm): Accept struct childbase.
* src/vmsjobs.c (child_execute_job): Implement new calling signature.
Modify the return value to be pid_t, not a boolean, and return the
PID rather than setting it in the child. This is OK because our only
caller immediately reset PID to -1 anyway if we return 0.
Paul Smith [Tue, 3 Sep 2019 20:17:50 +0000 (16:17 -0400)]
Show useful errors when posix_spawn() doesn't do so
The posix_spawn() function may not detect that the command to run is
invalid when it's invoked. Instead, it will run then exit with
error code 127. If that happens do our best to present the user
with a useful error message.
* src/job.h (struct child): Add cmd_name to hold the command we ran.
* src/job.c (start_job_command): On success, remember the cmd_name.
(reap_children): On exit 127, stat cmd_name and show a useful error.
(free_child): Free cmd_name.
projects / thirdparty / make.git / log
