Paul Smith [Sun, 8 Jan 2023 21:40:55 +0000 (16:40 -0500)]
[SV 61218] Ensure MAKEFLAGS is expanded even with -e
If -e was given we weren't expanding MAKEFLAGS before passing it
through the environment to jobs: we don't expand variables we
receive from the environment and when -e is given we set the
origin of MAKEFLAGS to "environment override". Check for MAKEFLAGS
specifically, which seems like a hack but I don't have a better
idea offhand.
* src/main.c (main): Drive-by: use o_default for MAKEOVERRIDES.
* src/variable.c (target_environment): Always expand MAKEFLAGS
regardless of the origin type.
* tests/scripts/options/dash-e: Create a test.
Paul Smith [Sun, 8 Jan 2023 02:50:59 +0000 (21:50 -0500)]
[SV 62654] Support GNU Make on z/OS
Original patches provided by Igor Todorovski <itodorov@ca.ibm.com>
Reworked by Paul Smith <psmith@gnu.org>.
Thanks to IBM for providing a test system.
* NEWS: Announce support.
* AUTHORS: Ditto.
* README.zOS: Provide details on building GNU Make on z/OS.
* build.sh (get_mk_var): z/OS sh has a strange bug which causes it to
generate extra lines of output: rework the function to print output
as we compute it instead of collecting it into a variable, which
works around this bug.
* src/makeint.h: Declare MK_OS_ZOS if we're building for z/OS.
* src/arscan.c: Don't include <ar.h> on z/OS.
* src/job.c: We can't change environ in ASCII mode on z/OS.
* src/main.c: Ditto. Also we can't use pselect() on z/OS.
* src/posixos.c: pselect() seems to hang on z/OS: don't use it.
* tests/run_make_tests.pl: Handle different exit codes on z/OS.
* tests/test_driver.pl: Preserve some special z/OS env.vars.
Add special checks to output comparisons when on z/OS.
* tests/scripts/features/archives: Don't validate names. Don't
try to compile empty files as IBM compilers complain.
* tests/scripts/features/shell_assignment: Fix octal value of #.
* tests/scripts/features/temp_stdin: Don't print "term".
* tests/scripts/functions/shell: Handle shell exit codes.
* tests/scripts/targets/ONESHELL: Ditto.
* tests/scripts/targets/POSIX: sh -x prints differently.
* tests/scripts/variables/SHELL: Ditto.
Paul Smith [Sun, 8 Jan 2023 14:38:59 +0000 (09:38 -0500)]
Create src/mkcustom.h which is included by config.h
Put declarations for missing functions which we create in src/misc.c
into a file which is included by config.h via AH_BOTTOM(). This
ensures those prototypes are available, even in files added to
lib/... from gnulib.
* src/mkcustom.h: Add a new file with function declarations.
* configure.ac: Include src/mkcustom.h in config.h with AH_BOTTOM()
* Makefile.am: Add the header to the SRCS list.
* src/makeint.h: Remove content that we added to src/mkcustom.h.
* src/config.ami: Add #include "mkcustom.h" to specialized config.h.
* src/config.h-vms: Ditto.
* src/config.h.W32: Ditto.
* src/configh.dos: Ditto.
Paul Smith [Sun, 8 Jan 2023 02:59:16 +0000 (21:59 -0500)]
Include <strings.h> globally
Various code uses str{,n}casecmp() so include <strings.h>, if it
exists, everywhere.
* configure.ac: Check for <strings.h> explicitly.
* src/makeint.h: Include it if HAVE_STRINGS_H, for str{,n}casecmp().
* src/job.c: Remove include of <strings.h>.
* src/main.c: Ditto.
Paul Smith [Tue, 3 Jan 2023 07:14:24 +0000 (02:14 -0500)]
[SV 63609] Avoid buffer overrun in --warn-undefined-variables
Reported by Dmitry Goncharov <dgoncharov@users.sf.net>
* src/variable.c (struct defined_vars): Create a struct that holds the
name and length of each variable name.
(warn_undefined): Check the lengths before comparing the contents.
* tests/scripts/options/warn-undefined-variables: Add a test.
Paul Smith [Tue, 3 Jan 2023 06:57:35 +0000 (01:57 -0500)]
[SV 14927] Allow parallel builds for archives
Compare the timestamp of the object file (if it exists) with the
archived object and if the object file is newer, ensure it's updated
in the archive.
* NEWS: Announce the new capability.
* doc/make.texi (Dangers When Using Archives): Explain how to enable
parallel builds with archives.
* src/remake.c (f_mtime): For archive element files check the mod
time of the object file (if it exists) against the archive object
(if it exists).
* tests/scripts/features/archives: Add tests for this capability.
If a parent target has an exported variable that is private, don't
export it in child targets.
* NEWS: Mention this change.
* src/variable.c (target_environment): Ignore private inherited
variables.
* tests/thelp.pl: Add a new "env" operation to show env.var. values.
* tests/scripts/variables/private: Verify this new behavior.
Paul Smith [Mon, 2 Jan 2023 05:52:32 +0000 (00:52 -0500)]
Update ancient glob/fnmatch implementations
I looked again at trying to use the latest gnulib implemenentations
of GNU glob and fnmatch, and the effort required to extract them
from gnulib and make them portable to systems which don't support
configure is simply far too daunting for me. However it's clear
that the previous implementations are growing too long on the tooth
to continue to be used without some maintenance, so perform some
upkeep on them.
- Remove support for pre-ANSI function definitions.
- Remove the obsolete "register" keyword.
- Assume standard ISO C90/C99 header file support.
- Assume standard ISO C "void" and "const" support.
- Avoid symbols prefixed with "__" as they're reserved.
* maintMakefile: Add a rule to verify lib has the latest content.
* src/dir.c: Use void* not __ptr_t which was removed.
* gl/lib/glob.c: See above.
* gl/lib/fnmatch.in.h: See above.
* gl/lib/glob.in.h: See above.
* gl/lib/fnmatch.c: See above. Remove __strchrnul(): it is not
checked anywhere and is only used in one place anyway.
Paul Smith [Sun, 1 Jan 2023 14:44:59 +0000 (09:44 -0500)]
Don't free uninitialized hash_table
* src/dir.c (clear_directory_contents): We use ht_vec to mark when
we have initialized the hash_table: don't free if it's NULL.
(everywhere): Use NULL instead of 0 when working with pointers.
Paul Smith [Sun, 1 Jan 2023 21:08:35 +0000 (16:08 -0500)]
Update to the latest gnulib stable branch
* bootstrap.conf: Request the latest gnulib stable branch.
* README.git: Recommend the argument form of autogen/autopull.
* bootstrap: Import the latest version of bootstrap.
* bootstrap-funclib.sh: Ditto.
* autogen.sh: Ditto.
* autopull.sh: Ditto.
Paul Smith [Sat, 31 Dec 2022 15:31:24 +0000 (10:31 -0500)]
Avoid using false(1) in regression tests
The POSIX standard only requires false(1) to return a "non-zero" exit
code; almost all systems return 1 but some (Solaris!!!) return 255 or
possibly even other values. Use our helper "fail" instead.
* tests/thelp.pl: Have the "fail" command obey -q.
* tests/scripts/features/parallelism: Helper -q no longer prints fail.
* tests/scripts/targets/POSIX: Replace false with #HELPER# -q fail 1.
* tests/scripts/variables/MAKEFLAGS: Ditto.
* tests/scripts/variables/SHELL: Ditto.
Paul Smith [Mon, 26 Dec 2022 18:10:26 +0000 (13:10 -0500)]
Make bootstrap.bat more portable
Using backslashes in a sed command line is tricky as different programs
use them differently as escape sequences. Eli Zaretskii points out
that Windows "echo" doesn't do any processing, so rework all our sed
invocations to use script files created by echo.
* bootstrap.bat: Use echo to create sed script files instead of -e.
* Basic.mk.template: Fix typo in the comments.
* .gitignore: Ignore any .sed scripts.
Paul Smith [Sat, 24 Dec 2022 15:30:13 +0000 (10:30 -0500)]
[SV 63439, SV 63452] Don't warn on undefined internal variables
Don't generate undefined variable warnings for variables that are
internal / special to make and where the empty string is valid.
Rather than defining them to empty, which could introduce unwanted
behavior, keep a list of variable names which we should never warn
about.
* src/variable.h (warn_undefined): Convert the macro to a function.
* src/variable.c (defined_vars): Always "defined" variable names.
(warn_undefined): Implement as a function and check against the
defined_vars before generating a warning.
* src/read.c (read_all_makefiles): No need to reset warning flag.
* src/vpath.c (build_vpath_lists): Ditto.
* tests/scripts/options/warn-undefined-variables: Expand all the
pre-defined variables to ensure warnings are not generated.
Paul Smith [Sat, 24 Dec 2022 14:26:24 +0000 (09:26 -0500)]
[SV 63552] Change directories before constructing include paths
* src/makeint.h (reset_makeflags): New function to handle changing
MAKEFLAGS from within makefiles. Remove decode_env_switches().
* src/variable.c (set_special_var): Call reset_makeflags() instead
of various internal methods.
* src/main.c (decode_env_switches): Only internal now so make static.
(decode_switches): Don't invoke construct_include_path() yet.
(reset_makeflags): Decode env switches and construct include paths.
(main): Construct include paths after we process -C options.
* tests/scripts/options/dash-C: Rewrite to use new test constructs.
Add a test using both -C and -I together.
Add a test for multiple -C options.
Paul Smith [Mon, 19 Dec 2022 05:24:42 +0000 (00:24 -0500)]
tests [WINDOWS32]: Support Strawberry Perl on Windows
Strawberry Perl has some different behaviors from ActiveState Perl
which impact the test suite:
- Avoid Perl's chomp() as it may not remove CRs; chomp() may remove
only the final NL but not the CR in a CRNL line ending.
- Strawberry Perl doesn't support ActiveState's system(1, ...) form.
- Strawberry Perl (or msys?) does something weird with "/tmp" when
provided to exec(), replacing it with the user's %TEMP%.
- Strawberry Perl uses msys paths like /c/foo instead of C:\foo.
* tests/test_driver.pl (get_osname): Strawberry Perl uses 'msys' as
its $^O so if we see that use a port of 'W32'.
(_run_with_timeout): Strawberry Perl doesn't support the special
system(1, ...) form of system() so use POSIX standard fork/exec.
(compare_answer): Paths generated by Strawberry Perl use msys path
format (e.g., /c/foo instead of C:\foo); check for those differences
and compare RE against both the unmodified and modified log.
* tests/run_make_tests.pl (set_defaults): Switch from chomp to s///
to remove CRNL and NL line endings.
* tests/scripts/features/errors: Executing directories on Strawberry
will give an error; translate it to Windows error output format.
* tests/scripts/features/output-sync: Ditto.
* tests/scripts/features/temp_stdin: Ditto.
* tests/scripts/functions/realpath: Ditto.
* tests/scripts/options/dash-I: Ditto.
* tests/scripts/variables/INCLUDE_DIRS: Ditto.
* tests/scripts/misc/close_stdout: /dev/full is reported as existing
on Strawberry Perl, but it doesn't do anything. Skip the test.
* tests/scripts/variables/MAKEFLAGS: When an argument containing
/tmp is passed to a program via exec(), something replaces it with
the expansion of the %TEMP% variable. Instead of using /tmp create
a local directory to use.
Paul Smith [Mon, 19 Dec 2022 05:20:06 +0000 (00:20 -0500)]
[WINDOWS32] Remove CRNL from FormatMessage() result string
Sometimes the error message is expected to contain a newline, other
times it is not. Also the result of FormatMessage() sometimes ends
in CRNL already: when printed via fprintf() the final newline is
converted to CRNL, giving an output of CRCRNL which causes tests
to fail to match. Remove any CR/NL chars from the result string.
* src/job.c (reap_children): Add \n to the error message fprintf.
(exec_command): Ditto.
* src/w32/subproc/w32err.c (map_windows32_error_to_string): Remove
any trailing CR or NL from the string before returning.
Dmitry Goncharov [Sun, 18 Dec 2022 15:29:17 +0000 (10:29 -0500)]
[SV 63537] Document and test flippable switches
* doc/make.texi (Options/Recursion): Clarify that MAKEFLAGS values
from the environment have precedence over those set in the makefile.
* tests/scripts/variables/MAKEFLAGS: Check boolean switches -k/-S,
-w/--no-print-directory and -s/--no-silent as follows:
1. A switch can be enabled or disabled on the command line.
2. A switch can be enabled or disabled in env.
3. A switch can be enabled or disabled in makefile.
4. Command line beats env and makefile.
5. Env beats makefile.
6. MAKEFLAGS contains each specified switch at parse and build time.
7. If switches are specified in multiple origins, MAKEFLAGS contains
the winning switch at parse and build time.
8. MAKEFLAGS does not contain the losing switch.
Also test that --debug settings from different origins are combined
together into one option.
Dmitry Goncharov [Sun, 18 Dec 2022 19:43:41 +0000 (14:43 -0500)]
[SV 63537] Pass enabled-by-default switches to submake
Certain switches, such as -S or --no-silent, turn on behavior that is
enabled by default. When a switch is specified via the command line,
makefile, or env, ensure the switch is added to MAKEFLAGS.
* src/main.c (struct command_switch): Add bit "specified".
(switches): Initialize command_switch->specified.
(decode_switches): Set command_switch->specified.
(define_makeflags): Check command_switch->specified.
Dmitry Goncharov [Sun, 18 Dec 2022 15:01:30 +0000 (10:01 -0500)]
[SV 63537] Remember the origin of command line options
Certain options can be flipped on and off: -k/-S, -s/--no-silent, and
-w/--no-print-directory. Ensure they behave as follows:
1. A switch can be enabled or disabled on the command line.
2. A switch can be enabled or disabled in env.
3. A switch can be enabled or disabled in makefile.
4. Command line beats env and makefile.
5. Env beats makefile.
* src/main.c: Add variables to hold the origin of relevant options.
(struct command_switch): Add origin field.
(switches): Set a pointer to hold the origin of relevant options.
(decode_switches): For any switch that can be specified in makefile or
env, honor the switch only if cs->origin is not set or the specified
origin beats cs->origin. Set cs->origin when relevant.
Dmitry Goncharov [Sun, 18 Dec 2022 14:49:34 +0000 (09:49 -0500)]
[SV 63537] Fix setting -w in makefiles
* src/makeint.h: Replace print_directory flag with should_print_dir().
* src/main.c (main): Remove print_directory flag and related code.
(should_print_dir): Create.
* src/output.c (output_dump): Use should_print_dir().
(output_start): Ditto.
Dmitry Goncharov [Sat, 17 Dec 2022 19:12:41 +0000 (14:12 -0500)]
[SV 63484] Force included makefiles to be explicit
Ensure included makefiles are not treated as intermediate, even if
they are created by an implicit rule.
Reported by Patrick Oppenlander <patrick.oppenlander@gmail.com>.
* src/read.c (eval_makefile): Mark makefiles as explicit.
* tests/scripts/features/include: Add a test.
Paul Smith [Sat, 17 Dec 2022 16:01:31 +0000 (11:01 -0500)]
[SV 63516] [DOS] Support include files with drivespecs
* src/makeint.h (HAVE_DRIVESPEC): Create a macro to check.
* src/main.c (.FEATURES): Add "dospaths" as a feature.
* src/read.c (eval_makefile) [DOS]: If the included makefile name
starts with a drivespec, don't search the include directories.
* doc/make.texi (Include): Document this behavior.
* tests/scripts/features/include: Add a test.
Dmitry Goncharov [Sun, 27 Nov 2022 20:40:28 +0000 (15:40 -0500)]
[SV 63417] Ensure global .NOTINTERMEDIATE disables all intermediates
Fix .NOTINTERMEDIATE without prerequisites to disable intermediate
status for all targets.
* src/makeint.h: Declare extern no_intermediates.
* src/main.c: Add global definition of no_intermediates.
* src/file.c: Remove static no_intermediates to use global variable.
(remove_intermediates): Check no_intermediates.
* src/implicit.c (pattern_search): For a file found by implicit search
set file->notintermediate if no_intermediates is set.
* src/remake.c (update_file_1): Don't set file->secondary for a
pre-existing file if no_intermediates is set. The check for
no_intermediates here is redundant, but won't hurt: keep it in case
things change so that it matters.
* tests/scripts/targets/NOTINTERMEDIATE: Fix a test.
Dmitry Goncharov [Sun, 27 Nov 2022 19:09:17 +0000 (14:09 -0500)]
[SV 63347] Always add command line variable assignments to MAKEFLAGS
This commit introduces two visible changes:
1. Keep command line variable assignments in MAKEFLAGS at all times,
even while parsing makefiles.
2. Define makeflags immediately when a makefile modifies MAKEFLAGS.
The new MAKEFLAGS and MAKEOVERRIDES initialization procedure:
1. decode_switches (argc, argv, o_command) is called to parse command
line variable assignments.
2. Command line variable assignments go through quote_for_env.
Initialize -*-command-variables-*- to the quoted values.
3. MAKEOVERRIDES is initialized to refer to -*-command-variables-*-
with origin o_env to keep the definitions in the database intact.
4. define_makeflags() is called which adds MAKEOVERRIDES to MAKEFLAGS.
5. Makefiles are parsed. If a makefile modifies MAKEFLAGS, the new
value of MAKEFLAGS is defined right away.
6. Env switches are decoded again as o_env. The definitions set by
decode_switches at step 1 stay intact, as o_command beats o_env.
We must preserve the original intact definitions in order to detect
failure cases; for example:
$ cat makefile
all:; $(hello)
$ make hello='$(world'
makefile:1: *** unterminated variable reference. Stop.
* src/makeint.h: Declare enum variable_origin, struct variable and
define_makeflags(). Add parameter origin to decode_env_switches().
* src/main.c (define_makeflags): Remove "all". If a variable is
assigned on the command line then append MAKEOVERRIDES to MAKEFLAGS.
(decode_env_switches): Replace parameter env with origin.
(decode_switches): Replace parameter env with origin.
Treat origin == o_command as env == 0.
(handle_non_switch_argument): Replace parameter env with origin.
Treat origin == o_command as env == 0.
(main): Call decode_switches() with origin==o_command before parsing
makefiles. Call decode_switches() with origin==o_env after parsing
makefiles.
* src/variable.c (set_special_var): Define makeflags at parse time,
each time a makefile modifies MAKEFLAGS.
(do_variable_definition): Strip command line variable assignments from
MAKEFLAGS before appending extra flags. set_special_var() adds them
back.
* tests/scripts/variables/MAKEFLAGS: Add tests.
Paul Smith [Tue, 15 Nov 2022 15:50:34 +0000 (10:50 -0500)]
Add specific hints for errors due to invalid conditionals
* src/read.c (eval): If "missing separator" appears to be due to
missing space after ifeq/ifneq, give a hint about the error.
* tests/scripts/misc/failure: Check for these types of failures.
* tests/scripts/variables/special: Move error checking unrelated
to special variables, to misc/failure.
Dmitry Goncharov [Sun, 13 Nov 2022 21:20:33 +0000 (16:20 -0500)]
[SV 63333] Be more lenient when failing to create temporary files
If make cannot create a temporary lock file for output sync, continue
without output sync enabled rather than dying.
However, if make cannot store a makefile from stdin to a temporary
file that is still a fatal error.
* misc.c (get_tmppath): Keep running on failure to generate a
temporary file name.
(get_tmpfd): Keep running on failure to get a temporary file.
(get_tmpfile): Keep running on failure to open a temporary file.
Ensure memory is freed if we return an error.
* posixos.c (os_anontmp): Keep running on failure to open an
anonymous temporary file.
* output.c (setup_tmpfile): Print an error on failure to create an
output sync lock file.
* main.c (main): Die on failure to store makefile from stdin to a
temporary file.
* tests/scripts/features/output-sync: Add tests.
* tests/scripts/features/temp_stdin: Ditto.
Paul Smith [Mon, 7 Nov 2022 20:55:21 +0000 (15:55 -0500)]
Keep going if we can't connect to the jobserver
* src/posixos.c (jobserver_parse_auth): Don't invoke fatal() if we
can't connect to an existing jobserver: just keep going without it.
* src/w32/w32os.c (jobserver_parse_auth): Ditto.
* tests/scripts/features/jobserver: Add a test for invalid FIFO
auth files.
Paul Smith [Mon, 7 Nov 2022 18:08:11 +0000 (13:08 -0500)]
* src/dir.c (dir_contents_file_exists_p): Show dir name in error.
If we fail to read a directory show the directory name in the error
message. Pass struct directory instead of directory_contents to
allow that.
(dir_file_exists_p): Change dir_contents_file_exists_p caller.
(open_dirstrem): Ditto.
Paul Smith [Sun, 6 Nov 2022 19:14:28 +0000 (14:14 -0500)]
[SV 63315] tests: Simplify TERM signaling
Tests that try to kill the make process were not behaving as expected
on OpenBSD: the signal was sent from make to its children but the
sleep didn't die. Something odd about the way the shell treats TERM.
To reduce platform dependencies add "term" to the helper tool and run
that instead of kill / sleep.
* tests/thelp.pl: Add a new operation "term" that takes a PID.
* tests/scripts/features/output-sync: Use it.
* tests/scripts/features/temp_stdin: Ditto.
Paul Smith [Sat, 5 Nov 2022 16:56:52 +0000 (12:56 -0400)]
[SV 63315] Allocate function names when defining functions
* src/function.c (define_new_function): Don't keep a pointer to the
user-provided name of a user-defined function: if the .so is unloaded
it will point to garbage. Add the name to the strcache instead.
Paul Smith [Sat, 5 Nov 2022 16:46:29 +0000 (12:46 -0400)]
tests: Don't convert \ to / when checking regex's
When tests compare the output they will try converting backslashes
to slashes to see if that works. When we compare using regex's,
we can't do that because backslashes can escape special characters.
* tests/test_driver.pl (compare_output): Clean up this function.
(compare_answer_vms) [VMS]: Comparing answers on VMS is complex;
move all of it into its own function returning 0/1.
(compare_answer): A new function to compare answers: return 0/1.
Remember the CRLF->LF conversion forever; only check \ -> / when
we compare strings, not regex's.
Eli Zaretskii [Sat, 5 Nov 2022 14:46:58 +0000 (16:46 +0200)]
Fix build with Tiny C
* src/config.h.W32 (strtoll, strtoull) [__TINYC__]: Redirect to
_strtoi64 and _strtoui64, respectively. Reported by Christian
Jullien <eligis@orange.fr>.
Paul Smith [Mon, 31 Oct 2022 05:48:33 +0000 (01:48 -0400)]
Fix issues found by ASAN and Coverity
* tests/test_driver.pl: Preserve the LSAN_OPTIONS variable.
* tests/scripts/targets/ONESHELL: Don't set a local variable.
* tests/scripts/functions/let: Test empty let variable.
* src/posixos.c (osync_parse_mutex): Free existing osync_tmpfile.
* src/misc.c (get_tmpfd): Set umask() before invoking mkstemp().
* src/ar.c (ar_parse_name): Check invalid name (shouldn't happen).
* src/function.c (define_new_function): Free previous function entry
when replacing it with a new one.
* src/job.c (child_execute_job): Initialize pid for safety.
(construct_command_argv_internal): In oneshell mode ensure that the
returned argv has the right format (0th element is a pointer to the
entire buffer).
Paul Smith [Fri, 28 Oct 2022 17:03:18 +0000 (13:03 -0400)]
Avoid C99 constructs
Although gnulib requires C99, most of the code does compile with a
C90 compiler (perhaps with a lot of warnings). Reinstate our C90
configuration test, and clean up a few C99 things that crept in.
* src/job.c (construct_command_argv_internal): Don't use loop-local
variables or C++ comments.
* src/read.c (eval_makefile): Don't use loop-local variables.
Paul Smith [Fri, 28 Oct 2022 19:21:55 +0000 (15:21 -0400)]
tests: Avoid the use of File::Temp->newdir()
This was added in Perl 5.8 but some systems still only provide older
versions such as Perl 5.6. We don't really need it anyway.
Paul Eggert <eggert@cs.ucla.edu> reported this issue.
* tests/README: Update this to be a bit more modern.
* tests/test_driver.pl: Delete the $TEMPDIR variable.
* tests/scripts/features/temp_stdin: Use $temppath not $TEMPDIR.
Paul Smith [Fri, 28 Oct 2022 19:51:49 +0000 (15:51 -0400)]
[SV 62174] Force locale to be "C" before retrieving error messages
We attempt to do this with POSIX::setlocale() but apparently on some
systems (AIX) this isn't sufficient. So, in addition force the LC
environment variables to use "C".
Reported by Dmitry Goncharov <dgoncharov@users.sf.net>.
* tests/run_make_tests.pl: Move the global setup into set_default().
Force the %ENV locale variables to use the ones we'll use when running
make, then reset them back again after we find error messages.
Paul Smith [Tue, 25 Oct 2022 18:46:58 +0000 (14:46 -0400)]
* tests/scripts/features/exec: Don't test with C shells
Using C shells (csh, tcsh) with make is known to be problematic due
to incorrect ways it handles open file descriptors, at least. If
the user's shell is *csh then don't try it during exec tests.
Paul Smith [Tue, 25 Oct 2022 18:44:26 +0000 (14:44 -0400)]
Increase the test framework timeout from 5s to 60s
It seems that some of the test environments hit the 5s timeout on
some tests. Since it doesn't really matter, as long as we don't
hang forever, increase the timeout to 60s.
* tests/test_driver.pl: Increase $test_timout to 60. We don't need
to handle VMS timeouts specially anymore.
* tests/scripts/features/parallelism: We don't need to override the
default timeout anymore.
* tests/scripts/features/patternrules: Remove confusing comment.
Paul Smith [Mon, 24 Oct 2022 04:48:47 +0000 (00:48 -0400)]
[SV 63260] Don't recurse forever if setup_tmpfile() fails
If we fail during setup_tmpfile() we'll try to write an error, which
will invoke setup_tmpfile() again, etc. Avoid infinite recursion.
Original patch by Dmitry Goncharov <dgoncharov@users.sf.net>
* src/output.c (setup_tmpfile): Remember we're in this function and
return immediately if we enter it during recursion.
(message): Remember the starting location and use that instead of
fmtbuf.buffer.
(error): Ditto.
(fatal): Ditto.
Paul Smith [Sun, 23 Oct 2022 22:36:44 +0000 (18:36 -0400)]
Provide a maintainer-only debug method
Generating debug logs to stdout or stderr makes it impossible to
run tests etc. so create a dumb DBG facility to log to a temp file.
This exists only in maintainer mode and the DBG macro gives a
compile error if it's used in non-maintainer mode.
* src/makeint.h (DBG): Call dbg() in maintainer mode, else error.
(dbg): Define it in maintainer mode.
* src/misc.c (dbg): Open a log file for append, write, then close.
Paul Smith [Sat, 22 Oct 2022 23:42:07 +0000 (19:42 -0400)]
Set PATH_MAX on systems without a default value
Some systems (HURD) use fully-dynamic pathnames, with no limit.
We can't support this without significant effort so for now set
PATH_MAX to a large value.
* src/makeint.h: Set PATH_MAX to 4096 if not set and MAXPATHLEN
is also not set. Remove MAXPATHLEN setting: we won't use it.
* src/misc.c (get_path_max): If we can't get the path max via
pathconf() use the default PATH_MAX.
* src/dir.c (find_directory) [W32]: Use MAX_PATH not MAXPATHLEN.
(local_stat) [W32]: Ditto.
* src/job.c (create_batch_file) [W32]: Ditto.
* src/remake.c (name_mtime) [W32]: Ditto.
* src/w32/w32os.c (os_anontmp) [W32]: Ditto.
Paul Smith [Sat, 22 Oct 2022 23:09:44 +0000 (19:09 -0400)]
[SV 63098] Enhance detection of missing peer also-make targets
The previous attempt to detect missing peer targets for implicit
rules had some holes. Move the detection to notice_finished_file().
* src/remake.c (check_also_make): If we don't have the current mtime
for the file, obtain it.
(update_goal_chain): Don't call check_also_make() here.
(check_dep): Ditto.
(notice_finished_file): If we finished running an implicit rule that
has also_make targets, invoke check_also_make().
Paul Smith [Sat, 22 Oct 2022 21:23:17 +0000 (17:23 -0400)]
Collect failure details when the regression tests fail
* README.in: Add a section on running regression tests.
* Makefile.am (check-regression): Capture the test run output, and
on failure collect configure and test results into a tar file.
Paul Smith [Sat, 22 Oct 2022 19:35:16 +0000 (15:35 -0400)]
Enhance tests to work on different systems
The GNU platform testers reported a number of test errors on
different systems; try to address them.
* tests/thelp.pl: A number of tests timed out with a 4-second
timeout. Increase the default timeout to 10 seconds.
* tests/run_make_tests.pl: Executing directories on cygwin behaves
differently in Perl than make so skip these tests there.
* tests/scripts/options/symlinks: Check for the symlink feature
in make, rather than whether the system supports them.
* tests/scripts/features/implicit_search: On some systems "false"
exits with a different exit code. Use the helper instead.
* tests/scripts/features/loadapi: Ditto.
* tests/scripts/features/output-sync: Sleep before make -f bar in
the first test as well as the second one.
* tests/scripts/features/exec: Skip on cygwin, which seems to
be "UNIX" but where scripts don't run normally.
* tests/scripts/misc/fopen-fail: Skip on cygwin, where make
eventually exits with exit code 0 and no error messages.
Dmitry Goncharov [Fri, 21 Oct 2022 23:35:09 +0000 (19:35 -0400)]
[SV 63243] tests: Avoid SIGTERM racing with make error messages
Original patch from Frank Heckenbach <f.heckenbach@fh-soft.de>.
* tests/scripts/features/output-sync: Introduce a sleep to let make
write its error message. Some systems use different names for
SIGTERM so match with a regex.
* tests/scripts/features/temp_stdin: Ditto.
Dmitry Goncharov [Fri, 21 Oct 2022 22:54:39 +0000 (18:54 -0400)]
[SV 63248] Ignore SIGPIPE
Don't terminate when make's output is redirected to a pipe and the
reader exits early; e.g.:
$ echo 'all:; sleep 2' | make -f- -j2 -O |:
This lets us unlink temporary files, and tell the user that make was
not able to write its output.
Reported by Frank Heckenbach <f.heckenbach@fh-soft.de>.
Paul Smith [Tue, 18 Oct 2022 03:44:31 +0000 (23:44 -0400)]
Rework temp file handling to avoid GNU libc warnings
Original patch provided by Paul Eggert <eggert@cs.ucla.edu>.
GNU libc will generate a link-time warning if we use mktemp() even
though we are using it safely (we only use it with mkfifo()).
Avoid this and clean up some handling.
First, check all calls related to temporary files and exit with a
fatal error and a useful message if we can't obtain them. In some
situations it might be possible to continue with reduced capability
but it's not worth the effort.
On POSIX systems we can create anonymous temp files using O_TMPFILE
if it's supported, else if we're using the default location and we
have dup(2), we can use standard tmpfile() and get an FD from it.
If we need a named temp file and FILE* and we have mkstemp() we can
use that, else if we have fdopen() we can get a temp FD and open it.
If none of those are available all we can do is generate a temp name
then open it with fopen() which is not secure.
* src/makeint.h (get_tmpdir): Declare it for use elsewhere.
* src/misc.c (get_tmpdir): Make it public not static.
(get_tmptemplate): Simplify the implementation.
(get_tmppath): Only define this if we have to have it to avoid
warnings from GNU libc.
(get_tmpfd): Generate fatal errors on error.
(get_tmpfile): Ditto. Open files in "wb+" mode to match tmpfile().
Require a filename pointer (all callers want it).
* src/os.h (os_anontmp): Implement for posixos.c as well.
* src/posix.c (jobserver_setup): Don't use mktemp to avoid GNU libc
errors. Instead construct the FIFO name based on the PID.
(osync_setup): get_tmpfd() can't fail so don't check it.
(os_anontmp): If the system supports O_TMPFILE use it. If not, and
we want to create the temporary file in the default directory, we
can use tmpfile() then use dup() to copy the file descriptor.
* src/main.c (main): get_tmpfile() can't fail.
* src/vmsjobs.c (child_execute_job): get_tmpfile() can't fail.
Paul Smith [Tue, 18 Oct 2022 03:36:36 +0000 (23:36 -0400)]
Add ISDIRSEP() helper macro and use it
Create a ISDIRSEP() macro to check for directory separator chars
using the stopchar_map, and replace inline checks and explicit
STOP_SET calls with this macro.
Paul Smith [Mon, 17 Oct 2022 21:56:18 +0000 (17:56 -0400)]
Add preprocessor macros for different platforms
* configure.ac: MK_CONFIGURE shows config.h was generated by configure.
* src/config.ami: Define MK_AMIGAOS.
* src/config.h-vms: Define MK_VMS.
* src/configh.dos: Define MK_DJGPP.
* src/config.h.W32: Define MK_W32 and WINDOWS32.
* src/build_w32.bat: Let WINDOWS32 be defined by config.h. Remove
unused setting of WIN32.
* src/job.c: Clean up use of WIN32.
* src/main.c: Ditto.
* tests/scripts/features/default_names: Ditto.
Paul Smith [Mon, 10 Oct 2022 00:17:18 +0000 (17:17 -0700)]
Check for recipe line count overflow before it overflows
awk 'BEGIN {
print "x:"
for (i = 0; i < 65536; i++)
printf "\techo %d\n", i}
' | make -f -
Outputs only "make: 'x' is up to date." Larger values run only the
lines above 65536. Reported by Paul Eggert <eggert@cs.ucla.edu>.
* src/commands.c (chop_commands): Check the line count before it has
a chance to overflow. Use size_t for max count so it can't overflow.
Remove stray 'd' in diagnostic.
Paul Smith [Sun, 16 Oct 2022 19:13:41 +0000 (15:13 -0400)]
Remove template files to simplify distribution creation
The README templates were not useful since the replacement step
didn't have anything to replace: rename them.
Rather than creating template files for the config variants, create
mkconfig.h.in containg PACKAGE_* variables to be replaced, and have
config variant header files include it. Note on POSIX we don't use
this, and continue to generate a single config.h.in file.
Use config.status to convert the README.in and mkconfig.h.in files
during distribution creation.
Modify all users of VERSION to use PACKAGE_VERSION instead.
* configure.ac: Use GNU Make not GNU make as the package name.
* README.in: Use GNU Make not GNU make.
* README.git: Remove references to README.W32.template.
* .gitignore: Update for new behavior.
* Basic.mk.template: Remove unused posix_SOURCES and VERSION, and
references to unshipped mk/Posix.mk
* Makefile.am: Add src/mkconfig.h as an extra dist file.
* bootstrap.bat: Rewrite mkconfig.h.in to mkconfig.h
* maintMakefile: Remove obsolete template files; add mkconfig.h.in.
* prepare_vms.com: Rewrite mkconfig.h.in to mkconfig.h
* mk/VMS.mk: Fix incorrect header file prerequisite.
* src/mkconfig.h.in: New file containing PACKAGE variables.
* src/config.ami: Include mkconfig.h.
* src/config.h.W32: Ditto.
* src/configh.dos: Ditto.
* src/config.h-vms: Ditto.
* src/version.c: Use PACKAGE_VERSION not VERSION.
Juan M. Guerrero [Sat, 15 Oct 2022 23:14:49 +0000 (19:14 -0400)]
Support building with DJGPP 2.05
* builddos.bat: Use env var settings for paths. Fix a typo in
expand.o. Add the missing load.o compilation and link. Enable
Guile support. Copy Basic.mk from the correct location.
* src/configh.dos.templage: DJGPP supports strtoll() and ssize_t
given new enough versions. Set preprocessor variables to 1 not 0.
Paul Smith [Sat, 15 Oct 2022 22:11:21 +0000 (18:11 -0400)]
[SV 63215] Remember the random seed for shuffle mode
Basic fix provided by James Hilliard <james.hilliard1@gmail.com>.
Ensure we remember and propagate the random seed we generate during
shuffle mode. Also add a debug statement displaying the seed.
* src/shuffle.c (shuffle_set_mode): Init and save the randoms seed.
* src/misc.c (make_rand): Code cleanups.
* src/main.c (main): Show a debug message containing the seed.
Paul Smith [Sat, 15 Oct 2022 20:34:54 +0000 (16:34 -0400)]
[SV 63157] Ensure temporary files are removed when signaled
Original patch from Dmitry Goncharov <dgoncharov@users.sf.net>.
When handling a fatal signal ensure the temporary files for
stdin and the jobserver fifo (if in use) are deleted.
* src/makeint.h (temp_stdin_unlink): Declare a new method.
* src/main.c (temp_stdin_unlink): Delete the stdin temporary file
if it exists. If the unlink fails and we're not handling a signal
then show an error.
(main): Call temp_stdin_unlink() instead of unlinking by hand.
* src/commands.c (fatal_error_signal): Invoke cleanup methods if
we're handling a fatal signal.
* tests/scripts/features/output-sync: Test signal handling during
output sync and jobserver with FIFO.
* tests/scripts/features/temp_stdin: Test signal handling when
makefiles are read from stdin.
(jobserver_setup): Set job_root to true.
(jobserver_clear): if we're the root instance and we're using a
FIFO, unlink it. If we're not in a signal handler, free memory.
(jobserver_acquire_all): Call jobserver_clear().
(sync_root): Rename from sync_parent for consistency.
Paul Smith [Sun, 9 Oct 2022 17:41:21 +0000 (13:41 -0400)]
Set the proper type for variables set in signal handlers
* bootstrap.conf: Get gnulib's sig_atomic_t type checking M4 macro.
* configure.ac: Invoke it.
* src/makeint.h (handling_fatal_signal): Set the type correctly.
* src/commands.c (handling_fatal_signal): Ditto.
Paul Smith [Mon, 3 Oct 2022 19:06:12 +0000 (15:06 -0400)]
Clean up some warnings on Windows builds
* src/arscan.c (parse_int): Avoid conversion from int to char and
check for overflow given a max value.
(ar_scan): Check intmax sizes then cast to the proper type.
(ar_member_touch): Get proper return type from ar_scan and cast it
to off_t.
* src/function.c (a_word_hash_cmp): Don't cast from size_t to int.
(func_filter_filterout): Count using unsigned long to avoid casts.
(construct_include_path): Explicitly cast to void*.
* src/shuffle.c (reverse_shuffle_array): Use size_t index.
* src/w32/subproc/sub_proc.c (process_wait_for_multiple_objects):
Initialize the return value in case the count is 0.
Pacify 12.2.1 20220819 (Red Hat 12.2.1-2) -Wsign-compare by checking
the readbuf() return explicitly for errors then casting from ssize_t.
Initial patch provided by Paul Eggert <eggert@cs.ucla.edu>.
Paul Eggert [Sun, 2 Oct 2022 21:24:53 +0000 (17:24 -0400)]
Support systems with 32-bit long and 64-bit time_t
Don't assume that time_t fits in long, as some hosts (e.g.,
glibc x86 -D_TIME_BITS=64) have 32-bit long and 64-bit time_t.
* bootstrap.conf (gnulib_modules): Add largefile, to support files
with timestamps after Y2038 on hosts with 32-bit long.
* configure.ac: Do not call AC_SYS_LARGEFILE, as the largefile module
does that for us.
* src/makeint.h (PRIdMAX, PRIuMAX, SCNdMAX): Define if not already
defined (taken from gnulib).
* src/ar.c: Include intprops.h, for TYPE_MAXIMUM, as
INTEGER_TYPE_MAXIMUM does not work on time_t without issuing a bunch
of warnings.
(ar_member_date): Check that result is in time_t range.
(ar_member_date_1): Use intmax_t to hold the date.
(ar_glob_match): Ditto.
* src/arscan.c (VMS_function, VMS_function_ret, ar_scan)
(parse_int, ar_scan, ar_member_pos, ar_member_touch)
(describe_member): Convert long int to intmax_t.
* src/file.c (file_timestamp_sprintf): Use intmax_t/uintmax_t instead
of long/unsigned long for values that might be time_t.
* src/arscan.c (ar_member_touch): Fix buffer overrun if the timestamp
is too large.
Paul Smith [Sun, 2 Oct 2022 14:18:21 +0000 (10:18 -0400)]
[SV 63098] Temporarily revert the change to pattern rule behavior
The fix for SV 12078 caused a backward-compatibility issue with some
makefiles. In order to allow users to resolve this issue, revert
that change for this release cycle: it will be reinstated in the
next release cycle. Introduce a warning if we detect that the recipe
of a multi-target pattern rule doesn't create all the targets.
* NEWS: Announce the future backward-incompatibility.
* doc/make.texi (Pattern Intro): Describe the behavior and that it
will change in the future.
* src/remake.c (check_also_make): Check for also_make targets that
were not created and generate a warning.
(update_goal_chain): Call the new function.
(check_dep): Ditto.
(update_file_1): Defer implicit rule detection until after we check
all the also_make files (as it used to be).
* tests/scripts/features/patternrules: Add tests of the new warning.
Skip the tests for SV 12078.
projects / thirdparty / make.git / log
