Joe Crayne [Sat, 21 May 2016 21:26:00 +0000 (17:26 -0400)]
[SV 44742] Fix double-colon rules plus parallel builds.
* remake.c (update_file): Don't update double-colon target status
if we're still building targets.
(ftime_t): Don't propagate timestamps for double-colon targets that
we've not examined yet.
* tests/scripts/features/double_colon: Add parallel build tests.
Paul Smith [Sun, 1 May 2016 23:24:20 +0000 (19:24 -0400)]
Separate the GNU make load ABI from internal types.
Create an internal type "floc" and convert all users to that type.
* gnumake.h (gmk_floc): Remove the offset field from this type.
* loadapi.c (gmk_eval): Convert gmk_floc to internal floc.
Paul Eggert [Sun, 1 May 2016 19:52:58 +0000 (15:52 -0400)]
Fixes for enhanced GCC warnings.
Move function prototypes into header files and out of .c files.
Use void argument lists for functions that accept no args.
Remove unused macros. Make private functions static. Align
types with printf format characters.
Paul Smith [Sun, 1 May 2016 19:34:42 +0000 (15:34 -0400)]
Fix compile issues with Windows and VMS.
* main.c (initialize_stopchar_map): isblank() is not part of C89.
Install bits for space and tab directly.
* makeint.h: Don't define vfork; autoconf handles this for us.
* vmsjobs.c: Rename NEXT_TOKEN to V_NEXT_TOKEN to avoid conflicts.
* dir.c (print_dir_data_base): Visual Studio C doesn't have int64_t.
* w32/subproc/sub_proc.c (process_begin): Missing arg to memset().
* build_w32.bat: Don't use obsolete Visual Studio flags.
Eli Zaretskii [Sun, 24 Apr 2016 06:53:38 +0000 (09:53 +0300)]
Fix the MS-Windows MinGW build
* build_w32.bat (GccCompile): Use -std=gnu99, as some code uses
C99 features ('for' loop initial declarations).
* dir.c (print_dir_data_base) [WINDOWS32]: Cast 'mtime' to
'int64_t', and use %I64d to print it, to avoid compile-time
warning about printing a 'time_t' value, which could be either
a 32-bit or a 64 bit integral type.
Paul Smith [Sun, 10 Apr 2016 21:12:48 +0000 (17:12 -0400)]
[SV 46433] Show recipe line offsets in line number messages.
While displaying line numbers, show the relevant line number inside
the recipe not just the first line of the entire recipe.
Sample changes suggested by Brian Vandenberg <phantall@gmail.com>
* gnumake.h (gmk_floc): Add an 'offset' to track the recipe offset.
* read.c (eval, eval_makefile, eval_buffer): Initialize 'offset'.
(record_files, install_pattern_rule): Ditto.
* job.c (new_job, job_next_command): Update 'offset' based on the
line of the recipe we're expanding or invoking.
(child_error): Add 'offset' when showing the line number.
* function.c (func_shell_base): Ditto.
* output.c (error, fatal): Ditto.
* NEWS: Mention the new ability.
* tests/scripts/features/errors: Check the line number on errors.
* tests/scripts/functions/warning: Check the line number on warnings.
* tests/scripts/features/output-sync,
tests/scripts/features/parallelism, tests/scripts/functions/shell,
tests/scripts/functions/error: Update line numbers.
Paul Smith [Sat, 9 Apr 2016 23:49:27 +0000 (19:49 -0400)]
[SV 102] Don't show unnecessary include file errors.
Delay the generation of error messages for included files until we
are sure that we can't rebuild that included file.
* dep.h (struct dep): Don't reuse "changed"; make a separate field
to keep "flags". Get rid of dontcare and use the flag.
(struct goaldep): Create a new structure for goal prereqs
that tracks an errno value and the floc where the include happened.
Rework the structures to ensure they are supersets as expected.
In maintainer mode with GCC, use inline to get type checking.
* read.c (eval_makefile): Return a struct goaldep for the new
makefile. Ensure errno is set properly to denote a failure.
(read_all_makefiles): Switch to goaldep and check errno.
(eval): Don't show included file errors; instead remember them.
* remake.c (update_goal_chain): Set global variables to the current
goaldep we're building, and the entire chain.
(show_goal_error): Check if the current failure is a consequence
of building an included makefile and if so print an error.
(complain): Call show_goal_error() on rule failure.
* job.c (child_error): Call show_goal_error() on child error.
* main.c (main): Switch from struct dep to goaldep.
* misc.c (free_dep_chain): Not used; make into a macro.
* tests/scripts/features/include: Update and include new tests.
* tests/scripts/options/dash-B, tests/scripts/options/dash-W,
tests/scripts/options/print-directory,
tests/scripts/variables/MAKE_RESTARTS: Update known-good-output.
Paul Smith [Mon, 4 Apr 2016 05:23:04 +0000 (01:23 -0400)]
Preserve the real value of -jN in MAKEFLAGS using jobserver.
Previously if the jobserver was active, MAKEFLAGS would contain only
the -j option but not the number (not -j5 or whatever) so users
could not discover that value. Allow that value to be provided in
MAKEFLAGS without error but still give warnings if -jN is provided
on the command line if the jobserver is already activated.
* NEWS: Discuss the new behavior.
* os.h, posixos.c, w32/w32os.c: Return success/failure from
jobserver_setup() and jobserver_parse_auth().
* main.c (main): Separate the command line storage of job slots (now
in arg_job_slots) from the control storage (in job_slots). Make a
distinction between -jN flags read from MAKEFLAGS and those seen
on the command line: for the latter if the jobserver is enabled then
warn and disable it, as before.
* tests/scripts/features/jobserver: Add new testing.
Paul Smith [Sat, 2 Apr 2016 20:29:48 +0000 (16:29 -0400)]
Change --jobserver-fds to more generic --jobserver-auth.
* NEWS: Mention the change.
* main.c: Rename jobserver_fds variable to jobserver_auth and
--jobserver-fds option to --jobserver-auth.
* os.h, posixos.c, w32/w32os.c: Rename jobserver_parse_arg() and
jobserver_get_arg() to jobserver_parse_auth()/jobserver_get_auth().
Paul Smith [Mon, 21 Mar 2016 04:36:55 +0000 (00:36 -0400)]
[SV 46995] Strip leading/trailing space from variable names
* makeint.h: Change MAP_SPACE to MAP_NEWLINE, and add MAP_PATHSEP
and MAP_SPACE which is now MAP_BLANK|MAP_NEWLINE. Create
NEW_TOKEN(), END_OF_TOKEN(), ISBLANK(), ISSPACE() macros.
* main.c (initialize_stopchar_map): Set MAP_NEWLINE only for
newline characters.
* Convert all uses of isblank() and isspace() to macros.
* Examine all uses of isblank() (doesn't accept newlines) and
change them wherever possible to ISSPACE() (does accept newlines).
* function.c (func_foreach): Strip leading/trailing space.
* variable.c (parse_variable_definition): Clean up.
* tests/scripts/functions/foreach: Test settings and errors.
* tests/scripts/functions/call: Rewrite to new-style.
* tests/scripts/misc/bs-nl: Add many more tests for newlines.
Paul Smith [Sat, 19 Mar 2016 21:26:08 +0000 (17:26 -0400)]
* function.c (func_file): Support reading from files.
* NEWS: Add information about reading files.
* make.texi (File Function): Describe reading files.
* tests/scripts/functions/file: Test new features for $(file ...)
Paul Smith [Sun, 13 Mar 2016 22:13:00 +0000 (18:13 -0400)]
[SV 45728] Detect changes in .VARIABLES more accurately.
For performance, we only recompute .VARIABLES when (a) it's expanded
and (b) when its value will change from a previous expansion. To
determine (b) we were checking the number of entries in the hash
table which used to work until we started undefining entries: now if
you undefine and redefine the same number of entries in between
expanding .VARIABLES, it doesn't detect any change. Instead, keep
an increasing change number.
* variables.c: Add variable_changenum.
(define_variable_in_set, merge_variable_sets): Increment
variable_changenum if adding a new variable to the global set.
(undefine_variable_in_set): Increment variable_changenum if
undefining a variable from the global set.
(lookup_special_var): Test variable_changenum not the hash table.
* tests/scripts/variables/special: Test undefining variables.
Paul Smith [Sun, 13 Mar 2016 07:02:00 +0000 (03:02 -0400)]
[SV 46581] Pre-define .LOADED to avoid warnings.
* main.c (main): Pre-define .LOADED as a default-level variable.
* load.c (load_file): Set the value rather than append it. Avoid
adding an extra initial whitespace.
* tests/scripts/features/load: Run with --warn-undefined-variables.
Paul Smith [Sun, 13 Mar 2016 06:12:07 +0000 (01:12 -0500)]
[SV 44555] Use vfork() instead of fork() where available.
Testing has shown that vfork() is actually significantly
more efficient on systems where it's supported, even for
copy-on-write implementations. If make is big enough,
duplicating the page tables is significant overhead.
* configure.ac: Check for fork/vfork.
* makeint.h: Include vfork.h and set up #define for it.
* os.h, posixos.c (get_bad_stdin): For children who can't use
the normal stdin file descriptor, get a broken one.
* job.c (start_job_command): Avoid so many ifdefs and simplify
the invocation of child_execute_job()
(child_execute_job): move the fork operation here so it can
return early for the parent process. Switch to use vfork().
* function.c (func_shell_base): Use new child_execute_job() and
simplify ifdefs.
* job.h, main.c, remote-cstms.c, vmsjobs.c, w32os.c: Update
declarations and calls.
Paul Smith [Wed, 9 Mar 2016 04:07:14 +0000 (23:07 -0500)]
[SV 46261] Use pselect() for jobserver where supported.
* Makefile.am, configure.ac: Check for pselect() and sys/select.h.
* main.c (main): Block SIGCHLD if we have pselect() support.
* posixos.c (jobserver_acquire): If we support pselect() then use
it to query the jobserver pipe, while also listening for SIGCHLD.
Also pselect() supports a timeout so avoid alarm() calls.
Paul Smith [Mon, 7 Mar 2016 06:30:21 +0000 (01:30 -0500)]
Clean up some compiler warnings.
* commands.c, commands.h: Use unsigned char for flags.
* dir.c: Use time_t and size_t, and char for a boolean value.
* job.c: Use unsigned and char.
* read.c: Return a signed type since -1 is a valid return code.
Paul Smith [Sat, 5 Mar 2016 20:21:59 +0000 (15:21 -0500)]
[SV 46261] Add more EINTRLOOP wrappers.
This cannot be a perfect solution because there are always other
possible places EINTR can happen, including external libraries
such as gettext, Guile etc.
Paul Smith [Mon, 29 Feb 2016 03:07:50 +0000 (22:07 -0500)]
* strcache.c (add_hugestring): [SV 46832] Support huge strings.
The strcache was limited to strings of length 65535 or less,
because the length is kept in an unsigned short. To support
huge strings add a new simple linked list, which we don't try
to hash.
If the very first string added to the string cache is more than
half the maximum size, we failed when moving the only strcache
buffer to the full list.
Paul Smith [Mon, 29 Feb 2016 01:20:18 +0000 (20:20 -0500)]
[SV 47151] Exit with 1 when checking recursive make -q
* job.h (struct child): New bit to mark recursive command lines.
* job.c (start_job_command): Set the recursive command line bit.
(reap_children): If the child is a recursive command and it exits
with 1 during question mode, don't print an error and exit with 1.
* tests/scripts/options/dash-q: Add a regression test.
Newer versions of binutils allow ar to be compiled to generate
"deterministic archives" by default: in this mode no timestamp
information is generated in the static archive, which utterly
breaks GNU make's archive updating capability. Debian and Ubuntu
have turned this feature on by default in their distributions
which causes the regression tests to fail.
Update the regression tests to check for the availability of the
"U" option to ar which disables deterministic archives and allows
GNU make's archive support to work properly again.
Eli Zaretskii [Wed, 23 Sep 2015 08:42:53 +0000 (11:42 +0300)]
[SV 45838] When invoking w32 programs, don't use free'd memory.
* w32/subproc/sub_proc.c (process_begin): Freeing argv[0] makes
the other argv[i] pointers invalid, so need to allocate a new
array and copy argv[i] for i != 0 first, replacing argv[0] with
the batch file name, before we can free argv[0].
Paul Smith [Mon, 13 Jul 2015 04:51:35 +0000 (00:51 -0400)]
* implicit.c (pattern_search): [SV 43677] Mark files secondary.
In order to fix SV 12267 we were marking the prerequisites of
implicit (pattern) targets that existed elsewhere in the makefile
as precious to keep them from being deleted as intermediate files.
However this also keeps them from being deleted on error. Instead
mark them as secondary.
* tests/scripts/targets/DELETE_ON_ERROR: Test DELETE_ON_ERROR.
Paul Smith [Mon, 13 Jul 2015 01:03:24 +0000 (21:03 -0400)]
[SV 28092] Preserve the exit status of the $(shell...) function.
Add a new variable .SHELLSTATUS which holds the exit status of the
last-invoked shell function or != assignment.
* NEWS, doc/make.texi: Document the change.
* function.c (shell_completed, msdos_openpipe, func_shell_base): Add
shell_completed() to handle the completion of the shell, by setting
.SHELLSTATUS. Call it where needed.
* job.c (child_handler): Call shell_completed().
* tests/scripts/functions/shell: Add tests for .SHELLSTATUS.
Paul Smith [Sun, 12 Jul 2015 17:25:16 +0000 (13:25 -0400)]
[SV 45049] Check for '$' being the last character in a string.
* expand.c (variable_expand_string): Add a single '$' if '$' ends the
string.
* read.c (find_char_unquote, get_next_mword): Stop if '$' ends the
string.
* variable.c (parse_variable_definition): Ditto.
Paul Smith [Sat, 11 Jul 2015 21:21:22 +0000 (17:21 -0400)]
* strcache.c: [SV 45275] Handle very long strings.
Our previous behavior for handling too-long strings involved
increasing the size of the default string cache buffer, but the
implementation was incomplete. Instead, create a one-off large
string cache entry and add it directly to the full cache list
without changing the default buffer size.
Eli Zaretskii [Sat, 28 Feb 2015 12:34:51 +0000 (14:34 +0200)]
[SV 44348] Fix handling of shell widlcards on MS-Windows.
* job.c (construct_command_argv_internal): If shell wildcard
characters are found inside a string quoted with "..", give up the
fast route and go through the shell. Fixes Savannah bug #44348.
John Malmberg [Fri, 28 Nov 2014 03:28:30 +0000 (21:28 -0600)]
Fix bs-nl handling, exit and Environment for VMS.
This fix required a complete rewrite of the command parser vmsjobs.c
child_execute_job. The old parser had too many incorrect assumptions
about DCL commands and could not be repaired to extended.
The parser now more closely parses VMS commands and handles quoted
commands and redirection. Command File mode has been improved, but can
not fully support bs-nl syntax.
VMS Unix shell simulation has been improved.
* commands.c: vms_comma_separator is now a run-time setting.
* function.c: vms_comma_separator is now a run-time setting.
* function.c(func_basename_dir) now reports "[]" or "./" based on
VMS crtl runtime setting.
* job.c(start_job_command): VMS Handle empty commands propery.
* main.c: Add VMS environment variables for run-time settings.
* vms_legacy_behavior - Force older behavior.
* vms_comma_separator - Commas or spaces for separators.
* vms_unix_simulation - Enhanced Posix shell simulation features.
* Detect if VMS CRTL is set to report Unix paths instead of VMS.
* ':' and '>' are also MAP_DIRSEP on VMS.
* makeint.h: Add VMS run-time option variables.
* readme.vms: Update to current behavior.
* variable.c(define_variable_in_set): Fix VMS Environment variable
lookup.
* variable.c(define_automatic_variables): Remove some VMS specific
automatic variables and use the Unix ones instead.
* vms_export_symbol.c: Set max symbol size correctly.
* vmsjobs.c: child_execute_job() complete rewrite of VMS comand
parsing.
* vmsjobs.c(build_vms_cmd): VMS commmand building with shell simulation.
John Malmberg [Wed, 8 Oct 2014 00:23:47 +0000 (19:23 -0500)]
Fix VMS implicit rules and UNIX paths.
This fixes VMS implicit rules and UNIX style pathname handling.
It also fixes some of the VMS style pathname handling, more work
there will be needed later.
TODO: There are other case insensitive platforms besides VMS.
We need to find out why there is extra VMS code for this. This
indicates either the extra VMS code is not needed, or the case
insensitive support may not be complete on the other case
insensitive platforms.
* default.c: Add missing definitions to default_suffix_rules[] and
default_variables[].
TODO: As it is important that VMS DCL mode definitions must always
be a superset of UNIX definitions, a better way of maintaining the
VMS DCL mode definitions should be devised.
* dir.c (downcase_inplace): Add a reentrant downcase() routine.
Add future support for VMS 8.2+ _USE_STD_STAT macro which will
disable a lot of VMS specific code from compiling.
(dir_file_exists_p): vmsify filename only if directory name has VMS
directory delimiters.
(file_exists_p): Handle both VMS and UNIX directories.
(file_impossible): Handle both VMS and Unix directories. Track
whether a VMS format path is needed for the return value.
* file.c (lookup_file): Check if vmsify is needed; handle UNIX paths.
* implicit.c (pattern_search): Enable UNIX paths.
* read.c (parse_file_seq): Enable UNIX paths.
* remake.c (f_mtime): Fix gpath_search call for VMS paths.
* rule.c (count_implicit_rule): Enable UNIX paths, Fix VMS paths.
* vpath.c (selective_vpath_search): Enable UNIX paths.
John Malmberg [Fri, 12 Sep 2014 03:39:34 +0000 (22:39 -0500)]
[SV 41758]: Fix archive support for VMS.
Upated to match change to run_make_tests and some future fixes to
make on VMS.
* arscan.c: Use ANSI compatible pragmas instead of VAX C extensions.
* tests/scripts/features/archives: Fix tests to use VMS rules and
answers when running on VMS and using DCL as a shell.
* tests/scripts/features/vpath3: Fix epected answer on test when
run on VMS.
* tests/scripts/vms/library: (New) Test the VMS library rules that
are not tested by existing tests.
John Malmberg [Sun, 14 Sep 2014 01:20:22 +0000 (20:20 -0500)]
[SV 42447]: VMS simulate exporting symbols
This also includes fixing the most of the exit handling code for VMS.
Self tests:
Previously about 94 Tests in 36 categories fail.
Now about 45 tests in 22 categories fail.
Because some tests do not properly clean up, the number of tests that
fail can vary by one or two test cases between consecutive runs.
* Makefile.am: Add new VMS files.
* job.c: add prototype for vms_strsignal().
* job.c: (child_error): Remove VMS specific code as no longer needed.
* job.c: (reap_children): The VMS specific code was setting the
status to 0 instead of setting it to the proper exit status.
* job.h: Add vms_launch_status to struct child.
* main.c: (main): Use environment variables for options to use MCR
* instead of a foreign command, and to always use command files for
subprocesses.
For VMS use (set_program_name) routine which is common to ports of
other GNU packages to VMS to set the program name used internally.
Use (vms_putenv_symbol) to set up symbols to be visible in child
programs, including recursive make launched by execve()
Start of Bash shell detection code for VMS.
* makefile.com: Need nested_include=none for building on VMS search
lists. Add vms_progname, vms_exit, and vms_export_symbol.
* makefile.vms: Need nested_include=none for building on VMS search
lists. Add vms_progname, vms_exit, vms_export_symbol.
* makeint.h: Make sure non-standard "VMS" macro is defined. Add
prototypes for new VMS routines. Remove VMS-specific failure codes.
* vmsjobs.c: Add VMS POSIX exit code constants.
(_is_unixy_shell): Detect Bash shell.
(vms_strsignal): simulate strsignal() on VMS.
(vmsHandleChildTerm): fix to properly report failed LIB$SPAWN() exit
status codes. Remove code that duplicated code in job.c.
(child_execute_job): Export environment symbols before spawning a
child and restore afterward unless option to use command files for
subprocesses is set. Improve handling of UNIX null commands ":".
* vms_exit.c: Provides vms_exit() to detect if an exit code is UNIX
or VMS, and converts the UNIX code into a VMS exit code.
* vms_export_symbol.c: Routines to create DCL symbols that work like
shell aliases or exported shell symbols and clean them up on exit.
* vms_export_symbol_test.com: Unit test for vms_export_symbol.c
* vms_progname.c: New file: VMS specific replace for progname.c that
is used in some GNU projects.
John Malmberg [Tue, 7 Oct 2014 02:46:34 +0000 (21:46 -0500)]
Set up for running tests on VMS.
* run_make_tests.pl: set $port_type to be 'VMS-DCL' when the test are
run from the VMS DCL Interpreter. When the tests are run from GNV
on VMS, the $port_type will be 'UNIX'.
* run_make_tests.com: VMS search list support. This is needed for
using a search list such as prj_root = lcl_root:,vms_root:,src_root:
for building and testing.
Eli Zaretskii [Wed, 1 Oct 2014 15:34:29 +0000 (18:34 +0300)]
Avoid compilation warnings.
* main.c (find_and_set_default_shell, main) [WINDOWS32]: Declare
variables 'const char *' to avoid compiler warnings.
* job.c (construct_command_argv_internal) [!NDEBUG]: Declare 'end'
and set it only if NDEBUG is not defined, to avoid compiler
warnings.
Eli Zaretskii [Wed, 1 Oct 2014 15:26:28 +0000 (18:26 +0300)]
Treat redirection of standard handles on Windows as it is done on Unix.
* job.c (start_job_command) [WINDOWS32]: Compute outfd and errfd
as on Posix platforms, and pass the results to process_easy.
* function.c (windows32_openpipe) [WINDOWS32]: Accept an
additional argument ERRFD and use it for redirecting the standard
error handle passed to the subprocess.
(func_shell_base) [WINDOWS32]: Pass the computed errfd to
windows32_openpipe.
Paul Smith [Sun, 14 Sep 2014 05:03:19 +0000 (01:03 -0400)]
* main.c (main): Set MAKE_TTYOUT and MAKE_TTYERR.
* configure.ac: Test for isatty() and ttyname()
* makeint.h: provide a substitute for ttyname() if it's not available.
* config.ami.template, config.h-vms.template, config.h.W32.template:
define/undefine HAVE_ISATTY/HAVE_TTYNAME macros.
* NEWS, doc/make.texi: Document these new variables.
* ar.c: fix VMS library search for members, which do not have
suffixes, aka filename extensions.
* arscan.c: fix time conversion and library callback routines.
* default.c: more suffixes and automatically create the VMS library
if it doesn't exists.
* config.h-vms.template: add feature macro USE_DCL_COM_FILE to always
write a DCL command file, enabled by default.
* vmsjobs.c: with USE_DCL_COM_FILE enabled write make variables as DCL
symbol assignments into the command file. This enables printing
directory and make level info for recursive use of make. This also
enables forced DCL symbol substitution in the actions.
* function.c: add VMS code to func_notdir_suffix and func_basename_dir
to work on comma separated lists; this fixes the expansion of
$(^D), $(+D) and the F variants for VMS.
Hartmut Becker [Sun, 31 Aug 2014 21:52:25 +0000 (23:52 +0200)]
Enhance/fix VMS ONESHELL implementation and command execution
* job.c, vmsjobs.c: fix some double quote and new line handling;
implement ONESHELL with writing multiple lines into one DCL command
procedure; in ONESHELL allow VMS/make internal redirection only on the
first line; fix the created DCL command procedure, which didn't abort
on errors; return correct exit status from the DCL command procedure;
preserve current procedure verification; make the generated command
procedure more robust.