From: Zack Weinberg Date: Sun, 3 Dec 2023 17:09:37 +0000 (-0500) Subject: Overhaul AC_PROG_LEX for precision and maintainability X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=refs%2Fheads%2Fzack%2Fac-prog-lex-rewrite;p=thirdparty%2Fautoconf.git Overhaul AC_PROG_LEX for precision and maintainability AC_PROG_LEX has three different operating modes (“yywrap”, “noyywrap”, and 2.69 compatibility mode) which were all tangled together and hard to understand. In addition, the way we were handling yywrap, in all three modes, was imprecise to the point where I am unsure we were actually testing the property people want to test. Refactor AC_PROG_LEX into a set of smaller macros: - AC_PROG_LEX and _AC_PROG_LEX_1 perform argument validation and normalization. _AC_PROG_LEX_1 also ensures that repeated invocation of AC_PROG_LEX is harmless. - _AC_PROG_LEX_2 is invoked with exactly one argument which is the name of one of the three _AC_PROG_LEX_LIBL_* macros (described below). It invokes, in sequence, AC_CHECK_PROGS([LEX]), _AC_PROG_LEX_OUTPUT_ROOT, the _AC_PROG_LEX_LIBL_ macro it was given as an argument, and finally _AC_PROG_LEX_YYTEXT_DECL. If any of these sets LEX to ‘:’, further tests are skipped. _AC_PROG_LEX_2 is also responsible for cleaning up conftest.l and lex.yy.c afterward. - _AC_PROG_LEX_YYTEXT_DECL now *only* tests whether yytext is a pointer. It reuses the conftest.l from previous tests. - _AC_PROG_LEX_OUTPUT_ROOT just does the test for the name of the file produced by running ‘lex conftest.l’ with no further arguments. - _AC_PROG_LEX_LIBL_YYWRAP, _AC_PROG_LEX_LIBL_NOYYWRAP, and _AC_PROG_LEX_LIBL_COMPAT determine whether a generated lexer needs to be linked with -ll or -lfl. Each is dedicated to one of the three operating modes, and uses a conftest.l specialized for that mode. In compat mode we do *not* look for ‘yywrap’ using AC_SEARCH_LIBS; instead we actually try to link a generated lexer that uses yywrap. - _AC_PROG_LEX_SEARCH_LIBL, _AC_PROG_LEX_TEST_COMMON, _AC_PROG_LEX_TEST_YYWRAP, and _AC_PROG_LEX_TEST_NOYYWRAP are subroutines of the _AC_PROG_LEX_LIBL_* macros. TODO: - Shrink and sharpen the test program used by _AC_PROG_LEX_YYTEXT_DECL and make it independent of previous operations. Possibly smush it back together with _AC_PROG_LEX_OUTPUT_ROOT. - Re-audit the “noyywrap” program, it might be better to declare “_static_ int yywrap(void)” in the %{ %} block instead of not declaring it at all. - Re-audit the “yywrap” program, it might be better to rely on the skeleton to declare yywrap. * lib/autoconf/programs.m4 (AC_PROG_LEX): Extensive refactor. Test for yywrap is now done by linking a generated lexer, not with AC_SEARCH_LIBS. (_AC_PROG_LEX_YYTEXT_DECL): Just test whether yytext is a pointer. (_AC_PROG_LEX): Removed. (_AC_PROG_LEX_1, _AC_PROG_LEX_2): New macros dividing up the labor formerly performed by _AC_PROG_LEX and the order of operations in the old _AC_PROG_LEX_YYTEXT_DECL. (_AC_PROG_LEX_OUTPUT_ROOT, _AC_PROG_LEX_SEARCH_LIBL) (_AC_PROG_LEX_LIBL_YYWRAP, _AC_PROG_LEX_LIBL_NOYYWRAP) (_AC_PROG_LEX_LIBL_COMPAT, _AC_PROG_LEX_TEST_COMMON) (_AC_PROG_LEX_TEST_YYWRAP, _AC_PROG_LEX_TEST_NOYYWRAP): New macros dividing up the rest of the labor formerly performed by _AC_PROG_LEX_YYTEXT_DECL. * tests/semantics.at (AC_PROG_LEX with noyywrap): Rename “AC_PROG_LEX: noyywrap”. (AC_PROG_LEX with yywrap): Rename “AC_PROG_LEX: yywrap”. No longer necessary to skip this test on OSes where there is no -ll nor -lfl. (AC_PROG_LEX in legacy mode): Rename “AC_PROG_LEX: compat mode.” Convert to AT_CHECK_MACRO test using -Wno-obsolete. (Invalid arguments to AC_PROG_LEX): Rename “AC_PROG_LEX: invalid arguments and warnings.” Move the test for the warning “AC_PROG_LEX without either yywrap or noyywrap is obsolete” here. Adjust expected stack traces. --- diff --git a/lib/autoconf/programs.m4 b/lib/autoconf/programs.m4 index d06d18c4..d40ffe6c 100644 --- a/lib/autoconf/programs.m4 +++ b/lib/autoconf/programs.m4 @@ -738,9 +738,9 @@ AC_DEFUN([AC_PROG_LEX], [0], [], [1], [], [m4_fatal([too many arguments to $0])])]dnl -[_$0(m4_normalize([$1]))]) +[_AC_PROG_LEX_1(m4_normalize([$1]))]) -AC_DEFUN([_AC_PROG_LEX], +AC_DEFUN([_AC_PROG_LEX_1], [m4_case([$1], [yywrap], [], [noyywrap], [], @@ -761,24 +761,19 @@ dnl warn if they don't. [dnl dnl _AC_PROG_LEX_options not defined: first use m4_define([_AC_PROG_LEX_options], [$1])dnl -AC_CHECK_PROGS(LEX, flex lex, :) - if test "x$LEX" != "x:"; then - _AC_PROG_LEX_YYTEXT_DECL([$1]) -fi])]) - - -# _AC_PROG_LEX_YYTEXT_DECL -# ------------------------ -# Check for the Lex output root, the Lex library, and whether Lex -# declares yytext as a char * by default. -AC_DEFUN([_AC_PROG_LEX_YYTEXT_DECL], -[cat >conftest.l <<_ACEOF[ -%{ -#ifdef __cplusplus -extern "C" -#endif -int yywrap(void); -%} +_AC_PROG_LEX_2(_AC_PROG_LEX_LIBL_[]m4_toupper(m4_default([$1], [compat])))])]) + +AC_DEFUN([_AC_PROG_LEX_2], +[AC_CHECK_PROGS([LEX], [flex lex], [:]) +AS_IF([test "x$LEX" != "x:"], [_AC_PROG_LEX_OUTPUT_ROOT]) +AS_IF([test "x$LEX" != "x:"], [AS_VAR_SET_IF([LEXLIB], [], [$1])]) +AS_IF([test "x$LEX" != "x:"], [_AC_PROG_LEX_YYTEXT_DECL]) +rm -f conftest.l $LEX_OUTPUT_ROOT.c]) + +AC_DEFUN([_AC_PROG_LEX_TEST_COMMON], +[m4_divert_text([INIT_PREPARE], +[[# Test code for probing (f)lex, base version. +ac_prog_lex_conftest_l_common=' %% a { ECHO; } b { REJECT; } @@ -798,89 +793,159 @@ f { unput (yytext[0]); } extern char *yytext; #endif int -yywrap (void) +main (void) { - return 1; + return ! yylex (); } +' +]])]) + +AC_DEFUN([_AC_PROG_LEX_TEST_YYWRAP], +[AC_REQUIRE([_AC_PROG_LEX_TEST_COMMON])]dnl +[m4_divert_text([INIT_PREPARE], +[[# Test code for lex (yywrap needed from -ll) +ac_prog_lex_conftest_l_yywrap=' +%{ +#ifdef __cplusplus +extern "C" +#endif +int yywrap(void); +%} +'"$ac_prog_lex_conftest_l_common" +]])]) + +AC_DEFUN([_AC_PROG_LEX_TEST_NOYYWRAP], +[AC_REQUIRE([_AC_PROG_LEX_TEST_COMMON])]dnl +[m4_divert_text([INIT_PREPARE], +[[# Test code for lex (yywrap defined by .l file) +ac_prog_lex_conftest_l_noyywrap="$ac_prog_lex_conftest_l_common"' int -main (void) +yywrap (void) { - return ! yylex (); + return 1; } -]_ACEOF -AC_CACHE_CHECK([for lex output file root], [ac_cv_prog_lex_root], [ +' +]])]) + +# _AC_PROG_LEX_OUTPUT_ROOT +# ------------------------ +# Determine what the name of the Lex output file is. +AC_DEFUN([_AC_PROG_LEX_OUTPUT_ROOT], +[AC_REQUIRE([_AC_PROG_LEX_TEST_COMMON])]dnl +[AC_CACHE_CHECK([for lex output file root], [ac_cv_prog_lex_root], [ ac_cv_prog_lex_root=unknown +cat >conftest.l <<_ACEOF +$ac_prog_lex_conftest_l_common +_ACEOF _AC_DO_VAR(LEX conftest.l) && if test -f lex.yy.c; then ac_cv_prog_lex_root=lex.yy elif test -f lexyy.c; then ac_cv_prog_lex_root=lexyy fi]) -AS_IF([test "$ac_cv_prog_lex_root" = unknown], +AC_SUBST([LEX_OUTPUT_ROOT], [$ac_cv_prog_lex_root])]dnl +[AS_IF([test "$ac_cv_prog_lex_root" = unknown], [AC_MSG_WARN([cannot find output from $LEX; giving up on $LEX]) - LEX=: LEXLIB=]) -AC_SUBST([LEX_OUTPUT_ROOT], [$ac_cv_prog_lex_root])dnl - -AS_VAR_SET_IF([LEXLIB], [], [ - AC_CACHE_CHECK([for lex library], [ac_cv_lib_lex], [ + LEX=: LEXLIB=])]) + +# _AC_PROG_LEX_SEARCH_LIBL(CACHE-VAR, LEX-INPUT-VAR, MESSAGE, +# ACTION-IF-NOT-FOUND) +# ----------------------------------------------------------- +# Subroutine of _AC_PROG_LEX_LIBL_*, does the actual search for the library. +m4_define([_AC_PROG_LEX_SEARCH_LIBL], +[AC_CACHE_CHECK([for lex library ($3)], [$1], [ + cat >conftest.l <<_ACEOF +$[]$2 +_ACEOF + AS_IF([_AC_DO_VAR(LEX conftest.l)], [ ac_save_LIBS="$LIBS" ac_found=false - for ac_cv_lib_lex in 'none needed' -lfl -ll 'not found'; do - AS_CASE([$ac_cv_lib_lex], - ['none needed'], [], - ['not found'], [break], - [*], [LIBS="$ac_cv_lib_lex $ac_save_LIBS"]) - - AC_LINK_IFELSE([AC_LANG_DEFINES_PROVIDED[`cat $LEX_OUTPUT_ROOT.c`]], + for $1 in 'none needed' -lfl -ll 'not found'; do + AS_CASE([$[]$1], + ['not found'], [break], + ['none needed'], [], + [*], [LIBS="$[]$1 $ac_save_LIBS"]) + AC_LINK_IFELSE([AC_LANG_DEFINES_PROVIDED[`cat $LEX_OUTPUT_ROOT.c`]], [ac_found=:]) - if $ac_found; then - break - fi + if $ac_found; then + break + fi done LIBS="$ac_save_LIBS" - ]) - AS_IF( - [test "$ac_cv_lib_lex" = 'not found'], - [AC_MSG_WARN([required lex library not found; giving up on $LEX]) - LEX=: LEXLIB=], - [test "$ac_cv_lib_lex" = 'none needed'], - [LEXLIB=''], - [LEXLIB=$ac_cv_lib_lex]) -dnl -dnl For compatibility with autoconf 2.69 and prior, if $1 is not 'noyywrap', -dnl and we didn't already set LEXLIB to -ll or -lfl, see if one of those -dnl libraries provides yywrap and set LEXLIB to it if so. If $1 is 'yywrap', -dnl and we don't find a library that provides yywrap, we fail. - m4_case([$1], - [noyywrap], - [], - [yywrap], - [ac_save_LIBS="$LIBS" - AS_IF([test -n "$LEXLIB"], - [LIBS="$LEXLIB" - AC_CHECK_FUNC([yywrap], - [:], - [AC_MSG_WARN([$LEXLIB does not contain yywrap; giving up on $LEX]) - LEX=: LEXLIB=]) - ], - [LIBS= - AC_SEARCH_LIBS([yywrap], [fl l], [LEXLIB="$LIBS"]) - AS_IF([test x"$ac_cv_search_yywrap" = xno], - [AC_MSG_WARN([yywrap not found; giving up on $LEX]) - LEX=: LEXLIB=])]) - LIBS="$ac_save_LIBS"], - [], - [ac_save_LIBS="$LIBS" - LIBS= - AC_SEARCH_LIBS([yywrap], [fl l], [LEXLIB="$LIBS"]) - LIBS="$ac_save_LIBS"])dnl + ], + [$1='lex error'])]) +AS_CASE([$[]$1], + ['not found'], + [m4_default([$4], + [AC_MSG_WARN([required lex library not found; giving up on $LEX]) + LEX=: + LEXLIB=''])], + ['lex error'], + [AC_MSG_WARN([$LEX rejected the test program; giving up on it]) + LEX=: + LEXLIB=''], + ['none needed'], + [LEXLIB=''], + [*], + [LEXLIB="$[]$1"]) ]) -AC_SUBST(LEXLIB) -dnl This test is done last so that we don't define YYTEXT_POINTER if -dnl any of the above tests gave up on lex. -AS_IF([test "$LEX" != :], [ -AC_CACHE_CHECK(whether yytext is a pointer, ac_cv_prog_lex_yytext_pointer, +# _AC_PROG_LEX_LIBL_NOYYWRAP +# -------------------------- +# If it is necessary to link a simple lex scanner, that defines yywrap +# itself, with a library, then set LEXLIB to that library. If a library +# is needed but not found, set LEXLIB to be empty and pretend we couldn't +# find a lex program. Assumes _AC_PROG_LEX_OUTPUT_ROOT has already executed. + +AC_DEFUN([_AC_PROG_LEX_LIBL_NOYYWRAP], +[AC_REQUIRE([_AC_PROG_LEX_TEST_NOYYWRAP])]dnl +[_AC_PROG_LEX_SEARCH_LIBL( + [ac_cv_lib_lex_noyywrap], + [ac_prog_lex_conftest_l_noyywrap], + [yywrap not needed])]) + +# _AC_PROG_LEX_LIBL_YYWRAP +# ------------------------ +# If it is necessary to link a simple lex scanner, that doesn't define +# yywrap, with a library, then set LEXLIB to that library. If a library +# is needed but not found, set LEXLIB to be empty and pretend we couldn't +# find a lex program. Assumes _AC_PROG_LEX_OUTPUT_ROOT has already executed. +AC_DEFUN([_AC_PROG_LEX_LIBL_YYWRAP], +[AC_REQUIRE([_AC_PROG_LEX_TEST_YYWRAP])]dnl +[_AC_PROG_LEX_SEARCH_LIBL( + [ac_cv_lib_lex_yywrap], + [ac_prog_lex_conftest_l_yywrap], + [with yywrap])]) + + +# _AC_PROG_LEX_LIBL_COMPAT +# ------------------------ +# If it is necessary to link a simple lex scanner, that doesn't define +# yywrap, with a library, then set LEXLIB to that library. However, +# if that library is needed and not found, and a simple lex scanner that +# defines yywrap itself *doesn't* need the library, then set LEXLIB to +# the empty string. Assumes _AC_PROG_LEX_OUTPUT_ROOT has already executed. +AC_DEFUN([_AC_PROG_LEX_LIBL_COMPAT], +[AC_REQUIRE([_AC_PROG_LEX_TEST_YYWRAP])]dnl +[AC_REQUIRE([_AC_PROG_LEX_TEST_NOYYWRAP])]dnl +[_AC_PROG_LEX_SEARCH_LIBL( + [ac_cv_lib_lex_yywrap], + [ac_prog_lex_conftest_l_yywrap], + [with yywrap], + [_AC_PROG_LEX_SEARCH_LIBL( + [ac_cv_lib_lex_noyywrap], + [ac_prog_lex_conftest_l_noyywrap], + [yywrap not needed])])]) + +# _AC_PROG_LEX_YYTEXT_DECL +# ------------------------ +# Determine whether Lex declares yytext as a char * by default. +# Assumes that _AC_PROG_LEX_OUTPUT_ROOT and one of the +# _AC_PROG_LEX_LIBL macros have executed *immediately* prior. +# This test is done last so that we don't define YYTEXT_POINTER if +# any of the preceding tests gave up on lex. +AC_DEFUN([_AC_PROG_LEX_YYTEXT_DECL], +[AC_CACHE_CHECK([whether yytext is a pointer], [ac_cv_prog_lex_yytext_pointer], [# POSIX says lex can declare yytext either as a pointer or an array; the # default is implementation-dependent. Figure out which it is, since # not all implementations provide the %pointer and %array declarations. @@ -888,20 +953,15 @@ ac_cv_prog_lex_yytext_pointer=no AC_COMPILE_IFELSE([AC_LANG_DEFINES_PROVIDED [#define YYTEXT_POINTER 1 `cat $LEX_OUTPUT_ROOT.c`]], - [ac_cv_prog_lex_yytext_pointer=yes]) -]) -dnl -if test $ac_cv_prog_lex_yytext_pointer = yes; then - AC_DEFINE(YYTEXT_POINTER, 1, + [ac_cv_prog_lex_yytext_pointer=yes])]) +AS_IF([test $ac_cv_prog_lex_yytext_pointer = yes], + [AC_DEFINE(YYTEXT_POINTER, 1, [Define to 1 if 'lex' declares 'yytext' as a 'char *' by default, - not a 'char[]'.]) -fi -]) -rm -f conftest.l $LEX_OUTPUT_ROOT.c -])# _AC_PROG_LEX_YYTEXT_DECL - + not a 'char []'.])])]) -# Require AC_PROG_LEX in case some people were just calling this macro. +# Former name of an internal macro that did most, but not all, of what +# AC_PROG_LEX does. Map to AC_PROG_LEX in case any configure scripts +# were using it. AU_DEFUN([AC_DECL_YYTEXT], [AC_PROG_LEX]) diff --git a/tests/semantics.at b/tests/semantics.at index 3a0e946c..c24e9d6d 100644 --- a/tests/semantics.at +++ b/tests/semantics.at @@ -1252,51 +1252,35 @@ AT_CLEANUP # their paces as much as the autogenerated AT_CHECK_MACRO invocation # used to, back when AC_PROG_LEX took no arguments. -AT_CHECK_MACRO([AC_PROG_LEX with noyywrap], [AC_PROG_LEX([noyywrap])]) +AT_CHECK_MACRO([AC_PROG_LEX: noyywrap], + [AC_PROG_LEX([noyywrap])]) -AT_CHECK_MACRO([AC_PROG_LEX with yywrap], [AC_PROG_LEX([yywrap])], - [], [], [], -[# Skip this test on OSes where there is no -ll nor -lfl. +AT_CHECK_MACRO([AC_PROG_LEX: yywrap], + [AC_PROG_LEX([yywrap])]) -AT_DATA([configure.ac], -[[AC_INIT([lexlib-probe], [1]) -AC_PROG_CC -AC_SEARCH_LIBS([yywrap], [l fl], [], [AS_EXIT(77)]) -AC_OUTPUT -]]) -AT_CHECK_AUTOCONF -AT_CHECK_CONFIGURE -]) +AT_CHECK_MACRO([AC_PROG_LEX: compat mode], + [AC_PROG_LEX], [], [-Wno-obsolete]) +## ---------------------------------- ## +## Invalid arguments to AC_PROG_LEX. ## +## ---------------------------------- ## -AT_SETUP([AC_PROG_LEX in legacy mode]) +AT_SETUP([AC_PROG_LEX: invalid arguments and warnings]) AT_CONFIGURE_AC([[AC_PROG_LEX]]) -AT_CHECK_AUTOHEADER([], [ignore]) AT_CHECK_AUTOCONF([], [], [], [[configure.ac:4: warning: AC_PROG_LEX without either yywrap or noyywrap is obsolete -programs.m4: _AC_PROG_LEX is expanded from... +programs.m4: _AC_PROG_LEX_1 is expanded from... programs.m4: AC_PROG_LEX is expanded from... configure.ac:4: the top level ]]) -AT_CHECK_CONFIGURE - -AT_CLEANUP - - -## ---------------------------------- ## -## Invalid arguments to AC_PROG_LEX. ## -## ---------------------------------- ## - -AT_SETUP([Invalid arguments to AC_PROG_LEX]) - AT_CONFIGURE_AC( [[AC_PROG_LEX([nonsense]) ]]) AT_CHECK_AUTOCONF([], [1], [], [[configure.ac:4: error: AC_PROG_LEX: unrecognized argument: nonsense -programs.m4: _AC_PROG_LEX is expanded from... +programs.m4: _AC_PROG_LEX_1 is expanded from... programs.m4: AC_PROG_LEX is expanded from... configure.ac:4: the top level autom4te: error: m4 failed with exit status: 1 @@ -1317,7 +1301,7 @@ AT_CONFIGURE_AC( ]]) AT_CHECK_AUTOCONF([], [1], [], [[configure.ac:4: error: AC_PROG_LEX: yywrap and noyywrap are mutually exclusive -programs.m4: _AC_PROG_LEX is expanded from... +programs.m4: _AC_PROG_LEX_1 is expanded from... programs.m4: AC_PROG_LEX is expanded from... configure.ac:4: the top level autom4te: error: m4 failed with exit status: 1 @@ -1328,7 +1312,7 @@ AT_CONFIGURE_AC( ]]) AT_CHECK_AUTOCONF([], [1], [], [[configure.ac:4: error: AC_PROG_LEX: yywrap and noyywrap are mutually exclusive -programs.m4: _AC_PROG_LEX is expanded from... +programs.m4: _AC_PROG_LEX_1 is expanded from... programs.m4: AC_PROG_LEX is expanded from... configure.ac:4: the top level autom4te: error: m4 failed with exit status: 1 @@ -1339,7 +1323,7 @@ AT_CONFIGURE_AC( ]]) AT_CHECK_AUTOCONF([], [1], [], [[configure.ac:4: error: AC_PROG_LEX: unrecognized argument: yywrap nonsense -programs.m4: _AC_PROG_LEX is expanded from... +programs.m4: _AC_PROG_LEX_1 is expanded from... programs.m4: AC_PROG_LEX is expanded from... configure.ac:4: the top level autom4te: error: m4 failed with exit status: 1 @@ -1350,7 +1334,7 @@ AT_CONFIGURE_AC( ]]) AT_CHECK_AUTOCONF([], [1], [], [[configure.ac:4: error: AC_PROG_LEX: unrecognized argument: nonsense noyywrap -programs.m4: _AC_PROG_LEX is expanded from... +programs.m4: _AC_PROG_LEX_1 is expanded from... programs.m4: AC_PROG_LEX is expanded from... configure.ac:4: the top level autom4te: error: m4 failed with exit status: 1 @@ -1371,7 +1355,7 @@ AC_PROG_LEX([noyywrap]) ]]) AT_CHECK_AUTOCONF([], [0], [], [[configure.ac:5: warning: AC_PROG_LEX used twice with mismatched options -programs.m4: _AC_PROG_LEX is expanded from... +programs.m4: _AC_PROG_LEX_1 is expanded from... programs.m4: AC_PROG_LEX is expanded from... configure.ac:5: the top level ]])