Bootstrap libltdl to fix libtool v2.4 + automake v1.17 (#1900)

Bootstrap libltdl to fix libtool v2.4 + automake v1.17 build

    gmake[3]: Entering directory .../libltdl
    .../cfgaux/missing: line 85: aclocal-1.16: command not found
    gmake[3]: *** [Makefile:561: .././../libltdl/aclocal.m4]
    Error 127

During bootstrap.sh run, libtoolize copies prepackaged
configure and Makefile.in files into our libltdl directory:

* libltdl/configure from libtool v2.4 has aclocal version set
  to 1.16;
* libltdl/Makefile.in from libtool v2.4 uses
  configure-set aclocal version to build aclocal.m4

Thus, libltdl/Makefile (generated from libltdl/Makefile.in
above) runs aclocal-1.16 if "make" needs to build
libltdl/aclocal.m4.

Normally, "make" does not need to build libltdl/aclocal.m4
because that file was created by libtoolize. However, libtool
v2.4 is packaged with (generated by packaging folks)
libltdl/Makefile.in that makes libltdl/aclocal.m4 target
dependent on files in libltld/../m4 directory. Squid does not
have that ./m4 directory, so "make" attempts to re-generate
libltdl/aclocal.m4. When it does, it uses aclocal-1.16.

Our bootstrap.sh generated new ./configure but preserved
copied libltdl/configure with its aclocal version set to
1.16. In other words, our bootstrap.sh did not bootstrap
libltdl sub-project. In build environments without
aclocal-1.16, Squid build failed.

Several solutions or workarounds have been tried or
considered:

* Adjust bootstrap.sh to bootstrap libltdl (this change).
  2008 attempt to do that was reverted in commit bfd6b6a9 with
  "better to fix libtool installation" rationale. Another
  potential argument against this option is that packages
  should be bootstrapped by their distributors, not "users". We
  are not distributing libtool, but this is a gray area because
  we do distribute files that libtoolize creates. Finally,
  libtool itself does not provide a bootstrapping script and
  does not explicitly recommend bootstrapping in documentation.

* "Fix libtool installation". We failed to find a good way to
  do that on MacOS (without building and installing newer
  libtool from sources).

* Place m4 files where libtool v2.4 expects to find them.
  That change fixes MacOS builds that use automake v1.17, but
  breaks Gentoo builds because Gentoo libtool installs a buggy
  libltdl/Makefile.in that must be regenerated by automake
  before it can work. Fixing m4 files location prevents that
  regeneration.

We picked the first option despite its drawbacks because the
third option did not work on Gentoo, and asking Squid
developers to install libtool from sources (i.e. the second
option) felt like a greater evil.

This old problem was exposed by recently introduced CI MacOS
tests that started to fail when MacOS brew updated automake
package from v1.16 without the corresponding libtoolize
package changes.

Also work around what seems to be a libtool packaging bug
affecting MacOS/brew environments, including GitHub Actions
runners we use for CI:

    libtool  (2.4.7) : glibtool libtool path :
    /opt/homebrew/bin Bootstrapping glibtoolize:   error:
    creating 'libltdl/configure.ac' ... failed glibtoolize:
    error: creating 'libltdl/configure' ... failed glibtoolize
    failed

That workaround will be removed after libtool package is
fixed.

Also removed a single-iteration "for dir" loop with several
stale hacks from bootstrap.sh: With only two directories to
bootstrap and with a directory-specific mkdir command, source
comments, and progress messages, it is best to unroll that
loop.

----
Backport of PR #1877

diff --git a/bootstrap.sh b/bootstrap.sh
index bf9a000fd45b85a9fb6708c40593d0cf1920e122..96d6e4b16d858f240ee15203819cc981186a8e72 100755 (executable)
--- a/bootstrap.sh
+++ b/bootstrap.sh
@@ -116,38 +116,47 @@ echo "autoconf ($acversion) : autoconf$acver"
 echo "libtool  ($ltversion) : ${LIBTOOL_BIN}${ltver}"
 echo "libtool path : $ltpath"
 
-for dir in \
-       ""
-do
-    if [ -z "$dir" ] || [ -d $dir ]; then
-       if (
-       echo "Bootstrapping $dir"
-       cd ./$dir
-       if [ -n "$dir" ] && [ -f bootstrap.sh ]; then
-           ./bootstrap.sh
-       elif [ ! -f $dir/configure ]; then
-           # Make sure cfgaux exists
-           mkdir -p cfgaux
-
-            if test -n "$ltpath"; then
-              acincludeflag="-I $ltpath/../share/aclocal"
-            else
-              acincludeflag=""
-            fi
-
-           # Bootstrap the autotool subsystems
-           bootstrap aclocal$amver $acincludeflag
-           bootstrap autoheader$acver
-           bootstrap_libtoolize ${LIBTOOL_BIN}ize${ltver}
-           bootstrap automake$amver --foreign --add-missing --copy -f
-           bootstrap autoconf$acver --force
-       fi ); then
-           : # OK
-       else
-           exit 1
-       fi
+if test -n "$ltpath"; then
+    acincludeflag="-I $ltpath/../share/aclocal"
+else
+    acincludeflag=""
+fi
+
+# bootstrap primary or subproject sources
+bootstrap_dir() {
+    dir="$1"
+    cd $dir || exit $?
+
+    bootstrap aclocal$amver $acincludeflag
+    bootstrap autoheader$acver
+
+    # Do not libtoolize ltdl
+    if grep -q '^LTDL_INIT' configure.ac
+    then
+        bootstrap_libtoolize ${LIBTOOL_BIN}ize${ltver}
     fi
-done
+
+    bootstrap automake$amver --foreign --add-missing --copy --force
+    bootstrap autoconf$acver --force
+
+    cd - > /dev/null
+}
+
+echo "Bootstrapping primary Squid sources"
+mkdir -p cfgaux || exit $?
+bootstrap_dir .
+
+# The above bootstrap_libtoolize step creates or updates libltdl. It copies
+# (with minor adjustments) configure.ac and configure, Makefile.am and
+# Makefile.in from libtool installation, but does not regenerate copied
+# configure from copied configure.ac and copied Makefile.in from Makefile.am.
+# We get libltdl/configure and libltdl/Makefile.in as they were bootstrapped
+# by libtool authors or package maintainers. Low-level idiosyncrasies in those
+# libtool files result in mismatches between copied code expectations and
+# Squid sub-project environment, leading to occasional build failures that
+# this bootstrapping addresses.
+echo "Bootstrapping libltdl sub-project"
+bootstrap_dir libltdl
 
 # Make a copy of SPONSORS we can package
 if test -f SPONSORS.list; then