ci: Check and confirm Vim feature flags exist before testing

Vim tests for features such as python3 relies on checking the feature
flag exists by doing `has('python3')`. However, if the feature itself is
broken and the flag returns 0, the relevant tests will simply silently
get ignored and CI will passed erroneously. As a preventive measure, as
basic checks to make sure certain feature flags are correct as a basic
smoke test.

Currently only checking two types of feature flags:

1. Features that depend on system packages being installed properly
   (e.g. sodium) and could be erroneously dropped if the CI environment
   changed or a bug exists in the configure script.
2. Scripting languages. When in dynamic mode, these feature flags (e.g.
   "ruby", "python3") will return 0 when the lib cannot be found or the
   code has an initialization bug. This happened in #16964 where CI
   still passed despite Python 3 being broken.

closes: #16998

Signed-off-by: Yee Cheng Chin <ychin.git@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 19eabc477d19530285443bfce1cff118f218387c..2b4a8ccc9c9f7aa39c27cc54b45e1d3f56f1dafa 100644 (file)
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -266,6 +266,12 @@ jobs:
           "${SRCDIR}"/vim --version
           "${SRCDIR}"/vim -u NONE -i NONE --not-a-term -esNX -V1 -S ci/if_ver-1.vim -c quit
           "${SRCDIR}"/vim -u NONE -i NONE --not-a-term -esNX -V1 -S ci/if_ver-2.vim -c quit
+          if ${{ matrix.features == 'huge' }}; then
+            # Also check that optional and dynamic features are configured and working
+            "${SRCDIR}"/vim -u NONE -i NONE --not-a-term -esNX -V1 \
+              -c "let g:required=['gettext', 'sodium', 'sound', 'perl', 'python3', 'lua', 'ruby', 'tcl']" \
+              -S ci/if_feat_check.vim -c quit
+          fi
 
       - name: Test
         if: matrix.architecture != 'arm64'
@@ -392,6 +398,12 @@ jobs:
           "${SRCDIR}"/vim --version
           "${SRCDIR}"/vim -u NONE -i NONE --not-a-term -esNX -V1 -S ci/if_ver-1.vim -c quit
           "${SRCDIR}"/vim -u NONE -i NONE --not-a-term -esNX -V1 -S ci/if_ver-2.vim -c quit
+          if ${{ matrix.features == 'huge' }}; then
+            # Also check that optional and dynamic features are configured and working
+            "${SRCDIR}"/vim -u NONE -i NONE --not-a-term -esNX -V1 \
+              -c "let g:required=['sound', 'perl', 'python3', 'lua', 'ruby', 'tcl']" \
+              -S ci/if_feat_check.vim -c quit
+          fi
 
       - name: Install packages for testing
         run: |
@@ -663,6 +675,11 @@ jobs:
             src\vim --version || exit 1
             src\vim -u NONE -i NONE --not-a-term -esNX -V1 -S ci/if_ver-1.vim -c quit
             src\vim -u NONE -i NONE --not-a-term -esNX -V1 -S ci/if_ver-2.vim -c quit
+            if "${{ matrix.features }}"=="HUGE" (
+              src\vim -u NONE -i NONE --not-a-term -esNX -V1 ^
+                -c "let g:required=['gettext', 'sodium', 'sound', 'python3', 'lua']" ^
+                -S ci/if_feat_check.vim -c quit
+            )
           )
 
       #- name: Prepare Artifact

diff --git a/Filelist b/Filelist
index 7302788fea3e787e351caae82ec479da22d4a78f..4067df5081ed5a83e30e1e81e8a8a0f1e6c63b56 100644 (file)
--- a/Filelist
+++ b/Filelist
@@ -23,6 +23,7 @@ SRC_ALL =     \
                ci/appveyor.bat \
                ci/config.mk*.sed \
                ci/if_ver*.vim \
+               ci/if_feat_check.vim \
                ci/setup-xvfb.sh \
                ci/remove_snap.sh \
                src/Make_all.mak \

diff --git a/ci/if_feat_check.vim b/ci/if_feat_check.vim
new file mode 100644 (file)
index 0000000..f846f7c
--- /dev/null
+++ b/ci/if_feat_check.vim
@@ -0,0 +1,15 @@
+if 1 " This prevents it from being run in tiny versions
+  " Check for required features
+  if exists("g:required")
+    for feature in g:required
+      if !has(feature)
+        echo "Error: Feature '" .. feature .. "' not found"
+        echo ''
+        cquit
+      endif
+    endfor
+    echo "\nChecked features: " .. string(g:required)
+    echo ''
+  endif
+endif
+" vim: sts=2 sw=2 et

diff --git a/ci/if_ver-1.vim b/ci/if_ver-1.vim
index adc40a7cb0b5ec956ab1f761ec81ed3482b42d5b..397c05a456a1069fdbcb2bf9fbc9ac011b80f79a 100644 (file)
--- a/ci/if_ver-1.vim
+++ b/ci/if_ver-1.vim
@@ -1,6 +1,6 @@
-" Print all interface versions for Ubuntu. Part 1.
+" Print all interface versions. Part 1.
 
-if 1
+if 1 " This prevents it from being run in tiny versions
   execute 'source' expand('<sfile>:h') .. '/if_ver-cmd.vim'
 
   echo "*** Interface versions ***\n"

diff --git a/ci/if_ver-2.vim b/ci/if_ver-2.vim
index 83e80e48fff8c59a097a43d1056e641db5801a0f..f717ec1e562c78d15869dfc81466690ad8d59250 100644 (file)
--- a/ci/if_ver-2.vim
+++ b/ci/if_ver-2.vim
@@ -1,6 +1,8 @@
-" Print py3 interface versions for Ubuntu. Part 2.
+" Print py3 interface versions. Part 2.
+" This is done separately from part 1 because Vim cannot concurrently load
+" Python 2 and 3 together.
 
-if 1
+if 1 " This prevents it from being run in tiny versions
   execute 'source' expand('<sfile>:h') .. '/if_ver-cmd.vim'
 
   echo 'Python 3:'