From: Viktor Szakats Date: Wed, 11 Sep 2024 10:41:57 +0000 (+0200) Subject: CI: move Azure jobs to GHA, fix fallouts, sshserver, runtests tweaks X-Git-Tag: curl-8_10_1~7 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=e53523fef07894991c69d907a7c7794c7ada4ff4;p=thirdparty%2Fcurl.git CI: move Azure jobs to GHA, fix fallouts, sshserver, runtests tweaks CI: - GHA/windows: enable OpenSSH server, SysInternals `handle`, `impacket`. Skip `impacket` on MSYS2 due to install failure. Skip OpenSSH server for old/standalone mingw-w64 (building curl without SSH there.) - GHA/windows: make test tool installs a separate step. - GHA/cygwin: enable OpenSSH server. Skip `impacket`: it's compiling for 7 minutes then breaks. Skip `stunnel` due to sluggish test run performance. (This update is unrelated to Azure jobs.) - GHA/linux: migrate Linux jobs from Azure CI. - GHA/linux: migrate scanbuild job from Azure CI. - GHA/linux: enable libssh2 in a job. Also enable valgrind. - CI/windows: enable SSPI in two jobs. - CI/windows: disable zlib in one more job. - CI/windows: improve `if` condition checking GnuTLS. - CI/windows: ignore SFTP/SCP tests as necessary. - universally ignore SCP tests, they fail everywhere. - ignore test 612. - ignore test 613 616 618 with MSYS2 mingw-w64. - ignore test 614 with libssh. - ignore all SFTP with MSYS2 native. - ignore all SFTP with vcpkg with `libssh2[core,zlib]`. - ignore a couple of SFTP tests with MSYS2 mingw-w64. (This matches settings on Azure CI.) - GHA/windows: ignore failing 1451 'Basic SMB request' test for old mingw-w64 7.3.0 (but not for 9.5.0!): ``` 2024-09-11 21:45:59,738 ERROR smbComNegotiate: b'NT LM 0.12\x00' is not in list [...] curl: (7) Could not connect to server [...] FAIL 1451: 'Basic SMB request' SMB ``` Ref: https://github.com/curl/curl/actions/runs/10816280746/job/30007130770#step:13:3546 - CI/Azure: delete, now moved to GHA. sshserver: - sshserver: fix permissions for SSH host key on Windows, allowing sshd to launch. - sshserver: fix initializing config paths with Cygwin/MSYS2 sshd. - sshserver: fix initializing config paths in parallel builds. - sshserver: delete redundant `DenyUsers`. This also opens the way to allow multiple usernames. - sshserver: fix `AllowUsers` for GHA/windows, by allowing the domainless username again. Follow-up to 3ee7c676ec8ef32e3a9cb4d0fb2929a8335d58db #5721 - sshserver: fix `Deprecated option` warnings in `sshd_config`: ``` D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 7: Deprecated option AuthorizedKeysFile2 D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 25: Deprecated option KeyRegenerationInterval D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 35: Deprecated option RhostsRSAAuthentication D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 36: Deprecated option RSAAuthentication D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 37: Deprecated option ServerKeyBits D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 41: Deprecated option UseLogin ``` - sshserver: fix `Deprecated option` warnings in `ssh_config` with Cygwin/MSYS2. - sshserver: fix dumping config files due to the filenames missing their full paths. - sshserver: add workaround to make `logmsg` messages visible. Before this patch they only went to a file and never shown. runtests: - runtests: log details when these Windows commands are called: `handle`, `taskkill`, `tasklist`. - runtests: add documentation links to Windows tools: `handle`, `taskkill`, `tasklist`, `icacls`. - runtests: add `-t` (kill whole tree) option to `taskkill` in `servers.pm`, syncing it with the other `taskkill` call. Follow-up to bc72a78a11764558639131d51fa8e7c81ee1e113 #14488 - runtests: show warning if Sysinternals `handle` tool is missing. - runtests: drop Windows XP Home compatibility `tskill` call. The call was made on all Windows versions. It's possibly overkill to do this, because XP Home is probably rarely used for running curl tests these days. In case it's needed, it'd be better to put it under an explicit option. - runtests: show Perl version and path. Fix/silence fallouts: - unit2603: fix building with disabled HTTP support. - unit2604: silence `-Woverlength-strings` warnings in C89 mode. - test437, test1614: fix to pass with no-IPv6 builds. Closes #14859 --- diff --git a/.azure-pipelines.yml b/.azure-pipelines.yml deleted file mode 100644 index f4a7039f6e..0000000000 --- a/.azure-pipelines.yml +++ /dev/null @@ -1,247 +0,0 @@ -#*************************************************************************** -# _ _ ____ _ -# Project ___| | | | _ \| | -# / __| | | | |_) | | -# | (__| |_| | _ <| |___ -# \___|\___/|_| \_\_____| -# -# Copyright (C) Daniel Stenberg, , et al. -# -# This software is licensed as described in the file COPYING, which -# you should have received as part of this distribution. The terms -# are also available at https://curl.se/docs/copyright.html. -# -# You may opt to use, copy, modify, merge, publish, distribute and/or sell -# copies of the Software, and permit persons to whom the Software is -# furnished to do so, under the terms of the COPYING file. -# -# This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY -# KIND, either express or implied. -# -# SPDX-License-Identifier: curl -# -########################################################################### -# https://dev.azure.com/daniel0244/curl/_build?view=runs -# -# Azure Pipelines configuration: -# https://aka.ms/yaml - -trigger: - branches: - include: - - 'master' - - '*/ci' - paths: - exclude: - - '.circleci/*' - - '.github/*' - - '.github/workflows/*' - - 'appveyor.*' - - 'packages/*' - - 'plan9/*' - -pr: - branches: - include: - - 'master' - paths: - exclude: - - '.circleci/*' - - '.github/*' - - '.github/workflows/*' - - 'appveyor.*' - - 'packages/*' - - 'plan9/*' - -variables: - MAKEFLAGS: '-j 2' - -stages: - - ########################################## - ### Linux jobs first - ########################################## - - - stage: linux - dependsOn: [] - jobs: - - job: ubuntu - # define defaults to make sure variables are always expanded/replaced - variables: - install: '' - configure: '' - tests: '!433' - timeoutInMinutes: 60 - pool: - vmImage: 'ubuntu-latest' - strategy: - matrix: - default: - name: default - install: - configure: --enable-debug --with-openssl - disable_ipv6: - name: w/o IPv6 - configure: --disable-ipv6 --with-openssl - disable_http_smtp_imap: - name: w/o HTTP/SMTP/IMAP - configure: --disable-http --disable-smtp --disable-imap --without-ssl - disable_threadres: - name: sync resolver - configure: --disable-threaded-resolver --with-openssl - https_only: - name: HTTPS only - configure: --disable-dict --disable-file --disable-ftp --disable-gopher --disable-imap --disable-ldap --disable-pop3 --disable-rtmp --disable-rtsp --disable-scp --disable-sftp --disable-smb --disable-smtp --disable-telnet --disable-tftp --with-openssl - torture: - name: torture - install: libnghttp2-dev - configure: --enable-debug --disable-shared --disable-threaded-resolver --with-openssl - tests: -n -t --shallow=25 !FTP - steps: - - script: sudo apt-get update && sudo apt-get install -y stunnel4 python3-impacket libzstd-dev libbrotli-dev libpsl-dev $(install) - displayName: 'apt install' - retryCountOnTaskFailure: 3 - - - script: autoreconf -fi && ./configure --disable-dependency-tracking --enable-warnings --enable-werror $(configure) - displayName: 'configure $(name)' - - - script: make V=1 && make V=1 examples && cd tests && make V=1 - displayName: 'compile' - - - script: make V=1 test-ci - displayName: 'test' - env: - AZURE_ACCESS_TOKEN: "$(System.AccessToken)" - TFLAGS: "-j0 -ac /usr/bin/curl -r $(tests)" - - - stage: scanbuild - dependsOn: [] - jobs: - - job: ubuntu - timeoutInMinutes: 30 - pool: - vmImage: 'ubuntu-latest' - steps: - - script: sudo apt-get update && sudo apt-get install -y clang-tools clang libssl-dev libssh2-1-dev libpsl-dev libbrotli-dev libzstd-dev - displayName: 'apt install' - retryCountOnTaskFailure: 3 - - - script: autoreconf -fi - displayName: 'autoreconf' - - - script: scan-build ./configure --disable-dependency-tracking --enable-debug --enable-werror --with-openssl --with-libssh2 - displayName: 'configure' - env: - CC: "clang" - CCX: "clang++" - - - script: scan-build --status-bugs make - displayName: 'make' - - - script: scan-build --status-bugs make examples - displayName: 'make examples' - - ########################################## - ### Windows jobs below - ########################################## - - - stage: windows - dependsOn: [] - variables: - agent.preferPowerShellOnContainers: true - jobs: - - job: msys2 - # define defaults to make sure variables are always expanded/replaced - variables: - container_img: '' - container_cmd: '' - configure: '' - tests: '' - timeoutInMinutes: 120 - pool: - vmImage: 'windows-2019' - strategy: - matrix: - mingw32_openssl: - name: 32-bit OpenSSL/libssh2 - container_img: ghcr.io/mback2k/curl-docker-winbuildenv/msys2-mingw32:ltsc2019 - container_cmd: C:\msys64\usr\bin\sh - prepare: pacman -S --needed --noconfirm --noprogressbar libssh2-devel mingw-w64-i686-libssh2 - configure: --host=i686-w64-mingw32 --build=i686-w64-mingw32 --prefix=/mingw32 --enable-debug --enable-werror --with-libssh2 --with-openssl --without-libpsl - tests: "~571" - mingw64_openssl: - name: 64-bit OpenSSL/libssh2 - container_img: ghcr.io/mback2k/curl-docker-winbuildenv/msys2-mingw64:ltsc2019 - container_cmd: C:\msys64\usr\bin\sh - prepare: pacman -S --needed --noconfirm --noprogressbar libssh2-devel mingw-w64-x86_64-libssh2 - configure: --host=x86_64-w64-mingw32 --build=x86_64-w64-mingw32 --prefix=/mingw64 --enable-debug --enable-werror --with-libssh2 --with-openssl --without-libpsl - tests: "~571" - mingw64_libssh: - name: 64-bit OpenSSL/libssh - container_img: ghcr.io/mback2k/curl-docker-winbuildenv/msys2-mingw64:ltsc2019 - container_cmd: C:\msys64\usr\bin\sh - prepare: pacman -S --needed --noconfirm --noprogressbar libssh-devel mingw-w64-x86_64-libssh - configure: --host=x86_64-w64-mingw32 --build=x86_64-w64-mingw32 --prefix=/mingw64 --enable-debug --enable-werror --with-libssh --with-openssl --without-libpsl - tests: "~571 ~614" - mingw32: - name: 32-bit w/o zlib - container_img: ghcr.io/mback2k/curl-docker-winbuildenv/msys2-mingw32:ltsc2019 - container_cmd: C:\msys64\usr\bin\sh - configure: --host=i686-w64-mingw32 --build=i686-w64-mingw32 --prefix=/mingw32 --enable-debug --enable-werror --without-zlib --without-ssl --without-libpsl - tests: "!203 !1143" - mingw64: - name: 64-bit w/o zlib - container_img: ghcr.io/mback2k/curl-docker-winbuildenv/msys2-mingw64:ltsc2019 - container_cmd: C:\msys64\usr\bin\sh - configure: --host=x86_64-w64-mingw32 --build=x86_64-w64-mingw32 --prefix=/mingw64 --enable-debug --enable-werror --without-zlib --without-ssl --without-libpsl - tests: "!203 !1143" - mingw32_schannel: - name: 32-bit Schannel/SSPI/WinIDN/libssh2 - container_img: ghcr.io/mback2k/curl-docker-winbuildenv/msys2-mingw32:ltsc2019 - container_cmd: C:\msys64\usr\bin\sh - prepare: pacman -S --needed --noconfirm --noprogressbar libssh2-devel mingw-w64-i686-libssh2 - configure: --host=i686-w64-mingw32 --build=i686-w64-mingw32 --prefix=/mingw32 --enable-debug --enable-werror --enable-sspi --with-schannel --with-winidn --with-libssh2 --without-libpsl - tests: "~571" - mingw64_schannel: - name: 64-bit Schannel/SSPI/WinIDN/libssh2 - container_img: ghcr.io/mback2k/curl-docker-winbuildenv/msys2-mingw64:ltsc2019 - container_cmd: C:\msys64\usr\bin\sh - prepare: pacman -S --needed --noconfirm --noprogressbar libssh2-devel mingw-w64-x86_64-libssh2 - configure: --host=x86_64-w64-mingw32 --build=x86_64-w64-mingw32 --prefix=/mingw64 --enable-debug --enable-werror --enable-sspi --with-schannel --with-winidn --with-libssh2 --without-libpsl - tests: "~571" - mingw32_schannel_nozlib: - name: 32-bit Schannel/SSPI/WinIDN w/o zlib - container_img: ghcr.io/mback2k/curl-docker-winbuildenv/msys2-mingw32:ltsc2019 - container_cmd: C:\msys64\usr\bin\sh - configure: --host=i686-w64-mingw32 --build=i686-w64-mingw32 --prefix=/mingw32 --enable-debug --enable-werror --enable-sspi --with-schannel --with-winidn --without-zlib --without-libpsl - tests: "!203 !1143" - mingw64_schannel_nozlib: - name: 64-bit Schannel/SSPI/WinIDN w/o zlib - container_img: ghcr.io/mback2k/curl-docker-winbuildenv/msys2-mingw64:ltsc2019 - container_cmd: C:\msys64\usr\bin\sh - configure: --host=x86_64-w64-mingw32 --build=x86_64-w64-mingw32 --prefix=/mingw64 --enable-debug --enable-werror --enable-sspi --with-schannel --with-winidn --without-zlib --without-libpsl - tests: "!203 !1143" - container: - image: $(container_img) - env: - MSYS2_PATH_TYPE: inherit - steps: - - script: $(container_cmd) -l -c "cd $(echo '%cd%') && $(prepare)" - displayName: 'prepare' - condition: variables.prepare - retryCountOnTaskFailure: 3 - - - script: $(container_cmd) -l -c "cd $(echo '%cd%') && autoreconf -fi && ./configure --disable-dependency-tracking $(configure)" - displayName: 'configure $(name)' - - - script: $(container_cmd) -l -c "cd $(echo '%cd%') && make V=1 && make V=1 examples && cd tests && make V=1" - displayName: 'compile' - - - script: $(container_cmd) -l -c "cd $(echo '%cd%') && make V=1 install && PATH=/usr/bin:/bin find . -type f -path '*/.libs/*.exe' -print -execdir mv -t .. {} \;" - displayName: 'install' - - - script: $(container_cmd) -l -c "cd $(echo '%cd%') && make V=1 test-ci" - displayName: 'test' - env: - AZURE_ACCESS_TOKEN: "$(System.AccessToken)" - TFLAGS: "-j0 -ac /usr/bin/curl.exe !SCP ~612 $(tests)" diff --git a/.github/labeler.yml b/.github/labeler.yml index 57e87aa917..a2c0247776 100644 --- a/.github/labeler.yml +++ b/.github/labeler.yml @@ -86,7 +86,6 @@ CI: - all: - changed-files: - any-glob-to-any-file: - - '.azure-pipelines.yml' - '.circleci/**' - '.github/**' - 'appveyor.*' diff --git a/.github/workflows/awslc.yml b/.github/workflows/awslc.yml index 5531507bee..305ae210f0 100644 --- a/.github/workflows/awslc.yml +++ b/.github/workflows/awslc.yml @@ -11,7 +11,6 @@ name: Linux AWS-LC - '*/ci' paths-ignore: - '**/*.md' - - '.azure-pipelines.yml' - '.circleci/**' - 'appveyor.*' - 'packages/**' @@ -23,7 +22,6 @@ name: Linux AWS-LC - master paths-ignore: - '**/*.md' - - '.azure-pipelines.yml' - '.circleci/**' - 'appveyor.*' - 'packages/**' diff --git a/.github/workflows/checksrc.yml b/.github/workflows/checksrc.yml index 5da0728426..87868b6318 100644 --- a/.github/workflows/checksrc.yml +++ b/.github/workflows/checksrc.yml @@ -13,7 +13,6 @@ name: Source - '*/ci' paths-ignore: - '**/*.md' - - '.azure-pipelines.yml' - '.circleci/**' - 'appveyor.*' - 'plan9/**' @@ -24,7 +23,6 @@ name: Source - master paths-ignore: - '**/*.md' - - '.azure-pipelines.yml' - '.circleci/**' - 'appveyor.*' - 'plan9/**' diff --git a/.github/workflows/codeql-analysis.yml b/.github/workflows/codeql-analysis.yml index 2b8f595352..f5e7088b4a 100644 --- a/.github/workflows/codeql-analysis.yml +++ b/.github/workflows/codeql-analysis.yml @@ -11,7 +11,6 @@ name: CodeQL - '*/ci' paths-ignore: - '**/*.md' - - '.azure-pipelines.yml' - '.circleci/**' - 'appveyor.*' - 'docs/**' @@ -25,7 +24,6 @@ name: CodeQL - master paths-ignore: - '**/*.md' - - '.azure-pipelines.yml' - '.circleci/**' - 'appveyor.*' - 'docs/**' diff --git a/.github/workflows/curl-for-win.yml b/.github/workflows/curl-for-win.yml index d5ada3ec8e..78592905f9 100644 --- a/.github/workflows/curl-for-win.yml +++ b/.github/workflows/curl-for-win.yml @@ -11,7 +11,6 @@ name: curl-for-win - '*/ci' paths-ignore: - '**/*.md' - - '.azure-pipelines.yml' - '.circleci/**' - 'appveyor.*' - 'packages/**' @@ -23,7 +22,6 @@ name: curl-for-win - master paths-ignore: - '**/*.md' - - '.azure-pipelines.yml' - '.circleci/**' - 'appveyor.*' - 'packages/**' diff --git a/.github/workflows/cygwin.yml b/.github/workflows/cygwin.yml index 1a16f0aa77..1f1f250cd8 100644 --- a/.github/workflows/cygwin.yml +++ b/.github/workflows/cygwin.yml @@ -11,7 +11,6 @@ name: Cygwin - '*/ci' paths-ignore: - '**/*.md' - - '.azure-pipelines.yml' - '.circleci/**' - 'appveyor.*' - 'packages/**' @@ -23,7 +22,6 @@ name: Cygwin - master paths-ignore: - '**/*.md' - - '.azure-pipelines.yml' - '.circleci/**' - 'appveyor.*' - 'packages/**' @@ -60,9 +58,18 @@ jobs: - uses: cygwin/cygwin-install-action@006ad0b0946ca6d0a3ea2d4437677fa767392401 # v4 with: platform: ${{ matrix.platform }} - # https://cygwin.com/cgi-bin2/package-grep.cgi - packages: autoconf libtool ${{ matrix.build }} gcc-core gcc-g++ binutils make ninja libssl-devel zlib-devel libbrotli-devel libnghttp2-devel libpsl-devel libssh2-devel site: https://mirrors.kernel.org/sourceware/cygwin/ + # https://cygwin.com/cgi-bin2/package-grep.cgi + packages: >- + autoconf libtool gcc-core gcc-g++ binutils + ${{ matrix.build }} make ninja + openssh + libssl-devel + libssh2-devel + libpsl-devel + zlib-devel + libbrotli-devel + libnghttp2-devel - name: 'autotools autoreconf' if: ${{ matrix.build == 'automake' }} @@ -120,7 +127,7 @@ jobs: if: ${{ matrix.build == 'automake' && matrix.tflags != 'skipall' && matrix.tflags != 'skiprun' }} timeout-minutes: 15 run: | - export TFLAGS='-j8 ${{ matrix.tflags }}' + export TFLAGS='-j8 ${{ matrix.tflags }} ~615' if [ -x "$(cygpath "${SYSTEMROOT}/System32/curl.exe")" ]; then TFLAGS+=" -ac $(cygpath "${SYSTEMROOT}/System32/curl.exe")" fi @@ -173,7 +180,7 @@ jobs: if: ${{ matrix.build == 'cmake' && matrix.tflags != 'skipall' && matrix.tflags != 'skiprun' }} timeout-minutes: 15 run: | - export TFLAGS='-j8 ${{ matrix.tflags }}' + export TFLAGS='-j8 ${{ matrix.tflags }} ~615' if [ -x "$(cygpath "${SYSTEMROOT}/System32/curl.exe")" ]; then TFLAGS+=" -ac $(cygpath "${SYSTEMROOT}/System32/curl.exe")" fi diff --git a/.github/workflows/fuzz.yml b/.github/workflows/fuzz.yml index 29a63d1964..fc647d887b 100644 --- a/.github/workflows/fuzz.yml +++ b/.github/workflows/fuzz.yml @@ -12,7 +12,6 @@ name: Fuzzer paths-ignore: - '**/*.md' - '**/CMakeLists.txt' - - '.azure-pipelines.yml' - '.circleci/**' - 'appveyor.*' - 'CMake/**' @@ -27,7 +26,6 @@ name: Fuzzer paths-ignore: - '**/*.md' - '**/CMakeLists.txt' - - '.azure-pipelines.yml' - '.circleci/**' - 'appveyor.*' - 'CMake/**' diff --git a/.github/workflows/http3-linux.yml b/.github/workflows/http3-linux.yml index 85dd8bde9f..c3b5c31efd 100644 --- a/.github/workflows/http3-linux.yml +++ b/.github/workflows/http3-linux.yml @@ -12,7 +12,6 @@ name: http3-linux paths-ignore: - '**/*.md' - '**/CMakeLists.txt' - - '.azure-pipelines.yml' - '.circleci/**' - 'appveyor.*' - 'CMake/**' @@ -26,7 +25,6 @@ name: http3-linux paths-ignore: - '**/*.md' - '**/CMakeLists.txt' - - '.azure-pipelines.yml' - '.circleci/**' - 'appveyor.*' - 'CMake/**' diff --git a/.github/workflows/linux-old.yml b/.github/workflows/linux-old.yml index c06b7d27b3..17c99a4426 100644 --- a/.github/workflows/linux-old.yml +++ b/.github/workflows/linux-old.yml @@ -25,7 +25,6 @@ name: Old Linux - '*/ci' paths-ignore: - '**/*.md' - - '.azure-pipelines.yml' - '.circleci/**' - 'appveyor.*' - 'packages/**' @@ -37,7 +36,6 @@ name: Old Linux - master paths-ignore: - '**/*.md' - - '.azure-pipelines.yml' - '.circleci/**' - 'appveyor.*' - 'packages/**' diff --git a/.github/workflows/linux.yml b/.github/workflows/linux.yml index f11a30c5a4..27ca2b5786 100644 --- a/.github/workflows/linux.yml +++ b/.github/workflows/linux.yml @@ -11,7 +11,6 @@ name: Linux - '*/ci' paths-ignore: - '**/*.md' - - '.azure-pipelines.yml' - '.circleci/**' - 'appveyor.*' - 'packages/**' @@ -23,7 +22,6 @@ name: Linux - master paths-ignore: - '**/*.md' - - '.azure-pipelines.yml' - '.circleci/**' - 'appveyor.*' - 'packages/**' @@ -114,6 +112,17 @@ jobs: configure: LDFLAGS="-Wl,-rpath,$HOME/msh3/lib -Wl,-rpath,$HOME/quictls/lib" --with-msh3=$HOME/msh3 --with-openssl=$HOME/quictls --enable-debug singleuse: --unit + - name: openssl default + install_steps: pytest + configure: --with-openssl --enable-debug + singleuse: --unit + + - name: openssl libssh2 sync-resolver valgrind + install_packages: zlib1g-dev libssh2-1-dev valgrind + install_steps: pytest + configure: --with-openssl --enable-debug --disable-threaded-resolver --with-libssh2 + singleuse: --unit + - name: openssl3 valgrind install_packages: zlib1g-dev valgrind install_steps: gcc-11 openssl3 pytest @@ -138,6 +147,34 @@ jobs: generate: -DOPENSSL_ROOT_DIR=$HOME/openssl3 -DCURL_USE_GSSAPI=ON -DENABLE_DEBUG=ON -DENABLE_WEBSOCKETS=ON singleuse: --unit + - name: openssl3 !ipv6 + install_steps: gcc-11 openssl3 + configure: LDFLAGS="-Wl,-rpath,$HOME/openssl3/lib" --with-openssl=$HOME/openssl3 --enable-debug --disable-ipv6 + singleuse: --unit + + - name: openssl3 https-only + install_steps: gcc-11 openssl3 + configure: > + LDFLAGS="-Wl,-rpath,$HOME/openssl3/lib" --with-openssl=$HOME/openssl3 --enable-debug + --disable-dict --disable-gopher --disable-ldap --disable-telnet + --disable-imap --disable-pop3 --disable-smtp + --disable-rtmp --disable-rtsp + --disable-scp --disable-sftp --disable-tftp --disable-ftp --disable-file --disable-smb + singleuse: --unit + + - name: '!ssl !http !smtp !imap' + install_steps: gcc-11 + configure: --without-ssl --enable-debug --disable-http --disable-smtp --disable-imap + singleuse: --unit + + - name: scanbuild + install_packages: clang-tools clang libssl-dev libssh2-1-dev + install_steps: skipall + configure: --with-openssl --enable-debug --with-libssh2 + configure-prefix: CC=clang scan-build + make-prefix: scan-build --status-bugs + singleuse: --unit + - name: address-sanitizer install_packages: zlib1g-dev libssh2-1-dev clang libssl-dev libubsan1 libasan8 libtsan2 install_steps: pytest @@ -459,7 +496,7 @@ jobs: if: ${{ matrix.build.configure }} name: 'autoreconf' - - run: ./configure --disable-dependency-tracking --enable-warnings --enable-werror ${{ matrix.build.configure }} + - run: ${{ matrix.build.configure-prefix }} ./configure --disable-dependency-tracking --enable-warnings --enable-werror ${{ matrix.build.configure }} if: ${{ matrix.build.configure }} name: 'configure (autotools)' @@ -482,7 +519,7 @@ jobs: cat tests/config || true cat tests/http/config.ini || true - - run: make V=1 VERBOSE=1 + - run: ${{ matrix.build.make-prefix }} make V=1 VERBOSE=1 name: 'make' - run: | @@ -498,19 +535,20 @@ jobs: - run: ./src/curl -V name: 'check curl -V output' - - run: make V=1 examples + - run: ${{ matrix.build.make-prefix }} make V=1 examples if: ${{ matrix.build.configure }} name: 'make examples' - run: make V=1 -C tests - if: ${{ matrix.build.configure }} + if: ${{ matrix.build.configure && matrix.build.install_steps != 'skipall' }} name: 'make tests (autotools)' - run: make VERBOSE=1 testdeps - if: ${{ matrix.build.generate }} + if: ${{ matrix.build.generate && matrix.build.install_steps != 'skipall' }} name: 'make tests (cmake)' - name: 'run tests' + if: ${{ matrix.build.install_steps != 'skipall' && matrix.build.install_steps != 'skiprun' }} run: | export TFLAGS='${{ matrix.build.tflags }}' if [[ '${{ matrix.build.install_packages }}' = *'heimdal-dev'* ]]; then diff --git a/.github/workflows/linux32.yml b/.github/workflows/linux32.yml index c276bb1bbb..71f377f162 100644 --- a/.github/workflows/linux32.yml +++ b/.github/workflows/linux32.yml @@ -12,7 +12,6 @@ name: Linux 32-bit paths-ignore: - '**/*.md' - '**/CMakeLists.txt' - - '.azure-pipelines.yml' - '.circleci/**' - 'appveyor.*' - 'CMake/**' @@ -26,7 +25,6 @@ name: Linux 32-bit paths-ignore: - '**/*.md' - '**/CMakeLists.txt' - - '.azure-pipelines.yml' - '.circleci/**' - 'appveyor.*' - 'CMake/**' diff --git a/.github/workflows/macos.yml b/.github/workflows/macos.yml index 1579d13a74..3383b35ee4 100644 --- a/.github/workflows/macos.yml +++ b/.github/workflows/macos.yml @@ -11,7 +11,6 @@ name: macOS - '*/ci' paths-ignore: - '**/*.md' - - '.azure-pipelines.yml' - '.circleci/**' - 'appveyor.*' - 'packages/**' @@ -23,7 +22,6 @@ name: macOS - master paths-ignore: - '**/*.md' - - '.azure-pipelines.yml' - '.circleci/**' - 'appveyor.*' - 'packages/**' diff --git a/.github/workflows/non-native.yml b/.github/workflows/non-native.yml index 44331f224c..abb974e583 100644 --- a/.github/workflows/non-native.yml +++ b/.github/workflows/non-native.yml @@ -11,7 +11,6 @@ name: non-native - '*/ci' paths-ignore: - '**/*.md' - - '.azure-pipelines.yml' - '.circleci/**' - 'appveyor.*' - 'packages/**' @@ -23,7 +22,6 @@ name: non-native - master paths-ignore: - '**/*.md' - - '.azure-pipelines.yml' - '.circleci/**' - 'appveyor.*' - 'packages/**' diff --git a/.github/workflows/torture.yml b/.github/workflows/torture.yml index 4f39aa6dfe..79e5fa40e8 100644 --- a/.github/workflows/torture.yml +++ b/.github/workflows/torture.yml @@ -12,7 +12,6 @@ name: Linux torture paths-ignore: - '**/*.md' - '**/CMakeLists.txt' - - '.azure-pipelines.yml' - '.circleci/**' - 'appveyor.*' - 'CMake/**' @@ -26,7 +25,6 @@ name: Linux torture paths-ignore: - '**/*.md' - '**/CMakeLists.txt' - - '.azure-pipelines.yml' - '.circleci/**' - 'appveyor.*' - 'CMake/**' diff --git a/.github/workflows/windows.yml b/.github/workflows/windows.yml index 2a259ba156..770fefb96c 100644 --- a/.github/workflows/windows.yml +++ b/.github/workflows/windows.yml @@ -11,7 +11,6 @@ name: Windows - '*/ci' paths-ignore: - '**/*.md' - - '.azure-pipelines.yml' - '.circleci/**' - 'appveyor.*' - 'packages/**' @@ -23,7 +22,6 @@ name: Windows - master paths-ignore: - '**/*.md' - - '.azure-pipelines.yml' - '.circleci/**' - 'appveyor.*' - 'packages/**' @@ -52,10 +50,10 @@ jobs: - { build: 'autotools', sys: 'msys' , env: 'x86_64' , tflags: '!19 !504 !704 !705 !1233', config: '--enable-debug --disable-threaded-resolver', name: 'default' } - { build: 'cmake' , sys: 'msys' , env: 'x86_64' , tflags: 'skiprun' , config: '-DENABLE_DEBUG=ON -DENABLE_THREADED_RESOLVER=OFF', name: 'default' } - { build: 'autotools', sys: 'msys' , env: 'x86_64' , tflags: '!19 !504 !704 !705 !1233', config: '', name: 'default R' } - - { build: 'autotools', sys: 'mingw64', env: 'x86_64' , tflags: 'skiprun' , config: '--enable-debug --disable-threaded-resolver --disable-curldebug --enable-static=no', name: 'default' } - - { build: 'autotools', sys: 'mingw64', env: 'x86_64' , tflags: '~472 ~1299 ~1613' , config: '--enable-debug --enable-windows-unicode --enable-ares', name: 'c-ares U' } + - { build: 'autotools', sys: 'mingw64', env: 'x86_64' , tflags: 'skiprun' , config: '--enable-debug --disable-threaded-resolver --disable-curldebug --enable-static=no --without-zlib', name: 'default' } + - { build: 'autotools', sys: 'mingw64', env: 'x86_64' , tflags: '~472 ~1299 ~1613' , config: '--enable-debug --enable-windows-unicode --enable-ares --enable-sspi', name: 'c-ares U' } # FIXME: WebSockets test results ignored due to frequent failures on native Windows: - - { build: 'cmake' , sys: 'mingw64', env: 'x86_64' , tflags: '~2301 ~2302' , config: '-DENABLE_DEBUG=ON -DBUILD_SHARED_LIBS=OFF -DCURL_USE_SCHANNEL=ON -DENABLE_UNICODE=ON -DENABLE_ARES=ON', type: 'Debug', name: 'schannel c-ares U' } + - { build: 'cmake' , sys: 'mingw64', env: 'x86_64' , tflags: '~2301 ~2302' , config: '-DENABLE_DEBUG=ON -DBUILD_SHARED_LIBS=OFF -DCURL_USE_SCHANNEL=ON -DENABLE_UNICODE=ON -DENABLE_ARES=ON -DCURL_WINDOWS_SSPI=ON', type: 'Debug', name: 'schannel c-ares U' } - { build: 'cmake' , sys: 'ucrt64' , env: 'ucrt-x86_64' , tflags: 'skiprun' , config: '-DENABLE_DEBUG=OFF -DBUILD_SHARED_LIBS=ON -DCURL_USE_SCHANNEL=ON -DENABLE_UNICODE=ON -DENABLE_CURLDEBUG=ON', type: 'Release', name: 'schannel R TrackMemory' } - { build: 'cmake' , sys: 'clang64', env: 'clang-x86_64', tflags: 'skiprun' , config: '-DENABLE_DEBUG=ON -DBUILD_SHARED_LIBS=OFF -DCURL_USE_OPENSSL=ON -DENABLE_UNICODE=OFF', type: 'Release', name: 'openssl' } - { build: 'cmake' , sys: 'mingw64', env: 'x86_64' , tflags: 'skiprun' , config: '-DENABLE_DEBUG=OFF -DBUILD_SHARED_LIBS=ON -DCURL_USE_SCHANNEL=ON -DENABLE_UNICODE=ON -DBUILD_EXAMPLES=OFF', type: 'Release', test: 'uwp', name: 'schannel R' } @@ -63,13 +61,25 @@ jobs: steps: - run: git config --global core.autocrlf input shell: pwsh + - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4 + - uses: msys2/setup-msys2@ddf331adaebd714795f1042345e6ca57bd66cea8 # v2 if: ${{ matrix.sys == 'msys' }} with: msystem: ${{ matrix.sys }} # https://packages.msys2.org/search - install: gcc ${{ matrix.build }} ${{ matrix.build == 'autotools' && 'make' || 'ninja' }} openssl-devel zlib-devel brotli-devel libnghttp2-devel libpsl-devel libssh2-devel + install: >- + gcc + ${{ matrix.build }} ${{ matrix.build == 'autotools' && 'make' || 'ninja' }} + openssh + openssl-devel + zlib-devel + brotli-devel + libnghttp2-devel + libpsl-devel + libssh2-devel + - uses: msys2/setup-msys2@ddf331adaebd714795f1042345e6ca57bd66cea8 # v2 if: ${{ matrix.sys != 'msys' }} with: @@ -77,6 +87,7 @@ jobs: install: >- mingw-w64-${{ matrix.env }}-cc mingw-w64-${{ matrix.env }}-${{ matrix.build }} ${{ matrix.build == 'autotools' && 'make' || '' }} + openssh mingw-w64-${{ matrix.env }}-openssl mingw-w64-${{ matrix.env }}-libssh2 mingw-w64-${{ matrix.env }}-libpsl @@ -133,18 +144,27 @@ jobs: timeout-minutes: 10 run: make -C bld -j5 V=1 -C tests + - name: 'install test tools' + if: ${{ matrix.build == 'autotools' && matrix.tflags != 'skipall' && matrix.tflags != 'skiprun' }} + timeout-minutes: 5 + run: | + /c/ProgramData/chocolatey/choco.exe install --yes --no-progress --limit-output --timeout 180 --force stunnel || true + curl --disable --fail --silent --show-error --connect-timeout 15 --max-time 60 --retry 3 https://live.sysinternals.com/handle64.exe --output /bin/handle64.exe + - name: 'autotools run tests' if: ${{ matrix.build == 'autotools' && matrix.tflags != 'skipall' && matrix.tflags != 'skiprun' }} timeout-minutes: 14 run: | - export TFLAGS='-j14 ${{ matrix.tflags }}' + export TFLAGS='-j14 ${{ matrix.tflags }} ~SCP' if [ '${{ matrix.sys }}' != 'msys' ]; then TFLAGS+=' ~TFTP ~MQTT ~WebSockets ~SMTP ~FTP' + TFLAGS+=' ~612 ~613 ~616 ~618' # SFTP + else + TFLAGS+=' ~SFTP' fi if [ -x "$(cygpath "${SYSTEMROOT}/System32/curl.exe")" ]; then TFLAGS+=" -ac $(cygpath "${SYSTEMROOT}/System32/curl.exe")" fi - /c/ProgramData/chocolatey/choco.exe install --yes --no-progress --limit-output --timeout 180 stunnel --force PATH="$PATH:/c/Program Files (x86)/stunnel/bin" make -C bld -j5 V=1 test-ci @@ -214,18 +234,27 @@ jobs: timeout-minutes: 10 run: cmake --build bld --config '${{ matrix.type }}' --parallel 5 --target testdeps + - name: 'install test tools' + if: ${{ matrix.build == 'cmake' && matrix.tflags != 'skipall' && matrix.tflags != 'skiprun' }} + timeout-minutes: 5 + run: | + /c/ProgramData/chocolatey/choco.exe install --yes --no-progress --limit-output --timeout 180 --force stunnel || true + curl --disable --fail --silent --show-error --connect-timeout 15 --max-time 60 --retry 3 https://live.sysinternals.com/handle64.exe --output /bin/handle64.exe + - name: 'cmake run tests' if: ${{ matrix.build == 'cmake' && matrix.tflags != 'skipall' && matrix.tflags != 'skiprun' }} timeout-minutes: 14 run: | - export TFLAGS='-j14 ${{ matrix.tflags }}' + export TFLAGS='-j14 ${{ matrix.tflags }} ~SCP' if [ '${{ matrix.sys }}' != 'msys' ]; then TFLAGS+=' ~TFTP ~MQTT ~WebSockets ~SMTP ~FTP' + TFLAGS+=' ~612 ~613 ~616 ~618' # SFTP + else + TFLAGS+=' ~SFTP' fi if [ -x "$(cygpath "${SYSTEMROOT}/System32/curl.exe")" ]; then TFLAGS+=" -ac $(cygpath "${SYSTEMROOT}/System32/curl.exe")" fi - /c/ProgramData/chocolatey/choco.exe install --yes --no-progress --limit-output --timeout 180 stunnel --force PATH="$PWD/bld/lib:$PATH:/c/Program Files (x86)/stunnel/bin" cmake --build bld --config '${{ matrix.type }}' --target test-ci @@ -252,7 +281,7 @@ jobs: url: 'https://downloads.sourceforge.net/mingw-w64/Toolchains%20targetting%20Win64/Personal%20Builds/mingw-builds/7.3.0/threads-win32/seh/x86_64-7.3.0-release-win32-seh-rt_v5-rev0.7z' config: '-DENABLE_DEBUG=ON -DBUILD_SHARED_LIBS=OFF -DCURL_USE_SCHANNEL=ON -DENABLE_UNICODE=ON' type: 'Release' - tflags: '~2301 ~2302 ~3027' + tflags: '~2301 ~2302 ~3027 ~1451' - name: 'schannel !unity' env: '6.4.0-i686' dir: 'mingw32' @@ -332,6 +361,14 @@ jobs: PATH="$(cygpath "${USERPROFILE}")/my-cache/${{ matrix.dir }}/bin:/c/msys64/usr/bin:$PATH" cmake --build bld --config '${{ matrix.type }}' --parallel 5 --target testdeps + - name: 'install test tools' + if: ${{ matrix.tflags != 'skipall' && matrix.tflags != 'skiprun' }} + timeout-minutes: 5 + run: | + /c/ProgramData/chocolatey/choco.exe install --yes --no-progress --limit-output --timeout 180 --force stunnel || true + curl --disable --fail --silent --show-error --connect-timeout 15 --max-time 60 --retry 3 https://live.sysinternals.com/handle64.exe --output /bin/handle64.exe + python3 -m pip --disable-pip-version-check --no-input --no-cache-dir install --progress-bar off --prefer-binary impacket + - name: 'cmake run tests' if: ${{ matrix.tflags != 'skipall' && matrix.tflags != 'skiprun' }} timeout-minutes: 14 @@ -341,7 +378,6 @@ jobs: if [ -x "$(cygpath "${SYSTEMROOT}/System32/curl.exe")" ]; then TFLAGS+=" -ac $(cygpath "${SYSTEMROOT}/System32/curl.exe")" fi - /c/ProgramData/chocolatey/choco.exe install --yes --no-progress --limit-output --timeout 180 stunnel --force PATH="$PWD/bld/lib:$PATH:/c/Program Files (x86)/stunnel/bin" cmake --build bld --config '${{ matrix.type }}' --target test-ci @@ -432,7 +468,7 @@ jobs: plat: 'windows' type: 'Debug' tflags: '~1516 ~2301 ~2302 ~2303 ~2307' - config: '-DENABLE_DEBUG=ON -DENABLE_UNICODE=OFF -DCURL_USE_SCHANNEL=OFF -DCURL_BROTLI=ON -DCURL_ZSTD=ON -DBUILD_SHARED_LIBS=OFF -DCURL_USE_LIBSSH2=ON -DCURL_USE_OPENSSL=ON -DUSE_OPENSSL_QUIC=ON -DCURL_USE_GSASL=ON -DENABLE_ARES=ON -DCURL_USE_LIBUV=ON' + config: '-DENABLE_DEBUG=ON -DENABLE_UNICODE=OFF -DCURL_USE_SCHANNEL=OFF -DCURL_BROTLI=ON -DCURL_ZSTD=ON -DBUILD_SHARED_LIBS=OFF -DCURL_USE_LIBSSH2=ON -DCURL_USE_OPENSSL=ON -DUSE_OPENSSL_QUIC=ON -DCURL_USE_GSASL=ON -DENABLE_ARES=ON -DCURL_USE_LIBUV=ON -DCURL_WINDOWS_SSPI=ON' - name: 'openssl' install: 'brotli zlib zstd nghttp2 nghttp3 openssl libssh2' arch: 'x64' @@ -543,15 +579,27 @@ jobs: timeout-minutes: 10 run: cmake --build bld --config '${{ matrix.type }}' --parallel 5 --target testdeps - - name: 'cmake run tests' + - name: 'install test tools' if: ${{ matrix.tflags != 'skipall' && matrix.tflags != 'skiprun' }} - timeout-minutes: 16 + timeout-minutes: 5 run: | - export TFLAGS='-j14 ~TFTP ~MQTT ~WebSockets ~SMTP ~FTP ${{ matrix.tflags }}' # GnuTLS is not fully functional on Windows, so skip the tests # https://github.com/ShiftMediaProject/gnutls/issues/23 - if [ '${{ matrix.name }}' != 'gnutls' ]; then - /c/ProgramData/chocolatey/choco.exe install --yes --no-progress --limit-output --timeout 180 stunnel --force + if [[ '${{ matrix.name }}' = *'gnutls'* ]]; then + /c/ProgramData/chocolatey/choco.exe install --yes --no-progress --limit-output --timeout 180 --force stunnel openssh || true fi - PATH="$PWD/bld/lib:$PATH:/c/Program Files (x86)/stunnel/bin" + curl --disable --fail --silent --show-error --connect-timeout 15 --max-time 60 --retry 3 https://live.sysinternals.com/handle64.exe --output /bin/handle64.exe + python3 -m pip --disable-pip-version-check --no-input --no-cache-dir install --progress-bar off --prefer-binary impacket + + - name: 'cmake run tests' + if: ${{ matrix.tflags != 'skipall' && matrix.tflags != 'skiprun' }} + timeout-minutes: 16 + run: | + export TFLAGS='-j14 ~TFTP ~MQTT ~WebSockets ~SMTP ~FTP ~SCP ~612 ${{ matrix.tflags }}' + if [[ '${{ matrix.install }}' = *'libssh2[core,zlib]'* ]]; then + TFLAGS+=' ~SFTP' + elif [[ '${{ matrix.install }}' = *'libssh '* ]]; then + TFLAGS+=' ~614' # 'SFTP pre-quote chmod' SFTP, pre-quote, directory + fi + PATH="$PWD/bld/lib:$PATH:/c/Program Files (x86)/stunnel/bin:/c/Program Files/OpenSSH-Win64" cmake --build bld --config '${{ matrix.type }}' --target test-ci diff --git a/.github/workflows/wolfssl.yml b/.github/workflows/wolfssl.yml index 0b3104bbc4..a156dbbf39 100644 --- a/.github/workflows/wolfssl.yml +++ b/.github/workflows/wolfssl.yml @@ -12,7 +12,6 @@ name: Linux wolfSSL paths-ignore: - '**/*.md' - '**/CMakeLists.txt' - - '.azure-pipelines.yml' - '.circleci/**' - 'appveyor.*' - 'CMake/**' @@ -26,7 +25,6 @@ name: Linux wolfSSL paths-ignore: - '**/*.md' - '**/CMakeLists.txt' - - '.azure-pipelines.yml' - '.circleci/**' - 'appveyor.*' - 'CMake/**' diff --git a/appveyor.yml b/appveyor.yml index 6a4e78697c..8410ceb073 100644 --- a/appveyor.yml +++ b/appveyor.yml @@ -207,7 +207,6 @@ branches: skip_commits: files: - - '.azure-pipelines.yml' - '.circleci/*' - '.github/**/*' - 'packages/**/*' diff --git a/tests/CI.md b/tests/CI.md index fe280a6268..16aba08305 100644 --- a/tests/CI.md +++ b/tests/CI.md @@ -68,16 +68,7 @@ These are each configured in different files in `.github/workflows`. ### Azure -The following tests are run in Microsoft Azure CI environment: - -- Ubuntu tests with a variety of different compilation options. -- Windows tests with a variety of different compilation options. - -These are all configured in `.azure-pipelines.yml`. - -As of November 2021 `@bagder` and `@mback2k` are the only people with -administrator access to the Azure CI environment. Additional admins/group -members can be added on request. +Not used anymore. ### AppVeyor diff --git a/tests/data/test437 b/tests/data/test437 index 4098b4c645..0f781e0b9c 100644 --- a/tests/data/test437 +++ b/tests/data/test437 @@ -28,6 +28,7 @@ Alt-Svc: h1="[ffff::1]:8181" Debug alt-svc +IPv6 http diff --git a/tests/processhelp.pm b/tests/processhelp.pm index 253fa8b34c..44c6411cda 100644 --- a/tests/processhelp.pm +++ b/tests/processhelp.pm @@ -59,6 +59,7 @@ use serverhelp qw( servername_id mainsockf_pidfilename datasockf_pidfilename + logmsg ); use pathhelp qw( @@ -136,6 +137,7 @@ sub pidexists { $pid -= 65536; if($^O ne 'MSWin32') { my $filter = "PID eq $pid"; + # https://ss64.com/nt/tasklist.html my $result = `tasklist -fi \"$filter\" 2>nul`; if(index($result, "$pid") != -1) { return -$pid; @@ -166,8 +168,10 @@ sub pidterm { $pid -= 65536; if($^O ne 'MSWin32') { my $filter = "PID eq $pid"; + # https://ss64.com/nt/tasklist.html my $result = `tasklist -fi \"$filter\" 2>nul`; if(index($result, "$pid") != -1) { + # https://ss64.com/nt/taskkill.html system("taskkill -fi \"$filter\" >nul 2>&1"); } return; @@ -192,11 +196,15 @@ sub pidkill { $pid -= 65536; if($^O ne 'MSWin32') { my $filter = "PID eq $pid"; - my $result = `tasklist -fi \"$filter\" 2>nul`; + # https://ss64.com/nt/tasklist.html + my $cmd = "tasklist -fi \"$filter\" 2>nul"; + logmsg "Executing: '$cmd'\n"; + my $result = `$cmd`; if(index($result, "$pid") != -1) { - system("taskkill -f -t -fi \"$filter\" >nul 2>&1"); - # Windows XP Home compatibility - system("tskill $pid >nul 2>&1"); + # https://ss64.com/nt/taskkill.html + my $cmd = "taskkill -f -t -fi \"$filter\" >nul 2>&1"; + logmsg "Executing: '$cmd'\n"; + system($cmd); } return; } diff --git a/tests/runtests.pl b/tests/runtests.pl index c9854a6001..1f39e8faad 100755 --- a/tests/runtests.pl +++ b/tests/runtests.pl @@ -851,7 +851,8 @@ sub checksystemfeatures { "* Disabled: $dis\n", "* Host: $hostname\n", "* System: $hosttype\n", - "* OS: $hostos\n"); + "* OS: $hostos\n", + "* Perl: $^V ($^X)\n"); if($jobs) { # Only show if not the default for now diff --git a/tests/servers.pm b/tests/servers.pm index 433f3f2340..195d663fa3 100644 --- a/tests/servers.pm +++ b/tests/servers.pm @@ -274,7 +274,10 @@ sub clearlocks { $handle = "handle64"; } if(checkcmd($handle)) { - my @handles = `$handle $dir -accepteula -nobanner`; + # https://learn.microsoft.com/sysinternals/downloads/handle#usage + my $cmd = "$handle $dir -accepteula -nobanner"; + logmsg "Executing: '$cmd'\n"; + my @handles = `$cmd`; for my $tryhandle (@handles) { # Skip the "No matching handles found." warning when returned if($tryhandle =~ /^(\S+)\s+pid:\s+(\d+)\s+type:\s+(\w+)\s+([0-9A-F]+):\s+(.+)\r\r/) { @@ -282,12 +285,18 @@ sub clearlocks { # Ignore stunnel since we cannot do anything about its locks if("$3" eq "File" && "$1" ne "tstunnel.exe") { logmsg "Killing IMAGENAME eq $1 and PID eq $2\n"; - system("taskkill.exe -f -fi \"IMAGENAME eq $1\" -fi \"PID eq $2\" >nul 2>&1"); + # https://ss64.com/nt/taskkill.html + my $cmd = "taskkill.exe -f -t -fi \"IMAGENAME eq $1\" -fi \"PID eq $2\" >nul 2>&1"; + logmsg "Executing: '$cmd'\n"; + system($cmd); $done = 1; } } } } + else { + logmsg "Warning: 'handle' tool not found.\n"; + } } return $done; } diff --git a/tests/sshserver.pl b/tests/sshserver.pl index fcfe90a620..86db0eb0ce 100755 --- a/tests/sshserver.pl +++ b/tests/sshserver.pl @@ -79,7 +79,6 @@ use sshhelp qw( # Subs imported from serverhelp module # use serverhelp qw( - logmsg $logfile server_pidfilename server_logfilename @@ -115,6 +114,14 @@ sub pp { # TODO: do Windows path conversion here } +#*************************************************************************** +# Save the message to the log and print it +sub logmsg { + my $msg = $_[0]; + serverhelp::logmsg $msg; + print $msg; +} + #*************************************************************************** # Parse command line options # @@ -385,6 +392,9 @@ if((($sshid =~ /OpenSSH/) && ($sshvernum < 299)) || # -q: quiet keygen : SunSSH 1.0.0 and later # -t: key type : SunSSH 1.0.0 and later +$sshdconfig = pp($sshdconfig); +$sshconfig = pp($sshconfig); +$sftpconfig = pp($sftpconfig); #*************************************************************************** # Generate host and client key files for curl's tests @@ -411,6 +421,13 @@ if((! -e pp($hstprvkeyf)) || (! -s pp($hstprvkeyf)) || # Make sure that permissions are restricted so openssh doesn't complain system "chmod 600 " . pp($hstprvkeyf); system "chmod 600 " . pp($cliprvkeyf); + if(pathhelp::os_is_win()) { + # https://ss64.com/nt/icacls.html + $ENV{'MSYS2_ARG_CONV_EXCL'} = '/reset'; + system("icacls \"" . pathhelp::sys_native_abs_path(pp($hstprvkeyf)) . "\" /reset"); + system("icacls \"" . pathhelp::sys_native_abs_path(pp($hstprvkeyf)) . "\" /grant:r \"$username:(R)\""); + system("icacls \"" . pathhelp::sys_native_abs_path(pp($hstprvkeyf)) . "\" /inheritance:r"); + } # Save md5 and sha256 hashes of public host key open(my $rsakeyfile, "<", pp($hstpubkeyf)); my @rsahostkey = do { local $/ = ' '; <$rsakeyfile> }; @@ -443,27 +460,30 @@ my $clipubkeyf_config; my $hstprvkeyf_config; my $pidfile_config; my $sftpsrv_config; +my $sshdconfig_abs; if ($sshdid =~ /OpenSSH-Windows/) { # Ensure to use native Windows paths with OpenSSH for Windows $clipubkeyf_config = pathhelp::sys_native_abs_path(pp($clipubkeyf)); $hstprvkeyf_config = pathhelp::sys_native_abs_path(pp($hstprvkeyf)); $pidfile_config = pathhelp::sys_native_abs_path($pidfile); $sftpsrv_config = pathhelp::sys_native_abs_path($sftpsrv); + $sshdconfig_abs = pathhelp::sys_native_abs_path($sshdconfig); } elsif (pathhelp::os_is_win()) { # Ensure to use MinGW/Cygwin paths - $clipubkeyf_config = pathhelp::build_sys_abs_path($clipubkeyf_config); - $hstprvkeyf_config = pathhelp::build_sys_abs_path($hstprvkeyf_config); - $pidfile_config = pathhelp::build_sys_abs_path($pidfile_config); + $clipubkeyf_config = pathhelp::build_sys_abs_path(pp($clipubkeyf)); + $hstprvkeyf_config = pathhelp::build_sys_abs_path(pp($hstprvkeyf)); + $pidfile_config = pathhelp::build_sys_abs_path($pidfile); $sftpsrv_config = "internal-sftp"; + $sshdconfig_abs = pathhelp::build_sys_abs_path($sshdconfig); } else { $clipubkeyf_config = abs_path(pp($clipubkeyf)); $hstprvkeyf_config = abs_path(pp($hstprvkeyf)); $pidfile_config = $pidfile; $sftpsrv_config = $sftpsrv; + $sshdconfig_abs = abs_path($sshdconfig); } -my $sshdconfig_abs = pathhelp::sys_native_abs_path(pp($sshdconfig)); #*************************************************************************** # ssh daemon configuration file options we might use and version support @@ -473,7 +493,7 @@ my $sshdconfig_abs = pathhelp::sys_native_abs_path(pp($sshdconfig)); # AllowTcpForwarding : OpenSSH 2.3.0 and later # AllowUsers : OpenSSH 1.2.1 and later # AuthorizedKeysFile : OpenSSH 2.9.9 and later -# AuthorizedKeysFile2 : OpenSSH 2.9.9 and later +# AuthorizedKeysFile2 : OpenSSH 2.9.9 till 5.9 # Banner : OpenSSH 2.5.0 and later # ChallengeResponseAuthentication : OpenSSH 2.5.0 and later # Ciphers : OpenSSH 2.1.0 and later [3] @@ -501,7 +521,7 @@ my $sshdconfig_abs = pathhelp::sys_native_abs_path(pp($sshdconfig)); # KerberosOrLocalPasswd : OpenSSH 1.2.1 and later [1] # KerberosTgtPassing : OpenSSH 1.2.1 and later [1] # KerberosTicketCleanup : OpenSSH 1.2.1 and later [1] -# KeyRegenerationInterval : OpenSSH 1.2.1 and later +# KeyRegenerationInterval : OpenSSH 1.2.1 till 7.3 # ListenAddress : OpenSSH 1.2.1 and later # LoginGraceTime : OpenSSH 1.2.1 and later # LogLevel : OpenSSH 1.2.1 and later @@ -524,16 +544,16 @@ my $sshdconfig_abs = pathhelp::sys_native_abs_path(pp($sshdconfig)); # Protocol : OpenSSH 2.1.0 and later # PubkeyAuthentication : OpenSSH 2.5.0 and later # RhostsAuthentication : OpenSSH 1.2.1 and later -# RhostsRSAAuthentication : OpenSSH 1.2.1 and later -# RSAAuthentication : OpenSSH 1.2.1 and later -# ServerKeyBits : OpenSSH 1.2.1 and later +# RhostsRSAAuthentication : OpenSSH 1.2.1 till 7.3 +# RSAAuthentication : OpenSSH 1.2.1 till 7.3 +# ServerKeyBits : OpenSSH 1.2.1 till 7.3 # SkeyAuthentication : OpenSSH 1.2.1 and later [1] # StrictModes : OpenSSH 1.2.1 and later # Subsystem : OpenSSH 2.2.0 and later # SyslogFacility : OpenSSH 1.2.1 and later # TCPKeepAlive : OpenSSH 3.8.0 and later # UseDNS : OpenSSH 3.7.0 and later -# UseLogin : OpenSSH 1.2.1 and later +# UseLogin : OpenSSH 1.2.1 till 7.3 # UsePAM : OpenSSH 3.7.0 and later [1][2] # UsePrivilegeSeparation : OpenSSH 3.2.2 and later # VerifyReverseMapping : OpenSSH 3.1.0 and later @@ -560,20 +580,21 @@ push @cfgarr, '#'; # and do not support quotes around values for some unknown reason. if ($sshdid =~ /OpenSSH-Windows/) { my $username_lc = lc $username; + push @cfgarr, "AllowUsers " . $username_lc =~ s/ /\?/gr; if (exists $ENV{USERDOMAIN}) { my $userdomain_lc = lc $ENV{USERDOMAIN}; $username_lc = "$userdomain_lc\\$username_lc"; + $username_lc =~ s/ /\?/g; # replace space with ? + push @cfgarr, "AllowUsers " . $username_lc =~ s/ /\?/gr; } - $username_lc =~ s/ /\?/g; # replace space with ? - push @cfgarr, "DenyUsers !$username_lc"; - push @cfgarr, "AllowUsers $username_lc"; } else { - push @cfgarr, "DenyUsers !$username"; push @cfgarr, "AllowUsers $username"; } push @cfgarr, "AuthorizedKeysFile $clipubkeyf_config"; -push @cfgarr, "AuthorizedKeysFile2 $clipubkeyf_config"; +if(!($sshdid =~ /OpenSSH/) || ($sshdvernum <= 730)) { + push @cfgarr, "AuthorizedKeysFile2 $clipubkeyf_config"; +} push @cfgarr, "HostKey $hstprvkeyf_config"; if ($sshdid !~ /OpenSSH-Windows/) { push @cfgarr, "PidFile $pidfile_config"; @@ -598,7 +619,6 @@ push @cfgarr, 'HostbasedAuthentication no'; push @cfgarr, 'HostbasedUsesNameFromPacketOnly no'; push @cfgarr, 'IgnoreRhosts yes'; push @cfgarr, 'IgnoreUserKnownHosts yes'; -push @cfgarr, 'KeyRegenerationInterval 0'; push @cfgarr, 'LoginGraceTime 30'; push @cfgarr, "LogLevel $loglevel"; push @cfgarr, 'MaxStartups 5'; @@ -608,13 +628,16 @@ push @cfgarr, 'PermitRootLogin no'; push @cfgarr, 'PrintLastLog no'; push @cfgarr, 'PrintMotd no'; push @cfgarr, 'PubkeyAuthentication yes'; -push @cfgarr, 'RhostsRSAAuthentication no'; -push @cfgarr, 'RSAAuthentication no'; -push @cfgarr, 'ServerKeyBits 768'; push @cfgarr, 'StrictModes no'; push @cfgarr, "Subsystem sftp \"$sftpsrv_config\""; push @cfgarr, 'SyslogFacility AUTH'; -push @cfgarr, 'UseLogin no'; +if(!($sshdid =~ /OpenSSH/) || ($sshdvernum <= 730)) { + push @cfgarr, 'KeyRegenerationInterval 0'; + push @cfgarr, 'RhostsRSAAuthentication no'; + push @cfgarr, 'RSAAuthentication no'; + push @cfgarr, 'ServerKeyBits 768'; + push @cfgarr, 'UseLogin no'; +} push @cfgarr, 'X11Forwarding no'; push @cfgarr, '#'; @@ -622,7 +645,7 @@ push @cfgarr, '#'; #*************************************************************************** # Write out initial sshd configuration file for curl's tests # -$error = dump_array(pp($sshdconfig), @cfgarr); +$error = dump_array($sshdconfig, @cfgarr); if($error) { logmsg "$error\n"; exit 1; @@ -645,14 +668,14 @@ sub sshd_supports_opt { } if(($sshdid =~ /OpenSSH/) && ($sshdvernum >= 299)) { # ssh daemon supports command line options -t and -f - $err = dump_array(pp($sshdconfig), (@cfgarr, "$option $value")); + $err = dump_array($sshdconfig, (@cfgarr, "$option $value")); if($err) { logmsg "$err\n"; return 0; } $err = grep /((Unsupported)|(Bad configuration)|(Deprecated)) option.*$option/, `\"$sshd\" -t -f $sshdconfig_abs 2>&1`; - unlink pp($sshdconfig); + unlink $sshdconfig; return !$err; } return 0; @@ -784,7 +807,7 @@ push @cfgarr, '#'; #*************************************************************************** # Write out resulting sshd configuration file for curl's tests # -$error = dump_array(pp($sshdconfig), @cfgarr); +$error = dump_array($sshdconfig, @cfgarr); if($error) { logmsg "$error\n"; exit 1; @@ -847,8 +870,8 @@ if ($sshdid =~ /OpenSSH-Windows/) { } elsif (pathhelp::os_is_win()) { # Ensure to use MinGW/Cygwin paths - $identity_config = pathhelp::build_sys_abs_path($identity); - $knownhosts_config = pathhelp::build_sys_abs_path($knownhosts); + $identity_config = pathhelp::build_sys_abs_path(pp($identity)); + $knownhosts_config = pathhelp::build_sys_abs_path(pp($knownhosts)); } else { $identity_config = abs_path(pp($identity)); @@ -977,7 +1000,7 @@ push @cfgarr, 'PreferredAuthentications publickey'; push @cfgarr, 'PubkeyAuthentication yes'; # RSA authentication options are not supported by OpenSSH for Windows -if (!($sshdid =~ /OpenSSH-Windows/)) { +if (!($sshdid =~ /OpenSSH-Windows/ || pathhelp::os_is_win())) { push @cfgarr, 'RhostsRSAAuthentication no'; push @cfgarr, 'RSAAuthentication no'; } @@ -1092,13 +1115,12 @@ push @cfgarr, '#'; #*************************************************************************** # Write out resulting ssh client configuration file for curl's tests # -$error = dump_array(pp($sshconfig), @cfgarr); +$error = dump_array($sshconfig, @cfgarr); if($error) { logmsg "$error\n"; exit 1; } - #*************************************************************************** # Initialize client sftp config with options actually supported. # @@ -1120,7 +1142,7 @@ for(my $i = scalar(@cfgarr) - 1; $i > 0; $i--) { #*************************************************************************** # Write out resulting sftp client configuration file for curl's tests # -$error = dump_array(pp($sftpconfig), @cfgarr); +$error = dump_array($sftpconfig, @cfgarr); if($error) { logmsg "$error\n"; exit 1; @@ -1194,6 +1216,6 @@ elsif($verbose && ($rc >> 8)) { # unlink(pp($hstprvkeyf), pp($hstpubkeyf), pp($hstpubmd5f), pp($hstpubsha256f), pp($cliprvkeyf), pp($clipubkeyf), pp($knownhosts), - pp($sshdconfig), pp($sshconfig), pp($sftpconfig)); + $sshdconfig, $sshconfig, $sftpconfig); exit 0; diff --git a/tests/unit/unit1614.c b/tests/unit/unit1614.c index fd6f5849c8..3258fd225e 100644 --- a/tests/unit/unit1614.c +++ b/tests/unit/unit1614.c @@ -68,6 +68,7 @@ UNITTEST_START { "192.160.0.1", "10.0.0.1", 0, FALSE}, { NULL, NULL, 0, FALSE} /* end marker */ }; +#ifdef USE_IPV6 struct check list6[]= { { "::1", "::1", 0, TRUE}, { "::1", "::1", 128, TRUE}, @@ -76,6 +77,7 @@ UNITTEST_START { "fe80::ab47:4396:55c9:8474", "fe80::ab47:4396:55c9:8474", 64, TRUE}, { NULL, NULL, 0, FALSE} /* end marker */ }; +#endif struct noproxy list[]= { { "www.example.com", "localhost .example.com .example.de", FALSE}, { "www.example.com", "localhost,.example.com,.example.de", TRUE}, @@ -113,6 +115,7 @@ UNITTEST_START { "192.168.1.1", "192.168.0.0/33", FALSE}, { "192.168.1.1", "foo, bar, 192.168.0.0/24", FALSE}, { "192.168.1.1", "foo, bar, 192.168.0.0/16", TRUE}, +#ifdef USE_IPV6 { "[::1]", "foo, bar, 192.168.0.0/16", FALSE}, { "[::1]", "foo, bar, ::1/64", TRUE}, { "[::1]", "::1/64", TRUE}, @@ -121,6 +124,7 @@ UNITTEST_START { "bar", "foo, bar, ::1/64", TRUE}, { "BAr", "foo, bar, ::1/64", TRUE}, { "BAr", "foo,,,,, bar, ::1/64", TRUE}, +#endif { "www.example.com", "foo, .example.com", TRUE}, { "www.example.com", "www2.example.com, .example.net", FALSE}, { "example.com", ".example.com, .example.net", TRUE}, @@ -136,6 +140,7 @@ UNITTEST_START err++; } } +#ifdef USE_IPV6 for(i = 0; list6[i].a; i++) { bool match = Curl_cidr6_match(list6[i].a, list6[i].n, list6[i].bits); if(match != list6[i].match) { @@ -145,6 +150,7 @@ UNITTEST_START err++; } } +#endif for(i = 0; list[i].a; i++) { bool match = Curl_check_noproxy(list[i].a, list[i].n); if(match != list[i].match) { diff --git a/tests/unit/unit2603.c b/tests/unit/unit2603.c index 3c61776360..2775a4565a 100644 --- a/tests/unit/unit2603.c +++ b/tests/unit/unit2603.c @@ -37,6 +37,7 @@ static void unit_stop(void) { } +#ifndef CURL_DISABLE_HTTP struct tcase { const char **input; const char *default_scheme; @@ -176,9 +177,11 @@ static const char *T6_INPUT[] = { static struct tcase TEST6a = { T6_INPUT, NULL, "PUT", NULL, NULL, "/path", 1, 3 }; +#endif UNITTEST_START +#ifndef CURL_DISABLE_HTTP parse_success(&TEST1a); parse_success(&TEST1b); parse_success(&TEST2); @@ -186,5 +189,6 @@ UNITTEST_START parse_success(&TEST4a); parse_success(&TEST5a); parse_success(&TEST6a); +#endif UNITTEST_STOP diff --git a/tests/unit/unit2604.c b/tests/unit/unit2604.c index 734827b0ea..fb8405a6b6 100644 --- a/tests/unit/unit2604.c +++ b/tests/unit/unit2604.c @@ -45,6 +45,11 @@ struct set { UNITTEST_START #ifdef USE_SSH { +#ifdef __GNUC__ +#pragma GCC diagnostic push +#pragma GCC diagnostic ignored "-Woverlength-strings" +#endif + /* 60 a's */ #define SA60 "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" /* 540 a's */ @@ -72,6 +77,10 @@ UNITTEST_START { NULL, NULL, NULL, NULL, CURLE_OK } }; +#ifdef __GNUC__ +#pragma GCC diagnostic warning "-Woverlength-strings" +#endif + list[0].cp = calloc(1, too_long + 1); fail_unless(list[0].cp, "could not alloc too long value"); memset((void *)list[0].cp, 'a', too_long);