Today GHA Windows runner images (all versions) deployed an upgrade
(
20250127.1.0 ->
20250203.1.0) that upgraded the default MSYS2, which
now seems to feature the October 2024 issue that caused curl runtests
run times increasing ~2.5x. It also causes test987 to fail, and vcpkg
jobs hitting their time limits and fail. Reliability also got a hit.
In October this issue came with a Git for Windows upgrade, and likely
the MSYS2 runtime update within it. It affected vcpkg jobs only, and
I mitigated it by switching them to use the default MSYS2 shell and
runtime (at `C:\msys64`):
5f9411f953f35ca84645b5a6824fb9fef20887ed #15380
After today's update this mitigation no longer works. The issue also
affects `dl-mingw` jobs now, though to a lesser extent than vcpkg ones.
Tried switching back to Git for Windows which received several updates
since October, but the performance issue is still present.
I managed to mitigate the slowdown in vcpkg by lowering test parallelism
to `-j4` (from `-j8`), after which the jobs are about *half the speed*
than before, and fit their time limits. `dl-mingw` builds run slower by
1-1.5 minutes per job, they were already using `-j4`.
Example jobs:
Before (ALL GOOD):
https://github.com/curl/curl/actions/runs/
13167230443/job/
36750175428 installed MSYS2, mingw (-j8): 3m50s (OK)
https://github.com/curl/curl/actions/runs/
13167230443/job/
36750158662 default MSYS2, dl-mingw (-j4): 4m22s (OK)
https://github.com/curl/curl/actions/runs/
13167230443/job/
36750163392 default MSYS2, vcpkg (-j8): 3m27s (OK)
runner: https://github.com/actions/runner-images/blob/win22/
20250127.1/images/windows/Windows2022-Readme.md
C:\msys64:
System: MSYS_NT-10.0-20348 fv-az1115-916 3.5.4-
0bc1222b.x86_64 2024-12-05 09:27 UTC x86_64 Msys
https://github.com/msys2/msys2-runtime/commit/
0bc1222b
After:
https://github.com/curl/curl/actions/runs/
13186498273/job/
36809747078 installed MSYS2, mingw (-j8): 3m48s (OK)
https://github.com/curl/curl/actions/runs/
13186498273/job/
36809728481 default MSYS2, dl-mingw (-j4): 5m56s (SLOW)
https://github.com/curl/curl/actions/runs/
13186498273/job/
36809736429 default MSYS2, vcpkg (-j8): 9m1s (SLOW)
runner: https://github.com/actions/runner-images/blob/win22/
20250203.1/images/windows/Windows2022-Readme.md
C:\msys64:
System: MSYS_NT-10.0-20348 fv-az1115-498 3.5.7-
2644508f.x86_64 2025-01-30 09:08 UTC x86_64 Msys
https://github.com/msys2/msys2-runtime/commit/
2644508f
windows-2025 image:
C:\msys64:
System: MSYS_NT-10.0-26100 fv-az2043-515 3.5.7-
2644508f.x86_64 2025-01-30 09:08 UTC x86_64 Msys
windows-2019 image:
C:\msys64:
System: MSYS_NT-10.0-17763 fv-az1434-677 3.5.7-
2644508f.x86_64 2025-01-30 09:08 UTC x86_64 Msys
This PR:
final: https://github.com/curl/curl/actions/runs/
13186498273/job/
36809736429 GfW, vcpkg (*-j4*): ~7m (SLOW)
test: https://github.com/curl/curl/actions/runs/
13187992987/job/
36814644852?pr=16217, GfW, vcpkg (-j8): ~11m (SLOWER)
Before and after (unused) Git for Windows (SLOW as tested in this PR):
C:\Program Files\Git
System: MINGW64_NT-10.0-20348 fv-az1760-186 3.5.4-
395fda67.x86_64 2024-11-25 09:49 UTC x86_64 Msys
https://github.com/msys2/msys2-runtime/commit/
395fda67 (fork)
Before and after (used) MSYS2 installed via msys2/setup-msys2 (OK):
D:\a\_temp\msys64
System: MINGW64_NT-10.0-20348 fv-az836-378 3.5.4-
0bc1222b.x86_64 2024-12-05 09:27 UTC x86_64 Msys
Perl pipe issue report from October, still open:
https://github.com/msys2/msys2-runtime/issues/230
ARM deadlock fixed by GfW 2.47.1(1), but for x86_64, on a quick glance:
https://github.com/msys2/msys2-runtime/commit/
290bea94d0d5c632c016f916a7b123a260fc02a8
Possibly interesting:
https://github.com/msys2/msys2-autobuild/issues/62
Closes #16217