curl: Retry on failures

`curl()` is being used for things like fetching Fedora rawhide's GPG
key, opensuse's `/history/latest` or CentOS' `.composeinfo`. These are
all volatile and thus hard to pre-download/cache. But we've seen several
transient download failures like

> curl: (35) Recv failure: Connection reset by peer

Make these more robust with curl's retry (with exponential back-off)
mechanism.

diff --git a/mkosi/curl.py b/mkosi/curl.py
index 0db38229af278aeff134cfd7d4a97b97f723fc81..401199d7003aabea5d0162d8327b939e37d9257d 100644 (file)
--- a/mkosi/curl.py
+++ b/mkosi/curl.py
@@ -39,6 +39,10 @@ def curl(config: Config, url: str, *, output_dir: Optional[Path] = None, log: bo
             *(["--remote-name"] if output_dir else []),
             "--no-progress-meter",
             "--fail",
+            # be resilient against transient network failures
+            "--retry", "5",
+            # include mid-transfer connection reset
+            "--retry-all-errors",
             *(["--silent"] if not log else []),
             *(["--proxy", config.proxy_url] if config.proxy_url else []),
             *(["--noproxy", ",".join(config.proxy_exclude)] if config.proxy_exclude else []),