From: Pauli <pauli@openssl.org>
Date: Tue, 17 Aug 2021 13:34:52 +0000 (+1000)
Subject: [github-ci] Add comment about our approach to GitHub Actions CI
X-Git-Tag: OpenSSL_1_1_1l~19
X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=a7ce0c00a2eb8fe88786c4eb28e9ed385581eab7;p=thirdparty%2Fopenssl.git

[github-ci] Add comment about our approach to GitHub Actions CI

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/16252)
---

diff --git a/.github/workflows/README.md b/.github/workflows/README.md
new file mode 100644
index 00000000000..ac956f0cf4f
--- /dev/null
+++ b/.github/workflows/README.md
@@ -0,0 +1,40 @@
+## Rationale about our design for the GitHub Actions CI
+
+The balance is between the time taken and the number of jobs.
+We're allowed 180 concurrent jobs in total across the entire project.
+Currently we're running about 60 on pull_request, a few more on push and
+a pile per day.
+So three simultaneous PRs should finish quickly enough.
+Given that most jobs run quickly, this could scale up to 5 or 6 without
+problem.
+
+Moving more jobs into the `pull_request` category will limit the number
+of parallel builds (from different PRs) we can handle.
+We got into quite some strife over this with our older CI hosts
+-- remember builds taking the best part of a day to run.
+We really want to avoid that again.
+
+I've been trying to limit total job time per job to around 20-30 minutes
+(there are some longer ones I know of), with most jobs running in the
+sub 5 minute range.
+There are some longer lived CIs -- up to an hour and I try to delegate
+these to push or daily rather than pull_request.
+
+Still, there is no hard and fast rule about what runs when or where.
+Make a suggestion about bettering the CIs -- Ideally I'd like the
+`pull_request` jobs to be the ones catching most of the problems and the
+push and daily being predictably boring successes.
+Just make an effort to rationally justify the inclusions/changes.
+
+Things like the sanitiser builds, we know catch problems often.
+So even though they are slow they are worthwhile on `pull_request`.
+A lot of the daily builds are unlikely to catch much since they are
+checking options can be turned off and on, so they are fine not running
+as much.
+The demarkation between `pull_request` and `pull_request + push` is the
+difficult choice.
+I believe we should do all pull_request jobs as part of push too.
+The question is how many more should there be.
+
+I don't have a good answer but I think we're converging on a practical
+number and we should get better as we gain experience.