t/lib-httpd: fix apply-one-time-script race under concurrent requests
apply-one-time-script.sh checks for the "one-time-script" marker, runs
it, captures the git-http-backend response in the fixed-name files "out"
and "out_modified", and removes the marker only after it has finished
serving the modified response. Because the client receives the response
body before that removal, it can start its next request while the marker
still exists. Apache can then run this CGI for two requests at once: a
partial fetch that receives a REF_DELTA against a missing promisor
object lazily fetches that base while the first response is still in
flight. The second request passes the marker check, the first request
then removes the marker, and the second fails to exec the now-missing
marker, emits no output, and the server answers HTTP 500:
fatal: ... The requested URL returned error: 500
fatal: could not fetch <oid> from promisor remote
This has been seen as a flaky failure of t5616.47 on the macOS CI
runners.
Claim the marker atomically with a rename, and only once the one-time
script has succeeded and actually changed the response; give the scratch
files per-request names. A request that loses the rename, or whose
script fails or leaves the response unchanged, serves the unmodified
body and keeps the marker for a later request. No path emits an empty
body, so the HTTP 500 no longer occurs.
Running the one-time script more than once is fine; the only thing to
avoid is serving a second, racing request's modified output. Two
requests can both find the marker and run the script before either
renames it away, but the rename is atomic, so exactly one of them wins:
it serves its modified body and consumes the marker. The loser's rename
fails because the marker is already gone, so it discards the modified
output it produced and serves the unmodified body instead. The rename,
not running the script, is what is serialized.
Add t5567 to lock this down. The overlap depends on timing, so a live
httpd test such as t5616.47 (the real code path) passes almost every
time even against the buggy helper; t5567 instead drives the helper
directly with a fake git-http-backend and forces the overlap with FIFOs.
Against the pre-fix helper it fails with the same shell error seen in
the field:
./one-time-script: No such file or directory
Signed-off-by: Michael Montalbo <mmontalbo@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>