]> git.ipfire.org Git - thirdparty/postgresql.git/commit
Use annotations to reduce instability of isolation-test results.
authorTom Lane <tgl@sss.pgh.pa.us>
Wed, 23 Jun 2021 01:43:12 +0000 (21:43 -0400)
committerTom Lane <tgl@sss.pgh.pa.us>
Wed, 23 Jun 2021 01:43:12 +0000 (21:43 -0400)
commita1417e4378b9a3a4e896766331eb45a1f4de0cfd
tree0cb843dad820d5aa4b94f8637a5d9253102a90a6
parent77200c569238dd88b4134b09cfa65e3598136f19
Use annotations to reduce instability of isolation-test results.

We've long contended with isolation test results that aren't entirely
stable.  Some test scripts insert long delays to try to force stable
results, which is not terribly desirable; but other erratic failure
modes remain, causing unrepeatable buildfarm failures.  I've spent a
fair amount of time trying to solve this by improving the server-side
support code, without much success: that way is fundamentally unable
to cope with diffs that stem from chance ordering of arrival of
messages from different server processes.

We can improve matters on the client side, however, by annotating
the test scripts themselves to show the desired reporting order
of events that might occur in different orders.  This patch adds
three types of annotations to deal with (a) test steps that might or
might not complete their waits before the isolationtester can see them
waiting; (b) test steps in different sessions that can legitimately
complete in either order; and (c) NOTIFY messages that might arrive
before or after the completion of a step in another session.  We might
need more annotation types later, but this seems to be enough to deal
with the instabilities we've seen in the buildfarm.  It also lets us
get rid of all the long delays that were previously used, cutting more
than a minute off the runtime of the isolation tests.

Back-patch to all supported branches, because the buildfarm
instabilities affect all the branches, and because it seems desirable
to keep isolationtester's capabilities the same across all branches
to simplify possible future back-patching of tests.

Discussion: https://postgr.es/m/327948.1623725828@sss.pgh.pa.us
39 files changed:
contrib/test_decoding/expected/concurrent_ddl_dml.out
src/test/isolation/README
src/test/isolation/expected/alter-table-3.out
src/test/isolation/expected/alter-table-4.out
src/test/isolation/expected/deadlock-hard.out
src/test/isolation/expected/deadlock-simple.out
src/test/isolation/expected/eval-plan-qual-trigger.out
src/test/isolation/expected/fk-deadlock2_1.out
src/test/isolation/expected/fk-deadlock2_2.out
src/test/isolation/expected/fk-deadlock_1.out
src/test/isolation/expected/insert-conflict-do-nothing-2.out
src/test/isolation/expected/lock-committed-keyupdate.out
src/test/isolation/expected/lock-update-delete_1.out
src/test/isolation/expected/multiple-cic.out
src/test/isolation/expected/multiple-cic_1.out [deleted file]
src/test/isolation/expected/multixact-no-forget_1.out
src/test/isolation/expected/nowait-4.out
src/test/isolation/expected/nowait-4_1.out
src/test/isolation/expected/nowait-5.out
src/test/isolation/expected/partition-key-update-1.out
src/test/isolation/expected/partition-key-update-3.out
src/test/isolation/expected/propagate-lock-delete.out
src/test/isolation/expected/read-write-unique-2.out
src/test/isolation/expected/read-write-unique-3.out
src/test/isolation/expected/read-write-unique-4.out
src/test/isolation/expected/read-write-unique.out
src/test/isolation/expected/sequence-ddl.out
src/test/isolation/expected/skip-locked-4_1.out
src/test/isolation/expected/timeouts.out
src/test/isolation/expected/tuplelock-update.out
src/test/isolation/isolationtester.c
src/test/isolation/isolationtester.h
src/test/isolation/specparse.y
src/test/isolation/specs/deadlock-hard.spec
src/test/isolation/specs/deadlock-soft-2.spec
src/test/isolation/specs/multiple-cic.spec
src/test/isolation/specs/timeouts.spec
src/test/isolation/specs/tuplelock-update.spec
src/test/isolation/specscanner.l