]> git.ipfire.org Git - thirdparty/postgresql.git/commit
Use annotations to reduce instability of isolation-test results.
authorTom Lane <tgl@sss.pgh.pa.us>
Wed, 23 Jun 2021 01:43:12 +0000 (21:43 -0400)
committerTom Lane <tgl@sss.pgh.pa.us>
Wed, 23 Jun 2021 01:43:12 +0000 (21:43 -0400)
commit13f3fd9e436d27af50aaebf5a1f2440740ee93eb
tree68a6571d6d873ece2c8a420ddfe0f97af5e81b72
parent0b29b41e5b9614397ba839552f35b4485ca49e0f
Use annotations to reduce instability of isolation-test results.

We've long contended with isolation test results that aren't entirely
stable.  Some test scripts insert long delays to try to force stable
results, which is not terribly desirable; but other erratic failure
modes remain, causing unrepeatable buildfarm failures.  I've spent a
fair amount of time trying to solve this by improving the server-side
support code, without much success: that way is fundamentally unable
to cope with diffs that stem from chance ordering of arrival of
messages from different server processes.

We can improve matters on the client side, however, by annotating
the test scripts themselves to show the desired reporting order
of events that might occur in different orders.  This patch adds
three types of annotations to deal with (a) test steps that might or
might not complete their waits before the isolationtester can see them
waiting; (b) test steps in different sessions that can legitimately
complete in either order; and (c) NOTIFY messages that might arrive
before or after the completion of a step in another session.  We might
need more annotation types later, but this seems to be enough to deal
with the instabilities we've seen in the buildfarm.  It also lets us
get rid of all the long delays that were previously used, cutting more
than a minute off the runtime of the isolation tests.

Back-patch to all supported branches, because the buildfarm
instabilities affect all the branches, and because it seems desirable
to keep isolationtester's capabilities the same across all branches
to simplify possible future back-patching of tests.

Discussion: https://postgr.es/m/327948.1623725828@sss.pgh.pa.us
36 files changed:
contrib/test_decoding/expected/concurrent_ddl_dml.out
src/test/isolation/README
src/test/isolation/expected/alter-table-3.out
src/test/isolation/expected/alter-table-4.out
src/test/isolation/expected/deadlock-hard.out
src/test/isolation/expected/deadlock-simple.out
src/test/isolation/expected/eval-plan-qual-trigger.out
src/test/isolation/expected/fk-deadlock2_1.out
src/test/isolation/expected/fk-deadlock2_2.out
src/test/isolation/expected/fk-deadlock_1.out
src/test/isolation/expected/insert-conflict-do-nothing-2.out
src/test/isolation/expected/lock-committed-keyupdate.out
src/test/isolation/expected/lock-update-delete_1.out
src/test/isolation/expected/multiple-cic.out
src/test/isolation/expected/multiple-cic_1.out [deleted file]
src/test/isolation/expected/multixact-no-forget_1.out
src/test/isolation/expected/nowait-4.out
src/test/isolation/expected/nowait-4_1.out
src/test/isolation/expected/nowait-5.out
src/test/isolation/expected/propagate-lock-delete.out
src/test/isolation/expected/read-write-unique-2.out
src/test/isolation/expected/read-write-unique-3.out
src/test/isolation/expected/read-write-unique-4.out
src/test/isolation/expected/read-write-unique.out
src/test/isolation/expected/skip-locked-4_1.out
src/test/isolation/expected/timeouts.out
src/test/isolation/expected/tuplelock-update.out
src/test/isolation/isolationtester.c
src/test/isolation/isolationtester.h
src/test/isolation/specparse.y
src/test/isolation/specs/deadlock-hard.spec
src/test/isolation/specs/deadlock-soft-2.spec
src/test/isolation/specs/multiple-cic.spec
src/test/isolation/specs/timeouts.spec
src/test/isolation/specs/tuplelock-update.spec
src/test/isolation/specscanner.l