From c678f68a19638c1e2bbfee6a7a1d8d728976ce66 Mon Sep 17 00:00:00 2001 From: Pauli Date: Sat, 12 Dec 2020 21:38:17 +1000 Subject: [PATCH] test: document the random test ordering env variable Reviewed-by: Tomas Mraz (Merged from https://github.com/openssl/openssl/pull/13672) --- test/README.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/test/README.md b/test/README.md index 43f8471120e..9a5c91953dd 100644 --- a/test/README.md +++ b/test/README.md @@ -148,3 +148,23 @@ for `TAP::Harness` to know more. To run up to four tests in parallel at any given time: $ make HARNESS_JOBS=4 test + +Randomisation of Test Ordering +------------------------------ + +By default, the test harness will execute tests in the order they were added. +By setting the `OPENSSL_TEST_RAND_ORDER` environment variable to zero, the +test ordering will be randomised. If a randomly ordered test fails, the +seed value used will be reported. Setting the `OPENSSL_TEST_RAND_ORDER` +environment variable to this value will rerun the tests in the same +order. This assures repeatability of randomly ordered test runs. +This repeatability is independent of the operating system, processor or +platform used. + +To randomise the test ordering: + + $ make OPENSSL_TEST_RAND_ORDER=0 test + +To run the tests using the order defined by the random seed `42`: + + $ make OPENSSL_TEST_RAND_ORDER=42 test -- 2.47.2