From: drh <>
Date: Thu, 1 Jan 2026 15:41:50 +0000 (+0000)
Subject: Updates to the testrunner documentation.
X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=6e8afa79e794f7127481a4c5839baa7d72e1b2cd;p=thirdparty%2Fsqlite.git

Updates to the testrunner documentation.

FossilOrigin-Name: bcac99f962c6403fddfcaeef309ea9f825fa661fee0b3e3fd24f04c861be1d1b
---

diff --git a/doc/testrunner.md b/doc/testrunner.md
index d1696e9d1d..a79e517547 100644
--- a/doc/testrunner.md
+++ b/doc/testrunner.md
@@ -81,24 +81,21 @@ Both of the commands above accomplish about the same thing, but the second
 one has the advantage of not requiring "watch" to be installed on your
 system.
 
-Sometimes testrunner.tcl uses the `testfixture` binary that it is run with
-to run tests (see "Binary Tests" below). Sometimes it builds testfixture and
+Sometimes testrunner.tcl uses the `testfixture` and `sqlite3` binaries that
+are in the directory from which testrunner.tcl is run.
+(see "Binary Tests" below). Sometimes it builds testfixture and
 other binaries in specific configurations to test (see "Source Tests").
 
 <a name=binary_tests></a>
 # 2. Binary Tests
 
 The commands described in this section all run various combinations of the Tcl
-test scripts using the `testfixture` binary used to run the testrunner.tcl
-script (i.e. they do not invoke the compiler to build new binaries, or the
-`make` command to run tests that are not Tcl scripts). The procedure to run
-these tests is therefore:
-
-  1. Build the "testfixture" (or "testfixture.exe" for windows) binary using
-     whatever method seems convenient.
-
-  2. Test the binary built in step 1 by running testrunner.tcl with it, 
-     perhaps with various options.
+test scripts using whatever `testfixture` binary (and maybe also the `sqlite3`
+binary, depending on the test) that is found in the directory from which
+testrunner.tcl is launched.  So typically, one must first run something
+like "`make testfixture sqlite3`" before launching binary tests.  In other
+words, testrunner.tcl does not build the binaries under test itself; it
+expects them to be available already.
 
 The following sub-sections describe the various options that can be
 passed to testrunner.tcl to test binary testfixture builds.
@@ -140,22 +137,22 @@ are defined in file *testrunner_data.tcl*.
 To run the "veryquick" test set, use either of the following:
 
 ```
-  ./testfixture $TESTDIR/testrunner.tcl
-  ./testfixture $TESTDIR/testrunner.tcl veryquick
+  tclsh $TESTDIR/testrunner.tcl
+  tclsh $TESTDIR/testrunner.tcl veryquick
 ```
 
 To run the "full" test suite:
 
 ```
-  ./testfixture $TESTDIR/testrunner.tcl full
+  tclsh $TESTDIR/testrunner.tcl full
 ```
 
 To run the subset of the "full" test suite for which the test file name matches
 a specified pattern (e.g. all tests that start with "fts5"), either of:
 
 ```
-  ./testfixture $TESTDIR/testrunner.tcl fts5%
-  ./testfixture $TESTDIR/testrunner.tcl 'fts5*'
+  tclsh $TESTDIR/testrunner.tcl fts5%
+  tclsh $TESTDIR/testrunner.tcl 'fts5*'
 ```
 
 Strictly speaking, for a test to be run the pattern must match the script
@@ -167,9 +164,14 @@ characters specified as part of the pattern are transformed to "\*".
 To run "all" tests (full + permutations):
 
 ```
-  ./testfixture $TESTDIR/testrunner.tcl all
+  tclsh $TESTDIR/testrunner.tcl all
 ```
 
+Note that in all of the previous examples, when running on Unix, the
+initial "<tt>tclsh</tt>" on the command-line may be omitted.  The
+testrunner.tcl is marked as executable and begins with an appropriate
+"<tt>#!</tt>" line to invoke tclsh automatically.
+
 <a name=binary_test_failures></a>
 ## 2.3. Investigating Binary Test Failures
 
@@ -180,16 +182,21 @@ be retrieved from either *testrunner.log* or *testrunner.db*.
 If there is no permutation, the individual test script may be run with:
 
 ```
-  ./testfixture $PATH_TO_SCRIPT
+  tclsh $PATH_TO_SCRIPT
 ```
 
 Or, if the failure occured as part of a permutation:
 
 ```
-  ./testfixture $TESTDIR/testrunner.tcl $PERMUTATION $PATH_TO_SCRIPT
+  tclsh $TESTDIR/testrunner.tcl $PERMUTATION $PATH_TO_SCRIPT
 ```
 
-TODO: An example instead of "$PERMUTATION" and $PATH\_TO\_SCRIPT?
+One can also run all tests that failed in the previous invocation by
+typing:
+
+```
+  tclsh $TESTDIR/testrunner.tcl retest
+```
 
 <a name=source_code_tests></a>
 # 3. Source Code Tests
@@ -204,11 +211,9 @@ other tests. The advantages of this are that:
   *  it ensures that tests are always run using binaries created with the
      same set of compiler options.
 
-The testrunner.tcl commands described in this section may be run using
-either a *testfixture* (or testfixture.exe) build, or with any other Tcl
-shell that supports SQLite 3.31.1 or newer via "package require sqlite3".
-
-TODO: ./configure + Makefile.msc build systems.
+The testrunner.tcl commands described in this section do not require that
+the testfixture and/or sqlite3 binaries be built ahead of time.  Those
+binaries will be constructed automatically.
 
 <a name=commands_to_run_tests></a>
 ## 3.1. Commands to Run SQLite Tests
@@ -227,13 +232,13 @@ In other words, it is equivalent to running:
   $TOP/configure --enable-all --enable-debug
   make fuzztest
   make testfixture
-  ./testfixture $TOP/test/testrunner.tcl veryquick
+  tclsh $TOP/test/testrunner.tcl veryquick
 
   # Then, after removing files created by the tests above:
   $TOP/configure --enable-all OPTS="-O0"
   make fuzztest
   make testfixture
-  ./testfixture $TOP/test/testrunner.tcl veryquick
+  tclsh $TOP/test/testrunner.tcl veryquick
 ```
 
 The **sdevtest** command is identical to the mdevtest command, except that the
@@ -380,7 +385,7 @@ When running either binary or source code tests, testrunner.tcl reports the
 number of jobs it intends to use to stdout. e.g.
 
 ```
-  $ ./testfixture $TESTDIR/testrunner.tcl
+  $ tclsh $TESTDIR/testrunner.tcl
   splitting work across 16 jobs
   ... more output ...
 ```
@@ -390,7 +395,7 @@ of real cores on the machine. This can be overridden using the "--jobs" (or -j)
 switch:
 
 ```
-  $ ./testfixture $TESTDIR/testrunner.tcl --jobs 8
+  $ tclsh $TESTDIR/testrunner.tcl --jobs 8
   splitting work across 8 jobs
   ... more output ...
 ```
@@ -400,5 +405,5 @@ running by exucuting the following command from the directory containing the
 testrunner.log and testrunner.db files:
 
 ```
-  $ ./testfixture $TESTDIR/testrunner.tcl njob $NEW_NUMBER_OF_JOBS
+  $ tclsh $TESTDIR/testrunner.tcl njob $NEW_NUMBER_OF_JOBS
 ```
diff --git a/manifest b/manifest
index d624956c3f..88e8b0bcae 100644
--- a/manifest
+++ b/manifest
@@ -1,5 +1,5 @@
-C Verify\sthe\seffect\sof\s"--titles\son"\sin\sthe\s"insert"\smode\sin\sthe\sCLI.
-D 2025-12-31T20:43:31.772
+C Updates\sto\sthe\stestrunner\sdocumentation.
+D 2026-01-01T15:41:50.965
 F .fossil-settings/binary-glob 61195414528fb3ea9693577e1980230d78a1f8b0a54c78cf1b9b24d0a409ed6a x
 F .fossil-settings/empty-dirs dbb81e8fc0401ac46a1491ab34a7f2c7c0452f2f06b54ebb845d024ca8283ef1
 F .fossil-settings/ignore-glob 35175cdfcf539b2318cb04a9901442804be81cd677d8b889fcc9149c21f239ea
@@ -63,7 +63,7 @@ F doc/jsonb.md acd77fc3a709f51242655ad7803510c886aa8304202fa9cf2abc5f5c4e9d7ae5
 F doc/lemon.html 2085fda0a90a94fe92159a79dccc5c30d5a2313524887a31659cd66162f17233
 F doc/pager-invariants.txt 83aa3a4724b2d7970cc3f3461f0295c46d4fc19a835a5781cbb35cb52feb0577
 F doc/tcl-extension-testing.md b88861804fc1eaf83249f8e206334189b61e150c360e1b80d0dcf91af82354f5
-F doc/testrunner.md 5ee928637e03f136a25fef852c5ed975932e31927bd9b05a574424ae18c31019
+F doc/testrunner.md 1232d8095a15a6aa6fbb6685313558038dde49e6b0b7b1daebd5cbeec5d0abf9
 F doc/trusted-schema.md 33625008620e879c7bcfbbfa079587612c434fa094d338b08242288d358c3e8a
 F doc/vdbesort-memory.md 4da2639c14cd24a31e0af694b1a8dd37eaf277aff3867e9a8cc14046bc49df56
 F doc/vfs-shm.txt 1a55f3f0e7b6745931b117ba5c9df3640d7a0536f532ef0052563100f4416f86
@@ -2189,8 +2189,8 @@ F tool/warnings-clang.sh bbf6a1e685e534c92ec2bfba5b1745f34fb6f0bc2a362850723a9ee
 F tool/warnings.sh d924598cf2f55a4ecbc2aeb055c10bd5f48114793e7ba25f9585435da29e7e98
 F tool/win/sqlite.vsix deb315d026cc8400325c5863eef847784a219a2f
 F tool/winmain.c 00c8fb88e365c9017db14c73d3c78af62194d9644feaf60e220ab0f411f3604c
-P d2c5df41285a9c8a66e9bdea4e28d86fe10665ad01cfbab5a7ce43b8e39b4e34
-R 0430ecb6b12c6a794879456e6400ecec
+P a5f922b0832d9544b083f3d55d4029aa80e320bf5505547c5a1ddb21f4685e3e
+R 9df0730403c6e3e10d8224307f93708f
 U drh
-Z 30b7479fb2fd2c8bcd423b72d9fa94de
+Z 4765444ce4544ead9159cfe583142a35
 # Remove this line to create a well-formed Fossil manifest.
diff --git a/manifest.uuid b/manifest.uuid
index e96ef06b5d..13a3fa8d35 100644
--- a/manifest.uuid
+++ b/manifest.uuid
@@ -1 +1 @@
-a5f922b0832d9544b083f3d55d4029aa80e320bf5505547c5a1ddb21f4685e3e
+bcac99f962c6403fddfcaeef309ea9f825fa661fee0b3e3fd24f04c861be1d1b