tests: add the timing-dependent keyword on several tests

These are ones likely to fail on heavily-loaded machines that alter the
normal test timing. Most of these tests already had the flaky keyword
since this condition makes them more likely to fail on CI.

diff --git a/tests/FILEFORMAT.md b/tests/FILEFORMAT.md
index 57bfb4f5bb1e42bb9970e578845c44d66cf9589f..665d93eb855a559de0ece21486b9375f70e9d9a5 100644 (file)
--- a/tests/FILEFORMAT.md
+++ b/tests/FILEFORMAT.md
@@ -190,7 +190,9 @@ tests. Try to use already used keywords. These keywords will be used for
 statistical/informational purposes and for choosing or skipping classes of
 tests. Keywords must begin with an alphabetic character, `-`, `[` or `{` and
 may actually consist of multiple words separated by spaces which are treated
-together as a single identifier.
+together as a single identifier. Most keywords are only there to provide a way
+for users to skip certain classes of tests, if desired, but a few are treated
+specially by the test harness or build system.
 
 When using curl built with Hyper, the keywords must include `HTTP` or `HTTPS`
 for 'hyper mode' to kick in and make line ending checks work for tests.
@@ -199,6 +201,12 @@ When running a unit test and the keywords include `unittest`, the `<tool>`
 section can be left empty to use the standard unit test tool name `unitN` where
 `N` is the test number.
 
+The `text-ci` make target automatically skips test with the `flaky` keyword.
+
+Tests that have strict timing dependencies have the `timing-dependent` keyword.
+These are intended to eventually be treated specially on CI builds which are
+often run on overloaded machines with unpredictable timing.
+
 ## `<reply>`
 
 ### `<data [nocheck="yes"] [sendzero="yes"] [base64="yes"] [hex="yes"] [nonewline="yes"] [crlf="yes"]>`

diff --git a/tests/data/test1112 b/tests/data/test1112
index eb4d6f02f85c4f744d89807cd766c11748cd86ad..0efcf42ba625842c3aa8939a88f4c3050941212c 100644 (file)
--- a/tests/data/test1112
+++ b/tests/data/test1112
@@ -1,4 +1,5 @@
 <testcase>
+# This is the ftps equivalent of test1086
 <info>
 <keywords>
 FTPS
@@ -7,6 +8,7 @@ RETR
 timeout
 FAILURE
 SLOWDOWN
+timing-dependent
 </keywords>
 </info>
 

diff --git a/tests/data/test1206 b/tests/data/test1206
index b192592700401798bd69df5f44edb784c339f60e..453b991084434e24f2ad343f40ddcb9ceb0242c7 100644 (file)
--- a/tests/data/test1206
+++ b/tests/data/test1206
@@ -46,6 +46,7 @@ SIZE %TESTNUMBER
 RETR %TESTNUMBER\r
 QUIT\r
 </protocol>
+10 is CURLE_FTP_ACCEPT_FAILED
 <errorcode>
 10
 </errorcode>

diff --git a/tests/data/test1208 b/tests/data/test1208
index 2de619d90d6cf223f24acf129ba8ef0d3c0ae4ee..57937a967222183d9543f29525e5b4e07d914cb9 100644 (file)
--- a/tests/data/test1208
+++ b/tests/data/test1208
@@ -5,9 +5,10 @@ FTP
 PORT
 RETR
 NODATACONN150
-flaky
 timeout
 FAILURE
+flaky
+timing-dependent
 </keywords>
 </info>
 # Server-side
@@ -53,6 +54,7 @@ TYPE I
 SIZE %TESTNUMBER\r
 RETR %TESTNUMBER\r
 </protocol>
+# 28 is CURLE_OPERATION_TIMEDOUT
 <errorcode>
 28
 </errorcode>

diff --git a/tests/data/test1474 b/tests/data/test1474
index 8ca2cb8cc34a2af3e226f4fd7dcc55eb69fe28dc..848f15211b6f105550d14cbb55ab58e835502925 100644 (file)
--- a/tests/data/test1474
+++ b/tests/data/test1474
@@ -32,6 +32,7 @@ HTTP
 HTTP PUT
 Expect
 flaky
+timing-dependent
 </keywords>
 </info>
 # Server-side

diff --git a/tests/data/test1592 b/tests/data/test1592
index 8de07341ef5b14880ef5ed1519da170ff8b9568e..6da96dc49228ba0557f8cafa0bd5e5ca44e785a9 100644 (file)
--- a/tests/data/test1592
+++ b/tests/data/test1592
@@ -6,6 +6,7 @@ multi
 resolve
 speedcheck
 flaky
+timing-dependent
 </keywords>
 </info>
 

diff --git a/tests/data/test2032 b/tests/data/test2032
index 72c7cc4d210aee01b16664ff81be3c3a607689c7..d3d199c14c1c6122906aa01bc1c06ccbf1d847c4 100644 (file)
--- a/tests/data/test2032
+++ b/tests/data/test2032
@@ -5,8 +5,9 @@ HTTP
 HTTP GET
 HTTP Basic auth
 HTTP NTLM auth
-flaky
 NTLM
+flaky
+timing-dependent
 </keywords>
 </info>
 # Server-side

diff --git a/tests/data/test2600 b/tests/data/test2600
index d6d07256d746cb418cdb135050c304aed100483b..829ea0c6fe7d88c99ba92711c441e116c536f5c8 100644 (file)
--- a/tests/data/test2600
+++ b/tests/data/test2600
@@ -3,6 +3,7 @@
 <keywords>
 unittest
 cfilter
+timing-dependent
 </keywords>
 </info>