]>
Commit | Line | Data |
---|---|---|
453dfd8d EK |
1 | # SSL tests |
2 | ||
3 | SSL testcases are configured in the `ssl-tests` directory. | |
4 | ||
5 | Each `ssl_*.conf.in` file contains a number of test configurations. These files | |
6 | are used to generate testcases in the OpenSSL CONF format. | |
7 | ||
8 | The precise test output can be dependent on the library configuration. The test | |
9 | harness generates the output files on the fly. | |
10 | ||
11 | However, for verification, we also include checked-in configuration outputs | |
12 | corresponding to the default configuration. These testcases live in | |
15269e56 | 13 | `test/ssl-tests/*.conf` files. |
453dfd8d EK |
14 | |
15 | For more details, see `ssl-tests/01-simple.conf.in` for an example. | |
16 | ||
17 | ## Configuring the test | |
18 | ||
19 | First, give your test a name. The names do not have to be unique. | |
20 | ||
21 | An example test input looks like this: | |
22 | ||
23 | ``` | |
24 | { | |
25 | name => "test-default", | |
26 | server => { "CipherString" => "DEFAULT" }, | |
27 | client => { "CipherString" => "DEFAULT" }, | |
28 | test => { "ExpectedResult" => "Success" }, | |
29 | } | |
30 | ``` | |
31 | ||
9f48bbac EK |
32 | The test section supports the following options |
33 | ||
34 | ### Test mode | |
35 | ||
36 | * Method - the method to test. One of DTLS or TLS. | |
37 | ||
38 | * HandshakeMode - which handshake flavour to test: | |
39 | - Simple - plain handshake (default) | |
40 | - Resume - test resumption | |
fe7dd553 MC |
41 | - RenegotiateServer - test server initiated renegotiation |
42 | - RenegotiateClient - test client initiated renegotiation | |
9f48bbac EK |
43 | |
44 | When HandshakeMode is Resume or Renegotiate, the original handshake is expected | |
45 | to succeed. All configured test expectations are verified against the second | |
46 | handshake. | |
47 | ||
6dc99745 EK |
48 | * ApplicationData - amount of application data bytes to send (integer, defaults |
49 | to 256 bytes). Applies to both client and server. Application data is sent in | |
50 | 64kB chunks (but limited by MaxFragmentSize and available parallelization, see | |
51 | below). | |
52 | ||
53 | * MaxFragmentSize - maximum send fragment size (integer, defaults to 512 in | |
54 | tests - see `SSL_CTX_set_max_send_fragment` for documentation). Applies to | |
55 | both client and server. Lowering the fragment size will split handshake and | |
56 | application data up between more `SSL_write` calls, thus allowing to exercise | |
57 | different code paths. In particular, if the buffer size (64kB) is at least | |
58 | four times as large as the maximum fragment, interleaved multi-buffer crypto | |
59 | implementations may be used on some platforms. | |
60 | ||
9f48bbac | 61 | ### Test expectations |
453dfd8d EK |
62 | |
63 | * ExpectedResult - expected handshake outcome. One of | |
64 | - Success - handshake success | |
65 | - ServerFail - serverside handshake failure | |
66 | - ClientFail - clientside handshake failure | |
67 | - InternalError - some other error | |
68 | ||
9f48bbac | 69 | * ExpectedClientAlert, ExpectedServerAlert - expected alert. See |
dd8e5a57 EK |
70 | `ssl_test_ctx.c` for known values. Note: the expected alert is currently |
71 | matched against the _last_ received alert (i.e., a fatal alert or a | |
72 | `close_notify`). Warning alert expectations are not yet supported. (A warning | |
73 | alert will not be correctly matched, if followed by a `close_notify` or | |
74 | another alert.) | |
453dfd8d | 75 | |
9f48bbac | 76 | * ExpectedProtocol - expected negotiated protocol. One of |
453dfd8d EK |
77 | SSLv3, TLSv1, TLSv1.1, TLSv1.2. |
78 | ||
5c753de6 TS |
79 | * SessionTicketExpected - whether or not a session ticket is expected |
80 | - Ignore - do not check for a session ticket (default) | |
81 | - Yes - a session ticket is expected | |
82 | - No - a session ticket is not expected | |
590ed3d7 | 83 | |
a84e5c9a TS |
84 | * SessionIdExpected - whether or not a session id is expected |
85 | - Ignore - do not check for a session id (default) | |
86 | - Yes - a session id is expected | |
87 | - No - a session id is not expected | |
88 | ||
590ed3d7 EK |
89 | * ResumptionExpected - whether or not resumption is expected (Resume mode only) |
90 | - Yes - resumed handshake | |
91 | - No - full handshake (default) | |
92 | ||
9f48bbac | 93 | * ExpectedNPNProtocol, ExpectedALPNProtocol - NPN and ALPN expectations. |
ce2cdac2 | 94 | |
b93ad05d DSH |
95 | * ExpectedTmpKeyType - the expected algorithm or curve of server temp key |
96 | ||
7289ab49 | 97 | * ExpectedServerCertType, ExpectedClientCertType - the expected algorithm or |
7f5f35af DSH |
98 | curve of server or client certificate |
99 | ||
54b7f2a5 | 100 | * ExpectedServerSignHash, ExpectedClientSignHash - the expected |
ee5b6a42 DSH |
101 | signing hash used by server or client certificate |
102 | ||
54b7f2a5 DSH |
103 | * ExpectedServerSignType, ExpectedClientSignType - the expected |
104 | signature type used by server or client when signing messages | |
105 | ||
2e21539b DSH |
106 | * ExpectedClientCANames - for client auth list of CA names the server must |
107 | send. If this is "empty" the list is expected to be empty otherwise it | |
108 | is a file of certificates whose subject names form the list. | |
109 | ||
f15b50c4 DSH |
110 | * ExpectedServerCANames - list of CA names the client must send, TLS 1.3 only. |
111 | If this is "empty" the list is expected to be empty otherwise it is a file | |
112 | of certificates whose subject names form the list. | |
113 | ||
453dfd8d EK |
114 | ## Configuring the client and server |
115 | ||
116 | The client and server configurations can be any valid `SSL_CTX` | |
117 | configurations. For details, see the manpages for `SSL_CONF_cmd`. | |
118 | ||
119 | Give your configurations as a dictionary of CONF commands, e.g. | |
120 | ||
121 | ``` | |
122 | server => { | |
123 | "CipherString" => "DEFAULT", | |
124 | "MinProtocol" => "TLSv1", | |
125 | } | |
126 | ``` | |
127 | ||
590ed3d7 EK |
128 | The following sections may optionally be defined: |
129 | ||
130 | * server2 - this section configures a secondary context that is selected via the | |
131 | ServerName test option. This context is used whenever a ServerNameCallback is | |
132 | specified. If the server2 section is not present, then the configuration | |
133 | matches server. | |
134 | * resume_server - this section configures the client to resume its session | |
135 | against a different server. This context is used whenever HandshakeMode is | |
11279b13 | 136 | Resume. If the resume_server section is not present, then the configuration |
590ed3d7 | 137 | matches server. |
11279b13 EK |
138 | * resume_client - this section configures the client to resume its session with |
139 | a different configuration. In practice this may occur when, for example, | |
140 | upgraded clients reuse sessions persisted on disk. This context is used | |
141 | whenever HandshakeMode is Resume. If the resume_client section is not present, | |
142 | then the configuration matches client. | |
5c753de6 | 143 | |
9f48bbac EK |
144 | ### Configuring callbacks and additional options |
145 | ||
146 | Additional handshake settings can be configured in the `extra` section of each | |
147 | client and server: | |
148 | ||
149 | ``` | |
150 | client => { | |
151 | "CipherString" => "DEFAULT", | |
152 | extra => { | |
153 | "ServerName" => "server2", | |
154 | } | |
155 | } | |
156 | ``` | |
157 | ||
158 | #### Supported client-side options | |
159 | ||
160 | * ClientVerifyCallback - the client's custom certificate verify callback. | |
161 | Used to test callback behaviour. One of | |
162 | - None - no custom callback (default) | |
163 | - AcceptAll - accepts all certificates. | |
164 | - RejectAll - rejects all certificates. | |
165 | ||
166 | * ServerName - the server the client should attempt to connect to. One of | |
167 | - None - do not use SNI (default) | |
168 | - server1 - the initial context | |
169 | - server2 - the secondary context | |
170 | - invalid - an unknown context | |
171 | ||
da085d27 EK |
172 | * CTValidation - Certificate Transparency validation strategy. One of |
173 | - None - no validation (default) | |
174 | - Permissive - SSL_CT_VALIDATION_PERMISSIVE | |
175 | - Strict - SSL_CT_VALIDATION_STRICT | |
176 | ||
9f48bbac EK |
177 | #### Supported server-side options |
178 | ||
179 | * ServerNameCallback - the SNI switching callback to use | |
180 | - None - no callback (default) | |
181 | - IgnoreMismatch - continue the handshake on SNI mismatch | |
182 | - RejectMismatch - abort the handshake on SNI mismatch | |
183 | ||
184 | * BrokenSessionTicket - a special test case where the session ticket callback | |
185 | does not initialize crypto. | |
186 | - No (default) | |
187 | - Yes | |
188 | ||
189 | #### Mutually supported options | |
190 | ||
191 | * NPNProtocols, ALPNProtocols - NPN and ALPN settings. Server and client | |
192 | protocols can be specified as a comma-separated list, and a callback with the | |
193 | recommended behaviour will be installed automatically. | |
194 | ||
ea1ecd98 EK |
195 | * SRPUser, SRPPassword - SRP settings. For client, this is the SRP user to |
196 | connect as; for server, this is a known SRP user. | |
197 | ||
453dfd8d EK |
198 | ### Default server and client configurations |
199 | ||
200 | The default server certificate and CA files are added to the configurations | |
201 | automatically. Server certificate verification is requested by default. | |
202 | ||
203 | You can override these options by redefining them: | |
204 | ||
205 | ``` | |
206 | client => { | |
207 | "VerifyCAFile" => "/path/to/custom/file" | |
208 | } | |
209 | ``` | |
210 | ||
211 | or by deleting them | |
212 | ||
213 | ``` | |
214 | client => { | |
215 | "VerifyCAFile" => undef | |
216 | } | |
217 | ``` | |
218 | ||
219 | ## Adding a test to the test harness | |
220 | ||
15269e56 EK |
221 | 1. Add a new test configuration to `test/ssl-tests`, following the examples of |
222 | existing `*.conf.in` files (for example, `01-simple.conf.in`). | |
223 | ||
224 | 2. Generate the generated `*.conf` test input file. You can do so by running | |
225 | `generate_ssl_tests.pl`: | |
226 | ||
227 | ``` | |
228 | $ ./config | |
229 | $ cd test | |
f90486f4 | 230 | $ TOP=.. perl -I ../util/perl/ generate_ssl_tests.pl ssl-tests/my.conf.in \ |
15269e56 EK |
231 | > ssl-tests/my.conf |
232 | ``` | |
233 | ||
234 | where `my.conf.in` is your test input file. | |
235 | ||
236 | For example, to generate the test cases in `ssl-tests/01-simple.conf.in`, do | |
237 | ||
238 | ``` | |
f90486f4 | 239 | $ TOP=.. perl -I ../util/perl/ generate_ssl_tests.pl ssl-tests/01-simple.conf.in > ssl-tests/01-simple.conf |
15269e56 EK |
240 | ``` |
241 | ||
242 | Alternatively (hackish but simple), you can comment out | |
243 | ||
244 | ``` | |
245 | unlink glob $tmp_file; | |
246 | ``` | |
247 | ||
248 | in `test/recipes/80-test_ssl_new.t` and run | |
249 | ||
250 | ``` | |
251 | $ make TESTS=test_ssl_new test | |
252 | ``` | |
253 | ||
254 | This will save the generated output in a `*.tmp` file in the build directory. | |
255 | ||
256 | 3. Update the number of tests planned in `test/recipes/80-test_ssl_new.t`. If | |
257 | the test suite has any skip conditions, update those too (see | |
258 | `test/recipes/80-test_ssl_new.t` for details). | |
453dfd8d EK |
259 | |
260 | ## Running the tests with the test harness | |
261 | ||
262 | ``` | |
263 | HARNESS_VERBOSE=yes make TESTS=test_ssl_new test | |
264 | ``` | |
265 | ||
266 | ## Running a test manually | |
267 | ||
268 | These steps are only needed during development. End users should run `make test` | |
269 | or follow the instructions above to run the SSL test suite. | |
270 | ||
271 | To run an SSL test manually from the command line, the `TEST_CERTS_DIR` | |
272 | environment variable to point to the location of the certs. E.g., from the root | |
273 | OpenSSL directory, do | |
274 | ||
275 | ``` | |
1329b952 MC |
276 | $ CTLOG_FILE=test/ct/log_list.conf TEST_CERTS_DIR=test/certs test/ssl_test \ |
277 | test/ssl-tests/01-simple.conf | |
453dfd8d EK |
278 | ``` |
279 | ||
280 | or for shared builds | |
281 | ||
282 | ``` | |
1329b952 MC |
283 | $ CTLOG_FILE=test/ct/log_list.conf TEST_CERTS_DIR=test/certs \ |
284 | util/shlib_wrap.sh test/ssl_test test/ssl-tests/01-simple.conf | |
453dfd8d EK |
285 | ``` |
286 | ||
287 | Note that the test expectations sometimes depend on the Configure settings. For | |
288 | example, the negotiated protocol depends on the set of available (enabled) | |
289 | protocols: a build with `enable-ssl3` has different test expectations than a | |
290 | build with `no-ssl3`. | |
291 | ||
292 | The Perl test harness automatically generates expected outputs, so users who | |
293 | just run `make test` do not need any extra steps. | |
294 | ||
295 | However, when running a test manually, keep in mind that the repository version | |
296 | of the generated `test/ssl-tests/*.conf` correspond to expected outputs in with | |
297 | the default Configure options. To run `ssl_test` manually from the command line | |
298 | in a build with a different configuration, you may need to generate the right | |
299 | `*.conf` file from the `*.conf.in` input first. |