]>
Commit | Line | Data |
---|---|---|
453dfd8d EK |
1 | # SSL tests |
2 | ||
3 | SSL testcases are configured in the `ssl-tests` directory. | |
4 | ||
5 | Each `ssl_*.conf.in` file contains a number of test configurations. These files | |
6 | are used to generate testcases in the OpenSSL CONF format. | |
7 | ||
8 | The precise test output can be dependent on the library configuration. The test | |
9 | harness generates the output files on the fly. | |
10 | ||
11 | However, for verification, we also include checked-in configuration outputs | |
12 | corresponding to the default configuration. These testcases live in | |
15269e56 | 13 | `test/ssl-tests/*.conf` files. |
453dfd8d EK |
14 | |
15 | For more details, see `ssl-tests/01-simple.conf.in` for an example. | |
16 | ||
17 | ## Configuring the test | |
18 | ||
19 | First, give your test a name. The names do not have to be unique. | |
20 | ||
21 | An example test input looks like this: | |
22 | ||
23 | ``` | |
24 | { | |
25 | name => "test-default", | |
26 | server => { "CipherString" => "DEFAULT" }, | |
27 | client => { "CipherString" => "DEFAULT" }, | |
28 | test => { "ExpectedResult" => "Success" }, | |
29 | } | |
30 | ``` | |
31 | ||
9f48bbac EK |
32 | The test section supports the following options |
33 | ||
34 | ### Test mode | |
35 | ||
36 | * Method - the method to test. One of DTLS or TLS. | |
37 | ||
38 | * HandshakeMode - which handshake flavour to test: | |
39 | - Simple - plain handshake (default) | |
40 | - Resume - test resumption | |
fe7dd553 MC |
41 | - RenegotiateServer - test server initiated renegotiation |
42 | - RenegotiateClient - test client initiated renegotiation | |
9f48bbac EK |
43 | |
44 | When HandshakeMode is Resume or Renegotiate, the original handshake is expected | |
45 | to succeed. All configured test expectations are verified against the second | |
46 | handshake. | |
47 | ||
6dc99745 EK |
48 | * ApplicationData - amount of application data bytes to send (integer, defaults |
49 | to 256 bytes). Applies to both client and server. Application data is sent in | |
50 | 64kB chunks (but limited by MaxFragmentSize and available parallelization, see | |
51 | below). | |
52 | ||
53 | * MaxFragmentSize - maximum send fragment size (integer, defaults to 512 in | |
54 | tests - see `SSL_CTX_set_max_send_fragment` for documentation). Applies to | |
55 | both client and server. Lowering the fragment size will split handshake and | |
56 | application data up between more `SSL_write` calls, thus allowing to exercise | |
57 | different code paths. In particular, if the buffer size (64kB) is at least | |
58 | four times as large as the maximum fragment, interleaved multi-buffer crypto | |
59 | implementations may be used on some platforms. | |
60 | ||
9f48bbac | 61 | ### Test expectations |
453dfd8d EK |
62 | |
63 | * ExpectedResult - expected handshake outcome. One of | |
64 | - Success - handshake success | |
65 | - ServerFail - serverside handshake failure | |
66 | - ClientFail - clientside handshake failure | |
67 | - InternalError - some other error | |
68 | ||
9f48bbac | 69 | * ExpectedClientAlert, ExpectedServerAlert - expected alert. See |
dd8e5a57 EK |
70 | `ssl_test_ctx.c` for known values. Note: the expected alert is currently |
71 | matched against the _last_ received alert (i.e., a fatal alert or a | |
72 | `close_notify`). Warning alert expectations are not yet supported. (A warning | |
73 | alert will not be correctly matched, if followed by a `close_notify` or | |
74 | another alert.) | |
453dfd8d | 75 | |
9f48bbac | 76 | * ExpectedProtocol - expected negotiated protocol. One of |
453dfd8d EK |
77 | SSLv3, TLSv1, TLSv1.1, TLSv1.2. |
78 | ||
5c753de6 TS |
79 | * SessionTicketExpected - whether or not a session ticket is expected |
80 | - Ignore - do not check for a session ticket (default) | |
81 | - Yes - a session ticket is expected | |
82 | - No - a session ticket is not expected | |
590ed3d7 EK |
83 | |
84 | * ResumptionExpected - whether or not resumption is expected (Resume mode only) | |
85 | - Yes - resumed handshake | |
86 | - No - full handshake (default) | |
87 | ||
9f48bbac | 88 | * ExpectedNPNProtocol, ExpectedALPNProtocol - NPN and ALPN expectations. |
ce2cdac2 | 89 | |
b93ad05d DSH |
90 | * ExpectedTmpKeyType - the expected algorithm or curve of server temp key |
91 | ||
7289ab49 | 92 | * ExpectedServerCertType, ExpectedClientCertType - the expected algorithm or |
7f5f35af DSH |
93 | curve of server or client certificate |
94 | ||
54b7f2a5 | 95 | * ExpectedServerSignHash, ExpectedClientSignHash - the expected |
ee5b6a42 DSH |
96 | signing hash used by server or client certificate |
97 | ||
54b7f2a5 DSH |
98 | * ExpectedServerSignType, ExpectedClientSignType - the expected |
99 | signature type used by server or client when signing messages | |
100 | ||
2e21539b DSH |
101 | * ExpectedClientCANames - for client auth list of CA names the server must |
102 | send. If this is "empty" the list is expected to be empty otherwise it | |
103 | is a file of certificates whose subject names form the list. | |
104 | ||
f15b50c4 DSH |
105 | * ExpectedServerCANames - list of CA names the client must send, TLS 1.3 only. |
106 | If this is "empty" the list is expected to be empty otherwise it is a file | |
107 | of certificates whose subject names form the list. | |
108 | ||
453dfd8d EK |
109 | ## Configuring the client and server |
110 | ||
111 | The client and server configurations can be any valid `SSL_CTX` | |
112 | configurations. For details, see the manpages for `SSL_CONF_cmd`. | |
113 | ||
114 | Give your configurations as a dictionary of CONF commands, e.g. | |
115 | ||
116 | ``` | |
117 | server => { | |
118 | "CipherString" => "DEFAULT", | |
119 | "MinProtocol" => "TLSv1", | |
120 | } | |
121 | ``` | |
122 | ||
590ed3d7 EK |
123 | The following sections may optionally be defined: |
124 | ||
125 | * server2 - this section configures a secondary context that is selected via the | |
126 | ServerName test option. This context is used whenever a ServerNameCallback is | |
127 | specified. If the server2 section is not present, then the configuration | |
128 | matches server. | |
129 | * resume_server - this section configures the client to resume its session | |
130 | against a different server. This context is used whenever HandshakeMode is | |
11279b13 | 131 | Resume. If the resume_server section is not present, then the configuration |
590ed3d7 | 132 | matches server. |
11279b13 EK |
133 | * resume_client - this section configures the client to resume its session with |
134 | a different configuration. In practice this may occur when, for example, | |
135 | upgraded clients reuse sessions persisted on disk. This context is used | |
136 | whenever HandshakeMode is Resume. If the resume_client section is not present, | |
137 | then the configuration matches client. | |
5c753de6 | 138 | |
9f48bbac EK |
139 | ### Configuring callbacks and additional options |
140 | ||
141 | Additional handshake settings can be configured in the `extra` section of each | |
142 | client and server: | |
143 | ||
144 | ``` | |
145 | client => { | |
146 | "CipherString" => "DEFAULT", | |
147 | extra => { | |
148 | "ServerName" => "server2", | |
149 | } | |
150 | } | |
151 | ``` | |
152 | ||
153 | #### Supported client-side options | |
154 | ||
155 | * ClientVerifyCallback - the client's custom certificate verify callback. | |
156 | Used to test callback behaviour. One of | |
157 | - None - no custom callback (default) | |
158 | - AcceptAll - accepts all certificates. | |
159 | - RejectAll - rejects all certificates. | |
160 | ||
161 | * ServerName - the server the client should attempt to connect to. One of | |
162 | - None - do not use SNI (default) | |
163 | - server1 - the initial context | |
164 | - server2 - the secondary context | |
165 | - invalid - an unknown context | |
166 | ||
da085d27 EK |
167 | * CTValidation - Certificate Transparency validation strategy. One of |
168 | - None - no validation (default) | |
169 | - Permissive - SSL_CT_VALIDATION_PERMISSIVE | |
170 | - Strict - SSL_CT_VALIDATION_STRICT | |
171 | ||
9f48bbac EK |
172 | #### Supported server-side options |
173 | ||
174 | * ServerNameCallback - the SNI switching callback to use | |
175 | - None - no callback (default) | |
176 | - IgnoreMismatch - continue the handshake on SNI mismatch | |
177 | - RejectMismatch - abort the handshake on SNI mismatch | |
178 | ||
179 | * BrokenSessionTicket - a special test case where the session ticket callback | |
180 | does not initialize crypto. | |
181 | - No (default) | |
182 | - Yes | |
183 | ||
184 | #### Mutually supported options | |
185 | ||
186 | * NPNProtocols, ALPNProtocols - NPN and ALPN settings. Server and client | |
187 | protocols can be specified as a comma-separated list, and a callback with the | |
188 | recommended behaviour will be installed automatically. | |
189 | ||
ea1ecd98 EK |
190 | * SRPUser, SRPPassword - SRP settings. For client, this is the SRP user to |
191 | connect as; for server, this is a known SRP user. | |
192 | ||
453dfd8d EK |
193 | ### Default server and client configurations |
194 | ||
195 | The default server certificate and CA files are added to the configurations | |
196 | automatically. Server certificate verification is requested by default. | |
197 | ||
198 | You can override these options by redefining them: | |
199 | ||
200 | ``` | |
201 | client => { | |
202 | "VerifyCAFile" => "/path/to/custom/file" | |
203 | } | |
204 | ``` | |
205 | ||
206 | or by deleting them | |
207 | ||
208 | ``` | |
209 | client => { | |
210 | "VerifyCAFile" => undef | |
211 | } | |
212 | ``` | |
213 | ||
214 | ## Adding a test to the test harness | |
215 | ||
15269e56 EK |
216 | 1. Add a new test configuration to `test/ssl-tests`, following the examples of |
217 | existing `*.conf.in` files (for example, `01-simple.conf.in`). | |
218 | ||
219 | 2. Generate the generated `*.conf` test input file. You can do so by running | |
220 | `generate_ssl_tests.pl`: | |
221 | ||
222 | ``` | |
223 | $ ./config | |
224 | $ cd test | |
f90486f4 | 225 | $ TOP=.. perl -I ../util/perl/ generate_ssl_tests.pl ssl-tests/my.conf.in \ |
15269e56 EK |
226 | > ssl-tests/my.conf |
227 | ``` | |
228 | ||
229 | where `my.conf.in` is your test input file. | |
230 | ||
231 | For example, to generate the test cases in `ssl-tests/01-simple.conf.in`, do | |
232 | ||
233 | ``` | |
f90486f4 | 234 | $ TOP=.. perl -I ../util/perl/ generate_ssl_tests.pl ssl-tests/01-simple.conf.in > ssl-tests/01-simple.conf |
15269e56 EK |
235 | ``` |
236 | ||
237 | Alternatively (hackish but simple), you can comment out | |
238 | ||
239 | ``` | |
240 | unlink glob $tmp_file; | |
241 | ``` | |
242 | ||
243 | in `test/recipes/80-test_ssl_new.t` and run | |
244 | ||
245 | ``` | |
246 | $ make TESTS=test_ssl_new test | |
247 | ``` | |
248 | ||
249 | This will save the generated output in a `*.tmp` file in the build directory. | |
250 | ||
251 | 3. Update the number of tests planned in `test/recipes/80-test_ssl_new.t`. If | |
252 | the test suite has any skip conditions, update those too (see | |
253 | `test/recipes/80-test_ssl_new.t` for details). | |
453dfd8d EK |
254 | |
255 | ## Running the tests with the test harness | |
256 | ||
257 | ``` | |
258 | HARNESS_VERBOSE=yes make TESTS=test_ssl_new test | |
259 | ``` | |
260 | ||
261 | ## Running a test manually | |
262 | ||
263 | These steps are only needed during development. End users should run `make test` | |
264 | or follow the instructions above to run the SSL test suite. | |
265 | ||
266 | To run an SSL test manually from the command line, the `TEST_CERTS_DIR` | |
267 | environment variable to point to the location of the certs. E.g., from the root | |
268 | OpenSSL directory, do | |
269 | ||
270 | ``` | |
1329b952 MC |
271 | $ CTLOG_FILE=test/ct/log_list.conf TEST_CERTS_DIR=test/certs test/ssl_test \ |
272 | test/ssl-tests/01-simple.conf | |
453dfd8d EK |
273 | ``` |
274 | ||
275 | or for shared builds | |
276 | ||
277 | ``` | |
1329b952 MC |
278 | $ CTLOG_FILE=test/ct/log_list.conf TEST_CERTS_DIR=test/certs \ |
279 | util/shlib_wrap.sh test/ssl_test test/ssl-tests/01-simple.conf | |
453dfd8d EK |
280 | ``` |
281 | ||
282 | Note that the test expectations sometimes depend on the Configure settings. For | |
283 | example, the negotiated protocol depends on the set of available (enabled) | |
284 | protocols: a build with `enable-ssl3` has different test expectations than a | |
285 | build with `no-ssl3`. | |
286 | ||
287 | The Perl test harness automatically generates expected outputs, so users who | |
288 | just run `make test` do not need any extra steps. | |
289 | ||
290 | However, when running a test manually, keep in mind that the repository version | |
291 | of the generated `test/ssl-tests/*.conf` correspond to expected outputs in with | |
292 | the default Configure options. To run `ssl_test` manually from the command line | |
293 | in a build with a different configuration, you may need to generate the right | |
294 | `*.conf` file from the `*.conf.in` input first. |