]>
Commit | Line | Data |
---|---|---|
257e9d03 RS |
1 | SSL tests |
2 | ========= | |
453dfd8d EK |
3 | |
4 | SSL testcases are configured in the `ssl-tests` directory. | |
5 | ||
433deaff | 6 | Each `ssl_*.cnf.in` file contains a number of test configurations. These files |
453dfd8d EK |
7 | are used to generate testcases in the OpenSSL CONF format. |
8 | ||
9 | The precise test output can be dependent on the library configuration. The test | |
10 | harness generates the output files on the fly. | |
11 | ||
12 | However, for verification, we also include checked-in configuration outputs | |
13 | corresponding to the default configuration. These testcases live in | |
433deaff | 14 | `test/ssl-tests/*.cnf` files. |
453dfd8d | 15 | |
433deaff | 16 | For more details, see `ssl-tests/01-simple.cnf.in` for an example. |
453dfd8d | 17 | |
257e9d03 RS |
18 | Configuring the test |
19 | -------------------- | |
453dfd8d EK |
20 | |
21 | First, give your test a name. The names do not have to be unique. | |
22 | ||
23 | An example test input looks like this: | |
24 | ||
453dfd8d EK |
25 | { |
26 | name => "test-default", | |
27 | server => { "CipherString" => "DEFAULT" }, | |
28 | client => { "CipherString" => "DEFAULT" }, | |
29 | test => { "ExpectedResult" => "Success" }, | |
30 | } | |
453dfd8d | 31 | |
9f48bbac EK |
32 | The test section supports the following options |
33 | ||
34 | ### Test mode | |
35 | ||
36 | * Method - the method to test. One of DTLS or TLS. | |
37 | ||
38 | * HandshakeMode - which handshake flavour to test: | |
39 | - Simple - plain handshake (default) | |
40 | - Resume - test resumption | |
fe7dd553 MC |
41 | - RenegotiateServer - test server initiated renegotiation |
42 | - RenegotiateClient - test client initiated renegotiation | |
9f48bbac EK |
43 | |
44 | When HandshakeMode is Resume or Renegotiate, the original handshake is expected | |
45 | to succeed. All configured test expectations are verified against the second | |
46 | handshake. | |
47 | ||
6dc99745 EK |
48 | * ApplicationData - amount of application data bytes to send (integer, defaults |
49 | to 256 bytes). Applies to both client and server. Application data is sent in | |
50 | 64kB chunks (but limited by MaxFragmentSize and available parallelization, see | |
51 | below). | |
52 | ||
53 | * MaxFragmentSize - maximum send fragment size (integer, defaults to 512 in | |
54 | tests - see `SSL_CTX_set_max_send_fragment` for documentation). Applies to | |
55 | both client and server. Lowering the fragment size will split handshake and | |
56 | application data up between more `SSL_write` calls, thus allowing to exercise | |
57 | different code paths. In particular, if the buffer size (64kB) is at least | |
58 | four times as large as the maximum fragment, interleaved multi-buffer crypto | |
59 | implementations may be used on some platforms. | |
60 | ||
9f48bbac | 61 | ### Test expectations |
453dfd8d EK |
62 | |
63 | * ExpectedResult - expected handshake outcome. One of | |
64 | - Success - handshake success | |
65 | - ServerFail - serverside handshake failure | |
66 | - ClientFail - clientside handshake failure | |
67 | - InternalError - some other error | |
68 | ||
9f48bbac | 69 | * ExpectedClientAlert, ExpectedServerAlert - expected alert. See |
dd8e5a57 EK |
70 | `ssl_test_ctx.c` for known values. Note: the expected alert is currently |
71 | matched against the _last_ received alert (i.e., a fatal alert or a | |
72 | `close_notify`). Warning alert expectations are not yet supported. (A warning | |
73 | alert will not be correctly matched, if followed by a `close_notify` or | |
74 | another alert.) | |
453dfd8d | 75 | |
9f48bbac | 76 | * ExpectedProtocol - expected negotiated protocol. One of |
453dfd8d EK |
77 | SSLv3, TLSv1, TLSv1.1, TLSv1.2. |
78 | ||
5c753de6 TS |
79 | * SessionTicketExpected - whether or not a session ticket is expected |
80 | - Ignore - do not check for a session ticket (default) | |
81 | - Yes - a session ticket is expected | |
82 | - No - a session ticket is not expected | |
590ed3d7 | 83 | |
a84e5c9a TS |
84 | * SessionIdExpected - whether or not a session id is expected |
85 | - Ignore - do not check for a session id (default) | |
86 | - Yes - a session id is expected | |
87 | - No - a session id is not expected | |
88 | ||
590ed3d7 EK |
89 | * ResumptionExpected - whether or not resumption is expected (Resume mode only) |
90 | - Yes - resumed handshake | |
91 | - No - full handshake (default) | |
92 | ||
9f48bbac | 93 | * ExpectedNPNProtocol, ExpectedALPNProtocol - NPN and ALPN expectations. |
ce2cdac2 | 94 | |
b93ad05d DSH |
95 | * ExpectedTmpKeyType - the expected algorithm or curve of server temp key |
96 | ||
7289ab49 | 97 | * ExpectedServerCertType, ExpectedClientCertType - the expected algorithm or |
7f5f35af DSH |
98 | curve of server or client certificate |
99 | ||
54b7f2a5 | 100 | * ExpectedServerSignHash, ExpectedClientSignHash - the expected |
ee5b6a42 DSH |
101 | signing hash used by server or client certificate |
102 | ||
54b7f2a5 DSH |
103 | * ExpectedServerSignType, ExpectedClientSignType - the expected |
104 | signature type used by server or client when signing messages | |
105 | ||
2e21539b DSH |
106 | * ExpectedClientCANames - for client auth list of CA names the server must |
107 | send. If this is "empty" the list is expected to be empty otherwise it | |
108 | is a file of certificates whose subject names form the list. | |
109 | ||
f15b50c4 DSH |
110 | * ExpectedServerCANames - list of CA names the client must send, TLS 1.3 only. |
111 | If this is "empty" the list is expected to be empty otherwise it is a file | |
112 | of certificates whose subject names form the list. | |
113 | ||
257e9d03 RS |
114 | Configuring the client and server |
115 | --------------------------------- | |
453dfd8d EK |
116 | |
117 | The client and server configurations can be any valid `SSL_CTX` | |
118 | configurations. For details, see the manpages for `SSL_CONF_cmd`. | |
119 | ||
120 | Give your configurations as a dictionary of CONF commands, e.g. | |
121 | ||
257e9d03 RS |
122 | server => { |
123 | "CipherString" => "DEFAULT", | |
124 | "MinProtocol" => "TLSv1", | |
125 | } | |
453dfd8d | 126 | |
590ed3d7 EK |
127 | The following sections may optionally be defined: |
128 | ||
129 | * server2 - this section configures a secondary context that is selected via the | |
130 | ServerName test option. This context is used whenever a ServerNameCallback is | |
131 | specified. If the server2 section is not present, then the configuration | |
132 | matches server. | |
133 | * resume_server - this section configures the client to resume its session | |
134 | against a different server. This context is used whenever HandshakeMode is | |
11279b13 | 135 | Resume. If the resume_server section is not present, then the configuration |
590ed3d7 | 136 | matches server. |
11279b13 EK |
137 | * resume_client - this section configures the client to resume its session with |
138 | a different configuration. In practice this may occur when, for example, | |
139 | upgraded clients reuse sessions persisted on disk. This context is used | |
140 | whenever HandshakeMode is Resume. If the resume_client section is not present, | |
141 | then the configuration matches client. | |
5c753de6 | 142 | |
9f48bbac EK |
143 | ### Configuring callbacks and additional options |
144 | ||
145 | Additional handshake settings can be configured in the `extra` section of each | |
146 | client and server: | |
147 | ||
257e9d03 RS |
148 | client => { |
149 | "CipherString" => "DEFAULT", | |
150 | extra => { | |
151 | "ServerName" => "server2", | |
152 | } | |
9f48bbac | 153 | } |
9f48bbac EK |
154 | |
155 | #### Supported client-side options | |
156 | ||
157 | * ClientVerifyCallback - the client's custom certificate verify callback. | |
158 | Used to test callback behaviour. One of | |
159 | - None - no custom callback (default) | |
160 | - AcceptAll - accepts all certificates. | |
161 | - RejectAll - rejects all certificates. | |
162 | ||
163 | * ServerName - the server the client should attempt to connect to. One of | |
164 | - None - do not use SNI (default) | |
165 | - server1 - the initial context | |
166 | - server2 - the secondary context | |
167 | - invalid - an unknown context | |
168 | ||
da085d27 EK |
169 | * CTValidation - Certificate Transparency validation strategy. One of |
170 | - None - no validation (default) | |
171 | - Permissive - SSL_CT_VALIDATION_PERMISSIVE | |
172 | - Strict - SSL_CT_VALIDATION_STRICT | |
173 | ||
9f48bbac EK |
174 | #### Supported server-side options |
175 | ||
176 | * ServerNameCallback - the SNI switching callback to use | |
177 | - None - no callback (default) | |
178 | - IgnoreMismatch - continue the handshake on SNI mismatch | |
179 | - RejectMismatch - abort the handshake on SNI mismatch | |
180 | ||
181 | * BrokenSessionTicket - a special test case where the session ticket callback | |
182 | does not initialize crypto. | |
183 | - No (default) | |
184 | - Yes | |
185 | ||
186 | #### Mutually supported options | |
187 | ||
188 | * NPNProtocols, ALPNProtocols - NPN and ALPN settings. Server and client | |
189 | protocols can be specified as a comma-separated list, and a callback with the | |
190 | recommended behaviour will be installed automatically. | |
191 | ||
ea1ecd98 EK |
192 | * SRPUser, SRPPassword - SRP settings. For client, this is the SRP user to |
193 | connect as; for server, this is a known SRP user. | |
194 | ||
453dfd8d EK |
195 | ### Default server and client configurations |
196 | ||
197 | The default server certificate and CA files are added to the configurations | |
198 | automatically. Server certificate verification is requested by default. | |
199 | ||
200 | You can override these options by redefining them: | |
201 | ||
257e9d03 RS |
202 | client => { |
203 | "VerifyCAFile" => "/path/to/custom/file" | |
204 | } | |
453dfd8d EK |
205 | |
206 | or by deleting them | |
207 | ||
257e9d03 RS |
208 | client => { |
209 | "VerifyCAFile" => undef | |
210 | } | |
453dfd8d | 211 | |
257e9d03 RS |
212 | Adding a test to the test harness |
213 | --------------------------------- | |
453dfd8d | 214 | |
15269e56 | 215 | 1. Add a new test configuration to `test/ssl-tests`, following the examples of |
433deaff | 216 | existing `*.cnf.in` files (for example, `01-simple.cnf.in`). |
15269e56 | 217 | |
433deaff | 218 | 2. Generate the generated `*.cnf` test input file. You can do so by running |
15269e56 EK |
219 | `generate_ssl_tests.pl`: |
220 | ||
257e9d03 RS |
221 | $ ./config |
222 | $ cd test | |
223 | $ TOP=.. perl -I ../util/perl/ generate_ssl_tests.pl \ | |
224 | ssl-tests/my.cnf.in default > ssl-tests/my.cnf | |
15269e56 | 225 | |
9df3dd6b MC |
226 | where `my.cnf.in` is your test input file and `default` is the provider to use. |
227 | For all the pre-generated test files you should use the default provider. | |
15269e56 | 228 | |
433deaff | 229 | For example, to generate the test cases in `ssl-tests/01-simple.cnf.in`, do |
15269e56 | 230 | |
257e9d03 RS |
231 | $ TOP=.. perl -I ../util/perl/ generate_ssl_tests.pl \ |
232 | ssl-tests/01-simple.cnf.in default > ssl-tests/01-simple.cnf | |
15269e56 EK |
233 | |
234 | Alternatively (hackish but simple), you can comment out | |
235 | ||
257e9d03 | 236 | unlink glob $tmp_file; |
15269e56 EK |
237 | |
238 | in `test/recipes/80-test_ssl_new.t` and run | |
239 | ||
257e9d03 | 240 | $ make TESTS=test_ssl_new test |
15269e56 EK |
241 | |
242 | This will save the generated output in a `*.tmp` file in the build directory. | |
243 | ||
244 | 3. Update the number of tests planned in `test/recipes/80-test_ssl_new.t`. If | |
245 | the test suite has any skip conditions, update those too (see | |
246 | `test/recipes/80-test_ssl_new.t` for details). | |
453dfd8d | 247 | |
257e9d03 RS |
248 | Running the tests with the test harness |
249 | --------------------------------------- | |
453dfd8d | 250 | |
257e9d03 | 251 | HARNESS_VERBOSE=yes make TESTS=test_ssl_new test |
453dfd8d | 252 | |
257e9d03 RS |
253 | Running a test manually |
254 | ----------------------- | |
453dfd8d EK |
255 | |
256 | These steps are only needed during development. End users should run `make test` | |
257 | or follow the instructions above to run the SSL test suite. | |
258 | ||
259 | To run an SSL test manually from the command line, the `TEST_CERTS_DIR` | |
260 | environment variable to point to the location of the certs. E.g., from the root | |
261 | OpenSSL directory, do | |
262 | ||
257e9d03 RS |
263 | $ CTLOG_FILE=test/ct/log_list.cnf TEST_CERTS_DIR=test/certs test/ssl_test \ |
264 | test/ssl-tests/01-simple.cnf | |
453dfd8d EK |
265 | |
266 | or for shared builds | |
267 | ||
257e9d03 RS |
268 | $ CTLOG_FILE=test/ct/log_list.cnf TEST_CERTS_DIR=test/certs \ |
269 | util/wrap.pl test/ssl_test test/ssl-tests/01-simple.cnf | |
453dfd8d EK |
270 | |
271 | Note that the test expectations sometimes depend on the Configure settings. For | |
272 | example, the negotiated protocol depends on the set of available (enabled) | |
273 | protocols: a build with `enable-ssl3` has different test expectations than a | |
274 | build with `no-ssl3`. | |
275 | ||
276 | The Perl test harness automatically generates expected outputs, so users who | |
277 | just run `make test` do not need any extra steps. | |
278 | ||
279 | However, when running a test manually, keep in mind that the repository version | |
433deaff | 280 | of the generated `test/ssl-tests/*.cnf` correspond to expected outputs in with |
453dfd8d EK |
281 | the default Configure options. To run `ssl_test` manually from the command line |
282 | in a build with a different configuration, you may need to generate the right | |
433deaff | 283 | `*.cnf` file from the `*.cnf.in` input first. |