]>
Commit | Line | Data |
---|---|---|
d507436a | 1 | Engines |
1dc1ea18 | 2 | ======= |
b22bda21 | 3 | |
d507436a DMSP |
4 | Deprecation Note |
5 | ---------------- | |
6 | ||
7 | The ENGINE API was introduced in OpenSSL version 0.9.6 as a low level | |
8 | interface for adding alternative implementations of cryptographic | |
9 | primitives, most notably for integrating hardware crypto devices. | |
10 | ||
11 | The ENGINE interface has its limitations and it has been superseeded | |
62829f9f | 12 | by the [PROVIDER API](README-PROVIDERS.md), it is deprecated in OpenSSL |
d507436a DMSP |
13 | version 3.0. The following documentation is retained as an aid for |
14 | users who need to maintain or support existing ENGINE implementations. | |
15 | Support for new hardware devices or new algorithms should be added | |
16 | via providers, and existing engines should be converted to providers | |
17 | as soon as possible. | |
18 | ||
19 | Built-in ENGINE implementations | |
20 | ------------------------------- | |
dc589dae DMSP |
21 | |
22 | There are currently built-in ENGINE implementations for the following | |
23 | crypto devices: | |
b22bda21 | 24 | |
1dc1ea18 DDO |
25 | * Microsoft CryptoAPI |
26 | * VIA Padlock | |
27 | * nCipher CHIL | |
a8a18782 | 28 | |
dc589dae DMSP |
29 | In addition, dynamic binding to external ENGINE implementations is now |
30 | provided by a special ENGINE called "dynamic". See the "DYNAMIC ENGINE" | |
31 | section below for details. | |
a8a18782 | 32 | |
dc589dae | 33 | At this stage, a number of things are still needed and are being worked on: |
a8a18782 | 34 | |
1dc1ea18 DDO |
35 | 1. Integration of EVP support. |
36 | 2. Configuration support. | |
37 | 3. Documentation! | |
a8a18782 | 38 | |
dc589dae DMSP |
39 | Integration of EVP support |
40 | -------------------------- | |
41 | ||
42 | With respect to EVP, this relates to support for ciphers and digests in | |
43 | the ENGINE model so that alternative implementations of existing | |
44 | algorithms/modes (or previously unimplemented ones) can be provided by | |
45 | ENGINE implementations. | |
46 | ||
47 | Configuration support | |
48 | --------------------- | |
49 | ||
50 | Configuration support currently exists in the ENGINE API itself, in the | |
51 | form of "control commands". These allow an application to expose to the | |
52 | user/admin the set of commands and parameter types a given ENGINE | |
53 | implementation supports, and for an application to directly feed string | |
54 | based input to those ENGINEs, in the form of name-value pairs. This is an | |
55 | extensible way for ENGINEs to define their own "configuration" mechanisms | |
56 | that are specific to a given ENGINE (eg. for a particular hardware | |
57 | device) but that should be consistent across *all* OpenSSL-based | |
58 | applications when they use that ENGINE. Work is in progress (or at least | |
59 | in planning) for supporting these control commands from the CONF (or | |
60 | NCONF) code so that applications using OpenSSL's existing configuration | |
61 | file format can have ENGINE settings specified in much the same way. | |
62 | Presently however, applications must use the ENGINE API itself to provide | |
63 | such functionality. To see first hand the types of commands available | |
64 | with the various compiled-in ENGINEs (see further down for dynamic | |
65 | ENGINEs), use the "engine" openssl utility with full verbosity, i.e.: | |
66 | ||
67 | openssl engine -vvvv | |
68 | ||
69 | Documentation | |
70 | ------------- | |
71 | ||
72 | Documentation? Volunteers welcome! The source code is reasonably well | |
73 | self-documenting, but some summaries and usage instructions are needed - | |
74 | moreover, they are needed in the same POD format the existing OpenSSL | |
75 | documentation is provided in. Any complete or incomplete contributions | |
76 | would help make this happen. | |
77 | ||
78 | STABILITY & BUG-REPORTS | |
79 | ======================= | |
80 | ||
81 | What already exists is fairly stable as far as it has been tested, but | |
82 | the test base has been a bit small most of the time. For the most part, | |
83 | the vendors of the devices these ENGINEs support have contributed to the | |
84 | development and/or testing of the implementations, and *usually* (with no | |
85 | guarantees) have experience in using the ENGINE support to drive their | |
86 | devices from common OpenSSL-based applications. Bugs and/or inexplicable | |
87 | behaviour in using a specific ENGINE implementation should be sent to the | |
88 | author of that implementation (if it is mentioned in the corresponding C | |
89 | file), and in the case of implementations for commercial hardware | |
90 | devices, also through whatever vendor support channels are available. If | |
91 | none of this is possible, or the problem seems to be something about the | |
92 | ENGINE API itself (ie. not necessarily specific to a particular ENGINE | |
93 | implementation) then you should mail complete details to the relevant | |
94 | OpenSSL mailing list. For a definition of "complete details", refer to | |
95 | the OpenSSL "README" file. As for which list to send it to: | |
a8a18782 | 96 | |
1dc1ea18 DDO |
97 | * openssl-users: if you are *using* the ENGINE abstraction, either in an |
98 | pre-compiled application or in your own application code. | |
a8a18782 | 99 | |
1dc1ea18 | 100 | * openssl-dev: if you are discussing problems with OpenSSL source code. |
a8a18782 | 101 | |
dc589dae DMSP |
102 | USAGE |
103 | ===== | |
104 | ||
105 | The default "openssl" ENGINE is always chosen when performing crypto | |
106 | operations unless you specify otherwise. You must actively tell the | |
107 | openssl utility commands to use anything else through a new command line | |
108 | switch called "-engine". Also, if you want to use the ENGINE support in | |
109 | your own code to do something similar, you must likewise explicitly | |
110 | select the ENGINE implementation you want. | |
111 | ||
112 | Depending on the type of hardware, system, and configuration, "settings" | |
113 | may need to be applied to an ENGINE for it to function as expected/hoped. | |
114 | The recommended way of doing this is for the application to support | |
115 | ENGINE "control commands" so that each ENGINE implementation can provide | |
116 | whatever configuration primitives it might require and the application | |
117 | can allow the user/admin (and thus the hardware vendor's support desk | |
118 | also) to provide any such input directly to the ENGINE implementation. | |
119 | This way, applications do not need to know anything specific to any | |
120 | device, they only need to provide the means to carry such user/admin | |
121 | input through to the ENGINE in question. Ie. this connects *you* (and | |
122 | your helpdesk) to the specific ENGINE implementation (and device), and | |
123 | allows application authors to not get buried in hassle supporting | |
124 | arbitrary devices they know (and care) nothing about. | |
125 | ||
126 | A new "openssl" utility, "openssl engine", has been added in that allows | |
127 | for testing and examination of ENGINE implementations. Basic usage | |
128 | instructions are available by specifying the "-?" command line switch. | |
129 | ||
130 | DYNAMIC ENGINES | |
131 | =============== | |
132 | ||
133 | The new "dynamic" ENGINE provides a low-overhead way to support ENGINE | |
134 | implementations that aren't pre-compiled and linked into OpenSSL-based | |
135 | applications. This could be because existing compiled-in implementations | |
136 | have known problems and you wish to use a newer version with an existing | |
137 | application. It could equally be because the application (or OpenSSL | |
138 | library) you are using simply doesn't have support for the ENGINE you | |
139 | wish to use, and the ENGINE provider (eg. hardware vendor) is providing | |
140 | you with a self-contained implementation in the form of a shared-library. | |
141 | The other use-case for "dynamic" is with applications that wish to | |
142 | maintain the smallest foot-print possible and so do not link in various | |
143 | ENGINE implementations from OpenSSL, but instead leaves you to provide | |
144 | them, if you want them, in the form of "dynamic"-loadable | |
145 | shared-libraries. It should be possible for hardware vendors to provide | |
146 | their own shared-libraries to support arbitrary hardware to work with | |
147 | applications based on OpenSSL 0.9.7 or later. If you're using an | |
148 | application based on 0.9.7 (or later) and the support you desire is only | |
149 | announced for versions later than the one you need, ask the vendor to | |
150 | backport their ENGINE to the version you need. | |
151 | ||
152 | How does "dynamic" work? | |
153 | ------------------------ | |
154 | ||
155 | The dynamic ENGINE has a special flag in its implementation such that | |
156 | every time application code asks for the 'dynamic' ENGINE, it in fact | |
157 | gets its own copy of it. As such, multi-threaded code (or code that | |
158 | multiplexes multiple uses of 'dynamic' in a single application in any | |
159 | way at all) does not get confused by 'dynamic' being used to do many | |
160 | independent things. Other ENGINEs typically don't do this so there is | |
161 | only ever 1 ENGINE structure of its type (and reference counts are used | |
162 | to keep order). The dynamic ENGINE itself provides absolutely no | |
163 | cryptographic functionality, and any attempt to "initialise" the ENGINE | |
164 | automatically fails. All it does provide are a few "control commands" | |
165 | that can be used to control how it will load an external ENGINE | |
166 | implementation from a shared-library. To see these control commands, | |
167 | use the command-line; | |
1dc1ea18 DDO |
168 | |
169 | openssl engine -vvvv dynamic | |
170 | ||
dc589dae DMSP |
171 | The "SO_PATH" control command should be used to identify the |
172 | shared-library that contains the ENGINE implementation, and "NO_VCHECK" | |
173 | might possibly be useful if there is a minor version conflict and you | |
174 | (or a vendor helpdesk) is convinced you can safely ignore it. | |
175 | "ID" is probably only needed if a shared-library implements | |
176 | multiple ENGINEs, but if you know the engine id you expect to be using, | |
177 | it doesn't hurt to specify it (and this provides a sanity check if | |
178 | nothing else). "LIST_ADD" is only required if you actually wish the | |
179 | loaded ENGINE to be discoverable by application code later on using the | |
180 | ENGINE's "id". For most applications, this isn't necessary - but some | |
181 | application authors may have nifty reasons for using it. The "LOAD" | |
182 | command is the only one that takes no parameters and is the command | |
183 | that uses the settings from any previous commands to actually *load* | |
184 | the shared-library ENGINE implementation. If this command succeeds, the | |
185 | (copy of the) 'dynamic' ENGINE will magically morph into the ENGINE | |
186 | that has been loaded from the shared-library. As such, any control | |
187 | commands supported by the loaded ENGINE could then be executed as per | |
188 | normal. Eg. if ENGINE "foo" is implemented in the shared-library | |
189 | "libfoo.so" and it supports some special control command "CMD_FOO", the | |
190 | following code would load and use it (NB: obviously this code has no | |
191 | error checking); | |
192 | ||
193 | ENGINE *e = ENGINE_by_id("dynamic"); | |
194 | ENGINE_ctrl_cmd_string(e, "SO_PATH", "/lib/libfoo.so", 0); | |
195 | ENGINE_ctrl_cmd_string(e, "ID", "foo", 0); | |
196 | ENGINE_ctrl_cmd_string(e, "LOAD", NULL, 0); | |
197 | ENGINE_ctrl_cmd_string(e, "CMD_FOO", "some input data", 0); | |
198 | ||
199 | For testing, the "openssl engine" utility can be useful for this sort | |
200 | of thing. For example the above code excerpt would achieve much the | |
201 | same result as; | |
202 | ||
203 | openssl engine dynamic \ | |
204 | -pre SO_PATH:/lib/libfoo.so \ | |
205 | -pre ID:foo \ | |
206 | -pre LOAD \ | |
207 | -pre "CMD_FOO:some input data" | |
208 | ||
209 | Or to simply see the list of commands supported by the "foo" ENGINE; | |
210 | ||
211 | openssl engine -vvvv dynamic \ | |
212 | -pre SO_PATH:/lib/libfoo.so \ | |
213 | -pre ID:foo \ | |
214 | -pre LOAD | |
215 | ||
216 | Applications that support the ENGINE API and more specifically, the | |
217 | "control commands" mechanism, will provide some way for you to pass | |
218 | such commands through to ENGINEs. As such, you would select "dynamic" | |
219 | as the ENGINE to use, and the parameters/commands you pass would | |
220 | control the *actual* ENGINE used. Each command is actually a name-value | |
221 | pair and the value can sometimes be omitted (eg. the "LOAD" command). | |
222 | Whilst the syntax demonstrated in "openssl engine" uses a colon to | |
223 | separate the command name from the value, applications may provide | |
224 | their own syntax for making that separation (eg. a win32 registry | |
225 | key-value pair may be used by some applications). The reason for the | |
226 | "-pre" syntax in the "openssl engine" utility is that some commands | |
227 | might be issued to an ENGINE *after* it has been initialised for use. | |
228 | Eg. if an ENGINE implementation requires a smart-card to be inserted | |
229 | during initialisation (or a PIN to be typed, or whatever), there may be | |
230 | a control command you can issue afterwards to "forget" the smart-card | |
231 | so that additional initialisation is no longer possible. In | |
232 | applications such as web-servers, where potentially volatile code may | |
233 | run on the same host system, this may provide some arguable security | |
234 | value. In such a case, the command would be passed to the ENGINE after | |
235 | it has been initialised for use, and so the "-post" switch would be | |
236 | used instead. Applications may provide a different syntax for | |
237 | supporting this distinction, and some may simply not provide it at all | |
238 | ("-pre" is almost always what you're after, in reality). | |
239 | ||
240 | How do I build a "dynamic" ENGINE? | |
241 | ---------------------------------- | |
242 | ||
243 | This question is trickier - currently OpenSSL bundles various ENGINE | |
244 | implementations that are statically built in, and any application that | |
245 | calls the "ENGINE_load_builtin_engines()" function will automatically | |
246 | have all such ENGINEs available (and occupying memory). Applications | |
247 | that don't call that function have no ENGINEs available like that and | |
248 | would have to use "dynamic" to load any such ENGINE - but on the other | |
249 | hand such applications would only have the memory footprint of any | |
250 | ENGINEs explicitly loaded using user/admin provided control commands. | |
251 | The main advantage of not statically linking ENGINEs and only using | |
252 | "dynamic" for hardware support is that any installation using no | |
253 | "external" ENGINE suffers no unnecessary memory footprint from unused | |
254 | ENGINEs. Likewise, installations that do require an ENGINE incur the | |
255 | overheads from only *that* ENGINE once it has been loaded. | |
256 | ||
257 | Sounds good? Maybe, but currently building an ENGINE implementation as | |
258 | a shared-library that can be loaded by "dynamic" isn't automated in | |
259 | OpenSSL's build process. It can be done manually quite easily however. | |
260 | Such a shared-library can either be built with any OpenSSL code it | |
261 | needs statically linked in, or it can link dynamically against OpenSSL | |
262 | if OpenSSL itself is built as a shared library. The instructions are | |
263 | the same in each case, but in the former (statically linked any | |
264 | dependencies on OpenSSL) you must ensure OpenSSL is built with | |
265 | position-independent code ("PIC"). The default OpenSSL compilation may | |
266 | already specify the relevant flags to do this, but you should consult | |
267 | with your compiler documentation if you are in any doubt. | |
268 | ||
269 | This example will show building the "atalla" ENGINE in the | |
270 | crypto/engine/ directory as a shared-library for use via the "dynamic" | |
271 | ENGINE. | |
1dc1ea18 DDO |
272 | |
273 | 1. "cd" to the crypto/engine/ directory of a pre-compiled OpenSSL | |
dc589dae | 274 | source tree. |
1dc1ea18 DDO |
275 | |
276 | 2. Recompile at least one source file so you can see all the compiler | |
dc589dae | 277 | flags (and syntax) being used to build normally. Eg; |
1dc1ea18 | 278 | |
dc589dae | 279 | touch hw_atalla.c ; make |
1dc1ea18 | 280 | |
dc589dae | 281 | will rebuild "hw_atalla.o" using all such flags. |
1dc1ea18 DDO |
282 | |
283 | 3. Manually enter the same compilation line to compile the | |
dc589dae DMSP |
284 | "hw_atalla.c" file but with the following two changes; |
285 | * add "-DENGINE_DYNAMIC_SUPPORT" to the command line switches, | |
286 | * change the output file from "hw_atalla.o" to something new, | |
287 | eg. "tmp_atalla.o" | |
1dc1ea18 DDO |
288 | |
289 | 4. Link "tmp_atalla.o" into a shared-library using the top-level | |
dc589dae DMSP |
290 | OpenSSL libraries to resolve any dependencies. The syntax for doing |
291 | this depends heavily on your system/compiler and is a nightmare | |
292 | known well to anyone who has worked with shared-library portability | |
293 | before. 'gcc' on Linux, for example, would use the following syntax; | |
1dc1ea18 | 294 | |
dc589dae | 295 | gcc -shared -o dyn_atalla.so tmp_atalla.o -L../.. -lcrypto |
1dc1ea18 DDO |
296 | |
297 | 5. Test your shared library using "openssl engine" as explained in the | |
dc589dae | 298 | previous section. Eg. from the top-level directory, you might try |
1dc1ea18 | 299 | |
dc589dae DMSP |
300 | apps/openssl engine -vvvv dynamic \ |
301 | -pre SO_PATH:./crypto/engine/dyn_atalla.so -pre LOAD | |
1dc1ea18 | 302 | |
dc589dae DMSP |
303 | If the shared-library loads successfully, you will see both "-pre" |
304 | commands marked as "SUCCESS" and the list of control commands | |
305 | displayed (because of "-vvvv") will be the control commands for the | |
306 | *atalla* ENGINE (ie. *not* the 'dynamic' ENGINE). You can also add | |
307 | the "-t" switch to the utility if you want it to try and initialise | |
308 | the atalla ENGINE for use to test any possible hardware/driver issues. | |
56245be4 | 309 | |
dc589dae DMSP |
310 | PROBLEMS |
311 | ======== | |
56245be4 | 312 | |
dc589dae DMSP |
313 | It seems like the ENGINE part doesn't work too well with CryptoSwift on Win32. |
314 | A quick test done right before the release showed that trying "openssl speed | |
315 | -engine cswift" generated errors. If the DSO gets enabled, an attempt is made | |
316 | to write at memory address 0x00000002. | |
56245be4 | 317 |