]>
Commit | Line | Data |
---|---|---|
5ab4f893 RL |
1 | How to add recipes |
2 | ================== | |
3 | ||
4 | For any test that you want to perform, you write a script located in | |
5 | test/recipes/, named {nn}-test_{name}.t, where {nn} is a two digit number and | |
6 | {name} is a unique name of your choice. | |
7 | ||
8 | Please note that if a test involves a new testing executable, you will need to | |
7a5f5fd3 FWH |
9 | do some additions in test/build.info. Please refer to the section "Changes to |
10 | test/build.info" below. | |
5ab4f893 RL |
11 | |
12 | ||
69b86d4b | 13 | Naming conventions |
5ab4f893 RL |
14 | ================= |
15 | ||
16 | A test executable is named test/{name}test.c | |
17 | ||
18 | A test recipe is named test/recipes/{nn}-test_{name}.t, where {nn} is a two | |
19 | digit number and {name} is a unique name of your choice. | |
20 | ||
21 | The number {nn} is (somewhat loosely) grouped as follows: | |
22 | ||
532e7b36 RL |
23 | 00-04 sanity, internal and essential API tests |
24 | 05-09 individual symmetric cipher algorithms | |
25 | 10-14 math (bignum) | |
26 | 15-19 individual asymmetric cipher algorithms | |
27 | 20-24 openssl commands (some otherwise not tested) | |
28 | 25-29 certificate forms, generation and verification | |
29 | 30-35 engine and evp | |
51a7c4b5 RL |
30 | 60-79 APIs: |
31 | 60 X509 subsystem | |
32 | 61 BIO subsystem | |
33 | 65 CMP subsystem | |
532e7b36 RL |
34 | 70 PACKET layer |
35 | 80-89 "larger" protocols (CA, CMS, OCSP, SSL, TSA) | |
d0823f7a AP |
36 | 90-98 misc |
37 | 99 most time consuming tests [such as test_fuzz] | |
5ab4f893 RL |
38 | |
39 | ||
40 | A recipe that just runs a test executable | |
41 | ========================================= | |
42 | ||
43 | A script that just runs a program looks like this: | |
44 | ||
45 | #! /usr/bin/perl | |
c2500f65 | 46 | |
5ab4f893 | 47 | use OpenSSL::Test::Simple; |
c2500f65 | 48 | |
5ab4f893 RL |
49 | simple_test("test_{name}", "{name}test", "{name}"); |
50 | ||
51 | {name} is the unique name you have chosen for your test. | |
52 | ||
53 | The second argument to `simple_test' is the test executable, and `simple_test' | |
54 | expects it to be located in test/ | |
55 | ||
56 | For documentation on OpenSSL::Test::Simple, do | |
f90486f4 | 57 | `perldoc util/perl/OpenSSL/Test/Simple.pm'. |
5ab4f893 RL |
58 | |
59 | ||
60 | A recipe that runs a more complex test | |
61 | ====================================== | |
62 | ||
63 | For more complex tests, you will need to read up on Test::More and | |
64 | OpenSSL::Test. Test::More is normally preinstalled, do `man Test::More' for | |
f90486f4 | 65 | documentation. For OpenSSL::Test, do `perldoc util/perl/OpenSSL/Test.pm'. |
5ab4f893 RL |
66 | |
67 | A script to start from could be this: | |
68 | ||
69 | #! /usr/bin/perl | |
c2500f65 | 70 | |
5ab4f893 RL |
71 | use strict; |
72 | use warnings; | |
73 | use OpenSSL::Test; | |
c2500f65 | 74 | |
5ab4f893 | 75 | setup("test_{name}"); |
c2500f65 | 76 | |
5ab4f893 | 77 | plan tests => 2; # The number of tests being performed |
c2500f65 | 78 | |
5ab4f893 RL |
79 | ok(test1, "test1"); |
80 | ok(test2, "test1"); | |
c2500f65 | 81 | |
5ab4f893 RL |
82 | sub test1 |
83 | { | |
84 | # test feature 1 | |
85 | } | |
c2500f65 | 86 | |
5ab4f893 RL |
87 | sub test2 |
88 | { | |
89 | # test feature 2 | |
90 | } | |
c2500f65 | 91 | |
5ab4f893 | 92 | |
2fae041d P |
93 | Changes to test/build.info |
94 | ========================== | |
5ab4f893 RL |
95 | |
96 | Whenever a new test involves a new test executable you need to do the | |
97 | following (at all times, replace {NAME} and {name} with the name of your | |
98 | test): | |
99 | ||
2fae041d | 100 | * add {name} to the list of programs under PROGRAMS_NO_INST |
5ab4f893 | 101 | |
2fae041d P |
102 | * create a three line description of how to build the test, you will have |
103 | to modify the include paths and source files if you don't want to use the | |
104 | basic test framework: | |
5ab4f893 | 105 | |
4db40c94 | 106 | SOURCE[{name}]={name}.c |
7a5f5fd3 | 107 | INCLUDE[{name}]=.. ../include ../apps/include |
4db40c94 | 108 | DEPEND[{name}]=../libcrypto libtestutil.a |
5ab4f893 | 109 | |
2fae041d P |
110 | Generic form of C test executables |
111 | ================================== | |
5ab4f893 | 112 | |
2fae041d | 113 | #include "testutil.h" |
5ab4f893 | 114 | |
2fae041d P |
115 | static int my_test(void) |
116 | { | |
117 | int testresult = 0; /* Assume the test will fail */ | |
118 | int observed; | |
119 | ||
120 | observed = function(); /* Call the code under test */ | |
f3448f54 | 121 | if (!TEST_int_eq(observed, 2)) /* Check the result is correct */ |
2fae041d P |
122 | goto end; /* Exit on failure - optional */ |
123 | ||
124 | testresult = 1; /* Mark the test case a success */ | |
125 | end: | |
126 | cleanup(); /* Any cleanup you require */ | |
127 | return testresult; | |
128 | } | |
129 | ||
ad887416 | 130 | int setup_tests(void) |
2fae041d P |
131 | { |
132 | ADD_TEST(my_test); /* Add each test separately */ | |
ad887416 | 133 | return 1; /* Indicate success */ |
2fae041d | 134 | } |
5ab4f893 | 135 | |
2fae041d P |
136 | You should use the TEST_xxx macros provided by testutil.h to test all failure |
137 | conditions. These macros produce an error message in a standard format if the | |
138 | condition is not met (and nothing if the condition is met). Additional | |
139 | information can be presented with the TEST_info macro that takes a printf | |
140 | format string and arguments. TEST_error is useful for complicated conditions, | |
d063add7 P |
141 | it also takes a printf format string and argument. In all cases the TEST_xxx |
142 | macros are guaranteed to evaluate their arguments exactly once. This means | |
143 | that expressions with side effects are allowed as parameters. Thus, | |
144 | ||
145 | if (!TEST_ptr(ptr = OPENSSL_malloc(..))) | |
146 | ||
147 | works fine and can be used in place of: | |
148 | ||
149 | ptr = OPENSSL_malloc(..); | |
150 | if (!TEST_ptr(ptr)) | |
151 | ||
152 | The former produces a more meaningful message on failure than the latter. | |
153 |