]>
Commit | Line | Data |
---|---|---|
5ab4f893 RL |
1 | How to add recipes |
2 | ================== | |
3 | ||
4 | For any test that you want to perform, you write a script located in | |
5 | test/recipes/, named {nn}-test_{name}.t, where {nn} is a two digit number and | |
6 | {name} is a unique name of your choice. | |
7 | ||
8 | Please note that if a test involves a new testing executable, you will need to | |
9 | do some additions in test/Makefile. More on this later. | |
10 | ||
11 | ||
69b86d4b | 12 | Naming conventions |
5ab4f893 RL |
13 | ================= |
14 | ||
15 | A test executable is named test/{name}test.c | |
16 | ||
17 | A test recipe is named test/recipes/{nn}-test_{name}.t, where {nn} is a two | |
18 | digit number and {name} is a unique name of your choice. | |
19 | ||
20 | The number {nn} is (somewhat loosely) grouped as follows: | |
21 | ||
532e7b36 RL |
22 | 00-04 sanity, internal and essential API tests |
23 | 05-09 individual symmetric cipher algorithms | |
24 | 10-14 math (bignum) | |
25 | 15-19 individual asymmetric cipher algorithms | |
26 | 20-24 openssl commands (some otherwise not tested) | |
27 | 25-29 certificate forms, generation and verification | |
28 | 30-35 engine and evp | |
29 | 60-79 APIs | |
30 | 70 PACKET layer | |
31 | 80-89 "larger" protocols (CA, CMS, OCSP, SSL, TSA) | |
d0823f7a AP |
32 | 90-98 misc |
33 | 99 most time consuming tests [such as test_fuzz] | |
5ab4f893 RL |
34 | |
35 | ||
36 | A recipe that just runs a test executable | |
37 | ========================================= | |
38 | ||
39 | A script that just runs a program looks like this: | |
40 | ||
41 | #! /usr/bin/perl | |
c2500f65 | 42 | |
5ab4f893 | 43 | use OpenSSL::Test::Simple; |
c2500f65 | 44 | |
5ab4f893 RL |
45 | simple_test("test_{name}", "{name}test", "{name}"); |
46 | ||
47 | {name} is the unique name you have chosen for your test. | |
48 | ||
49 | The second argument to `simple_test' is the test executable, and `simple_test' | |
50 | expects it to be located in test/ | |
51 | ||
52 | For documentation on OpenSSL::Test::Simple, do | |
f90486f4 | 53 | `perldoc util/perl/OpenSSL/Test/Simple.pm'. |
5ab4f893 RL |
54 | |
55 | ||
56 | A recipe that runs a more complex test | |
57 | ====================================== | |
58 | ||
59 | For more complex tests, you will need to read up on Test::More and | |
60 | OpenSSL::Test. Test::More is normally preinstalled, do `man Test::More' for | |
f90486f4 | 61 | documentation. For OpenSSL::Test, do `perldoc util/perl/OpenSSL/Test.pm'. |
5ab4f893 RL |
62 | |
63 | A script to start from could be this: | |
64 | ||
65 | #! /usr/bin/perl | |
c2500f65 | 66 | |
5ab4f893 RL |
67 | use strict; |
68 | use warnings; | |
69 | use OpenSSL::Test; | |
c2500f65 | 70 | |
5ab4f893 | 71 | setup("test_{name}"); |
c2500f65 | 72 | |
5ab4f893 | 73 | plan tests => 2; # The number of tests being performed |
c2500f65 | 74 | |
5ab4f893 RL |
75 | ok(test1, "test1"); |
76 | ok(test2, "test1"); | |
c2500f65 | 77 | |
5ab4f893 RL |
78 | sub test1 |
79 | { | |
80 | # test feature 1 | |
81 | } | |
c2500f65 | 82 | |
5ab4f893 RL |
83 | sub test2 |
84 | { | |
85 | # test feature 2 | |
86 | } | |
c2500f65 | 87 | |
5ab4f893 | 88 | |
2fae041d P |
89 | Changes to test/build.info |
90 | ========================== | |
5ab4f893 RL |
91 | |
92 | Whenever a new test involves a new test executable you need to do the | |
93 | following (at all times, replace {NAME} and {name} with the name of your | |
94 | test): | |
95 | ||
2fae041d | 96 | * add {name} to the list of programs under PROGRAMS_NO_INST |
5ab4f893 | 97 | |
2fae041d P |
98 | * create a three line description of how to build the test, you will have |
99 | to modify the include paths and source files if you don't want to use the | |
100 | basic test framework: | |
5ab4f893 | 101 | |
4db40c94 | 102 | SOURCE[{name}]={name}.c |
2fae041d | 103 | INCLUDE[{name}]=.. ../include |
4db40c94 | 104 | DEPEND[{name}]=../libcrypto libtestutil.a |
5ab4f893 | 105 | |
2fae041d P |
106 | Generic form of C test executables |
107 | ================================== | |
5ab4f893 | 108 | |
2fae041d | 109 | #include "testutil.h" |
5ab4f893 | 110 | |
2fae041d P |
111 | static int my_test(void) |
112 | { | |
113 | int testresult = 0; /* Assume the test will fail */ | |
114 | int observed; | |
115 | ||
116 | observed = function(); /* Call the code under test */ | |
117 | if (!TEST_int_equal(observed, 2)) /* Check the result is correct */ | |
118 | goto end; /* Exit on failure - optional */ | |
119 | ||
120 | testresult = 1; /* Mark the test case a success */ | |
121 | end: | |
122 | cleanup(); /* Any cleanup you require */ | |
123 | return testresult; | |
124 | } | |
125 | ||
ad887416 | 126 | int setup_tests(void) |
2fae041d P |
127 | { |
128 | ADD_TEST(my_test); /* Add each test separately */ | |
ad887416 | 129 | return 1; /* Indicate success */ |
2fae041d | 130 | } |
5ab4f893 | 131 | |
2fae041d P |
132 | You should use the TEST_xxx macros provided by testutil.h to test all failure |
133 | conditions. These macros produce an error message in a standard format if the | |
134 | condition is not met (and nothing if the condition is met). Additional | |
135 | information can be presented with the TEST_info macro that takes a printf | |
136 | format string and arguments. TEST_error is useful for complicated conditions, | |
d063add7 P |
137 | it also takes a printf format string and argument. In all cases the TEST_xxx |
138 | macros are guaranteed to evaluate their arguments exactly once. This means | |
139 | that expressions with side effects are allowed as parameters. Thus, | |
140 | ||
141 | if (!TEST_ptr(ptr = OPENSSL_malloc(..))) | |
142 | ||
143 | works fine and can be used in place of: | |
144 | ||
145 | ptr = OPENSSL_malloc(..); | |
146 | if (!TEST_ptr(ptr)) | |
147 | ||
148 | The former produces a more meaningful message on failure than the latter. | |
149 |