]>
Commit | Line | Data |
---|---|---|
5ab4f893 RL |
1 | How to add recipes |
2 | ================== | |
3 | ||
4 | For any test that you want to perform, you write a script located in | |
5 | test/recipes/, named {nn}-test_{name}.t, where {nn} is a two digit number and | |
6 | {name} is a unique name of your choice. | |
7 | ||
8 | Please note that if a test involves a new testing executable, you will need to | |
9 | do some additions in test/Makefile. More on this later. | |
10 | ||
11 | ||
69b86d4b | 12 | Naming conventions |
5ab4f893 RL |
13 | ================= |
14 | ||
15 | A test executable is named test/{name}test.c | |
16 | ||
17 | A test recipe is named test/recipes/{nn}-test_{name}.t, where {nn} is a two | |
18 | digit number and {name} is a unique name of your choice. | |
19 | ||
20 | The number {nn} is (somewhat loosely) grouped as follows: | |
21 | ||
532e7b36 RL |
22 | 00-04 sanity, internal and essential API tests |
23 | 05-09 individual symmetric cipher algorithms | |
24 | 10-14 math (bignum) | |
25 | 15-19 individual asymmetric cipher algorithms | |
26 | 20-24 openssl commands (some otherwise not tested) | |
27 | 25-29 certificate forms, generation and verification | |
28 | 30-35 engine and evp | |
29 | 60-79 APIs | |
30 | 70 PACKET layer | |
31 | 80-89 "larger" protocols (CA, CMS, OCSP, SSL, TSA) | |
d0823f7a AP |
32 | 90-98 misc |
33 | 99 most time consuming tests [such as test_fuzz] | |
5ab4f893 RL |
34 | |
35 | ||
36 | A recipe that just runs a test executable | |
37 | ========================================= | |
38 | ||
39 | A script that just runs a program looks like this: | |
40 | ||
41 | #! /usr/bin/perl | |
42 | ||
43 | use OpenSSL::Test::Simple; | |
44 | ||
45 | simple_test("test_{name}", "{name}test", "{name}"); | |
46 | ||
47 | {name} is the unique name you have chosen for your test. | |
48 | ||
49 | The second argument to `simple_test' is the test executable, and `simple_test' | |
50 | expects it to be located in test/ | |
51 | ||
52 | For documentation on OpenSSL::Test::Simple, do | |
53 | `perldoc test/testlib/OpenSSL/Test/Simple.pm'. | |
54 | ||
55 | ||
56 | A recipe that runs a more complex test | |
57 | ====================================== | |
58 | ||
59 | For more complex tests, you will need to read up on Test::More and | |
60 | OpenSSL::Test. Test::More is normally preinstalled, do `man Test::More' for | |
61 | documentation. For OpenSSL::Test, do `perldoc test/testlib/OpenSSL/Test.pm'. | |
62 | ||
63 | A script to start from could be this: | |
64 | ||
65 | #! /usr/bin/perl | |
66 | ||
67 | use strict; | |
68 | use warnings; | |
69 | use OpenSSL::Test; | |
70 | ||
71 | setup("test_{name}"); | |
72 | ||
73 | plan tests => 2; # The number of tests being performed | |
74 | ||
75 | ok(test1, "test1"); | |
76 | ok(test2, "test1"); | |
77 | ||
78 | sub test1 | |
79 | { | |
80 | # test feature 1 | |
81 | } | |
82 | ||
83 | sub test2 | |
84 | { | |
85 | # test feature 2 | |
86 | } | |
87 | ||
88 | ||
2fae041d P |
89 | Changes to test/build.info |
90 | ========================== | |
5ab4f893 RL |
91 | |
92 | Whenever a new test involves a new test executable you need to do the | |
93 | following (at all times, replace {NAME} and {name} with the name of your | |
94 | test): | |
95 | ||
2fae041d | 96 | * add {name} to the list of programs under PROGRAMS_NO_INST |
5ab4f893 | 97 | |
2fae041d P |
98 | * create a three line description of how to build the test, you will have |
99 | to modify the include paths and source files if you don't want to use the | |
100 | basic test framework: | |
5ab4f893 | 101 | |
2fae041d P |
102 | SOURCE[{name}]={name}.c testutil.c test_main.c |
103 | INCLUDE[{name}]=.. ../include | |
104 | DEPEND[{name}]=../libcrypto | |
5ab4f893 | 105 | |
2fae041d P |
106 | Generic form of C test executables |
107 | ================================== | |
5ab4f893 | 108 | |
2fae041d P |
109 | #include "test_main.h" |
110 | #include "testutil.h" | |
5ab4f893 | 111 | |
2fae041d P |
112 | static int my_test(void) |
113 | { | |
114 | int testresult = 0; /* Assume the test will fail */ | |
115 | int observed; | |
116 | ||
117 | observed = function(); /* Call the code under test */ | |
118 | if (!TEST_int_equal(observed, 2)) /* Check the result is correct */ | |
119 | goto end; /* Exit on failure - optional */ | |
120 | ||
121 | testresult = 1; /* Mark the test case a success */ | |
122 | end: | |
123 | cleanup(); /* Any cleanup you require */ | |
124 | return testresult; | |
125 | } | |
126 | ||
127 | void register_tests(void) | |
128 | { | |
129 | ADD_TEST(my_test); /* Add each test separately */ | |
130 | } | |
5ab4f893 | 131 | |
2fae041d P |
132 | You should use the TEST_xxx macros provided by testutil.h to test all failure |
133 | conditions. These macros produce an error message in a standard format if the | |
134 | condition is not met (and nothing if the condition is met). Additional | |
135 | information can be presented with the TEST_info macro that takes a printf | |
136 | format string and arguments. TEST_error is useful for complicated conditions, | |
137 | it also takes a printf format string and argument. |