]>
Commit | Line | Data |
---|---|---|
a9b5d2ee UD |
1 | README for libm-test math test suite |
2 | ==================================== | |
3 | ||
4 | The libm-test math test suite tests a number of function points of | |
5 | math functions in the GNU C library. The following sections contain a | |
6 | brief overview. Please note that the test drivers and the Perl script | |
7 | "gen-libm-test.pl" have some options. A full list of options is | |
8 | available with --help (for the test drivers) and -h for | |
9 | "gen-libm-test.pl". | |
10 | ||
11 | ||
12 | What is tested? | |
13 | =============== | |
14 | The tests just evaluate the functions at specified points and compare | |
15 | the results with precomputed values and the requirements of the ISO | |
16 | C99 standard. | |
17 | ||
18 | Besides testing the special values mandated by IEEE 754 (infinities, | |
19 | NaNs and minus zero), some more or less random values are tested. | |
20 | ||
21 | Files that are part of libm-test | |
22 | ================================ | |
23 | ||
527de9e4 MS |
24 | The main file is "libm-test.inc". It is independent of the target |
25 | platform and the specific real floating type and format and contains | |
26 | placeholder test "templates" for math functions defined in libm. | |
27 | The file, along with a generated file named "auto-libm-test-out", | |
28 | is preprocessed by the Perl script "gen-libm-test.pl" to expand | |
29 | the templates and produce a set of test cases for each math function | |
30 | that are specific to the target platform but still independent of | |
31 | the real floating type. The results of the processing are | |
32 | "libm-test.c" and a file "libm-test-ulps.h" with platform specific | |
33 | deltas by which the actual math function results may deviate from | |
34 | the expected results and still be considered correct. | |
35 | ||
36 | The test drivers "test-double.c", "test-float.c", and "test-ldouble.c" | |
37 | test the normal double, float and long double implementation of libm. | |
38 | The test drivers with an 'i' in their name ("test-idouble.c", | |
39 | "test-ifloat.c", and "test-ildoubl.c") test the corresponding inline | |
40 | functions (where available - otherwise they also test the real | |
41 | functions in libm). Each driver selects the desired real floating | |
42 | type to exercise the math functions to test with (float, double, or | |
43 | long double) by defining a small set of macros just before including | |
44 | the generic "libm-test.c" file. Each driver also either defines or | |
45 | undefines the __NO_MATH_INLINES macro just before including | |
46 | "libm-test.c" to select either the real or inline functions, | |
47 | respectively. Each driver is compiled into a single executable test | |
48 | program with the corresponding name. | |
49 | ||
50 | As mentioned above, the "gen-libm-test.pl" script looks for a file | |
51 | named "libm-test-ulps" in the platform specific sysdep directory (or | |
52 | its fpu or nofpu subdirectory) and for each variant (real floating | |
53 | type and rounding mode) of every tested function reads from it the | |
54 | maximum difference expressed as Units of Least Precision (ULP) the | |
55 | actual result of the function may deviate from the expected result | |
56 | before it's considered incorrect. | |
57 | ||
58 | The "auto-libm-test-out" file contains sets of test cases to exercise, | |
59 | the conditions under which to exercise each, and the expected results. | |
60 | The file is generated by the "gen-auto-libm-tests" program from the | |
61 | "auto-libm-test-in" file. See the comments in gen-auto-libm-tests.c | |
62 | for details about the content and format of the -in and -out files. | |
a9b5d2ee UD |
63 | |
64 | How can I generate "libm-test-ulps"? | |
65 | ==================================== | |
66 | ||
26510bdd CD |
67 | To automatically generate a new "libm-test-ulps" run "make regen-ulps". |
68 | This generates the file "math/NewUlps" in the build directory. The file | |
69 | contains the sorted results of all the tests. You can use the "NewUlps" | |
70 | file as the machine's updated "libm-test-ulps" file. Copy "NewUlps" to | |
71 | "libm-test-ulps" in the appropriate machine sysdep directory. Verify | |
72 | the changes, post your patch, and check it in after review. | |
73 | ||
74 | To manually generate a new "libm-test-ulps" file, first remove "ULPs" | |
75 | file in the current directory, then you can execute for example: | |
085b2d41 | 76 | ./testrun.sh math/test-double -u --ignore-max-ulp=yes |
a9b5d2ee | 77 | This generates a file "ULPs" with all double ULPs in it, ignoring any |
26510bdd CD |
78 | previously calculated ULPs, and running with the newly built dynamic |
79 | loader and math library (assumes you didn't install your build). Now | |
80 | generate the ULPs for all other formats, the tests will be appending the | |
81 | data to the "ULPs" file. As final step run "gen-libm-test.pl" with the | |
82 | file as input and ask to generate a pretty printed output in the file | |
83 | "NewUlps": | |
7e1e68b4 | 84 | gen-libm-test.pl -u ULPs -n NewUlps |
26510bdd CD |
85 | Copy "NewUlps" to "libm-test-ulps" in the appropriate machine sysdep |
86 | directory. | |
87 | ||
88 | Note that the test drivers have an option "-u" to output an unsorted | |
89 | list of all epsilons that the functions have. The output can be read | |
90 | in directly but it's better to pretty print it first. | |
91 | "gen-libm-test.pl" has an option to generate a pretty-printed and | |
92 | sorted new ULPs file from the output of the test drivers. | |
a9b5d2ee UD |
93 | |
94 | Contents of libm-test-ulps | |
95 | ========================== | |
a9b5d2ee | 96 | |
e6b6a857 JM |
97 | Since libm-test-ulps can be generated automatically, just a few notes. |
98 | The file contains lines for maximal errors of single functions, like: | |
99 | ||
a9b5d2ee | 100 | Function "yn": |
e6b6a857 | 101 | idouble: 6 |
a9b5d2ee UD |
102 | |
103 | The keywords are float, ifloat, double, idouble, ldouble and ildouble | |
b7dab1e4 | 104 | (the prefix i stands for inline). |
a9b5d2ee UD |
105 | |
106 | Adding tests to libm-test.inc | |
107 | ============================= | |
108 | ||
109 | The tests are evaluated by a set of special test macros. The macros | |
110 | start with "TEST_" followed by a specification the input values, an | |
111 | underscore and a specification of the output values. As an example, | |
112 | the test macro for a function with input of type FLOAT (FLOAT is | |
113 | either float, double, long double) and output of type FLOAT is | |
114 | "TEST_f_f". The macro's parameter are the name of the function, the | |
115 | input parameter, output parameter and optionally one exception | |
116 | parameter. | |
117 | ||
118 | The accepted parameter types are: | |
119 | - "f" for FLOAT | |
76475eae | 120 | - "j" for long double. |
a9b5d2ee UD |
121 | - "b" for boolean - just tests if the output parameter evaluates to 0 |
122 | or 1 (only for output). | |
123 | - "c" for complex. This parameter needs two values, first the real, | |
124 | then the imaginary part. | |
125 | - "i" for int. | |
126 | - "l" for long int. | |
127 | - "L" for long long int. | |
76475eae JM |
128 | - "u" for unsigned int. |
129 | - "M" for intmax_t. | |
130 | - "U" for uintmax_t. | |
131 | - "p" for an argument (described in the previous character) passed | |
132 | through a pointer rather than directly. | |
a9b5d2ee UD |
133 | - "F" for the address of a FLOAT (only as input parameter) |
134 | - "I" for the address of an int (only as input parameter) | |
76475eae JM |
135 | - "1" for an additional output (either output through a pointer passed |
136 | as an argument, or to a global variable such as signgam). | |
527de9e4 MS |
137 | |
138 | How to read the test output | |
139 | =========================== | |
140 | ||
141 | Running each test on its own at the default level of verbosity will | |
142 | print on stdout a line describing the implementation of math functions | |
143 | exercised by the test (float, double, or long double), along with | |
144 | whether the inline set has been selected, regardless of whether or | |
145 | not any inline functions actually exist. This is then followed by | |
146 | the details of test failures (if any). The output concludes by | |
147 | a summary listing the number of test cases exercised and the number | |
148 | of test failures uncovered. | |
149 | ||
150 | For each test failure (and for each test case at higher levels of | |
151 | verbosity), the output contains the name of the function under test | |
152 | and its arguments or conditions that triggered the failure. Note | |
153 | that the name of the function in the output need not correspond | |
154 | exactly to the name of the math function actually invoked. For example, | |
155 | the output will refer to the "acos" function even if the actual function | |
156 | under test is acosf (for the float version) or acosl (for the long | |
157 | double version). Also note that the function arguments may be shown | |
158 | in either the decimal or the hexadecimal floating point format which | |
159 | may or may not correspond to the format used in the auto-libm-test-in | |
160 | file. Besides the name of the function, for each test failure the | |
161 | output contains the actual and expected results and the difference | |
162 | between the two, printed in both the decimal and hexadecimal | |
163 | floating point format, and the ULP and maximum ULP for the test | |
164 | case. |