]>
Commit | Line | Data |
---|---|---|
a9b5d2ee UD |
1 | README for libm-test math test suite |
2 | ==================================== | |
3 | ||
4 | The libm-test math test suite tests a number of function points of | |
5 | math functions in the GNU C library. The following sections contain a | |
2813e41e JM |
6 | brief overview. Please note that the test drivers and the Python |
7 | script "gen-libm-test.py" have some options. A full list of options | |
8 | is available with --help (for the test drivers) and -h for | |
9 | "gen-libm-test.py". | |
a9b5d2ee UD |
10 | |
11 | ||
12 | What is tested? | |
13 | =============== | |
14 | The tests just evaluate the functions at specified points and compare | |
15 | the results with precomputed values and the requirements of the ISO | |
16 | C99 standard. | |
17 | ||
18 | Besides testing the special values mandated by IEEE 754 (infinities, | |
19 | NaNs and minus zero), some more or less random values are tested. | |
20 | ||
21 | Files that are part of libm-test | |
22 | ================================ | |
23 | ||
5cbb5849 JM |
24 | The main files are "libm-test-<func>.inc". They are independent of |
25 | the target platform and the specific real floating type and format and | |
26 | contain placeholder test "templates" for math functions defined in | |
27 | libm. These files, along with generated files named | |
2813e41e JM |
28 | "auto-libm-test-out-<func>", are preprocessed by the Python script |
29 | "gen-libm-test.py" to expand the templates and produce a set of test | |
5cbb5849 JM |
30 | cases for each math function that are specific to the target platform |
31 | but still independent of the real floating type. The results of the | |
32 | processing are "libm-test-<func>.c" and a file "libm-test-ulps.h" with | |
33 | platform specific deltas by which the actual math function results may | |
34 | deviate from the expected results and still be considered correct. | |
527de9e4 | 35 | |
92061bb0 JM |
36 | The test drivers "test-double-<func>.c", "test-float-<func>.c", and |
37 | "test-ldouble-<func>.c", generated by the Makefile, test the normal | |
1c15464c AZ |
38 | double, float and long double implementation of libm. Each driver |
39 | selects the desired real floating type to exercise the math functions | |
40 | to test with (float, double, or long double) by defining a small set | |
41 | of macros just before including the generic "libm-test.c" file. Each | |
42 | driver also either defines or undefines the __NO_MATH_INLINES macro | |
43 | just before including "libm-test-<func>.c" to select either the real | |
44 | or inline functions, respectively. Each driver is compiled into a | |
45 | single executable test program with the corresponding name. | |
527de9e4 | 46 | |
2813e41e | 47 | As mentioned above, the "gen-libm-test.py" script looks for a file |
527de9e4 MS |
48 | named "libm-test-ulps" in the platform specific sysdep directory (or |
49 | its fpu or nofpu subdirectory) and for each variant (real floating | |
50 | type and rounding mode) of every tested function reads from it the | |
51 | maximum difference expressed as Units of Least Precision (ULP) the | |
52 | actual result of the function may deviate from the expected result | |
53 | before it's considered incorrect. | |
54 | ||
4f1bc131 JM |
55 | The "auto-libm-test-out-<func>" files contain sets of test cases to |
56 | exercise, the conditions under which to exercise each, and the | |
57 | expected results. The files are generated by the | |
58 | "gen-auto-libm-tests" program from the "auto-libm-test-in" file. See | |
59 | the comments in gen-auto-libm-tests.c for details about the content | |
60 | and format of the -in and -out files. | |
a9b5d2ee UD |
61 | |
62 | How can I generate "libm-test-ulps"? | |
63 | ==================================== | |
64 | ||
26510bdd CD |
65 | To automatically generate a new "libm-test-ulps" run "make regen-ulps". |
66 | This generates the file "math/NewUlps" in the build directory. The file | |
67 | contains the sorted results of all the tests. You can use the "NewUlps" | |
68 | file as the machine's updated "libm-test-ulps" file. Copy "NewUlps" to | |
69 | "libm-test-ulps" in the appropriate machine sysdep directory. Verify | |
70 | the changes, post your patch, and check it in after review. | |
71 | ||
72 | To manually generate a new "libm-test-ulps" file, first remove "ULPs" | |
73 | file in the current directory, then you can execute for example: | |
085b2d41 | 74 | ./testrun.sh math/test-double -u --ignore-max-ulp=yes |
a9b5d2ee | 75 | This generates a file "ULPs" with all double ULPs in it, ignoring any |
26510bdd CD |
76 | previously calculated ULPs, and running with the newly built dynamic |
77 | loader and math library (assumes you didn't install your build). Now | |
78 | generate the ULPs for all other formats, the tests will be appending the | |
2813e41e | 79 | data to the "ULPs" file. As final step run "gen-libm-test.py" with the |
26510bdd CD |
80 | file as input and ask to generate a pretty printed output in the file |
81 | "NewUlps": | |
2813e41e | 82 | gen-libm-test.py -u ULPs -n NewUlps |
26510bdd CD |
83 | Copy "NewUlps" to "libm-test-ulps" in the appropriate machine sysdep |
84 | directory. | |
85 | ||
86 | Note that the test drivers have an option "-u" to output an unsorted | |
87 | list of all epsilons that the functions have. The output can be read | |
88 | in directly but it's better to pretty print it first. | |
2813e41e | 89 | "gen-libm-test.py" has an option to generate a pretty-printed and |
26510bdd | 90 | sorted new ULPs file from the output of the test drivers. |
a9b5d2ee UD |
91 | |
92 | Contents of libm-test-ulps | |
93 | ========================== | |
a9b5d2ee | 94 | |
e6b6a857 JM |
95 | Since libm-test-ulps can be generated automatically, just a few notes. |
96 | The file contains lines for maximal errors of single functions, like: | |
97 | ||
a9b5d2ee | 98 | Function "yn": |
1c15464c | 99 | double: 6 |
a9b5d2ee | 100 | |
1c15464c | 101 | The keywords are float, double, and ldouble. |
a9b5d2ee | 102 | |
5cbb5849 JM |
103 | Adding tests to libm-test-<func>.inc |
104 | ==================================== | |
a9b5d2ee UD |
105 | |
106 | The tests are evaluated by a set of special test macros. The macros | |
107 | start with "TEST_" followed by a specification the input values, an | |
108 | underscore and a specification of the output values. As an example, | |
109 | the test macro for a function with input of type FLOAT (FLOAT is | |
110 | either float, double, long double) and output of type FLOAT is | |
111 | "TEST_f_f". The macro's parameter are the name of the function, the | |
112 | input parameter, output parameter and optionally one exception | |
113 | parameter. | |
114 | ||
115 | The accepted parameter types are: | |
116 | - "f" for FLOAT | |
76475eae | 117 | - "j" for long double. |
8e554659 | 118 | - "a" for ARG_FLOAT, the argument type for narrowing functions. |
a9b5d2ee UD |
119 | - "b" for boolean - just tests if the output parameter evaluates to 0 |
120 | or 1 (only for output). | |
121 | - "c" for complex. This parameter needs two values, first the real, | |
122 | then the imaginary part. | |
123 | - "i" for int. | |
124 | - "l" for long int. | |
125 | - "L" for long long int. | |
76475eae JM |
126 | - "u" for unsigned int. |
127 | - "M" for intmax_t. | |
128 | - "U" for uintmax_t. | |
129 | - "p" for an argument (described in the previous character) passed | |
130 | through a pointer rather than directly. | |
a9b5d2ee UD |
131 | - "F" for the address of a FLOAT (only as input parameter) |
132 | - "I" for the address of an int (only as input parameter) | |
76475eae JM |
133 | - "1" for an additional output (either output through a pointer passed |
134 | as an argument, or to a global variable such as signgam). | |
527de9e4 MS |
135 | |
136 | How to read the test output | |
137 | =========================== | |
138 | ||
139 | Running each test on its own at the default level of verbosity will | |
140 | print on stdout a line describing the implementation of math functions | |
1c15464c AZ |
141 | exercised by the test (float, double, or long double). This is then |
142 | followed by the details of test failures (if any). The output concludes | |
143 | by a summary listing the number of test cases exercised and the number | |
527de9e4 MS |
144 | of test failures uncovered. |
145 | ||
146 | For each test failure (and for each test case at higher levels of | |
147 | verbosity), the output contains the name of the function under test | |
148 | and its arguments or conditions that triggered the failure. Note | |
149 | that the name of the function in the output need not correspond | |
150 | exactly to the name of the math function actually invoked. For example, | |
151 | the output will refer to the "acos" function even if the actual function | |
152 | under test is acosf (for the float version) or acosl (for the long | |
153 | double version). Also note that the function arguments may be shown | |
154 | in either the decimal or the hexadecimal floating point format which | |
155 | may or may not correspond to the format used in the auto-libm-test-in | |
156 | file. Besides the name of the function, for each test failure the | |
157 | output contains the actual and expected results and the difference | |
158 | between the two, printed in both the decimal and hexadecimal | |
159 | floating point format, and the ULP and maximum ULP for the test | |
160 | case. |