]>
Commit | Line | Data |
---|---|---|
a9b5d2ee UD |
1 | README for libm-test math test suite |
2 | ==================================== | |
3 | ||
4 | The libm-test math test suite tests a number of function points of | |
5 | math functions in the GNU C library. The following sections contain a | |
2813e41e JM |
6 | brief overview. Please note that the test drivers and the Python |
7 | script "gen-libm-test.py" have some options. A full list of options | |
8 | is available with --help (for the test drivers) and -h for | |
9 | "gen-libm-test.py". | |
a9b5d2ee UD |
10 | |
11 | ||
12 | What is tested? | |
13 | =============== | |
14 | The tests just evaluate the functions at specified points and compare | |
15 | the results with precomputed values and the requirements of the ISO | |
16 | C99 standard. | |
17 | ||
18 | Besides testing the special values mandated by IEEE 754 (infinities, | |
19 | NaNs and minus zero), some more or less random values are tested. | |
20 | ||
21 | Files that are part of libm-test | |
22 | ================================ | |
23 | ||
5cbb5849 JM |
24 | The main files are "libm-test-<func>.inc". They are independent of |
25 | the target platform and the specific real floating type and format and | |
26 | contain placeholder test "templates" for math functions defined in | |
27 | libm. These files, along with generated files named | |
2813e41e JM |
28 | "auto-libm-test-out-<func>", are preprocessed by the Python script |
29 | "gen-libm-test.py" to expand the templates and produce a set of test | |
5cbb5849 JM |
30 | cases for each math function that are specific to the target platform |
31 | but still independent of the real floating type. The results of the | |
32 | processing are "libm-test-<func>.c" and a file "libm-test-ulps.h" with | |
33 | platform specific deltas by which the actual math function results may | |
34 | deviate from the expected results and still be considered correct. | |
527de9e4 | 35 | |
92061bb0 JM |
36 | The test drivers "test-double-<func>.c", "test-float-<func>.c", and |
37 | "test-ldouble-<func>.c", generated by the Makefile, test the normal | |
1c15464c AZ |
38 | double, float and long double implementation of libm. Each driver |
39 | selects the desired real floating type to exercise the math functions | |
40 | to test with (float, double, or long double) by defining a small set | |
c10dde0d AZ |
41 | of macros just before including the generic "libm-test.c" file. Each |
42 | driver is compiled into a single executable test program with the | |
43 | corresponding name. | |
527de9e4 | 44 | |
2813e41e | 45 | As mentioned above, the "gen-libm-test.py" script looks for a file |
527de9e4 MS |
46 | named "libm-test-ulps" in the platform specific sysdep directory (or |
47 | its fpu or nofpu subdirectory) and for each variant (real floating | |
48 | type and rounding mode) of every tested function reads from it the | |
49 | maximum difference expressed as Units of Least Precision (ULP) the | |
50 | actual result of the function may deviate from the expected result | |
51 | before it's considered incorrect. | |
52 | ||
4f1bc131 JM |
53 | The "auto-libm-test-out-<func>" files contain sets of test cases to |
54 | exercise, the conditions under which to exercise each, and the | |
55 | expected results. The files are generated by the | |
56 | "gen-auto-libm-tests" program from the "auto-libm-test-in" file. See | |
57 | the comments in gen-auto-libm-tests.c for details about the content | |
58 | and format of the -in and -out files. | |
a9b5d2ee UD |
59 | |
60 | How can I generate "libm-test-ulps"? | |
61 | ==================================== | |
62 | ||
26510bdd CD |
63 | To automatically generate a new "libm-test-ulps" run "make regen-ulps". |
64 | This generates the file "math/NewUlps" in the build directory. The file | |
65 | contains the sorted results of all the tests. You can use the "NewUlps" | |
66 | file as the machine's updated "libm-test-ulps" file. Copy "NewUlps" to | |
67 | "libm-test-ulps" in the appropriate machine sysdep directory. Verify | |
68 | the changes, post your patch, and check it in after review. | |
69 | ||
70 | To manually generate a new "libm-test-ulps" file, first remove "ULPs" | |
71 | file in the current directory, then you can execute for example: | |
085b2d41 | 72 | ./testrun.sh math/test-double -u --ignore-max-ulp=yes |
a9b5d2ee | 73 | This generates a file "ULPs" with all double ULPs in it, ignoring any |
26510bdd CD |
74 | previously calculated ULPs, and running with the newly built dynamic |
75 | loader and math library (assumes you didn't install your build). Now | |
76 | generate the ULPs for all other formats, the tests will be appending the | |
2813e41e | 77 | data to the "ULPs" file. As final step run "gen-libm-test.py" with the |
26510bdd CD |
78 | file as input and ask to generate a pretty printed output in the file |
79 | "NewUlps": | |
2813e41e | 80 | gen-libm-test.py -u ULPs -n NewUlps |
26510bdd CD |
81 | Copy "NewUlps" to "libm-test-ulps" in the appropriate machine sysdep |
82 | directory. | |
83 | ||
84 | Note that the test drivers have an option "-u" to output an unsorted | |
85 | list of all epsilons that the functions have. The output can be read | |
86 | in directly but it's better to pretty print it first. | |
2813e41e | 87 | "gen-libm-test.py" has an option to generate a pretty-printed and |
26510bdd | 88 | sorted new ULPs file from the output of the test drivers. |
a9b5d2ee UD |
89 | |
90 | Contents of libm-test-ulps | |
91 | ========================== | |
a9b5d2ee | 92 | |
e6b6a857 JM |
93 | Since libm-test-ulps can be generated automatically, just a few notes. |
94 | The file contains lines for maximal errors of single functions, like: | |
95 | ||
a9b5d2ee | 96 | Function "yn": |
1c15464c | 97 | double: 6 |
a9b5d2ee | 98 | |
1c15464c | 99 | The keywords are float, double, and ldouble. |
a9b5d2ee | 100 | |
5cbb5849 JM |
101 | Adding tests to libm-test-<func>.inc |
102 | ==================================== | |
a9b5d2ee UD |
103 | |
104 | The tests are evaluated by a set of special test macros. The macros | |
105 | start with "TEST_" followed by a specification the input values, an | |
106 | underscore and a specification of the output values. As an example, | |
107 | the test macro for a function with input of type FLOAT (FLOAT is | |
108 | either float, double, long double) and output of type FLOAT is | |
109 | "TEST_f_f". The macro's parameter are the name of the function, the | |
110 | input parameter, output parameter and optionally one exception | |
111 | parameter. | |
112 | ||
113 | The accepted parameter types are: | |
114 | - "f" for FLOAT | |
76475eae | 115 | - "j" for long double. |
8e554659 | 116 | - "a" for ARG_FLOAT, the argument type for narrowing functions. |
a9b5d2ee UD |
117 | - "b" for boolean - just tests if the output parameter evaluates to 0 |
118 | or 1 (only for output). | |
119 | - "c" for complex. This parameter needs two values, first the real, | |
120 | then the imaginary part. | |
121 | - "i" for int. | |
122 | - "l" for long int. | |
123 | - "L" for long long int. | |
76475eae JM |
124 | - "u" for unsigned int. |
125 | - "M" for intmax_t. | |
126 | - "U" for uintmax_t. | |
127 | - "p" for an argument (described in the previous character) passed | |
128 | through a pointer rather than directly. | |
a9b5d2ee UD |
129 | - "F" for the address of a FLOAT (only as input parameter) |
130 | - "I" for the address of an int (only as input parameter) | |
76475eae JM |
131 | - "1" for an additional output (either output through a pointer passed |
132 | as an argument, or to a global variable such as signgam). | |
527de9e4 MS |
133 | |
134 | How to read the test output | |
135 | =========================== | |
136 | ||
137 | Running each test on its own at the default level of verbosity will | |
138 | print on stdout a line describing the implementation of math functions | |
1c15464c AZ |
139 | exercised by the test (float, double, or long double). This is then |
140 | followed by the details of test failures (if any). The output concludes | |
141 | by a summary listing the number of test cases exercised and the number | |
527de9e4 MS |
142 | of test failures uncovered. |
143 | ||
144 | For each test failure (and for each test case at higher levels of | |
145 | verbosity), the output contains the name of the function under test | |
146 | and its arguments or conditions that triggered the failure. Note | |
147 | that the name of the function in the output need not correspond | |
148 | exactly to the name of the math function actually invoked. For example, | |
149 | the output will refer to the "acos" function even if the actual function | |
150 | under test is acosf (for the float version) or acosl (for the long | |
151 | double version). Also note that the function arguments may be shown | |
152 | in either the decimal or the hexadecimal floating point format which | |
153 | may or may not correspond to the format used in the auto-libm-test-in | |
154 | file. Besides the name of the function, for each test failure the | |
155 | output contains the actual and expected results and the difference | |
156 | between the two, printed in both the decimal and hexadecimal | |
157 | floating point format, and the ULP and maximum ULP for the test | |
158 | case. |