]>
Commit | Line | Data |
---|---|---|
83ffe9cd | 1 | .. Copyright (C) 2014-2023 Free Software Foundation, Inc. |
35485da9 DM |
2 | Originally contributed by David Malcolm <dmalcolm@redhat.com> |
3 | ||
4 | This is free software: you can redistribute it and/or modify it | |
5 | under the terms of the GNU General Public License as published by | |
6 | the Free Software Foundation, either version 3 of the License, or | |
7 | (at your option) any later version. | |
8 | ||
9 | This program is distributed in the hope that it will be useful, but | |
10 | WITHOUT ANY WARRANTY; without even the implied warranty of | |
11 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU | |
12 | General Public License for more details. | |
13 | ||
14 | You should have received a copy of the GNU General Public License | |
15 | along with this program. If not, see | |
786973ce | 16 | <https://www.gnu.org/licenses/>. |
35485da9 DM |
17 | |
18 | Internals | |
19 | ========= | |
20 | ||
21 | Working on the JIT library | |
22 | -------------------------- | |
23 | Having checked out the source code (to "src"), you can configure and build | |
24 | the JIT library like this: | |
25 | ||
26 | .. code-block:: bash | |
27 | ||
28 | mkdir build | |
29 | mkdir install | |
30 | PREFIX=$(pwd)/install | |
31 | cd build | |
32 | ../src/configure \ | |
33 | --enable-host-shared \ | |
433d16df | 34 | --enable-languages=jit,c++ \ |
35485da9 DM |
35 | --disable-bootstrap \ |
36 | --enable-checking=release \ | |
37 | --prefix=$PREFIX | |
38 | nice make -j4 # altering the "4" to however many cores you have | |
39 | ||
40 | This should build a libgccjit.so within jit/build/gcc: | |
41 | ||
42 | .. code-block:: console | |
43 | ||
44 | [build] $ file gcc/libgccjit.so* | |
45 | gcc/libgccjit.so: symbolic link to `libgccjit.so.0' | |
46 | gcc/libgccjit.so.0: symbolic link to `libgccjit.so.0.0.1' | |
47 | gcc/libgccjit.so.0.0.1: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped | |
48 | ||
49 | Here's what those configuration options mean: | |
50 | ||
51 | .. option:: --enable-host-shared | |
52 | ||
53 | Configuring with this option means that the compiler is built as | |
54 | position-independent code, which incurs a slight performance hit, | |
55 | but it necessary for a shared library. | |
56 | ||
433d16df | 57 | .. option:: --enable-languages=jit,c++ |
35485da9 DM |
58 | |
59 | This specifies which frontends to build. The JIT library looks like | |
60 | a frontend to the rest of the code. | |
61 | ||
433d16df DM |
62 | The C++ portion of the JIT test suite requires the C++ frontend to be |
63 | enabled at configure-time, or you may see errors like this when | |
64 | running the test suite: | |
65 | ||
66 | .. code-block:: console | |
67 | ||
68 | xgcc: error: /home/david/jit/src/gcc/testsuite/jit.dg/test-quadratic.cc: C++ compiler not installed on this system | |
69 | c++: error trying to exec 'cc1plus': execvp: No such file or directory | |
70 | ||
35485da9 DM |
71 | .. option:: --disable-bootstrap |
72 | ||
73 | For hacking on the "jit" subdirectory, performing a full | |
74 | bootstrap can be overkill, since it's unused by a bootstrap. However, | |
75 | when submitting patches, you should remove this option, to ensure that | |
76 | the compiler can still bootstrap itself. | |
77 | ||
78 | .. option:: --enable-checking=release | |
79 | ||
80 | The compile can perform extensive self-checking as it runs, useful when | |
81 | debugging, but slowing things down. | |
82 | ||
83 | For maximum speed, configure with ``--enable-checking=release`` to | |
84 | disable this self-checking. | |
85 | ||
86 | Running the test suite | |
87 | ---------------------- | |
88 | ||
89 | .. code-block:: console | |
90 | ||
91 | [build] $ cd gcc | |
92 | [gcc] $ make check-jit RUNTESTFLAGS="-v -v -v" | |
93 | ||
94 | A summary of the tests can then be seen in: | |
95 | ||
96 | .. code-block:: console | |
97 | ||
98 | jit/build/gcc/testsuite/jit/jit.sum | |
99 | ||
100 | and detailed logs in: | |
101 | ||
102 | .. code-block:: console | |
103 | ||
104 | jit/build/gcc/testsuite/jit/jit.log | |
105 | ||
77f4ead7 DM |
106 | The test executables are normally deleted after each test is run. For |
107 | debugging, they can be preserved by setting :envvar:`PRESERVE_EXECUTABLES` | |
108 | in the environment. If so, they can then be seen as: | |
35485da9 DM |
109 | |
110 | .. code-block:: console | |
111 | ||
112 | jit/build/gcc/testsuite/jit/*.exe | |
113 | ||
114 | which can be run independently. | |
115 | ||
116 | You can compile and run individual tests by passing "jit.exp=TESTNAME" to RUNTESTFLAGS e.g.: | |
117 | ||
118 | .. code-block:: console | |
119 | ||
77f4ead7 DM |
120 | [gcc] $ PRESERVE_EXECUTABLES= \ |
121 | make check-jit \ | |
122 | RUNTESTFLAGS="-v -v -v jit.exp=test-factorial.c" | |
35485da9 DM |
123 | |
124 | and once a test has been compiled, you can debug it directly: | |
125 | ||
126 | .. code-block:: console | |
127 | ||
128 | [gcc] $ PATH=.:$PATH \ | |
129 | LD_LIBRARY_PATH=. \ | |
130 | LIBRARY_PATH=. \ | |
131 | gdb --args \ | |
50bb6c8e | 132 | testsuite/jit/test-factorial.c.exe |
35485da9 | 133 | |
2712de78 DM |
134 | Running under valgrind |
135 | ********************** | |
136 | ||
77f4ead7 | 137 | The jit testsuite detects if :envvar:`RUN_UNDER_VALGRIND` is present in the |
2712de78 | 138 | environment (with any value). If it is present, it runs the test client |
786973ce | 139 | code under `valgrind <https://valgrind.org>`_, |
2712de78 | 140 | specifcally, the default |
786973ce | 141 | `memcheck <https://valgrind.org/docs/manual/mc-manual.html>`_ |
2712de78 DM |
142 | tool with |
143 | `--leak-check=full | |
786973ce | 144 | <https://valgrind.org/docs/manual/mc-manual.html#opt.leak-check>`_. |
2712de78 DM |
145 | |
146 | It automatically parses the output from valgrind, injecting XFAIL results if | |
147 | any issues are found, or PASS results if the output is clean. The output | |
148 | is saved to ``TESTNAME.exe.valgrind.txt``. | |
149 | ||
150 | For example, the following invocation verbosely runs the testcase | |
151 | ``test-sum-of-squares.c`` under valgrind, showing an issue: | |
152 | ||
153 | .. code-block:: console | |
154 | ||
155 | $ RUN_UNDER_VALGRIND= \ | |
156 | make check-jit \ | |
157 | RUNTESTFLAGS="-v -v -v jit.exp=test-sum-of-squares.c" | |
158 | ||
159 | (...verbose log contains detailed valgrind errors, if any...) | |
160 | ||
161 | === jit Summary === | |
162 | ||
163 | # of expected passes 28 | |
164 | # of expected failures 2 | |
165 | ||
166 | $ less testsuite/jit/jit.sum | |
167 | (...other results...) | |
50bb6c8e DM |
168 | XFAIL: jit.dg/test-sum-of-squares.c: test-sum-of-squares.c.exe.valgrind.txt: definitely lost: 8 bytes in 1 blocks |
169 | XFAIL: jit.dg/test-sum-of-squares.c: test-sum-of-squares.c.exe.valgrind.txt: unsuppressed errors: 1 | |
2712de78 DM |
170 | (...other results...) |
171 | ||
50bb6c8e | 172 | $ less testsuite/jit/test-sum-of-squares.c.exe.valgrind.txt |
2712de78 DM |
173 | (...shows full valgrind report for this test case...) |
174 | ||
175 | When running under valgrind, it's best to have configured gcc with | |
d6e95a12 | 176 | :option:`--enable-valgrind-annotations`, which automatically suppresses |
2712de78 DM |
177 | various known false positives. |
178 | ||
35485da9 DM |
179 | Environment variables |
180 | --------------------- | |
181 | When running client code against a locally-built libgccjit, three | |
182 | environment variables need to be set up: | |
183 | ||
184 | .. envvar:: LD_LIBRARY_PATH | |
185 | ||
186 | `libgccjit.so` is dynamically linked into client code, so if running | |
187 | against a locally-built library, ``LD_LIBRARY_PATH`` needs to be set | |
188 | up appropriately. The library can be found within the "gcc" | |
189 | subdirectory of the build tree: | |
190 | ||
191 | .. code-block:: console | |
192 | ||
193 | $ file libgccjit.so* | |
194 | libgccjit.so: symbolic link to `libgccjit.so.0' | |
195 | libgccjit.so.0: symbolic link to `libgccjit.so.0.0.1' | |
196 | libgccjit.so.0.0.1: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, not stripped | |
197 | ||
198 | .. envvar:: PATH | |
199 | ||
200 | The library uses a driver executable for converting from .s assembler | |
201 | files to .so shared libraries. Specifically, it looks for a name | |
202 | expanded from | |
203 | ``${target_noncanonical}-gcc-${gcc_BASEVER}${exeext}`` | |
204 | such as ``x86_64-unknown-linux-gnu-gcc-5.0.0``. | |
205 | ||
206 | Hence ``PATH`` needs to include a directory where the library can | |
207 | locate this executable. | |
208 | ||
209 | The executable is normally installed to the installation bindir | |
210 | (e.g. /usr/bin), but a copy is also created within the "gcc" | |
211 | subdirectory of the build tree for running the testsuite, and for ease | |
212 | of development. | |
213 | ||
214 | .. envvar:: LIBRARY_PATH | |
215 | ||
216 | The driver executable invokes the linker, and the latter needs to locate | |
217 | support libraries needed by the generated code, or you will see errors | |
218 | like: | |
219 | ||
220 | .. code-block:: console | |
221 | ||
222 | ld: cannot find crtbeginS.o: No such file or directory | |
223 | ld: cannot find -lgcc | |
224 | ld: cannot find -lgcc_s | |
225 | ||
226 | Hence if running directly from a locally-built copy (without installing), | |
227 | ``LIBRARY_PATH`` needs to contain the "gcc" subdirectory of the build | |
228 | tree. | |
229 | ||
230 | For example, to run a binary that uses the library against a non-installed | |
231 | build of the library in LIBGCCJIT_BUILD_DIR you need an invocation of the | |
232 | client code like this, to preprend the dir to each of the environment | |
233 | variables: | |
234 | ||
235 | .. code-block:: console | |
236 | ||
237 | $ LD_LIBRARY_PATH=$(LIBGCCJIT_BUILD_DIR):$(LD_LIBRARY_PATH) \ | |
238 | PATH=$(LIBGCCJIT_BUILD_DIR):$(PATH) \ | |
239 | LIBRARY_PATH=$(LIBGCCJIT_BUILD_DIR):$(LIBRARY_PATH) \ | |
240 | ./jit-hello-world | |
241 | hello world | |
242 | ||
18eb0d13 DM |
243 | Packaging notes |
244 | --------------- | |
245 | The configure-time option :option:`--enable-host-shared` is needed when | |
246 | building the jit in order to get position-independent code. This will | |
247 | slow down the regular compiler by a few percent. Hence when packaging gcc | |
248 | with libgccjit, please configure and build twice: | |
249 | ||
250 | * once without :option:`--enable-host-shared` for most languages, and | |
251 | ||
252 | * once with :option:`--enable-host-shared` for the jit | |
253 | ||
254 | For example: | |
255 | ||
256 | .. code-block:: bash | |
257 | ||
258 | # Configure and build with --enable-host-shared | |
259 | # for the jit: | |
260 | mkdir configuration-for-jit | |
261 | pushd configuration-for-jit | |
262 | $(SRCDIR)/configure \ | |
263 | --enable-host-shared \ | |
264 | --enable-languages=jit \ | |
265 | --prefix=$(DESTDIR) | |
266 | make | |
267 | popd | |
268 | ||
269 | # Configure and build *without* --enable-host-shared | |
270 | # for maximum speed: | |
271 | mkdir standard-configuration | |
272 | pushd standard-configuration | |
273 | $(SRCDIR)/configure \ | |
274 | --enable-languages=all \ | |
275 | --prefix=$(DESTDIR) | |
276 | make | |
277 | popd | |
278 | ||
279 | # Both of the above are configured to install to $(DESTDIR) | |
280 | # Install the configuration with --enable-host-shared first | |
281 | # *then* the one without, so that the faster build | |
282 | # of "cc1" et al overwrites the slower build. | |
283 | pushd configuration-for-jit | |
284 | make install | |
285 | popd | |
286 | ||
287 | pushd standard-configuration | |
288 | make install | |
289 | popd | |
290 | ||
35485da9 DM |
291 | Overview of code structure |
292 | -------------------------- | |
293 | ||
b8ce0c43 | 294 | The library is implemented in C++. |
1470e75f | 295 | |
e53b6e56 | 296 | * ``libgccjit.cc`` implements the API entrypoints. It performs error |
35485da9 | 297 | checking, then calls into classes of the gcc::jit::recording namespace |
e53b6e56 | 298 | within ``jit-recording.cc`` and ``jit-recording.h``. |
35485da9 | 299 | |
e53b6e56 | 300 | * The gcc::jit::recording classes (within ``jit-recording.cc`` and |
35485da9 DM |
301 | ``jit-recording.h``) record the API calls that are made: |
302 | ||
303 | .. literalinclude:: ../../jit-common.h | |
304 | :start-after: /* Recording types. */ | |
305 | :end-before: /* End of recording types. */ | |
306 | :language: c++ | |
307 | ||
308 | * When the context is compiled, the gcc::jit::playback classes (within | |
e53b6e56 | 309 | ``jit-playback.cc`` and ``jit-playback.h``) replay the API calls |
35485da9 DM |
310 | within langhook:parse_file: |
311 | ||
312 | .. literalinclude:: ../../jit-common.h | |
313 | :start-after: /* Playback types. */ | |
314 | :end-before: /* End of playback types. */ | |
315 | :language: c++ | |
316 | ||
317 | .. literalinclude:: ../../notes.txt | |
318 | :lines: 1- | |
319 | ||
320 | Here is a high-level summary from ``jit-common.h``: | |
321 | ||
322 | .. include:: ../../jit-common.h | |
323 | :start-after: This comment is included by the docs. | |
324 | :end-before: End of comment for inclusion in the docs. */ | |
eb4c16eb DM |
325 | |
326 | .. _example-of-log-file: | |
327 | ||
328 | Another way to understand the structure of the code is to enable logging, | |
329 | via :c:func:`gcc_jit_context_set_logfile`. Here is an example of a log | |
330 | generated via this call: | |
331 | ||
332 | .. literalinclude:: test-hello-world.exe.log.txt | |
333 | :lines: 1- | |
86d0ac88 DM |
334 | |
335 | Design notes | |
336 | ------------ | |
337 | It should not be possible for client code to cause an internal compiler | |
338 | error. If this *does* happen, the root cause should be isolated (perhaps | |
339 | using :c:func:`gcc_jit_context_dump_reproducer_to_file`) and the cause | |
340 | should be rejected via additional checking. The checking ideally should | |
e53b6e56 | 341 | be within the libgccjit API entrypoints in libgccjit.cc, since this is as |
86d0ac88 | 342 | close as possible to the error; failing that, a good place is within |
e53b6e56 | 343 | ``recording::context::validate ()`` in jit-recording.cc. |
1470e75f DM |
344 | |
345 | Submitting patches | |
346 | ------------------ | |
347 | Please read the contribution guidelines for gcc at | |
348 | https://gcc.gnu.org/contribute.html. | |
349 | ||
350 | Patches for the jit should be sent to both the | |
351 | gcc-patches@gcc.gnu.org and jit@gcc.gnu.org mailing lists, | |
352 | with "jit" and "PATCH" in the Subject line. | |
353 | ||
354 | You don't need to do a full bootstrap for code that just touches the | |
355 | ``jit`` and ``testsuite/jit.dg`` subdirectories. However, please run | |
356 | ``make check-jit`` before submitting the patch, and mention the results | |
357 | in your email (along with the host triple that the tests were run on). | |
358 | ||
359 | A good patch should contain the information listed in the | |
360 | gcc contribution guide linked to above; for a ``jit`` patch, the patch | |
361 | shold contain: | |
362 | ||
363 | * the code itself (for example, a new API entrypoint will typically | |
364 | touch ``libgccjit.h`` and ``.c``, along with support code in | |
365 | ``jit-recording.[ch]`` and ``jit-playback.[ch]`` as appropriate) | |
366 | ||
367 | * test coverage | |
368 | ||
369 | * documentation for the C API | |
370 | ||
371 | * documentation for the C++ API | |
372 | ||
373 | A patch that adds new API entrypoints should also contain: | |
374 | ||
375 | * a feature macro in ``libgccjit.h`` so that client code that doesn't | |
376 | use a "configure" mechanism can still easily detect the presence of | |
377 | the entrypoint. See e.g. ``LIBGCCJIT_HAVE_SWITCH_STATEMENTS`` (for | |
378 | a category of entrypoints) and | |
379 | ``LIBGCCJIT_HAVE_gcc_jit_context_set_bool_allow_unreachable_blocks`` | |
380 | (for an individual entrypoint). | |
381 | ||
382 | * a new ABI tag containing the new symbols (in ``libgccjit.map``), so | |
383 | that we can detect client code that uses them | |
384 | ||
385 | * Support for :c:func:`gcc_jit_context_dump_reproducer_to_file`. Most | |
386 | jit testcases attempt to dump their contexts to a .c file; ``jit.exp`` | |
387 | then sanity-checks the generated c by compiling them (though | |
388 | not running them). A new API entrypoint | |
389 | needs to "know" how to write itself back out to C (by implementing | |
390 | ``gcc::jit::recording::memento::write_reproducer`` for the appropriate | |
391 | ``memento`` subclass). | |
392 | ||
393 | * C++ bindings for the new entrypoints (see ``libgccjit++.h``); ideally | |
394 | with test coverage, though the C++ API test coverage is admittedly | |
395 | spotty at the moment | |
396 | ||
397 | * documentation for the new C entrypoints | |
398 | ||
399 | * documentation for the new C++ entrypoints | |
400 | ||
401 | * documentation for the new ABI tag (see ``topics/compatibility.rst``). | |
402 | ||
403 | Depending on the patch you can either extend an existing test case, or | |
404 | add a new test case. If you add an entirely new testcase: ``jit.exp`` | |
405 | expects jit testcases to begin with ``test-``, or ``test-error-`` (for a | |
406 | testcase that generates an error on a :c:type:`gcc_jit_context`). | |
407 | ||
408 | Every new testcase that doesn't generate errors should also touch | |
409 | ``gcc/testsuite/jit.dg/all-non-failing-tests.h``: | |
410 | ||
411 | * Testcases that don't generate errors should ideally be added to the | |
412 | ``testcases`` array in that file; this means that, in addition | |
413 | to being run standalone, they also get run within | |
414 | ``test-combination.c`` (which runs all successful tests inside one | |
415 | big :c:type:`gcc_jit_context`), and ``test-threads.c`` (which runs all | |
416 | successful tests in one process, each one running in a different | |
417 | thread on a different :c:type:`gcc_jit_context`). | |
418 | ||
419 | .. note:: | |
420 | ||
421 | Given that exported functions within a :c:type:`gcc_jit_context` | |
422 | must have unique names, and most testcases are run within | |
423 | ``test-combination.c``, this means that every jit-compiled test | |
424 | function typically needs a name that's unique across the entire | |
425 | test suite. | |
426 | ||
427 | * Testcases that aren't to be added to the ``testcases`` array should | |
428 | instead add a comment to the file clarifying why they're not in that | |
429 | array. See the file for examples. | |
430 | ||
431 | Typically a patch that touches the .rst documentation will also need the | |
432 | texinfo to be regenerated. You can do this with | |
786973ce | 433 | `Sphinx 1.0 <https://sphinx-doc.org/>`_ or later by |
1470e75f DM |
434 | running ``make texinfo`` within ``SRCDIR/gcc/jit/docs``. Don't do this |
435 | within the patch sent to the mailing list; it can often be relatively | |
436 | large and inconsequential (e.g. anchor renumbering), rather like generated | |
437 | "configure" changes from configure.ac. You can regenerate it when | |
438 | committing to svn. |