gcc/doc/gccint/testsuites/support-for-testing-binary-compatibility.rst

   1 ..
   2   Copyright 1988-2022 Free Software Foundation, Inc.
   3   This is part of the GCC manual.
   4   For copying conditions, see the copyright.rst file.
   5
   6 .. _compat-testing:
   7
   8 Support for testing binary compatibility
   9 ****************************************
  10
  11 The file :samp:`compat.exp` provides language-independent support for
  12 binary compatibility testing.  It supports testing interoperability of
  13 two compilers that follow the same ABI, or of multiple sets of
  14 compiler options that should not affect binary compatibility.  It is
  15 intended to be used for testsuites that complement ABI testsuites.
  16
  17 A test supported by this framework has three parts, each in a
  18 separate source file: a main program and two pieces that interact
  19 with each other to split up the functionality being tested.
  20
  21 :samp:`{testname}_main.{suffix}`
  22   Contains the main program, which calls a function in file
  23   :samp:`{testname}_x.{suffix}`.
  24
  25 :samp:`{testname}_x.{suffix}`
  26   Contains at least one call to a function in
  27   :samp:`{testname}_y.{suffix}`.
  28
  29 :samp:`{testname}_y.{suffix}`
  30   Shares data with, or gets arguments from,
  31   :samp:`{testname}_x.{suffix}`.
  32
  33 Within each test, the main program and one functional piece are
  34 compiled by the GCC under test.  The other piece can be compiled by
  35 an alternate compiler.  If no alternate compiler is specified,
  36 then all three source files are all compiled by the GCC under test.
  37 You can specify pairs of sets of compiler options.  The first element
  38 of such a pair specifies options used with the GCC under test, and the
  39 second element of the pair specifies options used with the alternate
  40 compiler.  Each test is compiled with each pair of options.
  41
  42 :samp:`compat.exp` defines default pairs of compiler options.
  43 These can be overridden by defining the environment variable
  44 :envvar:`COMPAT_OPTIONS` as:
  45
  46 .. code-block::
  47
  48   COMPAT_OPTIONS="[list [list {tst1} {alt1}]
  49     ...[list {tstn} {altn}]]"
  50
  51 where :samp:`{tsti}` and :samp:`{alti}` are lists of options, with :samp:`{tsti}`
  52 used by the compiler under test and :samp:`{alti}` used by the alternate
  53 compiler.  For example, with
  54 ``[list [list {-g -O0} {-O3}] [list {-fpic} {-fPIC -O2}]]``,
  55 the test is first built with :option:`-g -O0` by the compiler under
  56 test and with :option:`-O3` by the alternate compiler.  The test is
  57 built a second time using :option:`-fpic` by the compiler under test
  58 and :option:`-fPIC -O2` by the alternate compiler.
  59
  60 An alternate compiler is specified by defining an environment
  61 variable to be the full pathname of an installed compiler; for C
  62 define :envvar:`ALT_CC_UNDER_TEST`, and for C++ define
  63 :envvar:`ALT_CXX_UNDER_TEST`.  These will be written to the
  64 :samp:`site.exp` file used by DejaGnu.  The default is to build each
  65 test with the compiler under test using the first of each pair of
  66 compiler options from :envvar:`COMPAT_OPTIONS`.  When
  67 :envvar:`ALT_CC_UNDER_TEST` or
  68 :envvar:`ALT_CXX_UNDER_TEST` is ``same``, each test is built using
  69 the compiler under test but with combinations of the options from
  70 :envvar:`COMPAT_OPTIONS`.
  71
  72 To run only the C++ compatibility suite using the compiler under test
  73 and another version of GCC using specific compiler options, do the
  74 following from :samp:`{objdir}/gcc`:
  75
  76 .. code-block::
  77
  78   rm site.exp
  79   make -k \
  80     ALT_CXX_UNDER_TEST=${alt_prefix}/bin/g++ \
  81     COMPAT_OPTIONS="lists as shown above" \
  82     check-c++ \
  83     RUNTESTFLAGS="compat.exp"
  84
  85 A test that fails when the source files are compiled with different
  86 compilers, but passes when the files are compiled with the same
  87 compiler, demonstrates incompatibility of the generated code or
  88 runtime support.  A test that fails for the alternate compiler but
  89 passes for the compiler under test probably tests for a bug that was
  90 fixed in the compiler under test but is present in the alternate
  91 compiler.
  92
  93 The binary compatibility tests support a small number of test framework
  94 commands that appear within comments in a test file.
  95
  96 ``dg-require-*``
  97   These commands can be used in :samp:`{testname}_main.{suffix}`
  98   to skip the test if specific support is not available on the target.
  99
 100 ``dg-options``
 101   The specified options are used for compiling this particular source
 102   file, appended to the options from :envvar:`COMPAT_OPTIONS`.  When this
 103   command appears in :samp:`{testname}_main.{suffix}` the options
 104   are also used to link the test program.
 105
 106 ``dg-xfail-if``
 107   This command can be used in a secondary source file to specify that
 108   compilation is expected to fail for particular options on particular
 109   targets.