gcc/doc/cppinternals/marking-tokens-ineligible-for-future-expansion.rst

   1 ..
   2   Copyright 1988-2022 Free Software Foundation, Inc.
   3   This is part of the GCC manual.
   4   For copying conditions, see the copyright.rst file.
   5
   6 Marking tokens ineligible for future expansion
   7 **********************************************
   8
   9 As discussed above, cpplib needs a way of marking tokens as
  10 unexpandable.  Since the tokens cpplib handles are read-only once they
  11 have been lexed, it instead makes a copy of the token and adds the
  12 flag ``NO_EXPAND`` to the copy.
  13
  14 For efficiency and to simplify memory management by avoiding having to
  15 remember to free these tokens, they are allocated as temporary tokens
  16 from the lexer's current token run (see :ref:`lexing-a-line`) using the
  17 function ``_cpp_temp_token``.  The tokens are then re-used once the
  18 current line of tokens has been read in.
  19
  20 This might sound unsafe.  However, tokens runs are not re-used at the
  21 end of a line if it happens to be in the middle of a macro argument
  22 list, and cpplib only wants to back-up more than one lexer token in
  23 situations where no macro expansion is involved, so the optimization
  24 is safe.