Various changes to ensure that the lexers and parsers can be correctly
generated when REBUILD_PARSERS is enabled.
Some notes:
* Because of the version of flex we are using to generate the lexers
(2.5.35) some post-processing in the Makefile is still required.
* The generated lexers do not contain the problematic C99 check that
was being replaced by the call to sed in the respective Makefiles so
it was removed.
* Since these files are generated, they will include trailing
whitespace in some places. This does not need to be corrected.