tailc: Fix up musttail calls vs. -fsanitize=thread [PR119801]

tailc: Fix up musttail calls vs. -fsanitize=thread [PR119801]

Calls with musttail attribute don't really work with -fsanitize=thread in
GCC.  The problem is that TSan instrumentation adds
  __tsan_func_entry (__builtin_return_address (0));
calls at the start of each instrumented function and
  __tsan_func_exit ();
call at the end of those and the latter stands in a way of normal tail calls
as well as musttail tail calls.

Looking at what LLVM does, for normal calls -fsanitize=thread also prevents
tail calls like in GCC (well, the __tsan_func_exit () call itself can be
tail called in GCC (and from what I see not in clang)).
But for [[clang::musttail]] calls it arranges to move the
__tsan_func_exit () before the musttail call instead of after it.

The following patch handles it similarly.  If we for -fsanitize=thread
instrumented function detect __builtin_tsan_func_exit () call, we process
it normally (so that the call can be tail called in function returning void)
but set a flag that the builtin has been seen (only for cfun->has_musttail
in the diag_musttail phase).  And then let tree_optimize_tail_calls_1
call find_tail_calls again in a new mode where the __tsan_func_exit ()
call is ignored and so we are able to find calls before it, but only
accept that if the call before it is actually a musttail.  For C++ it needs
to verify that EH cleanup if any also has the __tsan_func_exit () call
and if all goes well, the musttail call is registered for tailcalling with
a flag that it has __tsan_func_exit () after it and when optimizing that
we emit __tsan_func_exit (); call before the musttail tail call (or musttail
tail recursion).

2025-04-15  Jakub Jelinek  <jakub@redhat.com>

	PR sanitizer/119801
	* sanitizer.def (BUILT_IN_TSAN_FUNC_EXIT): Use BT_FN_VOID rather
	than BT_FN_VOID_PTR.
	* tree-tailcall.cc: Include attribs.h and asan.h.
	(struct tailcall): Add has_tsan_func_exit member.
	(empty_eh_cleanup): Add eh_has_tsan_func_exit argument, set what
	it points to to 1 if there is exactly one __tsan_func_exit call
	and ignore that call otherwise.  Adjust recursive call.
	(find_tail_calls): Add RETRY_TSAN_FUNC_EXIT argument, pass it
	to recursive calls.  When seeing __tsan_func_exit call with
	RETRY_TSAN_FUNC_EXIT 0, set it to -1.  If RETRY_TSAN_FUNC_EXIT
	is 1, initially ignore __tsan_func_exit calls.  Adjust
	empty_eh_cleanup caller.  When looking through stmts after the call,
	ignore exactly one __tsan_func_exit call but remember it in
	t->has_tsan_func_exit.  Diagnose if EH cleanups didn't have
	__tsan_func_exit and normal path did or vice versa.
	(optimize_tail_call): Emit __tsan_func_exit before the tail call
	or tail recursion.
	(tree_optimize_tail_calls_1): Adjust find_tail_calls callers.  If
	find_tail_calls changes retry_tsan_func_exit to -1, set it to 1
	and call it again with otherwise the same arguments.

	* c-c++-common/tsan/pr119801.c: New test.