ira: Add new hooks for callee-save vs spills [PR117477]

ira: Add new hooks for callee-save vs spills [PR117477]

Following on from the discussion in:

  https://gcc.gnu.org/pipermail/gcc-patches/2025-February/675256.html

this patch removes TARGET_IRA_CALLEE_SAVED_REGISTER_COST_SCALE and
replaces it with two hooks: one that controls the cost of using an
extra callee-saved register and one that controls the cost of allocating
a frame for the first spill.

(The patch does not attempt to address the shrink-wrapping part of
the thread above.)

On AArch64, this is enough to fix PR117477, as verified by the new tests.
The patch does not change the SPEC2017 scores significantly.  (I saw a
slight improvement in fotonik3d and roms, but I'm not convinced that
the improvements are real.)

The patch makes IRA use caller saves for gcc.target/aarch64/pr103350-1.c,
which is a scan-dump correctness test that relies on not using
caller saves.  The decision to use caller saves looks appropriate,
and saves an instruction, so I've just added -fno-caller-saves
to the test options.

The x86 parts were written by Honza.  ix86_callee_save_cost is updated
by H.J. to replace gcc_checking_assert with returning 1 if mem_cost <= 2.

gcc/
	PR rtl-optimization/117477
	* config/aarch64/aarch64.cc (aarch64_count_saves): New function.
	(aarch64_count_above_hard_fp_saves, aarch64_callee_save_cost)
	(aarch64_frame_allocation_cost): Likewise.
	(TARGET_CALLEE_SAVE_COST): Define.
	(TARGET_FRAME_ALLOCATION_COST): Likewise.
	* config/i386/i386.cc (ix86_ira_callee_saved_register_cost_scale):
	Replace with...
	(ix86_callee_save_cost): ...this new hook.
	(TARGET_IRA_CALLEE_SAVED_REGISTER_COST_SCALE): Delete.
	(TARGET_CALLEE_SAVE_COST): Define.
	* target.h (spill_cost_type, frame_cost_type): New enums.
	* target.def (callee_save_cost, frame_allocation_cost): New hooks.
	(ira_callee_saved_register_cost_scale): Delete.
	* doc/tm.texi.in (TARGET_IRA_CALLEE_SAVED_REGISTER_COST_SCALE): Delete.
	(TARGET_CALLEE_SAVE_COST, TARGET_FRAME_ALLOCATION_COST): New hooks.
	* doc/tm.texi: Regenerate.
	* hard-reg-set.h (hard_reg_set_popcount): New function.
	* ira-color.cc (allocated_memory_p): New variable.
	(allocated_callee_save_regs): Likewise.
	(record_allocation): New function.
	(assign_hard_reg): Use targetm.frame_allocation_cost to model
	the cost of the first spill or first caller save.  Use
	targetm.callee_save_cost to model the cost of using new callee-saved
	registers.  Apply the exit rather than entry frequency to the cost
	of restoring a register or deallocating the frame.  Update the
	new variables above.
	(improve_allocation): Use record_allocation.
	(color): Initialize allocated_callee_save_regs.
	(ira_color): Initialize allocated_memory_p.
	* targhooks.h (default_callee_save_cost): Declare.
	(default_frame_allocation_cost): Likewise.
	* targhooks.cc (default_callee_save_cost): New function.
	(default_frame_allocation_cost): Likewise.

gcc/testsuite/
	PR rtl-optimization/117477
	* gcc.target/aarch64/callee_save_1.c: New test.
	* gcc.target/aarch64/callee_save_2.c: Likewise.
	* gcc.target/aarch64/callee_save_3.c: Likewise.
	* gcc.target/aarch64/pr103350-1.c: Add -fno-caller-saves.

Co-authored-by: Jan Hubicka <hubicka@ucw.cz>
Co-authored-by: H.J. Lu <hjl.tools@gmail.com>