]> git.ipfire.org Git - thirdparty/gcc.git/commit
Add target-independent store forwarding avoidance pass
authorKonstantinos Eleftheriou <konstantinos.eleftheriou@vrull.eu>
Wed, 16 Oct 2024 08:31:39 +0000 (10:31 +0200)
committerPhilipp Tomsich <philipp.tomsich@vrull.eu>
Mon, 25 Nov 2024 02:19:45 +0000 (03:19 +0100)
commit1d8de1e93ea00f7797f61cf8e05c47ca86f21f8c
treedec2c35b382d6ce2ba65e07a72e2cb8ea5d96c71
parentba4cf2e296d8d5950c3d356fa6b6efcad00d0189
Add target-independent store forwarding avoidance pass

This pass detects cases of expensive store forwarding and tries to
avoid them by reordering the stores and using suitable bit insertion
sequences.  For example it can transform this:

     strb    w2, [x1, 1]
     ldr     x0, [x1]      # Expensive store forwarding to larger load.

To:

     ldr     x0, [x1]
     strb    w2, [x1]
     bfi     x0, x2, 0, 8

Assembly like this can appear with bitfields or type punning / unions.
On stress-ng when running the cpu-union microbenchmark the following
speedups have been observed.

  Neoverse-N1:      +29.4%
  Intel Coffeelake: +13.1%
  AMD 5950X:        +17.5%

The transformation is rejected on cases that cause store_bit_field to
generate subreg expressions on different register classes.  Files
avoid-store-forwarding-4.c and avoid-store-forwarding-5.c contain such
cases and have been marked as XFAIL.

Due to biasing of its operands in store_bit_field, there is a special
handling for machines with BITS_BIG_ENDIAN != BYTES_BIG_ENDIAN. The
need for this was exosed by an issue exposed on the H8 architecture,
which uses big-endian ordering, but BITS_BIG_ENDIAN is false. In that
case, the START parameter of store_bit_field needs to be calculated
from the end of the destination register.

gcc/ChangeLog:

* Makefile.in (OBJS): Add avoid-store-forwarding.o.
* common.opt (favoid-store-forwarding): New option.
* common.opt.urls: Regenerate.
* doc/invoke.texi: New param store-forwarding-max-distance.
* doc/passes.texi: Document new pass.
* doc/tm.texi: Regenerate.
* doc/tm.texi.in: Document new pass.
* params.opt (store-forwarding-max-distance): New param.
* passes.def: Add pass_rtl_avoid_store_forwarding before
pass_early_remat.
* target.def (avoid_store_forwarding_p): New DEFHOOK.
* target.h (struct store_fwd_info): Declare.
* targhooks.cc (default_avoid_store_forwarding_p): New function.
* targhooks.h (default_avoid_store_forwarding_p): Declare.
* tree-pass.h (make_pass_rtl_avoid_store_forwarding): Declare.
* avoid-store-forwarding.cc: New file.
* avoid-store-forwarding.h: New file.
* timevar.def (TV_AVOID_STORE_FORWARDING): New timevar.

gcc/testsuite/ChangeLog:

* gcc.target/aarch64/avoid-store-forwarding-1.c: New test.
* gcc.target/aarch64/avoid-store-forwarding-2.c: New test.
* gcc.target/aarch64/avoid-store-forwarding-3.c: New test.
* gcc.target/aarch64/avoid-store-forwarding-4.c: New test.
* gcc.target/aarch64/avoid-store-forwarding-5.c: New test.
* gcc.target/x86_64/abi/callabi/avoid-store-forwarding-1.c: New test.
* gcc.target/x86_64/abi/callabi/avoid-store-forwarding-2.c: New test.

Co-authored-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
Signed-off-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
Signed-off-by: Konstantinos Eleftheriou <konstantinos.eleftheriou@vrull.eu>
24 files changed:
gcc/Makefile.in
gcc/avoid-store-forwarding.cc [new file with mode: 0644]
gcc/avoid-store-forwarding.h [new file with mode: 0644]
gcc/common.opt
gcc/common.opt.urls
gcc/doc/invoke.texi
gcc/doc/passes.texi
gcc/doc/tm.texi
gcc/doc/tm.texi.in
gcc/params.opt
gcc/passes.def
gcc/target.def
gcc/target.h
gcc/targhooks.cc
gcc/targhooks.h
gcc/testsuite/gcc.target/aarch64/avoid-store-forwarding-1.c [new file with mode: 0644]
gcc/testsuite/gcc.target/aarch64/avoid-store-forwarding-2.c [new file with mode: 0644]
gcc/testsuite/gcc.target/aarch64/avoid-store-forwarding-3.c [new file with mode: 0644]
gcc/testsuite/gcc.target/aarch64/avoid-store-forwarding-4.c [new file with mode: 0644]
gcc/testsuite/gcc.target/aarch64/avoid-store-forwarding-5.c [new file with mode: 0644]
gcc/testsuite/gcc.target/x86_64/abi/callabi/avoid-store-forwarding-1.c [new file with mode: 0644]
gcc/testsuite/gcc.target/x86_64/abi/callabi/avoid-store-forwarding-2.c [new file with mode: 0644]
gcc/timevar.def
gcc/tree-pass.h