]> git.ipfire.org Git - thirdparty/gcc.git/commit
tree-nested: Do not inline or clone functions with nested functions with VM return...
authorJoseph Myers <josmyers@redhat.com>
Fri, 15 Nov 2024 14:08:42 +0000 (14:08 +0000)
committerJoseph Myers <josmyers@redhat.com>
Fri, 15 Nov 2024 14:08:42 +0000 (14:08 +0000)
commit3320319ede5ce1cb34ba016f4a0d1e6588059980
tree6e346a607ba1687cc464fc9c8ac6e9640ee167df
parentc57b2f88c93a00f4248b4e5da4de90d142967b4c
tree-nested: Do not inline or clone functions with nested functions with VM return type [PR117164]

Bug 117164 is an ICE on an existing test with -std=gnu23 involving a
nested function returning a variable-size structure (and I think the
last bug needing to be resolved before switching to -std=gnu23 as the
default, as without fixing this would be a clear regression from a
change in default).

The problem is a GIMPLE verification failure where (after type
remapping from inlining / cloning) the return type of the function no
longer exactly matches the type to which it is assigned (these types
use structural equality, which means GIMPLE verification can't use
TYPE_CANONICAL and expects an exact match).  Specifically, the nested
function itself is *not* inlined (the -fno-inline-small-functions in
the original test nested-func-12.c, I think, or the noinline attribute
in some of my variant tests), but the function containing it is either
cloned (the --param ipa-cp-eval-threshold=0 in the original test) or
inlined.  (I'm not sure what role -fno-guess-branch-probability plays
in getting the right situation for the ICE; maybe affecting when
inlining or cloning is considered profitable?)

There is in fact existing code in tree-nested.cc to prevent inlining
of a function containing a nested function with variably modified
*argument* types.  I think the same issue of ensuring consistency of
types means such prevention should also apply for a variably modified
return type.  Furthermore, exactly the same problem applies for
cloning for other reasons as it does for inlining.  Thus, change the
logic to include variably modified return types for nested functions
alongside those for arguments of those functions as a reason not to
inline, and also add the noclone attribute in these cases.

Bootstrapped with no regressions for x86-64-pc-linux-gnu.

PR c/117164

gcc/
* tree-nested.cc: Include "attribs.h".
(check_for_nested_with_variably_modified): Also return true for
variably modified return type.
(create_nesting_tree): If check_for_nested_with_variably_modified
returns true, also add noclone attribute.

gcc/testsuite/
* gcc.dg/nested-func-13.c, gcc.dg/nested-func-14.c:
gcc.dg/nested-func-15.c, gcc.dg/nested-func-16.c,
gcc.dg/nested-func-17.c: New tests.
gcc/testsuite/gcc.dg/nested-func-13.c [new file with mode: 0644]
gcc/testsuite/gcc.dg/nested-func-14.c [new file with mode: 0644]
gcc/testsuite/gcc.dg/nested-func-15.c [new file with mode: 0644]
gcc/testsuite/gcc.dg/nested-func-16.c [new file with mode: 0644]
gcc/testsuite/gcc.dg/nested-func-17.c [new file with mode: 0644]
gcc/tree-nested.cc