vect: Tighten check for impossible SLP layouts [PR113205]
During its forward pass, the SLP layout code tries to calculate
the cost of a layout change on an incoming edge. This is taken
as the minimum of two costs: one in which the source partition
keeps its current layout (chosen earlier during the pass) and
one in which the source partition switches to the new layout.
The latter can sometimes be arranged by the backward pass.
If only one of the costs is valid, the other cost was ignored.
But the PR shows that this is not safe. If the source partition
has layout 0 (the normal layout), we have to be prepared to handle
the case in which that ends up being the only valid layout.
Other code already accounts for this restriction, e.g. see
the code starting with:
/* Reject the layout if it would make layout 0 impossible
for later partitions. This amounts to testing that the
target supports reversing the layout change on edges
to later partitions.
gcc/
PR tree-optimization/113205
* tree-vect-slp.cc (vect_optimize_slp_pass::forward_cost): Reject
the proposed layout if it does not allow a source partition with
layout 2 to keep that layout.
gcc/testsuite/
PR tree-optimization/113205
* gcc.dg/torture/pr113205.c: New test.