Adjust inner loop cost scaling

This makes use of the estimated number of iterations of the inner loop
to limit --param vect-inner-loop-cost-factor scaling.  It also reduces
the maximum value of vect-inner-loop-cost-factor to 10000 making it
less likely to cause overflow of costs.

2021-08-23  Richard Biener  <rguenther@suse.de>

	* doc/invoke.texi (vect-inner-loop-cost-factor): Adjust.
	* params.opt (--param vect-inner-loop-cost-factor): Adjust
	maximum value.
	* tree-vect-loop.c (vect_analyze_loop_form): Initialize
	inner_loop_cost_factor to the minimum of the estimated number
	of iterations of the inner loop and vect-inner-loop-cost-factor.

diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index c057cc1e4ae5a4b993db931fc2a94185df155488..a9d56fecf4eb35501a54039ddf38df78c3be8c73 100644 (file)
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -14386,9 +14386,10 @@ The parameter only has an effect on targets that support partial
 vector loads and stores.
 
 @item vect-inner-loop-cost-factor
-The factor which the loop vectorizer applies to the cost of statements
-in an inner loop relative to the loop being vectorized.  The default
-value is 50.
+The maximum factor which the loop vectorizer applies to the cost of statements
+in an inner loop relative to the loop being vectorized.  The factor applied
+is the maximum of the estimated number of iterations of the inner loop and
+this parameter.  The default value of this parameter is 50.
 
 @item avoid-fma-max-bits
 Maximum number of bits for which we avoid creating FMAs.

diff --git a/gcc/params.opt b/gcc/params.opt
index f9264887b409710a02401998b96a663bf2635115..f414dc1a61cfa9d5b9ded75e96560fc1f73041a5 100644 (file)
--- a/gcc/params.opt
+++ b/gcc/params.opt
@@ -1114,7 +1114,7 @@ Common Joined UInteger Var(param_vect_partial_vector_usage) Init(2) IntegerRange
 Controls how loop vectorizer uses partial vectors.  0 means never, 1 means only for loops whose need to iterate can be removed, 2 means for all loops.  The default value is 2.
 
 -param=vect-inner-loop-cost-factor=
-Common Joined UInteger Var(param_vect_inner_loop_cost_factor) Init(50) IntegerRange(1, 999999) Param Optimization
-The factor which the loop vectorizer applies to the cost of statements in an inner loop relative to the loop being vectorized.
+Common Joined UInteger Var(param_vect_inner_loop_cost_factor) Init(50) IntegerRange(1, 10000) Param Optimization
+The maximum factor which the loop vectorizer applies to the cost of statements in an inner loop relative to the loop being vectorized.
 
 ; This comment is to ensure we retain the blank line above.

diff --git a/gcc/tree-vect-loop.c b/gcc/tree-vect-loop.c
index c521b43a47c17cd4ea2322cbe6c499a992e96505..0c8d992624b59ddd056aff594738305d6be5afa8 100644 (file)
--- a/gcc/tree-vect-loop.c
+++ b/gcc/tree-vect-loop.c
@@ -1519,6 +1519,13 @@ vect_analyze_loop_form (class loop *loop, vec_info_shared *shared)
       stmt_vec_info inner_loop_cond_info
        = loop_vinfo->lookup_stmt (inner_loop_cond);
       STMT_VINFO_TYPE (inner_loop_cond_info) = loop_exit_ctrl_vec_info_type;
+      /* If we have an estimate on the number of iterations of the inner
+        loop use that to limit the scale for costing, otherwise use
+        --param vect-inner-loop-cost-factor literally.  */
+      widest_int nit;
+      if (estimated_stmt_executions (loop->inner, &nit))
+       LOOP_VINFO_INNER_LOOP_COST_FACTOR (loop_vinfo)
+         = wi::smin (nit, param_vect_inner_loop_cost_factor).to_uhwi ();
     }
 
   gcc_assert (!loop->aux);