vect: Make all exit conditions early breaks for uncounted loops

For uncounted loops, given how no information can be derived from the
max number of iterations, we wish to make no distinction between the
"main" exit and any additional exits.

That is, an epilogue is required across all exits to establish when
the exit condition was met within the final vectorized loop iteration.

This can be accomplished via a two-fold approach:

     1. The need for all exits to go to the epilogue is shared with
     counted loops with early-break exits after the IV-counting exit.
     Such counted loops have the `LOOP_VINFO_EARLY_BREAKS_VECT_PEELED'
     flag set.  By modifying the flag's definition to encompass
     uncounted loops, we can make considerable use of the code written
     for this category of counted loops.

     2. Currently, when populating the `loop_vinfo' struct for a given
     loop, there is an assumption that the first exit condition in the
     `vect_loop_form_info' struct is the IV condition and any
     subsequent conditions are early break conditions.  This
     assumption breaks down for uncounted loops where _all_ exits
     should be treated as being "early break" exits.

     This is fixed by populating `LOOP_VINFO_LOOP_IV_COND (loop_vinfo)'
     conditionally on the false evaluation of the
     `LOOP_VINFO_NITERS_UNCOUNTED_P (loop_vinfo)' predicate, such that
     if we do have an uncounted loop we leave this field unpopulated,
     storing all conditions in `LOOP_VINFO_LOOP_CONDS (loop_vinfo)'.
     This approach has a further benefit in that, give how
     `LOOP_VINFO_EARLY_BREAKS' is defined by a non-empty list of early
     break exit conditions (`LOOP_VINFO_LOOP_CONDS (loop_vinfo)'),
     having LOOP_VINFO_LOOP_CONDS populated even for single-exit
     uncounted loops means that `LOOP_VINFO_EARLY_BREAKS' evaluates to
     true for all uncounted loops, irrespective of the number of exits
     it has.

gcc/ChangeLog:

	* tree-vectorizer.h (LOOP_VINFO_EARLY_BREAKS_VECT_PEELED): OR
	its current definition with `LOOP_VINFO_NITERS_UNCOUNTED_P(L)'
	* tree-vect-loop.cc (vect_create_loop_vinfo): Don't populate
	`LOOP_VINFO_LOOP_IV_COND' for uncounted loops.

diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc
index eb89d8e6ebe828fe10b2000b3aea55d8b73fc9b4..3a38732e74634dd885232e2833f7d5e138b8f18a 100644 (file)
--- a/gcc/tree-vect-loop.cc
+++ b/gcc/tree-vect-loop.cc
@@ -1653,9 +1653,12 @@ vect_create_loop_vinfo (class loop *loop, vec_info_shared *shared,
       STMT_VINFO_DEF_TYPE (loop_cond_info) = vect_condition_def;
     }
 
-  for (unsigned i = 1; i < info->conds.length (); i ++)
-    LOOP_VINFO_LOOP_CONDS (loop_vinfo).safe_push (info->conds[i]);
-  LOOP_VINFO_LOOP_IV_COND (loop_vinfo) = info->conds[0];
+  unsigned cond_id = 0;
+  if (!LOOP_VINFO_NITERS_UNCOUNTED_P (loop_vinfo))
+      LOOP_VINFO_LOOP_IV_COND (loop_vinfo) = info->conds[cond_id++];
+
+  for (; cond_id < info->conds.length (); cond_id ++)
+    LOOP_VINFO_LOOP_CONDS (loop_vinfo).safe_push (info->conds[cond_id]);
 
   LOOP_VINFO_IV_EXIT (loop_vinfo) = info->loop_exit;
 

diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h
index 5b68c3b46fe0f51a2013a82cc09895fe7e623882..c9dee90c4701c295eee227fa3115052ca076cb76 100644 (file)
--- a/gcc/tree-vectorizer.h
+++ b/gcc/tree-vectorizer.h
@@ -1313,7 +1313,8 @@ public:
 #define LOOP_VINFO_EARLY_BREAKS(L)         (L)->early_breaks
 #define LOOP_VINFO_EARLY_BRK_STORES(L)     (L)->early_break_stores
 #define LOOP_VINFO_EARLY_BREAKS_VECT_PEELED(L)  \
-  (single_pred ((L)->loop->latch) != (L)->vec_loop_iv_exit->src)
+  ((single_pred ((L)->loop->latch) != (L)->vec_loop_iv_exit->src) \
+   || LOOP_VINFO_NITERS_UNCOUNTED_P (L))
 #define LOOP_VINFO_EARLY_BREAKS_LIVE_IVS(L)  \
   (L)->early_break_live_ivs
 #define LOOP_VINFO_EARLY_BRK_DEST_BB(L)    (L)->early_break_dest_bb