i386: Fix ICE on out of bounds vector elt access [PR126446]

The following testcase ICEs on x86_64.
The isel pass has a check for out of bounds constant index before
optimizing into .VEC_SET, but it does it using
      // if index is a constant, then check the bounds
      poly_uint64 idx_poly;
      if (poly_int_tree_p (idx, &idx_poly))
        {
          poly_uint64 nelts = TYPE_VECTOR_SUBPARTS (TREE_TYPE (view_op0));
          if (known_gt (idx_poly, nelts))
            return false;
        }
In the testcase below, idx is INTEGER_CST with long long type and
negative value, that doesn't fit into poly_uint64, so we happily convert
it into .VEC_SET.

And another problem is that the x86 backend isn't trying to be careful
and handle out of bounds elt gracefully (I think it could still in theory
happen, if GIMPLE lets it through but e.g. something during expansion
figures out the index is constant or whatever).

The following patch fixes it in the backend to avoid triggering UB at compile
time by doing HOST_WIDE_INT_1U << elt etc. when elt is negative or too
large.  In order to avoid ICE, we need to emit something, so I emit
a no-op move, out of bounds vector set shouldn't change anything in
the target.

gimple-isel.cc will be changed incrementally.

2026-07-29  Jakub Jelinek  <jakub@redhat.com>

	PR target/126446
	* config/i386/i386-expand.cc (ix86_expand_vector_set): If elt is
	out of bounds, emit a no-op move.

	* gcc.target/i386/avx2-pr126446.c: New test.

Reviewed-by: Uros Bizjak <ubizjak@gmail.com>

diff --git a/gcc/config/i386/i386-expand.cc b/gcc/config/i386/i386-expand.cc
index 20914fcbad9dd572075f4f213b133baa08b1a887..9a37607146471581cd55c7c80331efc994291bd8 100644 (file)
--- a/gcc/config/i386/i386-expand.cc
+++ b/gcc/config/i386/i386-expand.cc
@@ -19157,6 +19157,11 @@ ix86_expand_vector_set (bool mmx_ok, rtx target, rtx val, int elt)
   machine_mode mmode = VOIDmode;
   rtx (*gen_blendm) (rtx, rtx, rtx, rtx);
 
+  if (!IN_RANGE (elt, 0, GET_MODE_NUNITS (mode)))
+    {
+      emit_move_insn (target, target);
+      return;
+    }
   if (TARGET_SSE4_1 && mode == V4SImode && val == const0_rtx)
     {
       emit_insn (gen_sse4_1_insertps_v4si_zero (target, target,

diff --git a/gcc/testsuite/gcc.target/i386/avx2-pr126446.c b/gcc/testsuite/gcc.target/i386/avx2-pr126446.c
new file mode 100644 (file)
index 0000000..4af8cfd
--- /dev/null
+++ b/gcc/testsuite/gcc.target/i386/avx2-pr126446.c
@@ -0,0 +1,14 @@
+/* PR target/126446 */
+/* { dg-do compile } */
+/* { dg-options "-O1 -mavx2" } */
+
+typedef signed char V __attribute__((vector_size (16)));
+
+signed char
+foo ()
+{
+  V b = {};
+  long long c = ~2878966870562407444LL;
+  b[c] = 1;
+  return b[0];
+}