wide-int: Fix up wi::shifted_mask [PR106144]

As the following self-test testcase shows, wi::shifted_mask sometimes
doesn't create canonicalized wide_ints, which then fail to compare equal
to canonicalized wide_ints with the same value.
In particular, wi::mask (128, false, 128) gives { -1 } with len 1 and prec 128,
while wi::shifted_mask (0, 128, false, 128) gives { -1, -1 } with len 2
and prec 128.
The problem is that the code is written with the assumption that there are
3 bit blocks (or 2 if start is 0), but doesn't consider the possibility
where there are 2 bit blocks (or 1 if start is 0) where the highest block
isn't present.  In that case, there is the optional block of negate ? 0 : -1
elts, followed by just one elt (either one from the if (shift) or just
negate ? -1 : 0) and the rest is implicit sign-extension.
Only if end < prec there is 1 or more bits above it that have different bit
value and so we need to emit all the elts till end and then one more elt.

if (end == prec) would work too, because we have:
  if (width > prec - start)
    width = prec - start;
  unsigned int end = start + width;
so end is guaranteed to be end <= prec, dunno what is preferred.

2022-07-01  Jakub Jelinek  <jakub@redhat.com>

	PR middle-end/106144
	* wide-int.cc (wi::shifted_mask): If end >= prec, return right after
	emitting element for shift or if shift is 0 first element after start.
	(wide_int_cc_tests): Add tests for equivalency of wi::mask and
	wi::shifted_mask with 0 start.

(cherry picked from commit e52592073f6df3d7a3acd9f0436dcc32a8b7493d)

diff --git a/gcc/wide-int.cc b/gcc/wide-int.cc
index c377e68e4b97f39ca012dc4f3ad02eb8ce28e4ae..b173d0247a52ad39ff22510f38f707cda7249570 100644 (file)
--- a/gcc/wide-int.cc
+++ b/gcc/wide-int.cc
@@ -842,6 +842,13 @@ wi::shifted_mask (HOST_WIDE_INT *val, unsigned int start, unsigned int width,
        val[i++] = negate ? block : ~block;
     }
 
+  if (end >= prec)
+    {
+      if (!shift)
+       val[i++] = negate ? 0 : -1;
+      return i;
+    }
+
   while (i < end / HOST_BITS_PER_WIDE_INT)
     /* 1111111 */
     val[i++] = negate ? 0 : -1;
@@ -853,7 +860,7 @@ wi::shifted_mask (HOST_WIDE_INT *val, unsigned int start, unsigned int width,
       HOST_WIDE_INT block = (HOST_WIDE_INT_1U << shift) - 1;
       val[i++] = negate ? ~block : block;
     }
-  else if (end < prec)
+  else
     val[i++] = negate ? -1 : 0;
 
   return i;
@@ -2549,6 +2556,10 @@ wide_int_cc_tests ()
   run_all_wide_int_tests <widest_int> ();
   test_overflow ();
   test_round_for_mask ();
+  ASSERT_EQ (wi::mask (128, false, 128),
+            wi::shifted_mask (0, 128, false, 128));
+  ASSERT_EQ (wi::mask (128, true, 128),
+            wi::shifted_mask (0, 128, true, 128));
 }
 
 } // namespace selftest