From: Jakub Jelinek <jakub@redhat.com>
Date: Thu, 15 Jul 2021 16:53:20 +0000 (+0200)
Subject: c++: Optimize away NULLPTR_TYPE comparisons [PR101443]
X-Git-Tag: basepoints/gcc-13~5982
X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=7094a69bd62a14dfa311eaa2fea468f221c7c9f3;p=thirdparty%2Fgcc.git

c++: Optimize away NULLPTR_TYPE comparisons [PR101443]

Comparisons of NULLPTR_TYPE operands cause all kinds of problems in the
middle-end and in fold-const.c, various optimizations assume that if they
see e.g. a non-equality comparison with one of the operands being
INTEGER_CST and it is not INTEGRAL_TYPE_P (which has TYPE_{MIN,MAX}_VALUE),
they can build_int_cst (type, 1) to find a successor.

The following patch fixes it by making sure they don't appear in the IL,
optimize them away at cp_fold time as all can be folded.

Though, I've just noticed that clang++ rejects the non-equality comparisons
instead, foo () > 0 with
invalid operands to binary expression ('decltype(nullptr)' (aka 'nullptr_t') and 'int')
and foo () > nullptr with
invalid operands to binary expression ('decltype(nullptr)' (aka 'nullptr_t') and 'nullptr_t')

Shall we reject those too, in addition or instead of parts of this patch?
If so, wouldn't this patch be still useful for backports, I bet we don't
want to start reject it on the release branches when we used to accept it.

2021-07-15  Jakub Jelinek  <jakub@redhat.com>

	PR c++/101443
	* cp-gimplify.c (cp_fold): For comparisons with NULLPTR_TYPE
	operands, fold them right away to true or false.

	* g++.dg/cpp0x/nullptr46.C: New test.
---

diff --git a/gcc/cp/cp-gimplify.c b/gcc/cp/cp-gimplify.c
index de37f2cdfdca..ff0bff771df6 100644
--- a/gcc/cp/cp-gimplify.c
+++ b/gcc/cp/cp-gimplify.c
@@ -2423,6 +2423,32 @@ cp_fold (tree x)
       op0 = cp_fold_maybe_rvalue (TREE_OPERAND (x, 0), rval_ops);
       op1 = cp_fold_rvalue (TREE_OPERAND (x, 1));
 
+      /* decltype(nullptr) has only one value, so optimize away all comparisons
+	 with that type right away, keeping them in the IL causes troubles for
+	 various optimizations.  */
+      if (COMPARISON_CLASS_P (org_x)
+	  && TREE_CODE (TREE_TYPE (op0)) == NULLPTR_TYPE
+	  && TREE_CODE (TREE_TYPE (op1)) == NULLPTR_TYPE)
+	{
+	  switch (code)
+	    {
+	    case EQ_EXPR:
+	    case LE_EXPR:
+	    case GE_EXPR:
+	      x = constant_boolean_node (true, TREE_TYPE (x));
+	      break;
+	    case NE_EXPR:
+	    case LT_EXPR:
+	    case GT_EXPR:
+	      x = constant_boolean_node (false, TREE_TYPE (x));
+	      break;
+	    default:
+	      gcc_unreachable ();
+	    }
+	  return omit_two_operands_loc (loc, TREE_TYPE (x), x,
+					op0, op1);
+	}
+
       if (op0 != TREE_OPERAND (x, 0) || op1 != TREE_OPERAND (x, 1))
 	{
 	  if (op0 == error_mark_node || op1 == error_mark_node)
diff --git a/gcc/testsuite/g++.dg/cpp0x/nullptr46.C b/gcc/testsuite/g++.dg/cpp0x/nullptr46.C
new file mode 100644
index 000000000000..1514cee3c3bb
--- /dev/null
+++ b/gcc/testsuite/g++.dg/cpp0x/nullptr46.C
@@ -0,0 +1,11 @@
+// PR c++/101443
+// { dg-do compile { target c++11 } }
+// { dg-options "-O2" }
+
+decltype(nullptr) foo ();
+
+bool
+bar ()
+{
+  return foo () > nullptr || foo () < nullptr;
+}