From: Joseph Myers <joseph@codesourcery.com>
Date: Thu, 9 Oct 2014 01:07:10 +0000 (+0000)
Subject: soft-fp: Fix _FP_TO_INT latent bug in overflow handling.
X-Git-Tag: glibc-2.21~508
X-Git-Url: http://git.ipfire.org/?a=commitdiff_plain;h=0022e688d082761a1e15b19121303cafbdad33ec;p=thirdparty%2Fglibc.git

soft-fp: Fix _FP_TO_INT latent bug in overflow handling.

This patch fixes a latent bug in _FP_TO_INT regarding handling of
arguments with maximum exponent (infinities and NaNs).  If the maximum
exponent is below that calculated as an overflow threshold, such
values would incorrectly be treated as normal values for the purposes
of the conversion.  This could not occur for any of the conversions
actually occurring in glibc, libgcc or the Linux kernel (the maximum
exponent for float is, just, big enough to ensure overflow for
unsigned __int128), but would apply if soft-fp were used for IEEE
binary16.  Appropriate checks are inserted to ensure that the maximum
exponent is always treated as an overflowing exponent, and never as a
normal one.

Tested for powerpc-nofpu that the disassembly of installed shared
libraries is unchanged by this patch.

	* soft-fp/op-common.h (_FP_TO_INT): Ensure maximum exponent is
	treated as invalid conversion, not as normal exponent.
---

diff --git a/ChangeLog b/ChangeLog
index f95e3ed79ab..18d1cbcfb6f 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,8 @@
 2014-10-09  Joseph Myers  <joseph@codesourcery.com>
 
+	* soft-fp/op-common.h (_FP_TO_INT): Ensure maximum exponent is
+	treated as invalid conversion, not as normal exponent.
+
 	* soft-fp/op-common.h (_FP_CMP_CHECK_NAN): New macro.
 	(_FP_CMP): Add extra argument EX.  Call _FP_CMP_CHECK_NAN.
 	(_FP_CMP_EQ): Likewise.
diff --git a/soft-fp/op-common.h b/soft-fp/op-common.h
index e62558e4b4a..e0a108a6ee3 100644
--- a/soft-fp/op-common.h
+++ b/soft-fp/op-common.h
@@ -1368,7 +1368,9 @@
 	  else								\
 	    FP_SET_EXCEPTION (FP_EX_INEXACT);				\
 	}								\
-      else if (X##_e >= _FP_EXPBIAS_##fs + rsize - (rsigned > 0 || X##_s) \
+      else if (X##_e >= (_FP_EXPMAX_##fs < _FP_EXPBIAS_##fs + rsize	\
+			 ? _FP_EXPMAX_##fs				\
+			 : _FP_EXPBIAS_##fs + rsize - (rsigned > 0 || X##_s)) \
 	       || (!rsigned && X##_s))					\
 	{								\
 	  /* Overflow or converting to the most negative integer.  */	\
@@ -1385,7 +1387,10 @@
 		r = ~r;							\
 	    }								\
 									\
-	  if (rsigned && X##_s && X##_e == _FP_EXPBIAS_##fs + rsize - 1) \
+	  if (_FP_EXPBIAS_##fs + rsize - 1 < _FP_EXPMAX_##fs		\
+	      && rsigned						\
+	      && X##_s							\
+	      && X##_e == _FP_EXPBIAS_##fs + rsize - 1)			\
 	    {								\
 	      /* Possibly converting to most negative integer; check the \
 		 mantissa.  */						\