From 07c1c6ec51a4474c22abbee731cfc8111fc09a43 Mon Sep 17 00:00:00 2001 From: David Rowley Date: Sun, 4 Jan 2026 20:33:14 +1300 Subject: [PATCH] Fix selectivity estimation integer overflow in contrib/intarray This fixes a poorly written integer comparison function which was performing subtraction in an attempt to return a negative value when a < b and a positive value when a > b, and 0 when the values were equal. Unfortunately that didn't always work correctly due to two's complement having the INT_MIN 1 further from zero than INT_MAX. This could result in an overflow and cause the comparison function to return an incorrect result, which would result in the binary search failing to find the value being searched for. This could cause poor selectivity estimates when the statistics stored the value of INT_MAX (2147483647) and the value being searched for was large enough to result in the binary search doing a comparison with that INT_MAX value. Author: Chao Li Reviewed-by: David Rowley Discussion: https://postgr.es/m/CAEoWx2ng1Ot5LoKbVU-Dh---dFTUZWJRH8wv2chBu29fnNDMaQ@mail.gmail.com Backpatch-through: 14 --- contrib/intarray/_int_selfuncs.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/contrib/intarray/_int_selfuncs.c b/contrib/intarray/_int_selfuncs.c index 9bf64486242..4f64e4cf2f1 100644 --- a/contrib/intarray/_int_selfuncs.c +++ b/contrib/intarray/_int_selfuncs.c @@ -326,7 +326,12 @@ static int compare_val_int4(const void *a, const void *b) { int32 key = *(int32 *) a; - const Datum *t = (const Datum *) b; + int32 value = DatumGetInt32(*(const Datum *) b); - return key - DatumGetInt32(*t); + if (key < value) + return -1; + else if (key > value) + return 1; + else + return 0; } -- 2.47.3