From 8f60f5499e10d19218cada082e0909516ebf0e74 Mon Sep 17 00:00:00 2001 From: Jakub Jelinek Date: Fri, 8 Dec 2023 08:56:33 +0100 Subject: [PATCH] haifa-sched: Avoid overflows in extend_h_i_d [PR112411] On Thu, Dec 07, 2023 at 09:36:23AM +0100, Jakub Jelinek wrote: > Without the dg-skip-if I got on 64-bit host with > -O3 --param min-nondebug-insn-uid=0x40000000: > cc1: out of memory allocating 571230784744 bytes after a total of 2772992 bytes I've looked at this and the problem is in haifa-sched.cc: 9047 h_i_d.safe_grow_cleared (3 * get_max_uid () / 2, true); get_max_uid () is 0x4000024d with the --param min-nondebug-insn-uid=0x40000000 and so 3 * get_max_uid () / 2 actually overflows to -536870028 but as vec.h then treats the value as unsigned, it attempts to allocate 0xe0000374U * 152UL bytes, i.e. those 532GB. If the above is fixed to do 3U * get_max_uid () / 2 instead, it will get slightly better and will only need 0x60000373U * 152UL bytes, i.e. 228GB. Perhaps more could be helped by making the vector indirect (contain pointers to haifa_insn_data_def rather than the structures themselves) and pool allocate those, but the more important question is how sparse are uids in normal compilations without those large --param min-nondebug-insn-uid= parameters. Because if they aren't enough, such a change would increase compile time memory just to help the unusual case. 2023-12-08 Jakub Jelinek PR middle-end/112411 * haifa-sched.cc (extend_h_i_d): Use 3U instead of 3 in 3 * get_max_uid () / 2 calculation. --- gcc/haifa-sched.cc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gcc/haifa-sched.cc b/gcc/haifa-sched.cc index 8e8add709b3a..35c2c9f2bdc7 100644 --- a/gcc/haifa-sched.cc +++ b/gcc/haifa-sched.cc @@ -9044,7 +9044,7 @@ extend_h_i_d (void) if (reserve > 0 && ! h_i_d.space (reserve)) { - h_i_d.safe_grow_cleared (3 * get_max_uid () / 2, true); + h_i_d.safe_grow_cleared (3U * get_max_uid () / 2, true); sched_extend_target (); } } -- 2.47.2