configure: Enable -fno-omit-frame-pointer for backtrace module

The backtrace module is written assuming that the frame pointer is in %ebp.
By default, -Os optimization level is used, which enables the gcc option
-fomit-frame-pointer. This breaks the backtrace functionality. Enabling
this may cause an unnoticeable performance cost and virtually no size increase.

The backtrace command on x86_64 and probably i386 is broken due to the
above rationale. I've not verified, but presumably the backtrace that used
to be printed for an unhandled CPU exception is also broken. Do any distros
handle this?

Considering that, to my knowledge, no one has complained about this in the
over 13 years that -Os has been used, has this code actually been useful?
Is it worth disabling -fomit-frame-pointer? Though, I don't see much downside
right now in disabling it. Alternatively, we could disable/remove the
backtrace code. I think it would be nice to keep it and have it working.

Nowadays, presumably QEMU makes the GDB stub rarely used as I imagine most
are developing in a virtual machines. Also, the GDB stub does not work in UEFI.
So, if anyone is using it on real hardware, they are doing so on pretty old
machines. The lack of a GDB stub does not seem to be a pain point because
no one has got it working on UEFI.

This patch gets the backtrace command working on x86_64-efi in QEMU for me.
However, it hangs when run on my laptop. Not sure what's going on there.

Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>

diff --git a/configure.ac b/configure.ac
index 27c5918faa1000732354f78cea26978682cb2da0..77e20ad0251106a642dfbcb57871cca1be550774 100644 (file)
--- a/configure.ac
+++ b/configure.ac
@@ -1020,6 +1020,19 @@ if test x"$target_cpu" = xsparc64 ; then
   TARGET_LDFLAGS="$TARGET_LDFLAGS $grub_cv_target_cc_mno_relax"
 fi
 
+# The backtrace module relies on frame pointers and the default optimization
+# level, -Os, omits them. Make sure they are enabled.
+AC_CACHE_CHECK([whether -fno-omit-frame-pointer works], [grub_cv_cc_fno_omit_frame_pointer], [
+  CFLAGS="$TARGET_CFLAGS -fno-omit-frame-pointer"
+  AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[]], [[]])],
+      [grub_cv_cc_fno_omit_frame_pointer=yes],
+      [grub_cv_cc_fno_omit_frame_pointer=no])
+])
+
+if test "x$grub_cv_cc_fno_omit_frame_pointer" = xyes; then
+  TARGET_CFLAGS="$TARGET_CFLAGS -fno-omit-frame-pointer"
+fi
+
 # By default, GCC 4.4 generates .eh_frame sections containing unwind
 # information in some cases where it previously did not. GRUB doesn't need
 # these and they just use up vital space. Restore the old compiler