[gdb/tui] Handle error in tui_enable

Say we simulate an error:
...
static void error_once () {
  static int v = 0;
  if (v == 1)
    return;

  v = 1;
  error (_("Oh no!!!"));
}
...
in the call to tui_set_initial_layout in tui_enable:
...
       tui_show_frame_info (deprecated_safe_get_selected_frame ());
+      error_once ();
       tui_set_initial_layout ();
...

After doing "tui enable"
...
$ gdb
(gdb) tui enable
...
the screen is cleared, and we have:
...
❌️ Oh no!!!
           (gdb) <blinking cursor>
...

After typing "apropos tui" (which is not echoed) and pressing enter, I run into
a segmentation violation here in tui_inject_newline_into_command_window:
...
    at /data/vries/gdb/leap-16-0/build/../../src/gdb/tui/tui-io.c:1084
1084	  WINDOW *w = tui_cmd_win ()->handle.get ();
...
because:
...
$2 = (tui_cmd_window *) 0x0
(gdb) p tui_cmd_win ()
...

The problem is that tui_active is true, and so
tui_inject_newline_into_command_window get called:
...
static void
tui_command_line_handler (gdb::unique_xmalloc_ptr<char> &&rl)
{
 ...
  if (tui_active)
    tui_inject_newline_into_command_window ();
 ...
}
...

Fix this by catching the error in tui_enable, and resetting tui_active back to
false.

While this fixes the segmentation fault, and does allow "apropos tui" to run,
still the command is not echoed, and the output of the command is garbled by
runaway indentation.

Fix this by using endwin / delscreen, as borrowed from earlier in tui_enable:
...
      if (cap == NULL || cap == (char *) -1 || *cap == '\0')
        {
          endwin ();
          delscreen (s);
          error (_("Cannot enable the TUI: "
                   "terminal doesn't support cursor addressing [TERM=%s]"),
                 gdb_getenv_term ());
        }
...

Note that this doesn't allow a second attempt:
...
$ gdb -q
(gdb) tui enable
❌️ Oh no!!!
(gdb) tui enable
❌️ Cannot enable the TUI
(gdb)
...
because tui_finish_init is stuck at TRIBOOL_UNKNOWN.

IWBN to fix this in a way that allows the second "tui enable" to succeed.  I
tried this for a bit, but didn't get it to work.

Tested on x86_64-linux.

Suggested-By: Tom Tromey <tom@tromey.com> [1]
Approved-By: Tom Tromey <tom@tromey.com>
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=34100

[1] https://sourceware.org/pipermail/gdb-patches/2025-May/217660.html

diff --git a/gdb/tui/tui.c b/gdb/tui/tui.c
index 35c3b80603d06046078b50b123569b9e4961bdfd..ba4f6f687697084cea1616a42cd9a018575c97bb 100644 (file)
--- a/gdb/tui/tui.c
+++ b/gdb/tui/tui.c
@@ -491,22 +491,31 @@ tui_enable (void)
         rely on this flag being true in order to know that the window
         they are creating is currently valid.  */
       tui_active = true;
         rely on this flag being true in order to know that the window
         they are creating is currently valid.  */
       tui_active = true;

+      try
+       {
+         cbreak ();
+         noecho ();
+         /* timeout (1); */
+         nodelay (w, FALSE);
+         nl ();
+         keypad (w, TRUE);
+         tui_set_term_height_to (LINES);
+         tui_set_term_width_to (COLS);
+         def_prog_mode ();
+         tui_show_frame_info (deprecated_safe_get_selected_frame ());
+         tui_set_initial_layout ();
+         tui_set_win_focus_to (tui_src_win ());
+         keypad (tui_cmd_win ()->handle.get (), TRUE);
+         wrefresh (tui_cmd_win ()->handle.get ());
+       }
+      catch (const gdb_exception &)
+       {
+         endwin ();
+         delscreen (s);
+         tui_active = false;
+         throw;
+       }

 
 

-      cbreak ();
-      noecho ();
-      /* timeout (1); */
-      nodelay(w, FALSE);
-      nl();
-      keypad (w, TRUE);
-      tui_set_term_height_to (LINES);
-      tui_set_term_width_to (COLS);
-      def_prog_mode ();
-
-      tui_show_frame_info (deprecated_safe_get_selected_frame ());
-      tui_set_initial_layout ();
-      tui_set_win_focus_to (tui_src_win ());
-      keypad (tui_cmd_win ()->handle.get (), TRUE);
-      wrefresh (tui_cmd_win ()->handle.get ());

       tui_finish_init = TRIBOOL_FALSE;
     }
   else
       tui_finish_init = TRIBOOL_FALSE;
     }
   else