From: Peter Damianov <peter0x44@disroot.org>
Date: Mon, 3 Jun 2024 17:07:08 +0000 (-0700)
Subject: diagnostics: Enable escape sequence processing on windows consoles
X-Git-Tag: basepoints/gcc-16~8199
X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=e943a5da40cd4799908d3d29001e1325eb00b755;p=thirdparty%2Fgcc.git

diagnostics: Enable escape sequence processing on windows consoles

Since windows 10 release v1511, the windows console has had support for VT100
escape sequences. We should try to enable this, and utilize it where possible.

gcc/ChangeLog:
	* diagnostic-color.cc (should_colorize): Enable processing of VT100
	escape sequences on windows consoles

Signed-off-by: Peter Damianov <peter0x44@disroot.org>
---

diff --git a/gcc/diagnostic-color.cc b/gcc/diagnostic-color.cc
index cbe57ce763f..261a14b6c5f 100644
--- a/gcc/diagnostic-color.cc
+++ b/gcc/diagnostic-color.cc
@@ -300,12 +300,23 @@ should_colorize (void)
      pp_write_text_to_stream() in pretty-print.cc calls fputs() on
      that stream.  However, the code below for non-Windows doesn't seem
      to care about it either...  */
-  HANDLE h;
-  DWORD m;
+  HANDLE handle;
+  DWORD mode;
+  BOOL isconsole = false;
 
-  h = GetStdHandle (STD_ERROR_HANDLE);
-  return (h != INVALID_HANDLE_VALUE) && (h != NULL)
-	  && GetConsoleMode (h, &m);
+  handle = GetStdHandle (STD_ERROR_HANDLE);
+
+  if ((handle != INVALID_HANDLE_VALUE) && (handle != NULL))
+    isconsole = GetConsoleMode (handle, &mode);
+
+  if (isconsole)
+    {
+      /* Try to enable processing of VT100 escape sequences */
+      mode |= ENABLE_PROCESSED_OUTPUT | ENABLE_VIRTUAL_TERMINAL_PROCESSING;
+      SetConsoleMode (handle, mode);
+    }
+
+  return isconsole;
 #else
   char const *t = getenv ("TERM");
   /* emacs M-x shell sets TERM="dumb".  */