mingw: intercept isatty() to handle /dev/null as Git expects it

When Git's source code calls isatty(), it really asks whether the
respective file descriptor is connected to an interactive terminal.

Windows' _isatty() function, however, determines whether the file
descriptor is associated with a character device. And NUL, Windows'
equivalent of /dev/null, is a character device.

Which means that for years, Git mistakenly detected an associated
interactive terminal when being run through the test suite, which
almost always redirects stdin, stdout and stderr to /dev/null.

This bug only became obvious, and painfully so, when the new
bisect--helper entered the `pu` branch and made the automatic build & test
time out because t6030 was waiting for an answer.

For details, see

	https://msdn.microsoft.com/en-us/library/f4s0ddew.aspx

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

diff --git a/compat/mingw.h b/compat/mingw.h
index 034fff9479d03d2a2e3c7017a4fe4131461f0ec6..33501695550accdb08368aa867b9addb25c90d5f 100644 (file)
--- a/compat/mingw.h
+++ b/compat/mingw.h
@@ -384,6 +384,9 @@ int mingw_raise(int sig);
  * ANSI emulation wrappers
  */
 
+int winansi_isatty(int fd);
+#define isatty winansi_isatty
+
 void winansi_init(void);
 HANDLE winansi_get_osfhandle(int fd);
 

diff --git a/compat/winansi.c b/compat/winansi.c
index db4a5b0a37d687218ab085f22b57eb35e28893b8..cb725fb02f6e4f48c4f314c3c6c5e8535aa6eda8 100644 (file)
--- a/compat/winansi.c
+++ b/compat/winansi.c
@@ -7,6 +7,9 @@
 #include <wingdi.h>
 #include <winreg.h>
 
+/* In this file, we actually want to use Windows' own isatty(). */
+#undef isatty
+
 /*
  ANSI codes used by git: m, K
 
@@ -570,6 +573,36 @@ static void detect_msys_tty(int fd)
 
 #endif
 
+int winansi_isatty(int fd)
+{
+       int res = isatty(fd);
+
+       if (res) {
+               /*
+                * Make sure that /dev/null is not fooling Git into believing
+                * that we are connected to a terminal, as "_isatty() returns a
+                * nonzero value if the descriptor is associated with a
+                * character device."; for more information, see
+                *
+                * https://msdn.microsoft.com/en-us/library/f4s0ddew.aspx
+                */
+               HANDLE handle = (HANDLE)_get_osfhandle(fd);
+               if (fd == STDIN_FILENO) {
+                       DWORD dummy;
+
+                       if (!GetConsoleMode(handle, &dummy))
+                               res = 0;
+               } else if (fd == STDOUT_FILENO || fd == STDERR_FILENO) {
+                       CONSOLE_SCREEN_BUFFER_INFO dummy;
+
+                       if (!GetConsoleScreenBufferInfo(handle, &dummy))
+                               res = 0;
+               }
+       }
+
+       return res;
+}
+
 void winansi_init(void)
 {
        int con1, con2;