Make pg_mkdir_p() tolerant of a concurrent directory creation.

pg_mkdir_p creates each missing path component with a stat() followed
by mkdir().  If the stat() reports the component as absent but another
process creates it in the window before this process's mkdir(), mkdir()
fails with EEXIST and pg_mkdir_p treated that as a hard error -- unlike
"mkdir -p", which is meant to be idempotent and race-tolerant.

This shows up when several processes concurrently create paths that
share an ancestor directory: for example, parallel initdb runs whose
data directories live under a common temporary directory.  One process
wins the race to create the shared ancestor and the others fail with
    could not create directory "...": File exists

Fix this race condition by first trying mkdir() and only attempting
stat() if it fails with EEXIST.

On Windows, there's an additional problem: stat() opens a file handle
and participates in share-mode locking, which means it can transiently
fail on a directory another process is concurrently creating.  Use
GetFileAttributes() instead: it requests only FILE_READ_ATTRIBUTES
and is exempt from share-mode denial, so it reliably sees a
concurrently-created directory.

I (tgl) also chose to back-patch 039f7ee0f's effects on this function,
so that pgmkdirp.c remains identical in all live branches.

Author: Andrew Dunstan <andrew@dunslane.net>
Co-authored-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/3ca004de-e49b-4471-b8aa-fd656e70f68c@dunslane.net
Backpatch-through: 14

diff --git a/src/port/pgmkdirp.c b/src/port/pgmkdirp.c
index 7d7cea4dd0ea18670d09ac911a3bd742d46d04bf..3e6b06fce79ebaa1cbb31b271e8b3ec4c4ad8c5e 100644 (file)
--- a/src/port/pgmkdirp.c
+++ b/src/port/pgmkdirp.c
@@ -56,7 +56,6 @@
 int
 pg_mkdir_p(char *path, int omode)
 {
-       struct stat sb;
        mode_t          numask,
                                oumask;
        int                     last,
@@ -119,24 +118,46 @@ pg_mkdir_p(char *path, int omode)
                if (last)
                        (void) umask(oumask);
 
-               /* check for pre-existing directory */
-               if (stat(path, &sb) == 0)
+               if (mkdir(path, last ? omode : S_IRWXU | S_IRWXG | S_IRWXO) < 0)
                {
-                       if (!S_ISDIR(sb.st_mode))
+                       /*
+                        * If we got EEXIST because there's already a directory there,
+                        * don't complain.
+                        */
+#ifndef WIN32
+                       int                     save_errno = errno;
+                       struct stat sb;
+
+                       if (save_errno != EEXIST ||
+                               stat(path, &sb) != 0 ||
+                               !S_ISDIR(sb.st_mode))
                        {
-                               if (last)
-                                       errno = EEXIST;
-                               else
-                                       errno = ENOTDIR;
+                               /* Don't let stat replace mkdir's errno */
+                               errno = save_errno;
                                retval = -1;
                                break;
                        }
+#else                                                  /* WIN32 */
+                       /*
+                        * On Windows, stat() opens a handle and can transiently fail on a
+                        * directory another process is concurrently creating.  Probe with
+                        * a path-based attribute query instead: it requests only
+                        * FILE_READ_ATTRIBUTES and is exempt from share-mode denial, so
+                        * it reliably sees a concurrently-created directory.  We assume
+                        * GetFileAttributes() won't change errno.
+                        */
+                       DWORD           attr = GetFileAttributes(path);
+
+                       if (errno != EEXIST ||
+                               attr == INVALID_FILE_ATTRIBUTES ||
+                               !(attr & FILE_ATTRIBUTE_DIRECTORY))
+                       {
+                               retval = -1;
+                               break;
+                       }
+#endif                                                 /* WIN32 */
                }
-               else if (mkdir(path, last ? omode : S_IRWXU | S_IRWXG | S_IRWXO) < 0)
-               {
-                       retval = -1;
-                       break;
-               }
+
                if (!last)
                        *p = '/';
        }