mm/userfaultfd: fix uninitialized output field for -EAGAIN race

While discussing some userfaultfd relevant issues recently, Andrea noticed
a potential ABI breakage with -EAGAIN on almost all userfaultfd ioctl()s.

Quote from Andrea, explaining how -EAGAIN was processed, and how this
should fix it (taking example of UFFDIO_COPY ioctl):

  The "mmap_changing" and "stale pmd" conditions are already reported as
  -EAGAIN written in the copy field, this does not change it. This change
  removes the subnormal case that left copy.copy uninitialized and required
  apps to explicitly set the copy field to get deterministic
  behavior (which is a requirement contrary to the documentation in both
  the manpage and source code). In turn there's no alteration to backwards
  compatibility as result of this change because userland will find the
  copy field consistently set to -EAGAIN, and not anymore sometime -EAGAIN
  and sometime uninitialized.

  Even then the change only can make a difference to non cooperative users
  of userfaultfd, so when UFFD_FEATURE_EVENT_* is enabled, which is not
  true for the vast majority of apps using userfaultfd or this unintended
  uninitialized field may have been noticed sooner.

Meanwhile, since this bug existed for years, it also almost affects all
ioctl()s that was introduced later.  Besides UFFDIO_ZEROPAGE, these also
get affected in the same way:

  - UFFDIO_CONTINUE
  - UFFDIO_POISON
  - UFFDIO_MOVE

This patch should have fixed all of them.

Link: https://lkml.kernel.org/r/20250424215729.194656-2-peterx@redhat.com
Fixes: df2cc96e7701 ("userfaultfd: prevent non-cooperative events vs mcopy_atomic races")
Fixes: f619147104c8 ("userfaultfd: add UFFDIO_CONTINUE ioctl")
Fixes: fc71884a5f59 ("mm: userfaultfd: add new UFFDIO_POISON ioctl")
Fixes: adef440691ba ("userfaultfd: UFFDIO_MOVE uABI")
Signed-off-by: Peter Xu <peterx@redhat.com>
Reported-by: Andrea Arcangeli <aarcange@redhat.com>
Suggested-by: Andrea Arcangeli <aarcange@redhat.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c
index d80f943461992f3e6235052fb015507eb52f31dc..22f4bf956ba1c4e622a8576489d31dce7673fa95 100644 (file)
--- a/fs/userfaultfd.c
+++ b/fs/userfaultfd.c
@@ -1585,8 +1585,11 @@ static int userfaultfd_copy(struct userfaultfd_ctx *ctx,
        user_uffdio_copy = (struct uffdio_copy __user *) arg;
 
        ret = -EAGAIN;
-       if (atomic_read(&ctx->mmap_changing))
+       if (unlikely(atomic_read(&ctx->mmap_changing))) {
+               if (unlikely(put_user(ret, &user_uffdio_copy->copy)))
+                       return -EFAULT;
                goto out;
+       }
 
        ret = -EFAULT;
        if (copy_from_user(&uffdio_copy, user_uffdio_copy,
@@ -1641,8 +1644,11 @@ static int userfaultfd_zeropage(struct userfaultfd_ctx *ctx,
        user_uffdio_zeropage = (struct uffdio_zeropage __user *) arg;
 
        ret = -EAGAIN;
-       if (atomic_read(&ctx->mmap_changing))
+       if (unlikely(atomic_read(&ctx->mmap_changing))) {
+               if (unlikely(put_user(ret, &user_uffdio_zeropage->zeropage)))
+                       return -EFAULT;
                goto out;
+       }
 
        ret = -EFAULT;
        if (copy_from_user(&uffdio_zeropage, user_uffdio_zeropage,
@@ -1744,8 +1750,11 @@ static int userfaultfd_continue(struct userfaultfd_ctx *ctx, unsigned long arg)
        user_uffdio_continue = (struct uffdio_continue __user *)arg;
 
        ret = -EAGAIN;
-       if (atomic_read(&ctx->mmap_changing))
+       if (unlikely(atomic_read(&ctx->mmap_changing))) {
+               if (unlikely(put_user(ret, &user_uffdio_continue->mapped)))
+                       return -EFAULT;
                goto out;
+       }
 
        ret = -EFAULT;
        if (copy_from_user(&uffdio_continue, user_uffdio_continue,
@@ -1801,8 +1810,11 @@ static inline int userfaultfd_poison(struct userfaultfd_ctx *ctx, unsigned long
        user_uffdio_poison = (struct uffdio_poison __user *)arg;
 
        ret = -EAGAIN;
-       if (atomic_read(&ctx->mmap_changing))
+       if (unlikely(atomic_read(&ctx->mmap_changing))) {
+               if (unlikely(put_user(ret, &user_uffdio_poison->updated)))
+                       return -EFAULT;
                goto out;
+       }
 
        ret = -EFAULT;
        if (copy_from_user(&uffdio_poison, user_uffdio_poison,
@@ -1870,8 +1882,12 @@ static int userfaultfd_move(struct userfaultfd_ctx *ctx,
 
        user_uffdio_move = (struct uffdio_move __user *) arg;
 
-       if (atomic_read(&ctx->mmap_changing))
-               return -EAGAIN;
+       ret = -EAGAIN;
+       if (unlikely(atomic_read(&ctx->mmap_changing))) {
+               if (unlikely(put_user(ret, &user_uffdio_move->move)))
+                       return -EFAULT;
+               goto out;
+       }
 
        if (copy_from_user(&uffdio_move, user_uffdio_move,
                           /* don't copy "move" last field */