string: Add load_unaligned_zeropad() code path to sized_strscpy()

The call to read_word_at_a_time() in sized_strscpy() is problematic
with MTE because it may trigger a tag check fault when reading
across a tag granule (16 bytes) boundary. To make this code
MTE compatible, let's start using load_unaligned_zeropad()
on architectures where it is available (i.e. architectures that
define CONFIG_DCACHE_WORD_ACCESS). Because load_unaligned_zeropad()
takes care of page boundaries as well as tag granule boundaries,
also disable the code preventing crossing page boundaries when using
load_unaligned_zeropad().

Signed-off-by: Peter Collingbourne <pcc@google.com>
Link: https://linux-review.googlesource.com/id/If4b22e43b5a4ca49726b4bf98ada827fdf755548
Fixes: 94ab5b61ee16 ("kasan, arm64: enable CONFIG_KASAN_HW_TAGS")
Cc: stable@vger.kernel.org
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Link: https://lore.kernel.org/r/20250403000703.2584581-2-pcc@google.com
Signed-off-by: Kees Cook <kees@kernel.org>

diff --git a/lib/string.c b/lib/string.c
index eb4486ed40d259e43dfb63d76c2bd57ef74e2fd1..b632c71df1a5061256f556c40a30587cf45dce18 100644 (file)
--- a/lib/string.c
+++ b/lib/string.c
@@ -119,6 +119,7 @@ ssize_t sized_strscpy(char *dest, const char *src, size_t count)
        if (count == 0 || WARN_ON_ONCE(count > INT_MAX))
                return -E2BIG;
 
+#ifndef CONFIG_DCACHE_WORD_ACCESS
 #ifdef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
        /*
         * If src is unaligned, don't cross a page boundary,
@@ -133,12 +134,14 @@ ssize_t sized_strscpy(char *dest, const char *src, size_t count)
        /* If src or dest is unaligned, don't do word-at-a-time. */
        if (((long) dest | (long) src) & (sizeof(long) - 1))
                max = 0;
+#endif
 #endif
 
        /*
-        * read_word_at_a_time() below may read uninitialized bytes after the
-        * trailing zero and use them in comparisons. Disable this optimization
-        * under KMSAN to prevent false positive reports.
+        * load_unaligned_zeropad() or read_word_at_a_time() below may read
+        * uninitialized bytes after the trailing zero and use them in
+        * comparisons. Disable this optimization under KMSAN to prevent
+        * false positive reports.
         */
        if (IS_ENABLED(CONFIG_KMSAN))
                max = 0;
@@ -146,7 +149,11 @@ ssize_t sized_strscpy(char *dest, const char *src, size_t count)
        while (max >= sizeof(unsigned long)) {
                unsigned long c, data;
 
+#ifdef CONFIG_DCACHE_WORD_ACCESS
+               c = load_unaligned_zeropad(src+res);
+#else
                c = read_word_at_a_time(src+res);
+#endif
                if (has_zero(c, &data, &constants)) {
                        data = prep_zero_mask(c, data, &constants);
                        data = create_zero_mask(data);