arm64: Fix 5-level paging support in kexec/hibernate trampoline

Add the missing code to allocate P4D level page tables when cloning the
the kernel page tables. This fixes a crash that may be observed when
attempting to resume from hibernation on an LPA2 capable system with 4k
pages, which therefore uses 5 levels of paging.

Presumably, kexec is equally affected.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20250110175145.785702-2-ardb+git@google.com
Signed-off-by: Will Deacon <will@kernel.org>

diff --git a/arch/arm64/mm/trans_pgd.c b/arch/arm64/mm/trans_pgd.c
index 19c67ed1a21fee0fad2f20c35dc03aaf9350a37e..18543b603c77bddbb1b2c84ca7acb01a0d3e0c8b 100644 (file)
--- a/arch/arm64/mm/trans_pgd.c
+++ b/arch/arm64/mm/trans_pgd.c
@@ -162,6 +162,13 @@ static int copy_p4d(struct trans_pgd_info *info, pgd_t *dst_pgdp,
        unsigned long next;
        unsigned long addr = start;
 
+       if (pgd_none(READ_ONCE(*dst_pgdp))) {
+               dst_p4dp = trans_alloc(info);
+               if (!dst_p4dp)
+                       return -ENOMEM;
+               pgd_populate(NULL, dst_pgdp, dst_p4dp);
+       }
+
        dst_p4dp = p4d_offset(dst_pgdp, start);
        src_p4dp = p4d_offset(src_pgdp, start);
        do {