projects / thirdparty / kernel / stable-queue.git / blame
| Commit | Line | Data |
|---|---|---|
| dfbc6f9b GKH |
1 | From 2ce77f6d8a9ae9ce6d80397d88bdceb84a2004cd Mon Sep 17 00:00:00 2001 |
| 2 | From: Will Deacon <will.deacon@arm.com> | |
| 3 | Date: Tue, 13 Feb 2018 13:14:09 +0000 | |
| 4 | Subject: arm64: proc: Set PTE_NG for table entries to avoid traversing them twice | |
| 5 | ||
| 6 | From: Will Deacon <will.deacon@arm.com> | |
| 7 | ||
| 8 | commit 2ce77f6d8a9ae9ce6d80397d88bdceb84a2004cd upstream. | |
| 9 | ||
| 10 | When KASAN is enabled, the swapper page table contains many identical | |
| 11 | mappings of the zero page, which can lead to a stall during boot whilst | |
| 12 | the G -> nG code continually walks the same page table entries looking | |
| 13 | for global mappings. | |
| 14 | ||
| 15 | This patch sets the nG bit (bit 11, which is IGNORED) in table entries | |
| 16 | after processing the subtree so we can easily skip them if we see them | |
| 17 | a second time. | |
| 18 | ||
| 19 | Tested-by: Mark Rutland <mark.rutland@arm.com> | |
| 20 | Signed-off-by: Will Deacon <will.deacon@arm.com> | |
| 21 | Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> | |
| 22 | Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> | |
| 23 | ||
| 24 | --- | |
| 25 | arch/arm64/mm/proc.S | 14 +++++++++----- | |
| 26 | 1 file changed, 9 insertions(+), 5 deletions(-) | |
| 27 | ||
| 28 | --- a/arch/arm64/mm/proc.S | |
| 29 | +++ b/arch/arm64/mm/proc.S | |
| 30 | @@ -190,7 +190,8 @@ ENDPROC(idmap_cpu_replace_ttbr1) | |
| 31 | dc cvac, cur_\()\type\()p // Ensure any existing dirty | |
| 32 | dmb sy // lines are written back before | |
| 33 | ldr \type, [cur_\()\type\()p] // loading the entry | |
| 34 | - tbz \type, #0, next_\()\type // Skip invalid entries | |
| 35 | + tbz \type, #0, skip_\()\type // Skip invalid and | |
| 36 | + tbnz \type, #11, skip_\()\type // non-global entries | |
| 37 | .endm | |
| 38 | ||
| 39 | .macro __idmap_kpti_put_pgtable_ent_ng, type | |
| 40 | @@ -249,8 +250,9 @@ ENTRY(idmap_kpti_install_ng_mappings) | |
| 41 | add end_pgdp, cur_pgdp, #(PTRS_PER_PGD * 8) | |
| 42 | do_pgd: __idmap_kpti_get_pgtable_ent pgd | |
| 43 | tbnz pgd, #1, walk_puds | |
| 44 | - __idmap_kpti_put_pgtable_ent_ng pgd | |
| 45 | next_pgd: | |
| 46 | + __idmap_kpti_put_pgtable_ent_ng pgd | |
| 47 | +skip_pgd: | |
| 48 | add cur_pgdp, cur_pgdp, #8 | |
| 49 | cmp cur_pgdp, end_pgdp | |
| 50 | b.ne do_pgd | |
| 51 | @@ -278,8 +280,9 @@ walk_puds: | |
| 52 | add end_pudp, cur_pudp, #(PTRS_PER_PUD * 8) | |
| 53 | do_pud: __idmap_kpti_get_pgtable_ent pud | |
| 54 | tbnz pud, #1, walk_pmds | |
| 55 | - __idmap_kpti_put_pgtable_ent_ng pud | |
| 56 | next_pud: | |
| 57 | + __idmap_kpti_put_pgtable_ent_ng pud | |
| 58 | +skip_pud: | |
| 59 | add cur_pudp, cur_pudp, 8 | |
| 60 | cmp cur_pudp, end_pudp | |
| 61 | b.ne do_pud | |
| 62 | @@ -298,8 +301,9 @@ walk_pmds: | |
| 63 | add end_pmdp, cur_pmdp, #(PTRS_PER_PMD * 8) | |
| 64 | do_pmd: __idmap_kpti_get_pgtable_ent pmd | |
| 65 | tbnz pmd, #1, walk_ptes | |
| 66 | - __idmap_kpti_put_pgtable_ent_ng pmd | |
| 67 | next_pmd: | |
| 68 | + __idmap_kpti_put_pgtable_ent_ng pmd | |
| 69 | +skip_pmd: | |
| 70 | add cur_pmdp, cur_pmdp, #8 | |
| 71 | cmp cur_pmdp, end_pmdp | |
| 72 | b.ne do_pmd | |
| 73 | @@ -317,7 +321,7 @@ walk_ptes: | |
| 74 | add end_ptep, cur_ptep, #(PTRS_PER_PTE * 8) | |
| 75 | do_pte: __idmap_kpti_get_pgtable_ent pte | |
| 76 | __idmap_kpti_put_pgtable_ent_ng pte | |
| 77 | -next_pte: | |
| 78 | +skip_pte: | |
| 79 | add cur_ptep, cur_ptep, #8 | |
| 80 | cmp cur_ptep, end_ptep | |
| 81 | b.ne do_pte |