1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
|
From 3c4444a6a3599660bcc24c9df5b2eb7fd36914e7 Mon Sep 17 00:00:00 2001
From: Catalin Marinas <catalin.marinas@arm.com>
Date: Thu, 23 Feb 2012 17:16:41 +0000
Subject: [PATCH 12/12] ARM: LPAE: Invalidate the TLB for module addresses
during translation fault
commit cf9b5a26876adb266aa1ba8a0b08669389a2e301 in repository:
git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git
During the free_pgtables() call all user and modules/pkmap entries are
removed. If a translation fault for the modules/pkmap area occurs before
we switched away from the current pgd, do_translation_fault() would copy
the init_mm pud into the user pud.
There is a small window between pud_clear() and pmd_free_tlb() in
free_pmd_range() where the pud entry was cleared but the TLB has not
been invalidated yet and the CPU may have cached the original (valid)
pud entry in the TLB. A scenario like below would get stuck in
continuous prefetch abort:
1. Current process exiting. The modules pmd entries not populated
2. exit_mmap() -> ... -> pmd_free_tlb()
3. pud_clear() for the 1GB pud containing user stack and modules (no TLB
invalidation yet)
4. Interrupt -> module interrupt routine
5. Level 2 (pmd) translation fault occurs when executing the module
interrupt routine. The CPU previously cached (TLB) the old valid pud
value for the modules area, so we don't get a level 1 translation
fault
6. do_translation fault() copies the pud_k into the pud
7. Linux returns to the faulty instruction. Goes back to 5
At point 7, since the CPU still has the old pud value, it goes back to
point 5 and never gets out of this loop. With this patch, the stale pud
TLB entry is invalidated after point 6 and the subsequent prefetch abort
doesn't occur.
Reported-by: Tony Thompson <Anthony.Thompson@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
---
arch/arm/mm/fault.c | 13 +++++++++++--
1 files changed, 11 insertions(+), 2 deletions(-)
diff --git a/arch/arm/mm/fault.c b/arch/arm/mm/fault.c
index 5bb4835..2c07452 100644
--- a/arch/arm/mm/fault.c
+++ b/arch/arm/mm/fault.c
@@ -448,8 +448,16 @@ do_translation_fault(unsigned long addr, unsigned int fsr,
if (pud_none(*pud_k))
goto bad_area;
- if (!pud_present(*pud))
+ if (!pud_present(*pud)) {
set_pud(pud, *pud_k);
+ /*
+ * There is a small window during free_pgtables() where the
+ * user *pud entry is 0 but the TLB has not been invalidated
+ * and we get a level 2 (pmd) translation fault caused by the
+ * intermediate TLB caching of the old level 1 (pud) entry.
+ */
+ flush_tlb_kernel_page(addr);
+ }
pmd = pmd_offset(pud, addr);
pmd_k = pmd_offset(pud_k, addr);
@@ -472,8 +480,9 @@ do_translation_fault(unsigned long addr, unsigned int fsr,
#endif
if (pmd_none(pmd_k[index]))
goto bad_area;
+ if (!pmd_present(pmd[index]))
+ copy_pmd(pmd, pmd_k);
- copy_pmd(pmd, pmd_k);
return 0;
bad_area:
--
1.7.9.1
|
