1 ==============================
2 Memory Layout on AArch64 Linux
3 ==============================
5 Author: Catalin Marinas <catalin.marinas@arm.com>
7 This document describes the virtual memory layout used by the AArch64
8 Linux kernel. The architecture allows up to 4 levels of translation
9 tables with a 4KB page size and up to 3 levels with a 64KB page size.
11 AArch64 Linux uses either 3 levels or 4 levels of translation tables
12 with the 4KB page configuration, allowing 39-bit (512GB) or 48-bit
13 (256TB) virtual addresses, respectively, for both user and kernel. With
14 64KB pages, only 2 levels of translation tables, allowing 42-bit (4TB)
15 virtual address, are used but the memory layout is the same.
17 ARMv8.2 adds optional support for Large Virtual Address space. This is
18 only available when running with a 64KB page size and expands the
19 number of descriptors in the first level of translation.
21 User addresses have bits 63:48 set to 0 while the kernel addresses have
22 the same bits set to 1. TTBRx selection is given by bit 63 of the
23 virtual address. The swapper_pg_dir contains only kernel (global)
24 mappings while the user pgd contains only user (non-global) mappings.
25 The swapper_pg_dir address is written to TTBR1 and never written to
29 AArch64 Linux memory layout with 4KB pages + 4 levels (48-bit)::
32 -----------------------------------------------------------------------
33 0000000000000000 0000ffffffffffff 256TB user
34 ffff000000000000 ffff7fffffffffff 128TB kernel logical memory map
35 [ ffff600000000000 ffff7fffffffffff ] 32TB [ kasan shadow region ]
36 ffff800000000000 ffff800007ffffff 128MB bpf jit region
37 ffff800008000000 ffff80000fffffff 128MB modules
38 ffff800010000000 fffffdffbffeffff 125TB vmalloc
39 fffffdffbfff0000 fffffdfffe5f8fff ~998MB [guard region]
40 fffffdfffe5f9000 fffffdfffe9fffff 4124KB fixed mappings
41 fffffdfffea00000 fffffdfffebfffff 2MB [guard region]
42 fffffdfffec00000 fffffdffffbfffff 16MB PCI I/O space
43 fffffdffffc00000 fffffdffffdfffff 2MB [guard region]
44 fffffdffffe00000 ffffffffffdfffff 2TB vmemmap
45 ffffffffffe00000 ffffffffffffffff 2MB [guard region]
48 AArch64 Linux memory layout with 64KB pages + 3 levels (52-bit with HW support)::
51 -----------------------------------------------------------------------
52 0000000000000000 000fffffffffffff 4PB user
53 fff0000000000000 ffff7fffffffffff ~4PB kernel logical memory map
54 [ fffd800000000000 ffff7fffffffffff ] 512TB [ kasan shadow region ]
55 ffff800000000000 ffff800007ffffff 128MB bpf jit region
56 ffff800008000000 ffff80000fffffff 128MB modules
57 ffff800010000000 fffff81ffffeffff 120TB vmalloc
58 fffff81fffff0000 fffffc1ffe58ffff ~3TB [guard region]
59 fffffc1ffe590000 fffffc1ffe9fffff 4544KB fixed mappings
60 fffffc1ffea00000 fffffc1ffebfffff 2MB [guard region]
61 fffffc1ffec00000 fffffc1fffbfffff 16MB PCI I/O space
62 fffffc1fffc00000 fffffc1fffdfffff 2MB [guard region]
63 fffffc1fffe00000 ffffffffffdfffff 3968GB vmemmap
64 ffffffffffe00000 ffffffffffffffff 2MB [guard region]
67 Translation table lookup with 4KB pages::
69 +--------+--------+--------+--------+--------+--------+--------+--------+
70 |63 56|55 48|47 40|39 32|31 24|23 16|15 8|7 0|
71 +--------+--------+--------+--------+--------+--------+--------+--------+
74 | | | | | [11:0] in-page offset
75 | | | | +-> [20:12] L3 index
76 | | | +-----------> [29:21] L2 index
77 | | +---------------------> [38:30] L1 index
78 | +-------------------------------> [47:39] L0 index
79 +-------------------------------------------------> [63] TTBR0/1
82 Translation table lookup with 64KB pages::
84 +--------+--------+--------+--------+--------+--------+--------+--------+
85 |63 56|55 48|47 40|39 32|31 24|23 16|15 8|7 0|
86 +--------+--------+--------+--------+--------+--------+--------+--------+
89 | | | | [15:0] in-page offset
90 | | | +----------> [28:16] L3 index
91 | | +--------------------------> [41:29] L2 index
92 | +-------------------------------> [47:42] L1 index (48-bit)
93 | [51:42] L1 index (52-bit)
94 +-------------------------------------------------> [63] TTBR0/1
97 When using KVM without the Virtualization Host Extensions, the
98 hypervisor maps kernel pages in EL2 at a fixed (and potentially
99 random) offset from the linear mapping. See the kern_hyp_va macro and
100 kvm_update_va_mask function for more details. MMIO devices such as
101 GICv2 gets mapped next to the HYP idmap page, as do vectors when
102 ARM64_HARDEN_EL2_VECTORS is selected for particular CPUs.
104 When using KVM with the Virtualization Host Extensions, no additional
105 mappings are created, since the host kernel runs directly in EL2.
107 52-bit VA support in the kernel
108 -------------------------------
109 If the ARMv8.2-LVA optional feature is present, and we are running
110 with a 64KB page size; then it is possible to use 52-bits of address
111 space for both userspace and kernel addresses. However, any kernel
112 binary that supports 52-bit must also be able to fall back to 48-bit
113 at early boot time if the hardware feature is not present.
115 This fallback mechanism necessitates the kernel .text to be in the
116 higher addresses such that they are invariant to 48/52-bit VAs. Due
117 to the kasan shadow being a fraction of the entire kernel VA space,
118 the end of the kasan shadow must also be in the higher half of the
119 kernel VA space for both 48/52-bit. (Switching from 48-bit to 52-bit,
120 the end of the kasan shadow is invariant and dependent on ~0UL,
121 whilst the start address will "grow" towards the lower addresses).
123 In order to optimise phys_to_virt and virt_to_phys, the PAGE_OFFSET
124 is kept constant at 0xFFF0000000000000 (corresponding to 52-bit),
125 this obviates the need for an extra variable read. The physvirt
126 offset and vmemmap offsets are computed at early boot to enable
129 As a single binary will need to support both 48-bit and 52-bit VA
130 spaces, the VMEMMAP must be sized large enough for 52-bit VAs and
131 also must be sized large enough to accommodate a fixed PAGE_OFFSET.
133 Most code in the kernel should not need to consider the VA_BITS, for
134 code that does need to know the VA size the variables are
137 VA_BITS constant the *maximum* VA space size
139 VA_BITS_MIN constant the *minimum* VA space size
141 vabits_actual variable the *actual* VA space size
144 Maximum and minimum sizes can be useful to ensure that buffers are
145 sized large enough or that addresses are positioned close enough for
150 To maintain compatibility with software that relies on the ARMv8.0
151 VA space maximum size of 48-bits, the kernel will, by default,
152 return virtual addresses to userspace from a 48-bit range.
154 Software can "opt-in" to receiving VAs from a 52-bit space by
155 specifying an mmap hint parameter that is larger than 48-bit.
161 maybe_high_address = mmap(~0UL, size, prot, flags,...);
163 It is also possible to build a debug kernel that returns addresses
164 from a 52-bit space by enabling the following kernel config options:
168 CONFIG_EXPERT=y && CONFIG_ARM64_FORCE_52BIT=y
170 Note that this option is only intended for debugging applications
171 and should not be used in production.