mm/memory_hotplug: introduce MEM_PREPARE_ONLINE/MEM_FINISH_OFFLINE notifiers

Patch series "implement "memmap on memory" feature on s390".

This series provides "memmap on memory" support on s390 platform.  "memmap
on memory" allows struct pages array to be allocated from the hotplugged
memory range instead of allocating it from main system memory.

s390 currently preallocates struct pages array for all potentially
possible memory, which ensures memory onlining always succeeds, but with
the cost of significant memory consumption from the available system
memory during boottime.  In certain extreme configuration, this could lead
to ipl failure.

"memmap on memory" ensures struct pages array are populated from self
contained hotplugged memory range instead of depleting the available
system memory and this could eliminate ipl failure on s390 platform.

On other platforms, system might go OOM when the physically hotplugged
memory depletes the available memory before it is onlined.  Hence, "memmap
on memory" feature was introduced as described in commit a08a2ae34613
("mm,memory_hotplug: allocate memmap from the added memory range").

Unlike other architectures, s390 memory blocks are not physically
accessible until it is online.  To make it physically accessible two new
memory notifiers MEM_PREPARE_ONLINE / MEM_FINISH_OFFLINE are added and
this notifier lets the hypervisor inform that the memory should be made
physically accessible.  This allows for "memmap on memory" initialization
during memory hotplug onlining phase, which is performed before calling
MEM_GOING_ONLINE notifier.

Patch 1 introduces MEM_PREPARE_ONLINE/MEM_FINISH_OFFLINE memory notifiers
to prepare the transition of memory to and from a physically accessible
state.  New mhp_flag MHP_OFFLINE_INACCESSIBLE is introduced to ensure
altmap cannot be written when adding memory - before it is set online.
This enhancement is crucial for implementing the "memmap on memory"
feature for s390 in a subsequent patch.

Patches 2 allocates vmemmap pages from self-contained memory range for
s390.  It allocates memory map (struct pages array) from the hotplugged
memory range, rather than using system memory by passing altmap to vmemmap
functions.

Patch 3 removes unhandled memory notifier types on s390.

Patch 4 implements MEM_PREPARE_ONLINE/MEM_FINISH_OFFLINE memory notifiers
on s390.  MEM_PREPARE_ONLINE memory notifier makes memory block physical
accessible via sclp assign command.  The notifier ensures self-contained
memory maps are accessible and hence enabling the "memmap on memory" on
s390.  MEM_FINISH_OFFLINE memory notifier shifts the memory block to an
inaccessible state via sclp unassign command.

Patch 5 finally enables MHP_MEMMAP_ON_MEMORY on s390.

This patch (of 5):

Introduce MEM_PREPARE_ONLINE/MEM_FINISH_OFFLINE memory notifiers to
prepare the transition of memory to and from a physically accessible
state.  This enhancement is crucial for implementing the "memmap on
memory" feature for s390 in a subsequent patch.

Platforms such as x86 can support physical memory hotplug via ACPI.  When
there is physical memory hotplug, ACPI event leads to the memory addition
with the following callchain:

acpi_memory_device_add()
  -> acpi_memory_enable_device()
     -> __add_memory()

After this, the hotplugged memory is physically accessible, and altmap
support prepared, before the "memmap on memory" initialization in
memory_block_online() is called.

On s390, memory hotplug works in a different way.  The available hotplug
memory has to be defined upfront in the hypervisor, but it is made
physically accessible only when the user sets it online via sysfs,
currently in the MEM_GOING_ONLINE notifier.  This is too late and "memmap
on memory" initialization is performed before calling MEM_GOING_ONLINE
notifier.

During the memory hotplug addition phase, altmap support is prepared and
during the memory onlining phase s390 requires memory to be physically
accessible and then subsequently initiate the "memmap on memory"
initialization process.

The memory provider will handle new MEM_PREPARE_ONLINE /
MEM_FINISH_OFFLINE notifications and make the memory accessible.

The mhp_flag MHP_OFFLINE_INACCESSIBLE is introduced and is relevant when
used along with MHP_MEMMAP_ON_MEMORY, because the altmap cannot be written
(e.g., poisoned) when adding memory -- before it is set online.  This
allows for adding memory with an altmap that is not currently made
available by a hypervisor.  When onlining that memory, the hypervisor can
be instructed to make that memory accessible via the new notifiers and the
onlining phase will not require any memory allocations, which is helpful
in low-memory situations.

All architectures ignore unknown memory notifiers.  Therefore, the
introduction of these new notifiers does not result in any functional
modifications across architectures.

Link: https://lkml.kernel.org/r/20240108132747.3238763-1-sumanthk@linux.ibm.com
Link: https://lkml.kernel.org/r/20240108132747.3238763-2-sumanthk@linux.ibm.com
Signed-off-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Suggested-by: Gerald Schaefer <gerald.schaefer@linux.ibm.com>
Suggested-by: David Hildenbrand <david@redhat.com>
Acked-by: David Hildenbrand <david@redhat.com>
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
Cc: Anshuman Khandual <anshuman.khandual@arm.com>
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

diff --git a/drivers/base/memory.c b/drivers/base/memory.c
index 14f964a..c0436f4 100644 (file)
--- a/drivers/base/memory.c
+++ b/drivers/base/memory.c
@@ -188,6 +188,7 @@ static int memory_block_online(struct memory_block *mem)
        unsigned long start_pfn = section_nr_to_pfn(mem->start_section_nr);
        unsigned long nr_pages = PAGES_PER_SECTION * sections_per_block;
        unsigned long nr_vmemmap_pages = 0;
+       struct memory_notify arg;
        struct zone *zone;
        int ret;
 
@@ -207,9 +208,19 @@ static int memory_block_online(struct memory_block *mem)
        if (mem->altmap)
                nr_vmemmap_pages = mem->altmap->free;
 
+       arg.altmap_start_pfn = start_pfn;
+       arg.altmap_nr_pages = nr_vmemmap_pages;
+       arg.start_pfn = start_pfn + nr_vmemmap_pages;
+       arg.nr_pages = nr_pages - nr_vmemmap_pages;
        mem_hotplug_begin();
+       ret = memory_notify(MEM_PREPARE_ONLINE, &arg);
+       ret = notifier_to_errno(ret);
+       if (ret)
+               goto out_notifier;
+
        if (nr_vmemmap_pages) {
-               ret = mhp_init_memmap_on_memory(start_pfn, nr_vmemmap_pages, zone);
+               ret = mhp_init_memmap_on_memory(start_pfn, nr_vmemmap_pages,
+                                               zone, mem->altmap->inaccessible);
                if (ret)
                        goto out;
        }
@@ -231,7 +242,11 @@ static int memory_block_online(struct memory_block *mem)
                                          nr_vmemmap_pages);
 
        mem->zone = zone;
+       mem_hotplug_done();
+       return ret;
 out:
+       memory_notify(MEM_FINISH_OFFLINE, &arg);
+out_notifier:
        mem_hotplug_done();
        return ret;
 }
@@ -244,6 +259,7 @@ static int memory_block_offline(struct memory_block *mem)
        unsigned long start_pfn = section_nr_to_pfn(mem->start_section_nr);
        unsigned long nr_pages = PAGES_PER_SECTION * sections_per_block;
        unsigned long nr_vmemmap_pages = 0;
+       struct memory_notify arg;
        int ret;
 
        if (!mem->zone)
@@ -275,6 +291,11 @@ static int memory_block_offline(struct memory_block *mem)
                mhp_deinit_memmap_on_memory(start_pfn, nr_vmemmap_pages);
 
        mem->zone = NULL;
+       arg.altmap_start_pfn = start_pfn;
+       arg.altmap_nr_pages = nr_vmemmap_pages;
+       arg.start_pfn = start_pfn + nr_vmemmap_pages;
+       arg.nr_pages = nr_pages - nr_vmemmap_pages;
+       memory_notify(MEM_FINISH_OFFLINE, &arg);
 out:
        mem_hotplug_done();
        return ret;

diff --git a/include/linux/memory.h b/include/linux/memory.h
index f53cfda..939a16b 100644 (file)
--- a/include/linux/memory.h
+++ b/include/linux/memory.h
@@ -96,8 +96,17 @@ int set_memory_block_size_order(unsigned int order);
 #define        MEM_GOING_ONLINE        (1<<3)
 #define        MEM_CANCEL_ONLINE       (1<<4)
 #define        MEM_CANCEL_OFFLINE      (1<<5)
+#define        MEM_PREPARE_ONLINE      (1<<6)
+#define        MEM_FINISH_OFFLINE      (1<<7)
 
 struct memory_notify {
+       /*
+        * The altmap_start_pfn and altmap_nr_pages fields are designated for
+        * specifying the altmap range and are exclusively intended for use in
+        * MEM_PREPARE_ONLINE/MEM_FINISH_OFFLINE notifiers.
+        */
+       unsigned long altmap_start_pfn;
+       unsigned long altmap_nr_pages;
        unsigned long start_pfn;
        unsigned long nr_pages;
        int status_change_nid_normal;

diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h
index 7d20765..ee00015 100644 (file)
--- a/include/linux/memory_hotplug.h
+++ b/include/linux/memory_hotplug.h
@@ -106,6 +106,22 @@ typedef int __bitwise mhp_t;
  * implies the node id (nid).
  */
 #define MHP_NID_IS_MGID                ((__force mhp_t)BIT(2))
+/*
+ * The hotplugged memory is completely inaccessible while the memory is
+ * offline. The memory provider will handle MEM_PREPARE_ONLINE /
+ * MEM_FINISH_OFFLINE notifications and make the memory accessible.
+ *
+ * This flag is only relevant when used along with MHP_MEMMAP_ON_MEMORY,
+ * because the altmap cannot be written (e.g., poisoned) when adding
+ * memory -- before it is set online.
+ *
+ * This allows for adding memory with an altmap that is not currently
+ * made available by a hypervisor. When onlining that memory, the
+ * hypervisor can be instructed to make that memory available, and
+ * the onlining phase will not require any memory allocations, which is
+ * helpful in low-memory situations.
+ */
+#define MHP_OFFLINE_INACCESSIBLE       ((__force mhp_t)BIT(3))
 
 /*
  * Extended parameters for memory hotplug:
@@ -154,7 +170,7 @@ extern void adjust_present_page_count(struct page *page,
                                      long nr_pages);
 /* VM interface that may be used by firmware interface */
 extern int mhp_init_memmap_on_memory(unsigned long pfn, unsigned long nr_pages,
-                                    struct zone *zone);
+                                    struct zone *zone, bool mhp_off_inaccessible);
 extern void mhp_deinit_memmap_on_memory(unsigned long pfn, unsigned long nr_pages);
 extern int online_pages(unsigned long pfn, unsigned long nr_pages,
                        struct zone *zone, struct memory_group *group);

diff --git a/include/linux/memremap.h b/include/linux/memremap.h
index 744c830..9837f3e 100644 (file)
--- a/include/linux/memremap.h
+++ b/include/linux/memremap.h
@@ -25,6 +25,7 @@ struct vmem_altmap {
        unsigned long free;
        unsigned long align;
        unsigned long alloc;
+       bool inaccessible;
 };
 
 /*

diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index 2189099..707027f 100644 (file)
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -1087,7 +1087,7 @@ void adjust_present_page_count(struct page *page, struct memory_group *group,
 }
 
 int mhp_init_memmap_on_memory(unsigned long pfn, unsigned long nr_pages,
-                             struct zone *zone)
+                             struct zone *zone, bool mhp_off_inaccessible)
 {
        unsigned long end_pfn = pfn + nr_pages;
        int ret, i;
@@ -1096,6 +1096,15 @@ int mhp_init_memmap_on_memory(unsigned long pfn, unsigned long nr_pages,
        if (ret)
                return ret;
 
+       /*
+        * Memory block is accessible at this stage and hence poison the struct
+        * pages now.  If the memory block is accessible during memory hotplug
+        * addition phase, then page poisining is already performed in
+        * sparse_add_section().
+        */
+       if (mhp_off_inaccessible)
+               page_init_poison(pfn_to_page(pfn), sizeof(struct page) * nr_pages);
+
        move_pfn_range_to_zone(zone, pfn, nr_pages, NULL, MIGRATE_UNMOVABLE);
 
        for (i = 0; i < nr_pages; i++)
@@ -1415,7 +1424,7 @@ static void __ref remove_memory_blocks_and_altmaps(u64 start, u64 size)
 }
 
 static int create_altmaps_and_memory_blocks(int nid, struct memory_group *group,
-                                           u64 start, u64 size)
+                                           u64 start, u64 size, mhp_t mhp_flags)
 {
        unsigned long memblock_size = memory_block_size_bytes();
        u64 cur_start;
@@ -1431,6 +1440,8 @@ static int create_altmaps_and_memory_blocks(int nid, struct memory_group *group,
                };
 
                mhp_altmap.free = memory_block_memmap_on_memory_pages();
+               if (mhp_flags & MHP_OFFLINE_INACCESSIBLE)
+                       mhp_altmap.inaccessible = true;
                params.altmap = kmemdup(&mhp_altmap, sizeof(struct vmem_altmap),
                                        GFP_KERNEL);
                if (!params.altmap) {
@@ -1516,7 +1527,7 @@ int __ref add_memory_resource(int nid, struct resource *res, mhp_t mhp_flags)
         */
        if ((mhp_flags & MHP_MEMMAP_ON_MEMORY) &&
            mhp_supports_memmap_on_memory(memory_block_size_bytes())) {
-               ret = create_altmaps_and_memory_blocks(nid, group, start, size);
+               ret = create_altmaps_and_memory_blocks(nid, group, start, size, mhp_flags);
                if (ret)
                        goto error;
        } else {

diff --git a/mm/sparse.c b/mm/sparse.c
index 338cf94..aed0951 100644 (file)
--- a/mm/sparse.c
+++ b/mm/sparse.c
@@ -908,7 +908,8 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,
         * Poison uninitialized struct pages in order to catch invalid flags
         * combinations.
         */
-       page_init_poison(memmap, sizeof(struct page) * nr_pages);
+       if (!altmap || !altmap->inaccessible)
+               page_init_poison(memmap, sizeof(struct page) * nr_pages);
 
        ms = __nr_to_section(section_nr);
        set_section_nid(section_nr, nid);