Patchwork [PATCHv2,1/7] x86/mm: concentrate the code to memblock allocator enabled

login
register
mail settings
Submitter Pingfan Liu
Date Jan. 11, 2019, 5:12 a.m.
Message ID <1547183577-20309-2-git-send-email-kernelfans@gmail.com>
Download mbox | patch
Permalink /patch/697417/
State New
Headers show

Comments

Pingfan Liu - Jan. 11, 2019, 5:12 a.m.
This patch identifies the point where memblock alloc start. It has no
functional.

Signed-off-by: Pingfan Liu <kernelfans@gmail.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: "Rafael J. Wysocki" <rjw@rjwysocki.net>
Cc: Len Brown <lenb@kernel.org>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Tejun Heo <tj@kernel.org>
Cc: Chao Fan <fanc.fnst@cn.fujitsu.com>
Cc: Baoquan He <bhe@redhat.com>
Cc: Juergen Gross <jgross@suse.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Mike Rapoport <rppt@linux.vnet.ibm.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Michal Hocko <mhocko@suse.com>
Cc: x86@kernel.org
Cc: linux-acpi@vger.kernel.org
Cc: linux-mm@kvack.org
---
 arch/x86/kernel/setup.c | 54 ++++++++++++++++++++++++-------------------------
 1 file changed, 26 insertions(+), 28 deletions(-)
Chao Fan - Jan. 11, 2019, 6:12 a.m.
On Fri, Jan 11, 2019 at 01:12:51PM +0800, Pingfan Liu wrote:
>This patch identifies the point where memblock alloc start. It has no
>functional.
[...]
>+#ifdef CONFIG_MEMORY_HOTPLUG
>+	/*
>+	 * Memory used by the kernel cannot be hot-removed because Linux
>+	 * cannot migrate the kernel pages. When memory hotplug is
>+	 * enabled, we should prevent memblock from allocating memory
>+	 * for the kernel.
>+	 *
>+	 * ACPI SRAT records all hotpluggable memory ranges. But before
>+	 * SRAT is parsed, we don't know about it.
>+	 *
>+	 * The kernel image is loaded into memory at very early time. We
>+	 * cannot prevent this anyway. So on NUMA system, we set any
>+	 * node the kernel resides in as un-hotpluggable.
>+	 *
>+	 * Since on modern servers, one node could have double-digit
>+	 * gigabytes memory, we can assume the memory around the kernel
>+	 * image is also un-hotpluggable. So before SRAT is parsed, just
>+	 * allocate memory near the kernel image to try the best to keep
>+	 * the kernel away from hotpluggable memory.
>+	 */
>+	if (movable_node_is_enabled())
>+		memblock_set_bottom_up(true);

Hi Pingfan,

In my understanding, 'movable_node' is based on the that memory near
kernel is considered as in the same node as kernel in high possibility.

If SRAT has been parsed early, do we still need the kernel parameter
'movable_node'? Since you have got the memory information about hot-remove,
so I wonder if it's OK to drop 'movable_node', and if memory-hotremove is
enabled, change memblock allocation according to SRAT.

If there is something wrong in my understanding, please let me know.

Thanks,
Chao Fan

>+#endif
> 	init_mem_mapping();
>+	memblock_set_current_limit(get_max_mapped());
> 
> 	idt_setup_early_pf();
> 
>@@ -1145,8 +1145,6 @@ void __init setup_arch(char **cmdline_p)
> 	 */
> 	mmu_cr4_features = __read_cr4() & ~X86_CR4_PCIDE;
> 
>-	memblock_set_current_limit(get_max_mapped());
>-
> 	/*
> 	 * NOTE: On x86-32, only from this point on, fixmaps are ready for use.
> 	 */
>-- 
>2.7.4
>
>
>
Pingfan Liu - Jan. 11, 2019, 10:06 a.m.
On Fri, Jan 11, 2019 at 2:13 PM Chao Fan <fanc.fnst@cn.fujitsu.com> wrote:
>
> On Fri, Jan 11, 2019 at 01:12:51PM +0800, Pingfan Liu wrote:
> >This patch identifies the point where memblock alloc start. It has no
> >functional.
> [...]
> >+#ifdef CONFIG_MEMORY_HOTPLUG
> >+      /*
> >+       * Memory used by the kernel cannot be hot-removed because Linux
> >+       * cannot migrate the kernel pages. When memory hotplug is
> >+       * enabled, we should prevent memblock from allocating memory
> >+       * for the kernel.
> >+       *
> >+       * ACPI SRAT records all hotpluggable memory ranges. But before
> >+       * SRAT is parsed, we don't know about it.
> >+       *
> >+       * The kernel image is loaded into memory at very early time. We
> >+       * cannot prevent this anyway. So on NUMA system, we set any
> >+       * node the kernel resides in as un-hotpluggable.
> >+       *
> >+       * Since on modern servers, one node could have double-digit
> >+       * gigabytes memory, we can assume the memory around the kernel
> >+       * image is also un-hotpluggable. So before SRAT is parsed, just
> >+       * allocate memory near the kernel image to try the best to keep
> >+       * the kernel away from hotpluggable memory.
> >+       */
> >+      if (movable_node_is_enabled())
> >+              memblock_set_bottom_up(true);
>
> Hi Pingfan,
>
> In my understanding, 'movable_node' is based on the that memory near
> kernel is considered as in the same node as kernel in high possibility.
>
> If SRAT has been parsed early, do we still need the kernel parameter
> 'movable_node'? Since you have got the memory information about hot-remove,
> so I wonder if it's OK to drop 'movable_node', and if memory-hotremove is
> enabled, change memblock allocation according to SRAT.
>
x86_32 still need this logic. Maybe it can be doable later.

Thanks,
Pingfan
> If there is something wrong in my understanding, please let me know.
>
> Thanks,
> Chao Fan
>
> >+#endif
> >       init_mem_mapping();
> >+      memblock_set_current_limit(get_max_mapped());
> >
> >       idt_setup_early_pf();
> >
> >@@ -1145,8 +1145,6 @@ void __init setup_arch(char **cmdline_p)
> >        */
> >       mmu_cr4_features = __read_cr4() & ~X86_CR4_PCIDE;
> >
> >-      memblock_set_current_limit(get_max_mapped());
> >-
> >       /*
> >        * NOTE: On x86-32, only from this point on, fixmaps are ready for use.
> >        */
> >--
> >2.7.4
> >
> >
> >
>
>
Pingfan Liu - Jan. 15, 2019, 7:06 a.m.
On Tue, Jan 15, 2019 at 7:07 AM Dave Hansen <dave.hansen@intel.com> wrote:
>
> On 1/10/19 9:12 PM, Pingfan Liu wrote:
> > This patch identifies the point where memblock alloc start. It has no
> > functional.
>
> It has no functional ... what?  Effects?
>
During re-organize the code, it takes me a long time to figure out why
memblock_set_bottom_up(true) is added here, and how far can it be
deferred. And finally, I realize that it only takes effect after
e820__memblock_setup(), the point where memblock allocator can work.
So I concentrate the related code, and hope this patch can classify
this truth.

> > -     memblock_set_current_limit(ISA_END_ADDRESS);
> > -     e820__memblock_setup();
> > -
> >       reserve_bios_regions();
> >
> >       if (efi_enabled(EFI_MEMMAP)) {
> > @@ -1113,6 +1087,8 @@ void __init setup_arch(char **cmdline_p)
> >               efi_reserve_boot_services();
> >       }
> >
> > +     memblock_set_current_limit(0, ISA_END_ADDRESS, false);
> > +     e820__memblock_setup();
>
> It looks like you changed the arguments passed to
> memblock_set_current_limit().  How can this even compile?  Did you mean
> that this patch is not functional?
>
Sorry that during rebasing, merge trivial fix by mistake. I will build
against each patch.

Best regards,
Pingfan

Patch

diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index d494b9b..ac432ae 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -962,29 +962,6 @@  void __init setup_arch(char **cmdline_p)
 
 	if (efi_enabled(EFI_BOOT))
 		efi_memblock_x86_reserve_range();
-#ifdef CONFIG_MEMORY_HOTPLUG
-	/*
-	 * Memory used by the kernel cannot be hot-removed because Linux
-	 * cannot migrate the kernel pages. When memory hotplug is
-	 * enabled, we should prevent memblock from allocating memory
-	 * for the kernel.
-	 *
-	 * ACPI SRAT records all hotpluggable memory ranges. But before
-	 * SRAT is parsed, we don't know about it.
-	 *
-	 * The kernel image is loaded into memory at very early time. We
-	 * cannot prevent this anyway. So on NUMA system, we set any
-	 * node the kernel resides in as un-hotpluggable.
-	 *
-	 * Since on modern servers, one node could have double-digit
-	 * gigabytes memory, we can assume the memory around the kernel
-	 * image is also un-hotpluggable. So before SRAT is parsed, just
-	 * allocate memory near the kernel image to try the best to keep
-	 * the kernel away from hotpluggable memory.
-	 */
-	if (movable_node_is_enabled())
-		memblock_set_bottom_up(true);
-#endif
 
 	x86_report_nx();
 
@@ -1096,9 +1073,6 @@  void __init setup_arch(char **cmdline_p)
 
 	cleanup_highmap();
 
-	memblock_set_current_limit(ISA_END_ADDRESS);
-	e820__memblock_setup();
-
 	reserve_bios_regions();
 
 	if (efi_enabled(EFI_MEMMAP)) {
@@ -1113,6 +1087,8 @@  void __init setup_arch(char **cmdline_p)
 		efi_reserve_boot_services();
 	}
 
+	memblock_set_current_limit(0, ISA_END_ADDRESS, false);
+	e820__memblock_setup();
 	/* preallocate 4k for mptable mpc */
 	e820__memblock_alloc_reserved_mpc_new();
 
@@ -1130,7 +1106,31 @@  void __init setup_arch(char **cmdline_p)
 	trim_platform_memory_ranges();
 	trim_low_memory_range();
 
+#ifdef CONFIG_MEMORY_HOTPLUG
+	/*
+	 * Memory used by the kernel cannot be hot-removed because Linux
+	 * cannot migrate the kernel pages. When memory hotplug is
+	 * enabled, we should prevent memblock from allocating memory
+	 * for the kernel.
+	 *
+	 * ACPI SRAT records all hotpluggable memory ranges. But before
+	 * SRAT is parsed, we don't know about it.
+	 *
+	 * The kernel image is loaded into memory at very early time. We
+	 * cannot prevent this anyway. So on NUMA system, we set any
+	 * node the kernel resides in as un-hotpluggable.
+	 *
+	 * Since on modern servers, one node could have double-digit
+	 * gigabytes memory, we can assume the memory around the kernel
+	 * image is also un-hotpluggable. So before SRAT is parsed, just
+	 * allocate memory near the kernel image to try the best to keep
+	 * the kernel away from hotpluggable memory.
+	 */
+	if (movable_node_is_enabled())
+		memblock_set_bottom_up(true);
+#endif
 	init_mem_mapping();
+	memblock_set_current_limit(get_max_mapped());
 
 	idt_setup_early_pf();
 
@@ -1145,8 +1145,6 @@  void __init setup_arch(char **cmdline_p)
 	 */
 	mmu_cr4_features = __read_cr4() & ~X86_CR4_PCIDE;
 
-	memblock_set_current_limit(get_max_mapped());
-
 	/*
 	 * NOTE: On x86-32, only from this point on, fixmaps are ready for use.
 	 */