18a23eb805
commit 763802b53a427ed3cbd419dbba255c414fdd9e7c upstream.
Commit 3f8fd02b1bf1 ("mm/vmalloc: Sync unmappings in
__purge_vmap_area_lazy()") introduced a call to vmalloc_sync_all() in
the vunmap() code-path. While this change was necessary to maintain
correctness on x86-32-pae kernels, it also adds additional cycles for
architectures that don't need it.
Specifically on x86-64 with CONFIG_VMAP_STACK=y some people reported
severe performance regressions in micro-benchmarks because it now also
calls the x86-64 implementation of vmalloc_sync_all() on vunmap(). But
the vmalloc_sync_all() implementation on x86-64 is only needed for newly
created mappings.
To avoid the unnecessary work on x86-64 and to gain the performance
back, split up vmalloc_sync_all() into two functions:
* vmalloc_sync_mappings(), and
* vmalloc_sync_unmappings()
Most call-sites to vmalloc_sync_all() only care about new mappings being
synchronized. The only exception is the new call-site added in the
above mentioned commit.
Shile Zhang directed us to a report of an 80% regression in reaim
throughput.
Fixes: 3f8fd02b1bf1 ("mm/vmalloc: Sync unmappings in __purge_vmap_area_lazy()")
Reported-by: kernel test robot <oliver.sang@intel.com>
Reported-by: Shile Zhang <shile.zhang@linux.alibaba.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Tested-by: Borislav Petkov <bp@suse.de>
Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com> [GHES]
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: <stable@vger.kernel.org>
Link: http://lkml.kernel.org/r/20191009124418.8286-1-joro@8bytes.org
Link: https://lists.01.org/hyperkitty/list/lkp@lists.01.org/thread/4D3JPPHBNOSPFK2KEPC6KGKS6J25AIDB/
Link: http://lkml.kernel.org/r/20191113095530.228959-1-shile.zhang@linux.alibaba.com
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
|
||
|---|---|---|
| .. | ||
| acpica | ||
| apei | ||
| pmic | ||
| ac.c | ||
| acpi_apd.c | ||
| acpi_cmos_rtc.c | ||
| acpi_extlog.c | ||
| acpi_ipmi.c | ||
| acpi_lpat.c | ||
| acpi_lpss.c | ||
| acpi_memhotplug.c | ||
| acpi_pad.c | ||
| acpi_platform.c | ||
| acpi_pnp.c | ||
| acpi_processor.c | ||
| acpi_video.c | ||
| battery.c | ||
| battery.h | ||
| bgrt.c | ||
| blacklist.c | ||
| bus.c | ||
| button.c | ||
| cm_sbs.c | ||
| container.c | ||
| cppc_acpi.c | ||
| custom_method.c | ||
| debugfs.c | ||
| device_pm.c | ||
| device_sysfs.c | ||
| dock.c | ||
| ec_sys.c | ||
| ec.c | ||
| event.c | ||
| fan.c | ||
| glue.c | ||
| gsi.c | ||
| hed.c | ||
| int340x_thermal.c | ||
| internal.h | ||
| ioapic.c | ||
| Kconfig | ||
| Makefile | ||
| nfit.c | ||
| nfit.h | ||
| numa.c | ||
| nvs.c | ||
| osl.c | ||
| pci_irq.c | ||
| pci_link.c | ||
| pci_root.c | ||
| pci_slot.c | ||
| power.c | ||
| proc.c | ||
| processor_core.c | ||
| processor_driver.c | ||
| processor_idle.c | ||
| processor_pdc.c | ||
| processor_perflib.c | ||
| processor_thermal.c | ||
| processor_throttling.c | ||
| property.c | ||
| reboot.c | ||
| resource.c | ||
| sbs.c | ||
| sbshc.c | ||
| sbshc.h | ||
| scan.c | ||
| sleep.c | ||
| sleep.h | ||
| sysfs.c | ||
| tables.c | ||
| thermal.c | ||
| utils.c | ||
| video_detect.c | ||
| wakeup.c | ||