leandrof/linux
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
0bdf525f04
linux/arch/x86/kernel
History
Alexander Duyck 0bdf525f04 x86: Improve __phys_addr performance by making use of carry flags and inlining 
This patch is meant to improve overall system performance when making use of
the __phys_addr call.  To do this I have implemented several changes.

First if CONFIG_DEBUG_VIRTUAL is not defined __phys_addr is made an inline,
similar to how this is currently handled in 32 bit.  However in order to do
this it is required to export phys_base so that it is available if __phys_addr
is used in kernel modules.

The second change was to streamline the code by making use of the carry flag
on an add operation instead of performing a compare on a 64 bit value.  The
advantage to this is that it allows us to significantly reduce the overall
size of the call.  On my Xeon E5 system the entire __phys_addr inline call
consumes a little less than 32 bytes and 5 instructions.  I also applied
similar logic to the debug version of the function.  My testing shows that the
debug version of the function with this patch applied is slightly faster than
the non-debug version without the patch.

Finally I also applied the same logic changes to __virt_addr_valid since it
used the same general code flow as __phys_addr and could achieve similar gains
though these changes.

Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
Link: http://lkml.kernel.org/r/20121116215315.8521.46270.stgit@ahduyck-cp1.jf.intel.com
Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
2012-11-16 16:42:08 -08:00
..
acpi Merge branch 'x86-smap-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2012-10-01 13:59:17 -07:00
apic x86: Move some contents of page_64_types.h into pgtable_64.h and page_64.h 2012-11-16 16:40:34 -08:00
cpu perf/x86: Remove unused variable in nhmex_rbox_alter_er() 2012-10-24 12:51:40 +02:00
.gitignore
alternative.c Merge branch 'x86-cleanups-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2012-10-01 10:47:45 -07:00
amd_gart_64.c X86 & IA64: adapt for dma_map_ops changes 2012-03-28 16:36:31 +02:00
amd_nb.c Merge branch 'x86-mce-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2012-07-22 16:07:45 -07:00
apb_timer.c Merge branch 'timers-clocksource-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2011-07-23 10:34:47 -07:00
aperture_64.c x86/gart: Fix kmemleak warning 2012-06-06 11:58:38 +02:00
apm_32.c x86/debug: Add KERN_<LEVEL> to bare printks, convert printks to pr_<level> 2012-06-06 09:17:22 +02:00
asm-offsets_32.c x86: Generate system call tables and unistd_*.h from tables 2011-11-17 13:35:37 -08:00
asm-offsets_64.c x32: If configured, add x32 system calls to system call tables 2012-02-20 12:52:06 -08:00
asm-offsets.c x86, um/x86: switch to generic sys_execve and kernel_execve 2012-09-30 22:53:32 -04:00
audit_64.c
bootflag.c
check.c x86: kernel/check.c simple_strtoul cleanup 2012-05-15 15:36:41 -07:00
cpuid.c Use get_online_cpus to avoid races involving CPU hotplug 2012-09-23 07:43:56 -07:00
crash_dump_32.c x86: remove the second argument of k[un]map_atomic() 2012-03-20 21:48:15 +08:00
crash_dump_64.c
crash.c x86, nmi: Wire up NMI handlers to new routines 2011-10-10 06:56:57 +02:00
devicetree.c x86: dt: Use linear irq domain for ioapic(s) 2012-08-21 22:16:57 +02:00
doublefault_32.c
dumpstack_32.c x86: Move call to print_modules() out of show_regs() 2012-06-20 14:33:48 +02:00
dumpstack_64.c x86: Move call to print_modules() out of show_regs() 2012-06-20 14:33:48 +02:00
dumpstack.c x86: Move call to print_modules() out of show_regs() 2012-06-20 14:33:48 +02:00
e820.c x86, mm: Trim memory in memblock to be page aligned 2012-10-24 11:52:21 -07:00
early_printk.c Revert "x86/early_printk: Replace obsolete simple_strtoul() usage with kstrtoint()" 2012-07-22 15:47:52 +02:00
early-quirks.c
entry_32.S Merge commit 'v3.7-rc1' into stable/for-linus-3.7 2012-10-19 15:19:19 -04:00
entry_64.S Merge commit 'v3.7-rc1' into stable/for-linus-3.7 2012-10-19 15:19:19 -04:00
ftrace.c ftrace/x86: Add save_regs for i386 function calls 2012-07-19 13:20:37 -04:00
head32.c x86, realmode: Move ACPI wakeup to unified realmode code 2012-05-08 11:46:05 -07:00
head64.c x86, realmode: Move ACPI wakeup to unified realmode code 2012-05-08 11:46:05 -07:00
head_32.S x86-32: Start out eflags and cr4 clean 2012-09-26 15:06:22 -07:00
head_64.S Merge branch 'x86-trampoline-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2012-05-29 20:14:53 -07:00
head.c memblock, x86: Replace memblock_x86_reserve/free_range() with generic ones 2011-07-14 11:47:53 -07:00
hpet.c x86: hpet: Fix copy-and-paste mistake in earlier change 2012-05-25 15:32:29 +02:00
hw_breakpoint.c
i386_ksyms_32.c
i387.c x86, kvm: fix kvm's usage of kernel_fpu_begin/end() 2012-09-21 16:59:04 -07:00
i8237.c
i8253.c x86: Use common i8253 clockevent 2011-07-01 10:37:14 +02:00
i8259.c x86/irq/i8259: Fix incorrect comment 2012-08-22 09:34:24 +02:00
io_delay.c
ioport.c
irq_32.c x86: Use common threadinfo allocator 2012-05-08 14:08:44 +02:00
irq_64.c x86: Add stack top margin for stack overflow checking 2011-12-07 09:27:11 +01:00
irq_work.c
irq.c Merge branch 'x86-mm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2012-10-01 11:13:33 -07:00
irqinit.c x86/tlb: replace INVALIDATE_TLB_VECTOR by CALL_FUNCTION_VECTOR 2012-06-27 19:29:13 -07:00
jump_label.c jump_label, x86: Fix section mismatch 2011-12-06 20:41:02 +01:00
kdebugfs.c arch/x86/kernel/kdebugfs.c: Ensure a consistent return value in error case 2012-07-26 15:07:20 +02:00
kgdb.c kgdb,x86: fix warning about unused variable 2012-10-12 06:37:34 -05:00
kprobes-common.h x86/kprobes: Split out optprobe related code to kprobes-opt.c 2012-03-06 09:49:49 +01:00
kprobes-opt.c x86/kprobes: Split out optprobe related code to kprobes-opt.c 2012-03-06 09:49:49 +01:00
kprobes.c kprobes/x86: Move skip_singlestep up 2012-09-20 14:48:16 +02:00
kvm.c KVM guest: exit idleness when handling KVM_PV_REASON_PAGE_NOT_PRESENT 2012-10-22 18:03:28 +02:00
kvmclock.c x86: kvmclock: remove check_and_clear_guest_paused warning 2012-06-11 23:18:33 -03:00
ldt.c Disintegrate asm/system.h for X86 2012-03-28 18:11:12 +01:00
machine_kexec_32.c Disintegrate asm/system.h for X86 2012-03-28 18:11:12 +01:00
machine_kexec_64.c
Makefile Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/signal 2012-10-10 12:02:25 +09:00
microcode_amd.c x86, microcode, AMD: Fix use after free in free_cache() 2012-09-19 18:06:25 +02:00
microcode_core.c Merge branch 'x86-microcode-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2012-10-01 11:15:17 -07:00
microcode_intel.c x86, microcode: Add a refresh firmware flag to ->request_microcode_fw 2012-08-22 16:15:58 -07:00
mmconf-fam10h_64.c
module.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial 2012-07-24 13:34:56 -07:00
mpparse.c Merge branch 'x86-trampoline-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2012-05-29 20:14:53 -07:00
msr.c Use get_online_cpus to avoid races involving CPU hotplug 2012-09-23 07:43:56 -07:00
nmi_selftest.c x86/nmi: Clean up register_nmi_handler() usage 2012-06-20 14:23:17 +02:00
nmi.c x86: Save cr2 in NMI in case NMIs take a page fault (for i386) 2012-06-08 18:51:12 -04:00
paravirt_patch_32.c
paravirt_patch_64.c
paravirt-spinlocks.c
paravirt.c x86, pvops: Remove hooks for {rd,wr}msr_safe_regs 2012-06-07 11:41:08 -07:00
pci-calgary_64.c x86/debug: Add KERN_<LEVEL> to bare printks, convert printks to pr_<level> 2012-06-06 09:17:22 +02:00
pci-dma.c iommu: Remove group_mf 2012-06-25 13:48:30 +02:00
pci-iommu_table.c
pci-nommu.c X86: integrate CMA with DMA-mapping subsystem 2012-05-21 15:09:38 +02:00
pci-swiotlb.c X86 & IA64: adapt for dma_map_ops changes 2012-03-28 16:36:31 +02:00
pcspeaker.c
perf_regs.c perf: Fix off by one test in perf_reg_value() 2012-09-19 17:08:40 +02:00
probe_roms.c x86/pci/probe_roms: Add missing __iomem annotation to pci_map_biosrom() 2012-09-05 10:52:25 +02:00
process_32.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/signal 2012-10-10 12:02:25 +09:00
process_64.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/signal 2012-10-10 12:02:25 +09:00
process.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/signal 2012-10-10 12:02:25 +09:00
ptrace.c Merge branch 'x86-fpu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2012-10-01 11:10:52 -07:00
pvclock.c
quirks.c x86/PCI: move fixup hooks from __init to __devinit 2012-06-12 09:10:54 -06:00
reboot_fixups_32.c
reboot.c x86/reboot: Remove quirk entry for SBC FITPC 2012-10-04 12:22:32 +02:00
relocate_kernel_32.S kexec, x86: Fix incorrect jump back address if not preserving context 2011-07-21 11:19:28 +02:00
relocate_kernel_64.S kexec, x86: Fix incorrect jump back address if not preserving context 2011-07-21 11:19:28 +02:00
resource.c
rtc.c sections: fix section conflicts in arch/x86 2012-10-06 03:04:40 +09:00
setup_percpu.c x86: Add read_mostly declaration/definition to variables from smp.h 2012-06-14 12:42:11 +02:00
setup.c EFI updates for 3.7 2012-10-26 10:17:38 +02:00
signal.c Merge branch 'uprobes/core' of git://git.kernel.org/pub/scm/linux/kernel/git/oleg/misc into perf/urgent 2012-10-21 18:18:17 +02:00
smp.c x86/reboot: Update nonmi_ipi parameter 2012-05-14 11:49:38 +02:00
smpboot.c x86/smp: Don't ever patch back to UP if we unplug cpus 2012-08-23 10:45:13 +02:00
stacktrace.c x86: Swap save_stack_trace_regs parameters 2011-06-14 22:48:51 -04:00
step.c uprobes/x86: Do not (ab)use TIF_SINGLESTEP/user_*_single_step() for single-stepping 2012-09-15 17:37:30 +02:00
sys_x86_64.c Merge branch 'x86-x32-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2012-03-29 18:12:23 -07:00
syscall_32.c x86, syscall: Re-fix typo in comment 2011-11-18 16:25:07 -08:00
syscall_64.c x32: If configured, add x32 system calls to system call tables 2012-02-20 12:52:06 -08:00
tboot.c x86, realmode: fixes compilation issue in tboot.c 2012-05-08 15:04:27 -07:00
tce_64.c Disintegrate asm/system.h for X86 2012-03-28 18:11:12 +01:00
test_nx.c
test_rodata.c x86, extable: Remove open-coded exception table entries in arch/x86/kernel/test_rodata.c 2012-04-20 13:51:38 -07:00
time.c MCA: delete all remaining traces of microchannel bus support. 2012-05-17 19:06:13 -04:00
tls.c Merge branch 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2012-03-29 14:28:26 -07:00
tls.h
topology.c x86: Fix files explicitly requiring export.h for EXPORT_SYMBOL/THIS_MODULE 2011-10-31 19:30:35 -04:00
traps.c Merge branch 'x86-fpu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2012-10-01 11:10:52 -07:00
tsc_sync.c x86/tsc: Reduce the TSC sync check time for core-siblings 2012-02-22 11:49:40 +01:00
tsc.c x86/debug: Add KERN_<LEVEL> to bare printks, convert printks to pr_<level> 2012-06-06 09:17:22 +02:00
uprobes.c uprobes/x86: Only rep+nop can be emulated correctly 2012-10-07 21:19:40 +02:00
verify_cpu.S
vm86_32.c x86: get rid of TIF_IRET hackery 2012-09-20 09:50:17 -04:00
vmlinux.lds.S x86, realmode: Move ACPI wakeup to unified realmode code 2012-05-08 11:46:05 -07:00
vsmp_64.c x86/apic/x2apic: Limit the vector reservation to the user specified mask 2012-07-06 11:00:22 +02:00
vsyscall_64.c time: Convert x86_64 to using new update_vsyscall 2012-09-24 12:38:09 -04:00
vsyscall_emu_64.S x86-64: Rework vsyscall emulation and add vsyscall= parameter 2011-08-10 19:26:46 -05:00
vsyscall_trace.h x86-64: Add vsyscall:emulate_vsyscall trace event 2011-08-04 16:13:53 -07:00
x86_init.c x86: xen: Cleanup and remove x86_init.paging.pagetable_setup_done() 2012-09-12 15:33:06 +02:00
x8664_ksyms_64.c x86: Improve __phys_addr performance by making use of carry flags and inlining 2012-11-16 16:42:08 -08:00
xsave.c x86, smap: Do not abuse the [f][x]rstor_checking() functions for user space 2012-09-25 15:42:18 -07:00