According to latest memory ordering specification documents from Intel
and AMD, both manufacturers are committed to in-order loads from
cacheable memory for the x86 architecture. Hence, smp_rmb() may be a
simple barrier.
Also according to those documents, and according to existing practice in
Linux (eg. spin_unlock doesn't enforce ordering), stores to cacheable
memory are visible in program order too. Special string stores are safe
-- their constituent stores may be out of order, but they must complete
in order WRT surrounding stores. Nontemporal stores to WB memory can go
out of order, and so they should be fenced explicitly to make them
appear in-order WRT other stores. Hence, smp_wmb() may be a simple
barrier.
http://developer.intel.com/products/processor/manuals/318147.pdfhttp://www.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/24593.pdf
In userspace microbenchmarks on a core2 system, fence instructions range
anywhere from around 15 cycles to 50, which may not be totally
insignificant in performance critical paths (code size will go down
too).
However the primary motivation for this is to have the canonical barrier
implementation for x86 architecture.
smp_rmb on buggy pentium pros remains a locked op, which is apparently
required.
Signed-off-by: Nick Piggin <npiggin@suse.de>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
wmb() on x86 must always include a barrier, because stores can go out of
order in many cases when dealing with devices (eg. WC memory).
Signed-off-by: Nick Piggin <npiggin@suse.de>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* master.kernel.org:/pub/scm/linux/kernel/git/gregkh/pci-2.6: (37 commits)
PCI: merge almost all of pci_32.h and pci_64.h together
PCI: X86: Introduce and enable PCI domain support
PCI: Add 'nodomains' boot option, and pci_domains_supported global
PCI: modify PCI bridge control ISA flag for clarity
PCI: use _CRS for PCI resource allocation
PCI: avoid P2P prefetch window for expansion ROMs
PCI: skip ISA ioresource alignment on some systems
PCI: remove transparent bridge sizing
pci: write file size to inode on proc bus file write
pci: use size stored in proc_dir_entry for proc bus files
pci: implement "pci=noaer"
PCI: fix IDE legacy mode resources
MSI: Use correct data offset for 32-bit MSI in read_msi_msg()
PCI: Fix incorrect argument order to list_add_tail() in PCI dynamic ID code
PCI: i386: Compaq EVO N800c needs PCI bus renumbering
PCI: Remove no longer correct documentation regarding MSI vector assignment
PCI: re-enable onboard sound on "MSI K8T Neo2-FIR"
PCI: quirk_vt82c586_acpi: Omit reading PCI revision ID
PCI: quirk amd_8131_mmrbc: Omit reading pci revision ID
cpqphp: Use PCI_CLASS_REVISION instead of PCI_REVISION_ID for read
...
* fix bug in pci_read() and pci_write() which prevented PCI domain
support from working (hardcoded domain 0).
* unconditionally enable CONFIG_PCI_DOMAINS
* implement pci_domain_nr() and pci_proc_domain(), as required of
all arches when CONFIG_PCI_DOMAINS is enabled.
* store domain in struct pci_sysdata, as assigned by ACPI
* support "pci=nodomains"
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>
Cc: Andi Kleen <ak@suse.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
force_enable hpet for ICH5.
[ Build fixes from Andrew Morton ]
Signed-off-by: Venkatesh Pallipadi <venkatesh.pallipadi@intel.com>
Cc: Andi Kleen <ak@suse.de>
Cc: john stultz <johnstul@us.ibm.com>
Cc: Greg KH <greg@kroah.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Arjan van de Ven <arjan@linux.intel.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Enable HPET later during boot, after the force detect in PCI quirks. Also add
a call to repeat the force enabling at resume time.
Signed-off-by: Venkatesh Pallipadi <venkatesh.pallipadi@intel.com>
Cc: Andi Kleen <ak@suse.de>
Cc: john stultz <johnstul@us.ibm.com>
Cc: Greg KH <greg@kroah.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Arjan van de Ven <arjan@linux.intel.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Force detect and/or enable HPET on ICH chipsets. This patch just handles the
detection part and following patches use this information. Adds a function to
repeat the force enabling during resume time.
Using HPET this way, instead of PIT increases the time CPUs can reside in
C-state when system is totally idle. On my test system with Core 2 Duo,
average C-state residency goes up from ~20mS to ~80mS.
[ Build fixed from Andrew Morton ]
Signed-off-by: Venkatesh Pallipadi <venkatesh.pallipadi@intel.com>
Cc: Andi Kleen <ak@suse.de>
Cc: john stultz <johnstul@us.ibm.com>
Cc: Greg KH <greg@kroah.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Arjan van de Ven <arjan@linux.intel.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Remove hpet_readl/writel from vsyscall.h, where it does not belong
anyway. Use the hpet code itself.
Signed-off-by: Chris Wright <chrisw@sous-sol.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Arjan van de Ven <arjan@linux.intel.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Chris Wright <chrisw@sous-sol.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Arjan van de Ven <arjan@linux.intel.com>
Remove the unused code after the switch to clock events.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Chris Wright <chrisw@sous-sol.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Arjan van de Ven <arjan@linux.intel.com>
Combine the timex.h variants and move the TSC related code into tsc.h.
Move the set_cyc2ns_scale() call into the tsc calibraction code, where
it belongs.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Arjan van de Ven <arjan@linux.intel.com>
Useless header file with 32 bit and 64 bit variants. Move the
single useful line to the place where it is used.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Arjan van de Ven <arjan@linux.intel.com>
AMDs C1E enabled CPUs stop the local apic timer, when both cores are
idle. This is a hardware feature which breaks highres/dynticks.
Add the same quirk as we have for 32 bit already.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Arjan van de Ven <arjan@linux.intel.com>
The clock events merge introduced a change to the nmi watchdog code to
handle the not longer increasing local apic timer count in the
broadcast mode. This is fine for UP, but on SMP it pampers over a
stuck CPU which is not handling the broadcast interrupt due to the
unconditional sum up of local apic timer count and irq0 count.
To cover all cases we need to keep track on which CPU irq0 is
handled. In theory this is CPU#0 due to the explicit disabling of irq
balancing for irq0, but there are systems which ignore this on the
hardware level. The per cpu irq0 accounting allows us to remove the
irq0 to CPU0 binding as well.
Add a per cpu counter for irq0 and evaluate this instead of the global
irq0 count in the nmi watchdog code.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Arjan van de Ven <arjan@linux.intel.com>
Finally switch to the clockevents code. Share code with i386 for
hpet and PIT.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Chris Wright <chrisw@sous-sol.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Arjan van de Ven <arjan@linux.intel.com>
[ tglx: arch/x86 adaptation ]
Signed-off-by: Yinghai Lu <yhlu.kernel@gmail.com>
Signed-off-by: Andi Kleen <ak@suse.de>
Cc: Christoph Lameter <clameter@sgi.com>
Cc: Len Brown <lenb@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Arjan van de Ven <arjan@linux.intel.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
PIT clock events work already and the PIT handling is the same for
i386 and x86_64. x86_64 does not support PIT as a clock source, so
disable the PIT clocksource for x86_64.
Use the i386 i8253.h include file for x86_64 as well to share the
exports and the PIT constants.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Chris Wright <chrisw@sous-sol.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Arjan van de Ven <arjan@linux.intel.com>
PIT clock events work already and the PIT handling is the same for
i386 and x86_64. x86_64 does not support PIT as a clock source, so
disable the PIT clocksource for x86_64.
Prepare i8253.h to be shared with x8664
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Chris Wright <chrisw@sous-sol.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Arjan van de Ven <arjan@linux.intel.com>
Move the TSC calibration code to tsc.c. Reimplement it so the
pm timer can be used as a reference as well.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Chris Wright <chrisw@sous-sol.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Arjan van de Ven <arjan@linux.intel.com>
This adds support for Multi-Function General Purpose Timers. It detects the
available timers during southbridge init, and provides an API for allocating
and setting the timers. They're higher resolution than the standard PIT, so
the MFGPTs come in handy for quite a few things.
Note that we never clobber the timers that the BIOS might have opted to use;
we just check for unused timers.
Signed-off-by: Jordan Crouse <jordan.crouse@amd.com>
Signed-off-by: Andres Salomon <dilinger@debian.org>
Cc: Andi Kleen <ak@suse.de>
Cc: Alan Cox <alan@lxorguk.ukuu.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Arjan van de Ven <arjan@linux.intel.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
The clock events merge introduced a change to the nmi watchdog code to
handle the not longer increasing local apic timer count in the
broadcast mode. This is fine for UP, but on SMP it pampers over a
stuck CPU which is not handling the broadcast interrupt due to the
unconditional sum up of local apic timer count and irq0 count.
To cover all cases we need to keep track on which CPU irq0 is
handled. In theory this is CPU#0 due to the explicit disabling of irq
balancing for irq0, but there are systems which ignore this on the
hardware level. The per cpu irq0 accounting allows us to remove the
irq0 to CPU0 binding as well.
Add a per cpu counter for irq0 and evaluate this instead of the global
irq0 count in the nmi watchdog code.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Arjan van de Ven <arjan@linux.intel.com>
Move the headers to include/asm-x86 and fixup the
header install make rules
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
