Patch from Nicolas Pitre
This patch provides a new implementation for optimized memory copy
functions on ARM. It is made of two levels: a template that consists of
the core copy code and separate files that define macros to be used with
the core code depending on the type of copy needed. This allows for best
performances while sharing the same core for implementing memcpy(),
copy_from_user() and copy_to_user() for instance.
Two reasons for this work:
1) the current copy_to_user/copy_from_user implementation assumes no
task switch will ever occur in the middle of each copied page making
it completely unsafe with CONFIG_PREEMPT=y.
2) current copy implementations are measurably suboptimal and optimizing
different implementations separately is a pain and more opportunities
for bugs.
The reason for (1) is the fact that copy inside user pages are performed
with the ldm instruction which has no mean for testing user protections
and could possibly race with process preemption bypassing the COW mechanism
for example. This is a longstanding issue that we said ought to be fixed
for about two years now. The solution is to substitute those ldm insns
with a series of ldrt or strt insns to enforce user memory protection.
At least on StrongARM and XScale cores the ldm is not faster than the
equivalent ldr/str insns with a warm i-cache so there is no measurable
performance degradation with that change. The fact that the copy code is
a template makes it pretty easy to reuse the same core code as for memcpy
and benefit from the same performance optimizations.
Now (2) is best demonstrated with actual throughput measurements.
First, here is a summary of memcopy tests performed on a StrongARM core:
PTR alignment buffer size kernel version this version
------------------------------------------------------------
aligned 32 59.73 107.43
unaligned 32 61.31 74.72
aligned 100 132.47 136.15
unaligned 100 103.84 123.76
aligned 4096 130.67 130.80
unaligned 4096 130.68 130.64
aligned 1048576 68.03 68.18
unaligned 1048576 68.03 68.18
The buffer size is in bytes and the measured speed in MB/s. The copy
was performed repeatedly with given buffer and throughput averaged over
3 seconds.
Here we can see that the current kernel version has a higher entry cost
that shows up with small buffers. As buffer size grows both implementation
converge to the same throughput.
Now here's the exact same test performed on an XScale core (PXA255):
PTR alignment buffer size kernel version this version
------------------------------------------------------------
aligned 32 46.99 77.58
unaligned 32 53.61 59.59
aligned 100 107.19 136.59
unaligned 100 83.61 97.58
aligned 4096 129.13 129.98
unaligned 4096 128.36 128.53
aligned 1048576 53.76 59.41
unaligned 1048576 33.67 56.96
Again we can see the entry setup cost being higher for the current kernel
before getting to the main copy loop. Then throughput results converge
as long as the buffer remains in the cache. Then the 1MB case shows more
differences probably due to better pld placement and/or less instruction
interlocks in this proposed implementation.
Disclaimer: The PXA system was running with slower clocks than the
StrongARM system so trying to infer any conclusion by comparing those
separate sets of results side by side would be completely inappropriate.
So... What this patch does is to replace both memcpy and memmove with
an implementation based on the provided copy code template. The memmove
code is kept separate since it is used only if the memory areas involved
do overlap in which case the code is a transposition of the template but
with the copy occurring in the opposite direction (trying to fit that
mode into the template turned it into a mess not worth it for memmove
alone). And obviously both memcpy and memmove were tested with all kinds
of pointer alignments and buffer sizes to exercise all code paths for
correctness.
The next patch will provide the now trivial replacement implementation
copy_to_user and copy_from_user.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Nicolas Pitre
Required for future enhancement patches.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from David Brownell
Lubbock updates:
* Provide an address for the SMC91x chip that doesn't generate
a boot-time warning (matching the EEPROM).
* Update MMC support to (a) detect card insert/remove, and
(b) report the readonly switch setting for SD cards.
Previously, MMC/SD cards had to be present at boot time else they
couldn't be detected.
Signed-off-by: David Brownell <dbrownell@users.sourceforge.net>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Ben Dooks
Platform data for the LCD/framebuffer driver for
the RX3715 LCD panel.
Signed-off-by: Ben Dooks <ben-linux@fluff.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Lennert Buytenhek
Switch the users of ixp2000_reg_write that depend on writes being
flushed out of the write buffer by the time that function returns
over to ixp2000_reg_wrb.
When using XCB=101, writes to the same functional unit are still
guaranteed to complete in order, so we only need to protect against:
- reordering of writes to different functional units
- masking an interrupt and then reenabling the IRQ bit in CPSR
Signed-off-by: Lennert Buytenhek <buytenh@wantstofly.org>
Signed-off-by: Deepak Saxena <dsaxena@plexity.net>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Lennert Buytenhek
The enp2611 version of the ixp2000 netdev driver needs to be able to
access a number of on-board peripherals. ioremap() is not suitable
for this, as that will cause XCB=000 mappings to be done, which will
make the cpu susceptible to crashing on ixp2400 erratum #66. Properly
aligned iotable mappings with MT_IXP2000_DEVICE will cause section
mappings with XCB=101 to be done, which is safe.
Signed-off-by: Lennert Buytenhek <buytenh@wantstofly.org>
Signed-off-by: Deepak Saxena <dsaxena@plexity.net>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
I recently picked up my older work to remove unnecessary #includes of
sched.h, starting from a patch by Dave Jones to not include sched.h
from module.h. This reduces the number of indirect includes of sched.h
by ~300. Another ~400 pointless direct includes can be removed after
this disentangling (patch to follow later).
However, quite a few indirect includes need to be fixed up for this.
In order to feed the patches through -mm with as little disturbance as
possible, I've split out the fixes I accumulated up to now (complete for
i386 and x86_64, more archs to follow later) and post them before the real
patch. This way this large part of the patch is kept simple with only
adding #includes, and all hunks are independent of each other. So if any
hunk rejects or gets in the way of other patches, just drop it. My scripts
will pick it up again in the next round.
Signed-off-by: Tim Schmielau <tim@physik3.uni-rostock.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Define jiffies_64 in kernel/timer.c rather than having 24 duplicated
defines in each architecture.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Make sure we always return, as all syscalls should. Also move the common
prototype to <linux/syscalls.h>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Miklos Szeredi <miklos@szeredi.hu>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
IXDP2[48]00 have only 1 UART on the board.
Signed-off-by: Deepak Saxena <dsaxena@plexity.net>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Nicolas Pitre
This patch gets rid of the last C implementations of needed libgcc
functions for the kernel, replacing them with optimized assembly
versions.
Those functions are:
__ashldi3
__ashrdi3
__lshrdi3
__muldi3
__ucmpdi2
The first 3 were lifted from gcc, the other two were written from scratch.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Encapsulate pool data into dmabounce_pool. Only account successful
allocations. Use dma_mapping_error().
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
We know what pgprot we're going to use, so don't #define it. Also,
since we select the nonaliasing/aliasing copypage implementation at
run time, there's no point having it globally visible.
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Richard Purdie
Add spitz irda platform support
Signed-off-by: Richard Purdie <rpurdie@rpsys.net>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Richard Purdie
Add corgi irda platform support
Signed-off-by: Richard Purdie <rpurdie@rpsys.net>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Richard Purdie
Add poodle irda platform support
Signed-off-by: Richard Purdie <rpurdie@rpsys.net>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Christoph Lameter demonstrated very poor scalability on the SGI 512-way, with
a many-threaded application which concurrently initializes different parts of
a large anonymous area.
This patch corrects that, by using a separate spinlock per page table page, to
guard the page table entries in that page, instead of using the mm's single
page_table_lock. (But even then, page_table_lock is still used to guard page
table allocation, and anon_vma allocation.)
In this implementation, the spinlock is tucked inside the struct page of the
page table page: with a BUILD_BUG_ON in case it overflows - which it would in
the case of 32-bit PA-RISC with spinlock debugging enabled.
Splitting the lock is not quite for free: another cacheline access. Ideally,
I suppose we would use split ptlock only for multi-threaded processes on
multi-cpu machines; but deciding that dynamically would have its own costs.
So for now enable it by config, at some number of cpus - since the Kconfig
language doesn't support inequalities, let preprocessor compare that with
NR_CPUS. But I don't think it's worth being user-configurable: for good
testing of both split and unsplit configs, split now at 4 cpus, and perhaps
change that to 8 later.
There is a benefit even for singly threaded processes: kswapd can be attacking
one part of the mm while another part is busy faulting.
Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Prepare arm for the split page_table_lock: three issues.
Signal handling's preserve and restore of iwmmxt context currently involves
reading and writing that context to and from user space, while holding
page_table_lock to secure the user page(s) against kswapd. If we split the
lock, then the structure might span two pages, secured by to read into and
write from a kernel stack buffer, copying that out and in without locking (the
structure is 160 bytes in size, and here we're near the top of the kernel
stack). Or would the overhead be noticeable?
arm_syscall's cmpxchg emulation use pte_offset_map_lock, instead of
pte_offset_map and mm-wide page_table_lock; and strictly, it should now also
take mmap_sem before descending to pmd, to guard against another thread
munmapping, and the page table pulled out beneath this thread.
Updated two comments in fault-armv.c. adjust_pte is interesting, since its
modification of a pte in one part of the mm depends on the lock held when
calling update_mmu_cache for a pte in some other part of that mm. This can't
be done with a split page_table_lock (and we've already taken the lowest lock
in the hierarchy here): so we'll have to disable split on arm, unless
CONFIG_CPU_CACHE_VIPT to ensures adjust_pte never used.
Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
check_user_page_readable is a problematic variant of follow_page. It's used
only by oprofile's i386 and arm backtrace code, at interrupt time, to
establish whether a userspace stackframe is currently readable.
This is problematic, because we want to push the page_table_lock down inside
follow_page, and later split it; whereas oprofile is doing a spin_trylock on
it (in the i386 case, forgotten in the arm case), and needs that to pin
perhaps two pages spanned by the stackframe (which might be covered by
different locks when we split).
I think oprofile is going about this in the wrong way: it doesn't need to know
the area is readable (neither i386 nor arm uses read protection of user
pages), it doesn't need to pin the memory, it should simply
__copy_from_user_inatomic, and see if that succeeds or not. Sorry, but I've
not got around to devising the sparse __user annotations for this.
Then we can eliminate check_user_page_readable, and return to a single
follow_page without the __follow_page variants.
Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Convert those few architectures which are calling pud_alloc, pmd_alloc,
pte_alloc_map on a user mm, not to take the page_table_lock first, nor drop it
after. Each of these can continue to use pte_alloc_map, no need to change
over to pte_alloc_map_lock, they're neither racy nor swappable.
In the sparc64 io_remap_pfn_range, flush_tlb_range then falls outside of the
page_table_lock: that's okay, on sparc64 it's like flush_tlb_mm, and that has
always been called from outside of page_table_lock in dup_mmap.
Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
First step in pushing down the page_table_lock. init_mm.page_table_lock has
been used throughout the architectures (usually for ioremap): not to serialize
kernel address space allocation (that's usually vmlist_lock), but because
pud_alloc,pmd_alloc,pte_alloc_kernel expect caller holds it.
Reverse that: don't lock or unlock init_mm.page_table_lock in any of the
architectures; instead rely on pud_alloc,pmd_alloc,pte_alloc_kernel to take
and drop it when allocating a new one, to check lest a racing task already
did. Similarly no page_table_lock in vmalloc's map_vm_area.
Some temporary ugliness in __pud_alloc and __pmd_alloc: since they also handle
user mms, which are converted only by a later patch, for now they have to lock
differently according to whether or not it's init_mm.
If sources get muddled, there's a danger that an arch source taking
init_mm.page_table_lock will be mixed with common source also taking it (or
neither take it). So break the rules and make another change, which should
break the build for such a mismatch: remove the redundant mm arg from
pte_alloc_kernel (ppc64 scrapped its distinct ioremap_mm in 2.6.13).
Exceptions: arm26 used pte_alloc_kernel on user mm, now pte_alloc_map; ia64
used pte_alloc_map on init_mm, now pte_alloc_kernel; parisc had bad args to
pmd_alloc and pte_alloc_kernel in unused USE_HPPA_IOREMAP code; ppc64
map_io_page forgot to unlock on failure; ppc mmu_mapin_ram and ppc64 im_free
took page_table_lock for no good reason.
Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Patch from Nicolas Pitre
Since vmlinux.lds.S is preprocessed, we can use the defines already
present in asm/memory.h (allowed by patch #3060) for the XIP kernel link
address instead of relying on a duplicated Makefile hardcoded value, and
also get rid of its dependency on awk to handle it at the same time.
While at it let's clean XIP stuff even further and make things clearer
in head.S with a nice code reduction.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Nicolas Pitre
This patch allows for assorted type of cleanups by letting assembly code
use the same set of defines for constant values and avoid duplicated
definitions that might not always be in sync, or that might simply be
confusing due to the different names for the same thing.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Nicolas Pitre
Fix XIP support after recent bootmem code refactoring.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
This got dropped from the SA1100 flash driver a while back and
never added to the platform support file. Add it back.
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
In PM v1, all devices were called at SUSPEND_DISABLE level. Then
all devices were called at SUSPEND_SAVE_STATE level, and finally
SUSPEND_POWER_DOWN level. However, with PM v2, to maintain
compatibility for platform devices, I arranged for the PM v2
suspend/resume callbacks to call the old PM v1 suspend/resume
callbacks three times with each level in order so that existing
drivers continued to work.
Since this is obsolete infrastructure which is no longer necessary,
we can remove it. Here's an (untested) patch to do exactly that.
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Patch from Bellido Nicolas
The AAED-2000 is equiped with an 640x480 LCD.
This adds the parameters that will be passed to the AAEC-2000 platform code.
Signed-off-by: Nicolas Bellido <ml@acolin.be>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Bellido Nicolas
The AAEC-2000 has an ARM PrimeCell PL110 Color LCD Controller.
This patch contains the platform glue that will be used by specific boards.
Signed-off-by: Nicolas Bellido <ml@acolin.be>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Bellido Nicolas
Here is a preliminary clock interface support for the AAEC-2000.
Signed-off-by: Nicolas Bellido <ml@acolin.be>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Bellido Nicolas
This adds platform code for MTD devices on AAEC-2000.
Signed-off-by: Nicolas Bellido <ml@acolin.be>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Bellido Nicolas
The AAED-2000 board has GPIO pins on an external port.
This patch adds the defines, and do the necessary mapping.
Signed-off-by: Nicolas Bellido <ml@acolin.be>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Nicolas Pitre
This is the PXA2xx common IRDA driver, plus platform support
for Lubbock and Mainstone.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Acked-by: Jean Tourrilhes <jt@hpl.hp.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Nicolas Pitre
From: Daniel Jacobowitz <dan@debian.org>
> I also fixed a bug that confused me greatly while trying to debug: one
> SIGILL has long been a SIGSEGV because of some broken RISCOS
> compatibility code.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Todd Poynor
Add symbols for PXA2xx PWRMODE register M field that selects low-power
mode, replace unadorned constants. Honor power mode parameter of
pxa_cpu_suspend(mode), no longer force to 3 (sleep). Full Deep Sleep
low-power mode support for PXA27x is pending generic PM interfaces to
select more than 2 suspend-to-RAM-style power modes, but this is
expected soon. This can be hardcoded in the meantime by replacing the
pxa_cpu_suspend() parameter value. From David Burrage and Todd Poynor.
Try #2 removes one of the register copies and moves the code to save the
pxa_cpu_suspend parameter to immediately surround the call that requires
the parameter value be preserved.
Signed-off-by: Todd Poynor <tpoynor@mvista.com>
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Jon Ringle
Updated 2898/1 per comments:
- Removed fixup
- Moved code in mach-mp1000/ to mach-clps711x/
- Cleaned up code in mp1000-seprom.c. Eliminated code that displayed the contents of the eeprom
Please comment.
Signed-off-by: Jon Ringle
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Ian Campbell
As noted by Uli Luckas in the comments of 3025 there is a typo in the i2s platform device. The i2s platform device refers to the i2c resources.
Signed-off-by: Ian Campbell <icampbell@arcom.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Ben Dooks
Add platform data for framebuffer for the
onboard LCD module
Signed-off-by: Ben Dooks <ben-linux@fluff.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
