Notable changes: - A large series to rewrite our SLB miss handling, replacing a lot of fairly complicated asm with much fewer lines of C. - Following on from that, we now maintain a cache of SLB entries for each process and preload them on context switch. Leading to a 27% speedup for our context switch benchmark on Power9. - Improvements to our handling of SLB multi-hit errors. We now print more debug information when they occur, and try to continue running by flushing the SLB and reloading, rather than treating them as fatal. - Enable THP migration on 64-bit Book3S machines (eg. Power7/8/9). - Add support for physical memory up to 2PB in the linear mapping on 64-bit Book3S. We only support up to 512TB as regular system memory, otherwise the percpu allocator runs out of vmalloc space. - Add stack protector support for 32 and 64-bit, with a per-task canary. - Add support for PTRACE_SYSEMU and PTRACE_SYSEMU_SINGLESTEP. - Support recognising "big cores" on Power9, where two SMT4 cores are presented to us as a single SMT8 core. - A large series to cleanup some of our ioremap handling and PTE flags. - Add a driver for the PAPR SCM (storage class memory) interface, allowing guests to operate on SCM devices (acked by Dan). - Changes to our ftrace code to handle very large kernels, where we need to use a trampoline to get to ftrace_caller(). Many other smaller enhancements and cleanups. Thanks to: Alan Modra, Alistair Popple, Aneesh Kumar K.V, Anton Blanchard, Aravinda Prasad, Bartlomiej Zolnierkiewicz, Benjamin Herrenschmidt, Breno Leitao, Cédric Le Goater, Christophe Leroy, Christophe Lombard, Dan Carpenter, Daniel Axtens, Finn Thain, Gautham R. Shenoy, Gustavo Romero, Haren Myneni, Hari Bathini, Jia Hongtao, Joel Stanley, John Allen, Laurent Dufour, Madhavan Srinivasan, Mahesh Salgaonkar, Mark Hairgrove, Masahiro Yamada, Michael Bringmann, Michael Neuling, Michal Suchanek, Murilo Opsfelder Araujo, Nathan Fontenot, Naveen N. Rao, Nicholas Piggin, Nick Desaulniers, Oliver O'Halloran, Paul Mackerras, Petr Vorel, Rashmica Gupta, Reza Arbab, Rob Herring, Sam Bobroff, Samuel Mendoza-Jonas, Scott Wood, Stan Johnson, Stephen Rothwell, Stewart Smith, Suraj Jitindar Singh, Tyrel Datwyler, Vaibhav Jain, Vasant Hegde, YueHaibing, zhong jiang, -----BEGIN PGP SIGNATURE----- iQIcBAABAgAGBQJb01vTAAoJEFHr6jzI4aWADsEP/jqL3+2qxs098ra80tpXCpXJ tgXCosEs4b35sGtyHeUWZZZfWXeisaPAIlP8zTx1n50HACZduDYRAl0Ew9XB7Xdw enDHRVccD21FsmHBOx/Ii1rVJlovWlj6EQCWHKeZmNjeRoFuClVZ7CYmf+mBifKR sw2Db2fKA/59wMTq2zIMy5pqYgqlAs4jTWS6uN5hKPoBmO/82ARnNG+qgLuloD3Z O8zSDM9QQ7PpuyDgTjO9SAo2YjmEfXlEG6cOCCejsU3DMctaEAK5PUZ+blsHYHBH BYZYKs/x4pcw0SO41GtTh0M2YqDYBVuBIpRw8lLZap97Xo9ucSkAm5WD3rGxk4CY YeZKEPUql6MHN3+DKl8mx2F0V+Et/tio2HNqc9KReR1tfoolZAbe+SFZHfgmc/Rq RD9nnG8KRd4K2K1BTqpkTmI1EtE7jPtPJPSV8gMGhgL/N5vPmH3mql/qyOtYx48E 6/hPzWESgs16VRZ/opLh8VvjlY1HBDODQhehhhl+o23/Vb8qEgRf8Uqhq50rQW1H EeOqyyYQ90txSU31Sgy1kQkvOgIFAsBObWT1ZCJ3RbfGbB4/tdEAvZqTZRlXo2OY 7P0Sqcw/9Le5eJkHIlLtBv0TF7y1OYemCbLgRQzFlcRP+UKtYyg8eFnFjqbPEEmP ulwhn/BfFVSgaYKQ503u =I0pj -----END PGP SIGNATURE----- Merge tag 'powerpc-4.20-1' of git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux Pull powerpc updates from Michael Ellerman: "Notable changes: - A large series to rewrite our SLB miss handling, replacing a lot of fairly complicated asm with much fewer lines of C. - Following on from that, we now maintain a cache of SLB entries for each process and preload them on context switch. Leading to a 27% speedup for our context switch benchmark on Power9. - Improvements to our handling of SLB multi-hit errors. We now print more debug information when they occur, and try to continue running by flushing the SLB and reloading, rather than treating them as fatal. - Enable THP migration on 64-bit Book3S machines (eg. Power7/8/9). - Add support for physical memory up to 2PB in the linear mapping on 64-bit Book3S. We only support up to 512TB as regular system memory, otherwise the percpu allocator runs out of vmalloc space. - Add stack protector support for 32 and 64-bit, with a per-task canary. - Add support for PTRACE_SYSEMU and PTRACE_SYSEMU_SINGLESTEP. - Support recognising "big cores" on Power9, where two SMT4 cores are presented to us as a single SMT8 core. - A large series to cleanup some of our ioremap handling and PTE flags. - Add a driver for the PAPR SCM (storage class memory) interface, allowing guests to operate on SCM devices (acked by Dan). - Changes to our ftrace code to handle very large kernels, where we need to use a trampoline to get to ftrace_caller(). And many other smaller enhancements and cleanups. Thanks to: Alan Modra, Alistair Popple, Aneesh Kumar K.V, Anton Blanchard, Aravinda Prasad, Bartlomiej Zolnierkiewicz, Benjamin Herrenschmidt, Breno Leitao, Cédric Le Goater, Christophe Leroy, Christophe Lombard, Dan Carpenter, Daniel Axtens, Finn Thain, Gautham R. Shenoy, Gustavo Romero, Haren Myneni, Hari Bathini, Jia Hongtao, Joel Stanley, John Allen, Laurent Dufour, Madhavan Srinivasan, Mahesh Salgaonkar, Mark Hairgrove, Masahiro Yamada, Michael Bringmann, Michael Neuling, Michal Suchanek, Murilo Opsfelder Araujo, Nathan Fontenot, Naveen N. Rao, Nicholas Piggin, Nick Desaulniers, Oliver O'Halloran, Paul Mackerras, Petr Vorel, Rashmica Gupta, Reza Arbab, Rob Herring, Sam Bobroff, Samuel Mendoza-Jonas, Scott Wood, Stan Johnson, Stephen Rothwell, Stewart Smith, Suraj Jitindar Singh, Tyrel Datwyler, Vaibhav Jain, Vasant Hegde, YueHaibing, zhong jiang" * tag 'powerpc-4.20-1' of git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux: (221 commits) Revert "selftests/powerpc: Fix out-of-tree build errors" powerpc/msi: Fix compile error on mpc83xx powerpc: Fix stack protector crashes on CPU hotplug powerpc/traps: restore recoverability of machine_check interrupts powerpc/64/module: REL32 relocation range check powerpc/64s/radix: Fix radix__flush_tlb_collapsed_pmd double flushing pmd selftests/powerpc: Add a test of wild bctr powerpc/mm: Fix page table dump to work on Radix powerpc/mm/radix: Display if mappings are exec or not powerpc/mm/radix: Simplify split mapping logic powerpc/mm/radix: Remove the retry in the split mapping logic powerpc/mm/radix: Fix small page at boundary when splitting powerpc/mm/radix: Fix overuse of small pages in splitting logic powerpc/mm/radix: Fix off-by-one in split mapping logic powerpc/ftrace: Handle large kernel configs powerpc/mm: Fix WARN_ON with THP NUMA migration selftests/powerpc: Fix out-of-tree build errors powerpc/time: no steal_time when CONFIG_PPC_SPLPAR is not selected powerpc/time: Only set CONFIG_ARCH_HAS_SCALED_CPUTIME on PPC64 powerpc/time: isolate scaled cputime accounting in dedicated functions. ...
264 lines
7.8 KiB
C
264 lines
7.8 KiB
C
/*
|
|
* Copyright (C) 2001 PPC64 Team, IBM Corp
|
|
*
|
|
* This struct defines the way the registers are stored on the
|
|
* kernel stack during a system call or other kernel entry.
|
|
*
|
|
* this should only contain volatile regs
|
|
* since we can keep non-volatile in the thread_struct
|
|
* should set this up when only volatiles are saved
|
|
* by intr code.
|
|
*
|
|
* Since this is going on the stack, *CARE MUST BE TAKEN* to insure
|
|
* that the overall structure is a multiple of 16 bytes in length.
|
|
*
|
|
* Note that the offsets of the fields in this struct correspond with
|
|
* the PT_* values below. This simplifies arch/powerpc/kernel/ptrace.c.
|
|
*
|
|
* This program is free software; you can redistribute it and/or
|
|
* modify it under the terms of the GNU General Public License
|
|
* as published by the Free Software Foundation; either version
|
|
* 2 of the License, or (at your option) any later version.
|
|
*/
|
|
#ifndef _ASM_POWERPC_PTRACE_H
|
|
#define _ASM_POWERPC_PTRACE_H
|
|
|
|
#include <uapi/asm/ptrace.h>
|
|
#include <asm/asm-const.h>
|
|
|
|
#ifndef __ASSEMBLY__
|
|
struct pt_regs
|
|
{
|
|
union {
|
|
struct user_pt_regs user_regs;
|
|
struct {
|
|
unsigned long gpr[32];
|
|
unsigned long nip;
|
|
unsigned long msr;
|
|
unsigned long orig_gpr3;
|
|
unsigned long ctr;
|
|
unsigned long link;
|
|
unsigned long xer;
|
|
unsigned long ccr;
|
|
#ifdef CONFIG_PPC64
|
|
unsigned long softe;
|
|
#else
|
|
unsigned long mq;
|
|
#endif
|
|
unsigned long trap;
|
|
unsigned long dar;
|
|
unsigned long dsisr;
|
|
unsigned long result;
|
|
};
|
|
};
|
|
|
|
#ifdef CONFIG_PPC64
|
|
unsigned long ppr;
|
|
#endif
|
|
};
|
|
#endif
|
|
|
|
#ifdef __powerpc64__
|
|
|
|
/*
|
|
* Size of redzone that userspace is allowed to use below the stack
|
|
* pointer. This is 288 in the 64-bit big-endian ELF ABI, and 512 in
|
|
* the new ELFv2 little-endian ABI, so we allow the larger amount.
|
|
*
|
|
* For kernel code we allow a 288-byte redzone, in order to conserve
|
|
* kernel stack space; gcc currently only uses 288 bytes, and will
|
|
* hopefully allow explicit control of the redzone size in future.
|
|
*/
|
|
#define USER_REDZONE_SIZE 512
|
|
#define KERNEL_REDZONE_SIZE 288
|
|
|
|
#define STACK_FRAME_OVERHEAD 112 /* size of minimum stack frame */
|
|
#define STACK_FRAME_LR_SAVE 2 /* Location of LR in stack frame */
|
|
#define STACK_FRAME_REGS_MARKER ASM_CONST(0x7265677368657265)
|
|
#define STACK_INT_FRAME_SIZE (sizeof(struct pt_regs) + \
|
|
STACK_FRAME_OVERHEAD + KERNEL_REDZONE_SIZE)
|
|
#define STACK_FRAME_MARKER 12
|
|
|
|
#ifdef PPC64_ELF_ABI_v2
|
|
#define STACK_FRAME_MIN_SIZE 32
|
|
#else
|
|
#define STACK_FRAME_MIN_SIZE STACK_FRAME_OVERHEAD
|
|
#endif
|
|
|
|
/* Size of dummy stack frame allocated when calling signal handler. */
|
|
#define __SIGNAL_FRAMESIZE 128
|
|
#define __SIGNAL_FRAMESIZE32 64
|
|
|
|
#else /* __powerpc64__ */
|
|
|
|
#define USER_REDZONE_SIZE 0
|
|
#define KERNEL_REDZONE_SIZE 0
|
|
#define STACK_FRAME_OVERHEAD 16 /* size of minimum stack frame */
|
|
#define STACK_FRAME_LR_SAVE 1 /* Location of LR in stack frame */
|
|
#define STACK_FRAME_REGS_MARKER ASM_CONST(0x72656773)
|
|
#define STACK_INT_FRAME_SIZE (sizeof(struct pt_regs) + STACK_FRAME_OVERHEAD)
|
|
#define STACK_FRAME_MARKER 2
|
|
#define STACK_FRAME_MIN_SIZE STACK_FRAME_OVERHEAD
|
|
|
|
/* Size of stack frame allocated when calling signal handler. */
|
|
#define __SIGNAL_FRAMESIZE 64
|
|
|
|
#endif /* __powerpc64__ */
|
|
|
|
#ifndef __ASSEMBLY__
|
|
|
|
#define GET_IP(regs) ((regs)->nip)
|
|
#define GET_USP(regs) ((regs)->gpr[1])
|
|
#define GET_FP(regs) (0)
|
|
#define SET_FP(regs, val)
|
|
|
|
#ifdef CONFIG_SMP
|
|
extern unsigned long profile_pc(struct pt_regs *regs);
|
|
#define profile_pc profile_pc
|
|
#endif
|
|
|
|
#include <asm-generic/ptrace.h>
|
|
|
|
#define kernel_stack_pointer(regs) ((regs)->gpr[1])
|
|
static inline int is_syscall_success(struct pt_regs *regs)
|
|
{
|
|
return !(regs->ccr & 0x10000000);
|
|
}
|
|
|
|
static inline long regs_return_value(struct pt_regs *regs)
|
|
{
|
|
if (is_syscall_success(regs))
|
|
return regs->gpr[3];
|
|
else
|
|
return -regs->gpr[3];
|
|
}
|
|
|
|
static inline void regs_set_return_value(struct pt_regs *regs, unsigned long rc)
|
|
{
|
|
regs->gpr[3] = rc;
|
|
}
|
|
|
|
#ifdef __powerpc64__
|
|
#define user_mode(regs) ((((regs)->msr) >> MSR_PR_LG) & 0x1)
|
|
#else
|
|
#define user_mode(regs) (((regs)->msr & MSR_PR) != 0)
|
|
#endif
|
|
|
|
#define force_successful_syscall_return() \
|
|
do { \
|
|
set_thread_flag(TIF_NOERROR); \
|
|
} while(0)
|
|
|
|
struct task_struct;
|
|
extern int ptrace_get_reg(struct task_struct *task, int regno,
|
|
unsigned long *data);
|
|
extern int ptrace_put_reg(struct task_struct *task, int regno,
|
|
unsigned long data);
|
|
|
|
#define current_pt_regs() \
|
|
((struct pt_regs *)((unsigned long)current_thread_info() + THREAD_SIZE) - 1)
|
|
/*
|
|
* We use the least-significant bit of the trap field to indicate
|
|
* whether we have saved the full set of registers, or only a
|
|
* partial set. A 1 there means the partial set.
|
|
* On 4xx we use the next bit to indicate whether the exception
|
|
* is a critical exception (1 means it is).
|
|
*/
|
|
#define FULL_REGS(regs) (((regs)->trap & 1) == 0)
|
|
#ifndef __powerpc64__
|
|
#define IS_CRITICAL_EXC(regs) (((regs)->trap & 2) != 0)
|
|
#define IS_MCHECK_EXC(regs) (((regs)->trap & 4) != 0)
|
|
#define IS_DEBUG_EXC(regs) (((regs)->trap & 8) != 0)
|
|
#endif /* ! __powerpc64__ */
|
|
#define TRAP(regs) ((regs)->trap & ~0xF)
|
|
#ifdef __powerpc64__
|
|
#define NV_REG_POISON 0xdeadbeefdeadbeefUL
|
|
#define CHECK_FULL_REGS(regs) BUG_ON(regs->trap & 1)
|
|
#else
|
|
#define NV_REG_POISON 0xdeadbeef
|
|
#define CHECK_FULL_REGS(regs) \
|
|
do { \
|
|
if ((regs)->trap & 1) \
|
|
printk(KERN_CRIT "%s: partial register set\n", __func__); \
|
|
} while (0)
|
|
#endif /* __powerpc64__ */
|
|
|
|
#define arch_has_single_step() (1)
|
|
#define arch_has_block_step() (!cpu_has_feature(CPU_FTR_601))
|
|
#define ARCH_HAS_USER_SINGLE_STEP_REPORT
|
|
|
|
/*
|
|
* kprobe-based event tracer support
|
|
*/
|
|
|
|
#include <linux/stddef.h>
|
|
#include <linux/thread_info.h>
|
|
extern int regs_query_register_offset(const char *name);
|
|
extern const char *regs_query_register_name(unsigned int offset);
|
|
#define MAX_REG_OFFSET (offsetof(struct pt_regs, dsisr))
|
|
|
|
/**
|
|
* regs_get_register() - get register value from its offset
|
|
* @regs: pt_regs from which register value is gotten
|
|
* @offset: offset number of the register.
|
|
*
|
|
* regs_get_register returns the value of a register whose offset from @regs.
|
|
* The @offset is the offset of the register in struct pt_regs.
|
|
* If @offset is bigger than MAX_REG_OFFSET, this returns 0.
|
|
*/
|
|
static inline unsigned long regs_get_register(struct pt_regs *regs,
|
|
unsigned int offset)
|
|
{
|
|
if (unlikely(offset > MAX_REG_OFFSET))
|
|
return 0;
|
|
return *(unsigned long *)((unsigned long)regs + offset);
|
|
}
|
|
|
|
/**
|
|
* regs_within_kernel_stack() - check the address in the stack
|
|
* @regs: pt_regs which contains kernel stack pointer.
|
|
* @addr: address which is checked.
|
|
*
|
|
* regs_within_kernel_stack() checks @addr is within the kernel stack page(s).
|
|
* If @addr is within the kernel stack, it returns true. If not, returns false.
|
|
*/
|
|
|
|
static inline bool regs_within_kernel_stack(struct pt_regs *regs,
|
|
unsigned long addr)
|
|
{
|
|
return ((addr & ~(THREAD_SIZE - 1)) ==
|
|
(kernel_stack_pointer(regs) & ~(THREAD_SIZE - 1)));
|
|
}
|
|
|
|
/**
|
|
* regs_get_kernel_stack_nth() - get Nth entry of the stack
|
|
* @regs: pt_regs which contains kernel stack pointer.
|
|
* @n: stack entry number.
|
|
*
|
|
* regs_get_kernel_stack_nth() returns @n th entry of the kernel stack which
|
|
* is specified by @regs. If the @n th entry is NOT in the kernel stack,
|
|
* this returns 0.
|
|
*/
|
|
static inline unsigned long regs_get_kernel_stack_nth(struct pt_regs *regs,
|
|
unsigned int n)
|
|
{
|
|
unsigned long *addr = (unsigned long *)kernel_stack_pointer(regs);
|
|
addr += n;
|
|
if (regs_within_kernel_stack(regs, (unsigned long)addr))
|
|
return *addr;
|
|
else
|
|
return 0;
|
|
}
|
|
|
|
#endif /* __ASSEMBLY__ */
|
|
|
|
#ifndef __powerpc64__
|
|
#else /* __powerpc64__ */
|
|
#define PT_FPSCR32 (PT_FPR0 + 2*32 + 1) /* each FP reg occupies 2 32-bit userspace slots */
|
|
#define PT_VR0_32 164 /* each Vector reg occupies 4 slots in 32-bit */
|
|
#define PT_VSCR_32 (PT_VR0 + 32*4 + 3)
|
|
#define PT_VRSAVE_32 (PT_VR0 + 33*4)
|
|
#define PT_VSR0_32 300 /* each VSR reg occupies 4 slots in 32-bit */
|
|
#endif /* __powerpc64__ */
|
|
#endif /* _ASM_POWERPC_PTRACE_H */
|