Minki/linux
1
0
Fork 1
mirror of https://github.com/torvalds/linux.git synced 2024-11-10 14:11:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

Updates for the MSI interrupt subsystem and RISC-V initial MSI support:

Browse Source 
- Core and platform-MSI
 
     The core changes have been adopted from previous work which converted
     ARM[64] to the new per device MSI domain model, which was merged to
     support multiple MSI domain per device. The ARM[64] changes are being
     worked on too, but have not been ready yet. The core and platform-MSI
     changes have been split out to not hold up RISC-V and to avoid that
     RISC-V builds on the scheduled for removal interfaces.
 
     The core support provides new interfaces to handle wire to MSI bridges
     in a straight forward way and introduces new platform-MSI interfaces
     which are built on top of the per device MSI domain model.
 
     Once ARM[64] is converted over the old platform-MSI interfaces and the
     related ugliness in the MSI core code will be removed.
 
   - Drivers:
 
     - Add a new driver for the Andes hart-level interrupt controller
 
     - Rework the SiFive PLIC driver to prepare for MSI suport
 
     - Expand the RISC-V INTC driver to support the new RISC-V AIA
       controller which provides the basis for MSI on RISC-V
 
     - A few fixup for the fallout of the core changes.
 
     The actual MSI parts for RISC-V were finalized late and have been
     post-poned for the next merge window.
 -----BEGIN PGP SIGNATURE-----
 
 iQJHBAABCgAxFiEEQp8+kY+LLUocC4bMphj1TA10mKEFAmXt7MsTHHRnbHhAbGlu
 dXRyb25peC5kZQAKCRCmGPVMDXSYofrMD/9Dag12ttmbE2uqzTzlTxc7RHC2MX5n
 VJLt84FNNwGPA4r7WLOOqHrfuvfoGjuWT9pYMrVaXCglRG1CMvL10kHMB2f28UWv
 Qpc5PzbJwpD6tqyfRSFHMoJp63DAI8IpS7J3I8bqnRD8+0PwYn3jMA1+iMZkH0B7
 8uO3mxlFhQ7BFvIAeMEAhR0szuAfvXqEtpi1iTgQTrQ4Je4Rf1pmLjEe2rkwDvF4
 p3SAmPIh4+F3IjO7vNsVkQ2yOarTP2cpSns6JmO8mrobLIVX7ZCQ6uVaVCfBhxfx
 WttuJO6Bmh/I15yDe/waH6q9ym+0VBwYRWi5lonMpViGdq4/D2WVnY1mNeLRIfjl
 X65aMWE1+bhiqyIIUfc24hacf0UgBIlMEW4kJ31VmQzb+OyLDXw+UvzWg1dO6XdA
 3L6j1nRgHk0ea5yFyH6SfH/mrfeyqHuwHqo17KFyHxD3jM2H1RRMplpbwXiOIepp
 KJJ/O06eMEzHqzn4B8GCT2EvX6L2ehgoWbLeEDNLQh/3LwA9OdcBzPr6gsweEl0U
 Q7szJgUWZHeMr39F2rnt0GmvkEuu6muEp/nQzfnohjoYZ0PhpMLSq++4Gi+Ko3fz
 2IyecJ+tlbSfyM5//8AdNnOSpsTG3f8u6B/WwhGp5lIDwMnMzCssgfQmRnc3Uyv5
 kU3pdMjURJaTUA==
 =7aXj
 -----END PGP SIGNATURE-----

Merge tag 'irq-msi-2024-03-10' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

Pull MSI updates from Thomas Gleixner:
 "Updates for the MSI interrupt subsystem and initial RISC-V MSI
  support.

  The core changes have been adopted from previous work which converted
  ARM[64] to the new per device MSI domain model, which was merged to
  support multiple MSI domain per device. The ARM[64] changes are being
  worked on too, but have not been ready yet. The core and platform-MSI
  changes have been split out to not hold up RISC-V and to avoid that
  RISC-V builds on the scheduled for removal interfaces.

  The core support provides new interfaces to handle wire to MSI bridges
  in a straight forward way and introduces new platform-MSI interfaces
  which are built on top of the per device MSI domain model.

  Once ARM[64] is converted over the old platform-MSI interfaces and the
  related ugliness in the MSI core code will be removed.

  The actual MSI parts for RISC-V were finalized late and have been
  post-poned for the next merge window.

  Drivers:

   - Add a new driver for the Andes hart-level interrupt controller

   - Rework the SiFive PLIC driver to prepare for MSI suport

   - Expand the RISC-V INTC driver to support the new RISC-V AIA
     controller which provides the basis for MSI on RISC-V

   - A few fixup for the fallout of the core changes"

* tag 'irq-msi-2024-03-10' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (29 commits)
  irqchip/riscv-intc: Fix low-level interrupt handler setup for AIA
  x86/apic/msi: Use DOMAIN_BUS_GENERIC_MSI for HPET/IO-APIC domain search
  genirq/matrix: Dynamic bitmap allocation
  irqchip/riscv-intc: Add support for RISC-V AIA
  irqchip/sifive-plic: Improve locking safety by using irqsave/irqrestore
  irqchip/sifive-plic: Parse number of interrupts and contexts early in plic_probe()
  irqchip/sifive-plic: Cleanup PLIC contexts upon irqdomain creation failure
  irqchip/sifive-plic: Use riscv_get_intc_hwnode() to get parent fwnode
  irqchip/sifive-plic: Use devm_xyz() for managed allocation
  irqchip/sifive-plic: Use dev_xyz() in-place of pr_xyz()
  irqchip/sifive-plic: Convert PLIC driver into a platform driver
  irqchip/riscv-intc: Introduce Andes hart-level interrupt controller
  irqchip/riscv-intc: Allow large non-standard interrupt number
  genirq/irqdomain: Don't call ops->select for DOMAIN_BUS_ANY tokens
  irqchip/imx-intmux: Handle pure domain searches correctly
  genirq/msi: Provide MSI_FLAG_PARENT_PM_DEV
  genirq/irqdomain: Reroute device MSI create_mapping
  genirq/msi: Provide allocation/free functions for "wired" MSI interrupts
  genirq/msi: Optionally use dev->fwnode for device domain
  genirq/msi: Provide DOMAIN_BUS_WIRED_TO_MSI
  ...
This commit is contained in:
Linus Torvalds 2024-03-11 14:03:03 -07:00
commit 4527e83780
21 changed files with 643 additions and 223 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
arch/x86/include/asm/hw_irq.h
View File

@ -16,8 +16,6 @@
#include <asm/irq_vectors.h>
#define IRQ_MATRIX_BITS NR_VECTORS
#ifndef __ASSEMBLY__
#include <linux/percpu.h>

2
arch/x86/kernel/apic/io_apic.c
View File

@ -2354,7 +2354,7 @@ static int mp_irqdomain_create(int ioapic)
fwspec.param_count = 1;
fwspec.param[0] = mpc_ioapic_id(ioapic);
parent = irq_find_matching_fwspec(&fwspec, DOMAIN_BUS_ANY);
parent = irq_find_matching_fwspec(&fwspec, DOMAIN_BUS_GENERIC_MSI);
if (!parent) {
if (!cfg->dev)
irq_domain_free_fwnode(fn);

2
arch/x86/kernel/hpet.c
View File

@ -568,7 +568,7 @@ static struct irq_domain *hpet_create_irq_domain(int hpet_id)
fwspec.param_count = 1;
fwspec.param[0] = hpet_id;
parent = irq_find_matching_fwspec(&fwspec, DOMAIN_BUS_ANY);
parent = irq_find_matching_fwspec(&fwspec, DOMAIN_BUS_GENERIC_MSI);
if (!parent) {
irq_domain_free_fwnode(fn);
kfree(domain_info);

119
drivers/base/platform-msi.c
View File

@ -13,6 +13,8 @@
#include <linux/msi.h>
#include <linux/slab.h>
/* Begin of removal area. Once everything is converted over. Cleanup the includes too! */
#define DEV_ID_SHIFT 21
#define MAX_DEV_MSIS (1 << (32 - DEV_ID_SHIFT))
@ -204,8 +206,8 @@ static void platform_msi_free_priv_data(struct device *dev)
* Returns:
* Zero for success, or an error code in case of failure
*/
int platform_msi_domain_alloc_irqs(struct device *dev, unsigned int nvec,
irq_write_msi_msg_t write_msi_msg)
static int platform_msi_domain_alloc_irqs(struct device *dev, unsigned int nvec,
irq_write_msi_msg_t write_msi_msg)
{
int err;
@ -219,18 +221,6 @@ int platform_msi_domain_alloc_irqs(struct device *dev, unsigned int nvec,
return err;
}
EXPORT_SYMBOL_GPL(platform_msi_domain_alloc_irqs);
/**
* platform_msi_domain_free_irqs - Free MSI interrupts for @dev
* @dev: The device for which to free interrupts
*/
void platform_msi_domain_free_irqs(struct device *dev)
{
msi_domain_free_irqs_all(dev, MSI_DEFAULT_DOMAIN);
platform_msi_free_priv_data(dev);
}
EXPORT_SYMBOL_GPL(platform_msi_domain_free_irqs);
/**
* platform_msi_get_host_data - Query the private data associated with
@ -350,3 +340,104 @@ int platform_msi_device_domain_alloc(struct irq_domain *domain, unsigned int vir
return msi_domain_populate_irqs(domain->parent, dev, virq, nr_irqs, &data->arg);
}
/* End of removal area */
/* Real per device domain interfaces */
/*
* This indirection can go when platform_device_msi_init_and_alloc_irqs()
* is switched to a proper irq_chip::irq_write_msi_msg() callback. Keep it
* simple for now.
*/
static void platform_msi_write_msi_msg(struct irq_data *d, struct msi_msg *msg)
{
irq_write_msi_msg_t cb = d->chip_data;
cb(irq_data_get_msi_desc(d), msg);
}
static void platform_msi_set_desc_byindex(msi_alloc_info_t *arg, struct msi_desc *desc)
{
arg->desc = desc;
arg->hwirq = desc->msi_index;
}
static const struct msi_domain_template platform_msi_template = {
.chip = {
.name = "pMSI",
.irq_mask = irq_chip_mask_parent,
.irq_unmask = irq_chip_unmask_parent,
.irq_write_msi_msg = platform_msi_write_msi_msg,
/* The rest is filled in by the platform MSI parent */
},
.ops = {
.set_desc = platform_msi_set_desc_byindex,
},
.info = {
.bus_token = DOMAIN_BUS_DEVICE_MSI,
},
};
/**
* platform_device_msi_init_and_alloc_irqs - Initialize platform device MSI
* and allocate interrupts for @dev
* @dev: The device for which to allocate interrupts
* @nvec: The number of interrupts to allocate
* @write_msi_msg: Callback to write an interrupt message for @dev
*
* Returns:
* Zero for success, or an error code in case of failure
*
* This creates a MSI domain on @dev which has @dev->msi.domain as
* parent. The parent domain sets up the new domain. The domain has
* a fixed size of @nvec. The domain is managed by devres and will
* be removed when the device is removed.
*
* Note: For migration purposes this falls back to the original platform_msi code
* up to the point where all platforms have been converted to the MSI
* parent model.
*/
int platform_device_msi_init_and_alloc_irqs(struct device *dev, unsigned int nvec,
irq_write_msi_msg_t write_msi_msg)
{
struct irq_domain *domain = dev->msi.domain;
if (!domain || !write_msi_msg)
return -EINVAL;
/* Migration support. Will go away once everything is converted */
if (!irq_domain_is_msi_parent(domain))
return platform_msi_domain_alloc_irqs(dev, nvec, write_msi_msg);
/*
* @write_msi_msg is stored in the resulting msi_domain_info::data.
* The underlying domain creation mechanism will assign that
* callback to the resulting irq chip.
*/
if (!msi_create_device_irq_domain(dev, MSI_DEFAULT_DOMAIN,
&platform_msi_template,
nvec, NULL, write_msi_msg))
return -ENODEV;
return msi_domain_alloc_irqs_range(dev, MSI_DEFAULT_DOMAIN, 0, nvec - 1);
}
EXPORT_SYMBOL_GPL(platform_device_msi_init_and_alloc_irqs);
/**
* platform_device_msi_free_irqs_all - Free all interrupts for @dev
* @dev: The device for which to free interrupts
*/
void platform_device_msi_free_irqs_all(struct device *dev)
{
struct irq_domain *domain = dev->msi.domain;
msi_domain_free_irqs_all(dev, MSI_DEFAULT_DOMAIN);
/* Migration support. Will go away once everything is converted */
if (!irq_domain_is_msi_parent(domain))
platform_msi_free_priv_data(dev);
}
EXPORT_SYMBOL_GPL(platform_device_msi_free_irqs_all);

8
drivers/dma/mv_xor_v2.c
View File

@ -747,8 +747,8 @@ static int mv_xor_v2_probe(struct platform_device *pdev)
if (IS_ERR(xor_dev->clk))
return PTR_ERR(xor_dev->clk);
ret = platform_msi_domain_alloc_irqs(&pdev->dev, 1,
mv_xor_v2_set_msi_msg);
ret = platform_device_msi_init_and_alloc_irqs(&pdev->dev, 1,
mv_xor_v2_set_msi_msg);
if (ret)
return ret;
@ -851,7 +851,7 @@ free_hw_desq:
xor_dev->desc_size * MV_XOR_V2_DESC_NUM,
xor_dev->hw_desq_virt, xor_dev->hw_desq);
free_msi_irqs:
platform_msi_domain_free_irqs(&pdev->dev);
platform_device_msi_free_irqs_all(&pdev->dev);
return ret;
}
@ -867,7 +867,7 @@ static void mv_xor_v2_remove(struct platform_device *pdev)
devm_free_irq(&pdev->dev, xor_dev->irq, xor_dev);
platform_msi_domain_free_irqs(&pdev->dev);
platform_device_msi_free_irqs_all(&pdev->dev);
tasklet_kill(&xor_dev->irq_tasklet);
}

6
drivers/dma/qcom/hidma.c
View File

@ -696,7 +696,7 @@ static void hidma_free_msis(struct hidma_dev *dmadev)
devm_free_irq(dev, virq, &dmadev->lldev);
}
platform_msi_domain_free_irqs(dev);
platform_device_msi_free_irqs_all(dev);
#endif
}
@ -706,8 +706,8 @@ static int hidma_request_msi(struct hidma_dev *dmadev,
#ifdef CONFIG_GENERIC_MSI_IRQ
int rc, i, virq;
rc = platform_msi_domain_alloc_irqs(&pdev->dev, HIDMA_MSI_INTS,
hidma_write_msi_msg);
rc = platform_device_msi_init_and_alloc_irqs(&pdev->dev, HIDMA_MSI_INTS,
hidma_write_msi_msg);
if (rc)
return rc;

5
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
View File

@ -3125,7 +3125,8 @@ static int arm_smmu_update_gbpa(struct arm_smmu_device *smmu, u32 set, u32 clr)
static void arm_smmu_free_msis(void *data)
{
struct device *dev = data;
platform_msi_domain_free_irqs(dev);
platform_device_msi_free_irqs_all(dev);
}
static void arm_smmu_write_msi_msg(struct msi_desc *desc, struct msi_msg *msg)
@ -3166,7 +3167,7 @@ static void arm_smmu_setup_msis(struct arm_smmu_device *smmu)
}
/* Allocate MSIs for evtq, gerror and priq. Ignore cmdq */
ret = platform_msi_domain_alloc_irqs(dev, nvec, arm_smmu_write_msi_msg);
ret = platform_device_msi_init_and_alloc_irqs(dev, nvec, arm_smmu_write_msi_msg);
if (ret) {
dev_warn(dev, "failed to allocate MSIs - falling back to wired irqs\n");
return;

6
drivers/irqchip/irq-gic-v3.c
View File

@ -1691,9 +1691,13 @@ static int gic_irq_domain_select(struct irq_domain *d,
irq_hw_number_t hwirq;
/* Not for us */
if (fwspec->fwnode != d->fwnode)
if (fwspec->fwnode != d->fwnode)
return 0;
/* Handle pure domain searches */
if (!fwspec->param_count)
return d->bus_token == bus_token;
/* If this is not DT, then we have a single domain */
if (!is_of_node(fwspec->fwnode))
return 1;

4
drivers/irqchip/irq-imx-intmux.c
View File

@ -166,6 +166,10 @@ static int imx_intmux_irq_select(struct irq_domain *d, struct irq_fwspec *fwspec
if (fwspec->fwnode != d->fwnode)
return false;
/* Handle pure domain searches */
if (!fwspec->param_count)
return d->bus_token == bus_token;
return irqchip_data->chanidx == fwspec->param[1];
}

104
drivers/irqchip/irq-riscv-intc.c
View File

@ -17,17 +17,29 @@
#include <linux/module.h>
#include <linux/of.h>
#include <linux/smp.h>
#include <linux/soc/andes/irq.h>
#include <asm/hwcap.h>
static struct irq_domain *intc_domain;
static unsigned int riscv_intc_nr_irqs __ro_after_init = BITS_PER_LONG;
static unsigned int riscv_intc_custom_base __ro_after_init = BITS_PER_LONG;
static unsigned int riscv_intc_custom_nr_irqs __ro_after_init;
static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
{
unsigned long cause = regs->cause & ~CAUSE_IRQ_FLAG;
if (unlikely(cause >= BITS_PER_LONG))
panic("unexpected interrupt cause");
if (generic_handle_domain_irq(intc_domain, cause))
pr_warn_ratelimited("Failed to handle interrupt (cause: %ld)\n", cause);
}
generic_handle_domain_irq(intc_domain, cause);
static asmlinkage void riscv_intc_aia_irq(struct pt_regs *regs)
{
unsigned long topi;
while ((topi = csr_read(CSR_TOPI)))
generic_handle_domain_irq(intc_domain, topi >> TOPI_IID_SHIFT);
}
/*
@ -39,12 +51,43 @@ static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
static void riscv_intc_irq_mask(struct irq_data *d)
{
csr_clear(CSR_IE, BIT(d->hwirq));
if (IS_ENABLED(CONFIG_32BIT) && d->hwirq >= BITS_PER_LONG)
csr_clear(CSR_IEH, BIT(d->hwirq - BITS_PER_LONG));
else
csr_clear(CSR_IE, BIT(d->hwirq));
}
static void riscv_intc_irq_unmask(struct irq_data *d)
{
csr_set(CSR_IE, BIT(d->hwirq));
if (IS_ENABLED(CONFIG_32BIT) && d->hwirq >= BITS_PER_LONG)
csr_set(CSR_IEH, BIT(d->hwirq - BITS_PER_LONG));
else
csr_set(CSR_IE, BIT(d->hwirq));
}
static void andes_intc_irq_mask(struct irq_data *d)
{
/*
* Andes specific S-mode local interrupt causes (hwirq)
* are defined as (256 + n) and controlled by n-th bit
* of SLIE.
*/
unsigned int mask = BIT(d->hwirq % BITS_PER_LONG);
if (d->hwirq < ANDES_SLI_CAUSE_BASE)
csr_clear(CSR_IE, mask);
else
csr_clear(ANDES_CSR_SLIE, mask);
}
static void andes_intc_irq_unmask(struct irq_data *d)
{
unsigned int mask = BIT(d->hwirq % BITS_PER_LONG);
if (d->hwirq < ANDES_SLI_CAUSE_BASE)
csr_set(CSR_IE, mask);
else
csr_set(ANDES_CSR_SLIE, mask);
}
static void riscv_intc_irq_eoi(struct irq_data *d)
@ -70,12 +113,21 @@ static struct irq_chip riscv_intc_chip = {
.irq_eoi = riscv_intc_irq_eoi,
};
static struct irq_chip andes_intc_chip = {
.name = "RISC-V INTC",
.irq_mask = andes_intc_irq_mask,
.irq_unmask = andes_intc_irq_unmask,
.irq_eoi = riscv_intc_irq_eoi,
};
static int riscv_intc_domain_map(struct irq_domain *d, unsigned int irq,
irq_hw_number_t hwirq)
{
struct irq_chip *chip = d->host_data;
irq_set_percpu_devid(irq);
irq_domain_set_info(d, irq, hwirq, &riscv_intc_chip, d->host_data,
handle_percpu_devid_irq, NULL, NULL);
irq_domain_set_info(d, irq, hwirq, chip, NULL, handle_percpu_devid_irq,
NULL, NULL);
return 0;
}
@ -93,6 +145,14 @@ static int riscv_intc_domain_alloc(struct irq_domain *domain,
if (ret)
return ret;
/*
* Only allow hwirq for which we have corresponding standard or
* custom interrupt enable register.
*/
if ((hwirq >= riscv_intc_nr_irqs && hwirq < riscv_intc_custom_base) ||
(hwirq >= riscv_intc_custom_base + riscv_intc_custom_nr_irqs))
return -EINVAL;
for (i = 0; i < nr_irqs; i++) {
ret = riscv_intc_domain_map(domain, virq + i, hwirq + i);
if (ret)
@ -113,18 +173,20 @@ static struct fwnode_handle *riscv_intc_hwnode(void)
return intc_domain->fwnode;
}
static int __init riscv_intc_init_common(struct fwnode_handle *fn)
static int __init riscv_intc_init_common(struct fwnode_handle *fn, struct irq_chip *chip)
{
int rc;
intc_domain = irq_domain_create_linear(fn, BITS_PER_LONG,
&riscv_intc_domain_ops, NULL);
intc_domain = irq_domain_create_tree(fn, &riscv_intc_domain_ops, chip);
if (!intc_domain) {
pr_err("unable to add IRQ domain\n");
return -ENXIO;
}
rc = set_handle_irq(&riscv_intc_irq);
if (riscv_isa_extension_available(NULL, SxAIA))
rc = set_handle_irq(&riscv_intc_aia_irq);
else
rc = set_handle_irq(&riscv_intc_irq);
if (rc) {
pr_err("failed to set irq handler\n");
return rc;
@ -132,7 +194,11 @@ static int __init riscv_intc_init_common(struct fwnode_handle *fn)
riscv_set_intc_hwnode_fn(riscv_intc_hwnode);
pr_info("%d local interrupts mapped\n", BITS_PER_LONG);
pr_info("%d local interrupts mapped%s\n",
riscv_isa_extension_available(NULL, SxAIA) ? 64 : riscv_intc_nr_irqs,
riscv_isa_extension_available(NULL, SxAIA) ? " using AIA" : "");
if (riscv_intc_custom_nr_irqs)
pr_info("%d custom local interrupts mapped\n", riscv_intc_custom_nr_irqs);
return 0;
}
@ -140,8 +206,9 @@ static int __init riscv_intc_init_common(struct fwnode_handle *fn)
static int __init riscv_intc_init(struct device_node *node,
struct device_node *parent)
{
int rc;
struct irq_chip *chip = &riscv_intc_chip;
unsigned long hartid;
int rc;
rc = riscv_of_parent_hartid(node, &hartid);
if (rc < 0) {
@ -166,10 +233,17 @@ static int __init riscv_intc_init(struct device_node *node,
return 0;
}
return riscv_intc_init_common(of_node_to_fwnode(node));
if (of_device_is_compatible(node, "andestech,cpu-intc")) {
riscv_intc_custom_base = ANDES_SLI_CAUSE_BASE;
riscv_intc_custom_nr_irqs = ANDES_RV_IRQ_LAST;
chip = &andes_intc_chip;
}
return riscv_intc_init_common(of_node_to_fwnode(node), chip);
}
IRQCHIP_DECLARE(riscv, "riscv,cpu-intc", riscv_intc_init);
IRQCHIP_DECLARE(andes, "andestech,cpu-intc", riscv_intc_init);
#ifdef CONFIG_ACPI
@ -196,7 +270,7 @@ static int __init riscv_intc_acpi_init(union acpi_subtable_headers *header,
return -ENOMEM;
}
return riscv_intc_init_common(fn);
return riscv_intc_init_common(fn, &riscv_intc_chip);
}
IRQCHIP_ACPI_DECLARE(riscv_intc, ACPI_MADT_TYPE_RINTC, NULL,

279
drivers/irqchip/irq-sifive-plic.c
View File

@ -3,7 +3,6 @@
* Copyright (C) 2017 SiFive
* Copyright (C) 2018 Christoph Hellwig
*/
#define pr_fmt(fmt) "plic: " fmt
#include <linux/cpu.h>
#include <linux/interrupt.h>
#include <linux/io.h>
@ -64,6 +63,7 @@
#define PLIC_QUIRK_EDGE_INTERRUPT 0
struct plic_priv {
struct device *dev;
struct cpumask lmask;
struct irq_domain *irqdomain;
void __iomem *regs;
@ -103,9 +103,11 @@ static void __plic_toggle(void __iomem *enable_base, int hwirq, int enable)
static void plic_toggle(struct plic_handler *handler, int hwirq, int enable)
{
raw_spin_lock(&handler->enable_lock);
unsigned long flags;
raw_spin_lock_irqsave(&handler->enable_lock, flags);
__plic_toggle(handler->enable_base, hwirq, enable);
raw_spin_unlock(&handler->enable_lock);
raw_spin_unlock_irqrestore(&handler->enable_lock, flags);
}
static inline void plic_irq_toggle(const struct cpumask *mask,
@ -242,6 +244,7 @@ static int plic_irq_set_type(struct irq_data *d, unsigned int type)
static int plic_irq_suspend(void)
{
unsigned int i, cpu;
unsigned long flags;
u32 __iomem *reg;
struct plic_priv *priv;
@ -259,12 +262,12 @@ static int plic_irq_suspend(void)
if (!handler->present)
continue;
raw_spin_lock(&handler->enable_lock);
raw_spin_lock_irqsave(&handler->enable_lock, flags);
for (i = 0; i < DIV_ROUND_UP(priv->nr_irqs, 32); i++) {
reg = handler->enable_base + i * sizeof(u32);
handler->enable_save[i] = readl(reg);
}
raw_spin_unlock(&handler->enable_lock);
raw_spin_unlock_irqrestore(&handler->enable_lock, flags);
}
return 0;
@ -273,6 +276,7 @@ static int plic_irq_suspend(void)
static void plic_irq_resume(void)
{
unsigned int i, index, cpu;
unsigned long flags;
u32 __iomem *reg;
struct plic_priv *priv;
@ -290,12 +294,12 @@ static void plic_irq_resume(void)
if (!handler->present)
continue;
raw_spin_lock(&handler->enable_lock);
raw_spin_lock_irqsave(&handler->enable_lock, flags);
for (i = 0; i < DIV_ROUND_UP(priv->nr_irqs, 32); i++) {
reg = handler->enable_base + i * sizeof(u32);
writel(handler->enable_save[i], reg);
}
raw_spin_unlock(&handler->enable_lock);
raw_spin_unlock_irqrestore(&handler->enable_lock, flags);
}
}
@ -376,9 +380,10 @@ static void plic_handle_irq(struct irq_desc *desc)
while ((hwirq = readl(claim))) {
int err = generic_handle_domain_irq(handler->priv->irqdomain,
hwirq);
if (unlikely(err))
pr_warn_ratelimited("can't find mapping for hwirq %lu\n",
hwirq);
if (unlikely(err)) {
dev_warn_ratelimited(handler->priv->dev,
"can't find mapping for hwirq %lu\n", hwirq);
}
}
chained_irq_exit(chip, desc);
@ -406,63 +411,122 @@ static int plic_starting_cpu(unsigned int cpu)
enable_percpu_irq(plic_parent_irq,
irq_get_trigger_type(plic_parent_irq));
else
pr_warn("cpu%d: parent irq not available\n", cpu);
dev_warn(handler->priv->dev, "cpu%d: parent irq not available\n", cpu);
plic_set_threshold(handler, PLIC_ENABLE_THRESHOLD);
return 0;
}
static int __init __plic_init(struct device_node *node,
struct device_node *parent,
unsigned long plic_quirks)
{
int error = 0, nr_contexts, nr_handlers = 0, i;
u32 nr_irqs;
struct plic_priv *priv;
struct plic_handler *handler;
unsigned int cpu;
static const struct of_device_id plic_match[] = {
{ .compatible = "sifive,plic-1.0.0" },
{ .compatible = "riscv,plic0" },
{ .compatible = "andestech,nceplic100",
.data = (const void *)BIT(PLIC_QUIRK_EDGE_INTERRUPT) },
{ .compatible = "thead,c900-plic",
.data = (const void *)BIT(PLIC_QUIRK_EDGE_INTERRUPT) },
{}
};
priv = kzalloc(sizeof(*priv), GFP_KERNEL);
static int plic_parse_nr_irqs_and_contexts(struct platform_device *pdev,
u32 *nr_irqs, u32 *nr_contexts)
{
struct device *dev = &pdev->dev;
int rc;
/*
* Currently, only OF fwnode is supported so extend this
* function for ACPI support.
*/
if (!is_of_node(dev->fwnode))
return -EINVAL;
rc = of_property_read_u32(to_of_node(dev->fwnode), "riscv,ndev", nr_irqs);
if (rc) {
dev_err(dev, "riscv,ndev property not available\n");
return rc;
}
*nr_contexts = of_irq_count(to_of_node(dev->fwnode));
if (WARN_ON(!(*nr_contexts))) {
dev_err(dev, "no PLIC context available\n");
return -EINVAL;
}
return 0;
}
static int plic_parse_context_parent(struct platform_device *pdev, u32 context,
u32 *parent_hwirq, int *parent_cpu)
{
struct device *dev = &pdev->dev;
struct of_phandle_args parent;
unsigned long hartid;
int rc;
/*
* Currently, only OF fwnode is supported so extend this
* function for ACPI support.
*/
if (!is_of_node(dev->fwnode))
return -EINVAL;
rc = of_irq_parse_one(to_of_node(dev->fwnode), context, &parent);
if (rc)
return rc;
rc = riscv_of_parent_hartid(parent.np, &hartid);
if (rc)
return rc;
*parent_hwirq = parent.args[0];
*parent_cpu = riscv_hartid_to_cpuid(hartid);
return 0;
}
static int plic_probe(struct platform_device *pdev)
{
int error = 0, nr_contexts, nr_handlers = 0, cpu, i;
struct device *dev = &pdev->dev;
unsigned long plic_quirks = 0;
struct plic_handler *handler;
u32 nr_irqs, parent_hwirq;
struct irq_domain *domain;
struct plic_priv *priv;
irq_hw_number_t hwirq;
bool cpuhp_setup;
if (is_of_node(dev->fwnode)) {
const struct of_device_id *id;
id = of_match_node(plic_match, to_of_node(dev->fwnode));
if (id)
plic_quirks = (unsigned long)id->data;
}
error = plic_parse_nr_irqs_and_contexts(pdev, &nr_irqs, &nr_contexts);
if (error)
return error;
priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
if (!priv)
return -ENOMEM;
priv->dev = dev;
priv->plic_quirks = plic_quirks;
priv->regs = of_iomap(node, 0);
if (WARN_ON(!priv->regs)) {
error = -EIO;
goto out_free_priv;
}
error = -EINVAL;
of_property_read_u32(node, "riscv,ndev", &nr_irqs);
if (WARN_ON(!nr_irqs))
goto out_iounmap;
priv->nr_irqs = nr_irqs;
priv->prio_save = bitmap_alloc(nr_irqs, GFP_KERNEL);
priv->regs = devm_platform_ioremap_resource(pdev, 0);
if (WARN_ON(!priv->regs))
return -EIO;
priv->prio_save = devm_bitmap_zalloc(dev, nr_irqs, GFP_KERNEL);
if (!priv->prio_save)
goto out_free_priority_reg;
nr_contexts = of_irq_count(node);
if (WARN_ON(!nr_contexts))
goto out_free_priority_reg;
error = -ENOMEM;
priv->irqdomain = irq_domain_add_linear(node, nr_irqs + 1,
&plic_irqdomain_ops, priv);
if (WARN_ON(!priv->irqdomain))
goto out_free_priority_reg;
return -ENOMEM;
for (i = 0; i < nr_contexts; i++) {
struct of_phandle_args parent;
irq_hw_number_t hwirq;
int cpu;
unsigned long hartid;
if (of_irq_parse_one(node, i, &parent)) {
pr_err("failed to parse parent for context %d.\n", i);
error = plic_parse_context_parent(pdev, i, &parent_hwirq, &cpu);
if (error) {
dev_warn(dev, "hwirq for context%d not found\n", i);
continue;
}
@ -470,7 +534,7 @@ static int __init __plic_init(struct device_node *node,
* Skip contexts other than external interrupts for our
* privilege level.
*/
if (parent.args[0] != RV_IRQ_EXT) {
if (parent_hwirq != RV_IRQ_EXT) {
/* Disable S-mode enable bits if running in M-mode. */
if (IS_ENABLED(CONFIG_RISCV_M_MODE)) {
void __iomem *enable_base = priv->regs +
@ -483,24 +547,17 @@ static int __init __plic_init(struct device_node *node,
continue;
}
error = riscv_of_parent_hartid(parent.np, &hartid);
if (error < 0) {
pr_warn("failed to parse hart ID for context %d.\n", i);
continue;
}
cpu = riscv_hartid_to_cpuid(hartid);
if (cpu < 0) {
pr_warn("Invalid cpuid for context %d\n", i);
dev_warn(dev, "Invalid cpuid for context %d\n", i);
continue;
}
/* Find parent domain and register chained handler */
if (!plic_parent_irq && irq_find_host(parent.np)) {
plic_parent_irq = irq_of_parse_and_map(node, i);
domain = irq_find_matching_fwnode(riscv_get_intc_hwnode(), DOMAIN_BUS_ANY);
if (!plic_parent_irq && domain) {
plic_parent_irq = irq_create_mapping(domain, RV_IRQ_EXT);
if (plic_parent_irq)
irq_set_chained_handler(plic_parent_irq,
plic_handle_irq);
irq_set_chained_handler(plic_parent_irq, plic_handle_irq);
}
/*
@ -510,7 +567,7 @@ static int __init __plic_init(struct device_node *node,
*/
handler = per_cpu_ptr(&plic_handlers, cpu);
if (handler->present) {
pr_warn("handler already present for context %d.\n", i);
dev_warn(dev, "handler already present for context %d.\n", i);
plic_set_threshold(handler, PLIC_DISABLE_THRESHOLD);
goto done;
}
@ -524,10 +581,10 @@ static int __init __plic_init(struct device_node *node,
i * CONTEXT_ENABLE_SIZE;
handler->priv = priv;
handler->enable_save = kcalloc(DIV_ROUND_UP(nr_irqs, 32),
sizeof(*handler->enable_save), GFP_KERNEL);
handler->enable_save = devm_kcalloc(dev, DIV_ROUND_UP(nr_irqs, 32),
sizeof(*handler->enable_save), GFP_KERNEL);
if (!handler->enable_save)
goto out_free_enable_reg;
goto fail_cleanup_contexts;
done:
for (hwirq = 1; hwirq <= nr_irqs; hwirq++) {
plic_toggle(handler, hwirq, 0);
@ -537,52 +594,60 @@ done:
nr_handlers++;
}
priv->irqdomain = irq_domain_add_linear(to_of_node(dev->fwnode), nr_irqs + 1,
&plic_irqdomain_ops, priv);
if (WARN_ON(!priv->irqdomain))
goto fail_cleanup_contexts;
/*
* We can have multiple PLIC instances so setup cpuhp state
* and register syscore operations only when context handler
* for current/boot CPU is present.
* and register syscore operations only once after context
* handlers of all online CPUs are initialized.
*/
handler = this_cpu_ptr(&plic_handlers);
if (handler->present && !plic_cpuhp_setup_done) {
cpuhp_setup_state(CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
"irqchip/sifive/plic:starting",
plic_starting_cpu, plic_dying_cpu);
register_syscore_ops(&plic_irq_syscore_ops);
plic_cpuhp_setup_done = true;
if (!plic_cpuhp_setup_done) {
cpuhp_setup = true;
for_each_online_cpu(cpu) {
handler = per_cpu_ptr(&plic_handlers, cpu);
if (!handler->present) {
cpuhp_setup = false;
break;
}
}
if (cpuhp_setup) {
cpuhp_setup_state(CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
"irqchip/sifive/plic:starting",
plic_starting_cpu, plic_dying_cpu);
register_syscore_ops(&plic_irq_syscore_ops);
plic_cpuhp_setup_done = true;
}
}
pr_info("%pOFP: mapped %d interrupts with %d handlers for"
" %d contexts.\n", node, nr_irqs, nr_handlers, nr_contexts);
dev_info(dev, "mapped %d interrupts with %d handlers for %d contexts.\n",
nr_irqs, nr_handlers, nr_contexts);
return 0;
out_free_enable_reg:
for_each_cpu(cpu, cpu_present_mask) {
fail_cleanup_contexts:
for (i = 0; i < nr_contexts; i++) {
if (plic_parse_context_parent(pdev, i, &parent_hwirq, &cpu))
continue;
if (parent_hwirq != RV_IRQ_EXT || cpu < 0)
continue;
handler = per_cpu_ptr(&plic_handlers, cpu);
kfree(handler->enable_save);
handler->present = false;
handler->hart_base = NULL;
handler->enable_base = NULL;
handler->enable_save = NULL;
handler->priv = NULL;
}
out_free_priority_reg:
kfree(priv->prio_save);
out_iounmap:
iounmap(priv->regs);
out_free_priv:
kfree(priv);
return error;
return -ENOMEM;
}
static int __init plic_init(struct device_node *node,
struct device_node *parent)
{
return __plic_init(node, parent, 0);
}
IRQCHIP_DECLARE(sifive_plic, "sifive,plic-1.0.0", plic_init);
IRQCHIP_DECLARE(riscv_plic0, "riscv,plic0", plic_init); /* for legacy systems */
static int __init plic_edge_init(struct device_node *node,
struct device_node *parent)
{
return __plic_init(node, parent, BIT(PLIC_QUIRK_EDGE_INTERRUPT));
}
IRQCHIP_DECLARE(andestech_nceplic100, "andestech,nceplic100", plic_edge_init);
IRQCHIP_DECLARE(thead_c900_plic, "thead,c900-plic", plic_edge_init);
static struct platform_driver plic_driver = {
.driver = {
.name = "riscv-plic",
.of_match_table = plic_match,
},
.probe = plic_probe,
};
builtin_platform_driver(plic_driver);

8
drivers/mailbox/bcm-flexrm-mailbox.c
View File

@ -1587,8 +1587,8 @@ static int flexrm_mbox_probe(struct platform_device *pdev)
}
/* Allocate platform MSIs for each ring */
ret = platform_msi_domain_alloc_irqs(dev, mbox->num_rings,
flexrm_mbox_msi_write);
ret = platform_device_msi_init_and_alloc_irqs(dev, mbox->num_rings,
flexrm_mbox_msi_write);
if (ret)
goto fail_destroy_cmpl_pool;
@ -1641,7 +1641,7 @@ skip_debugfs:
fail_free_debugfs_root:
debugfs_remove_recursive(mbox->root);
platform_msi_domain_free_irqs(dev);
platform_device_msi_free_irqs_all(dev);
fail_destroy_cmpl_pool:
dma_pool_destroy(mbox->cmpl_pool);
fail_destroy_bd_pool:
@ -1657,7 +1657,7 @@ static void flexrm_mbox_remove(struct platform_device *pdev)
debugfs_remove_recursive(mbox->root);
platform_msi_domain_free_irqs(dev);
platform_device_msi_free_irqs_all(dev);
dma_pool_destroy(mbox->cmpl_pool);
dma_pool_destroy(mbox->bd_pool);

4
drivers/perf/arm_smmuv3_pmu.c
View File

@ -716,7 +716,7 @@ static void smmu_pmu_free_msis(void *data)
{
struct device *dev = data;
platform_msi_domain_free_irqs(dev);
platform_device_msi_free_irqs_all(dev);
}
static void smmu_pmu_write_msi_msg(struct msi_desc *desc, struct msi_msg *msg)
@ -746,7 +746,7 @@ static void smmu_pmu_setup_msi(struct smmu_pmu *pmu)
if (!(readl(pmu->reg_base + SMMU_PMCG_CFGR) & SMMU_PMCG_CFGR_MSI))
return;
ret = platform_msi_domain_alloc_irqs(dev, 1, smmu_pmu_write_msi_msg);
ret = platform_device_msi_init_and_alloc_irqs(dev, 1, smmu_pmu_write_msi_msg);
if (ret) {
dev_warn(dev, "failed to allocate MSIs\n");
return;

8
drivers/ufs/host/ufs-qcom.c
View File

@ -1712,8 +1712,8 @@ static int ufs_qcom_config_esi(struct ufs_hba *hba)
* 2. Poll queues do not need ESI.
*/
nr_irqs = hba->nr_hw_queues - hba->nr_queues[HCTX_TYPE_POLL];
ret = platform_msi_domain_alloc_irqs(hba->dev, nr_irqs,
ufs_qcom_write_msi_msg);
ret = platform_device_msi_init_and_alloc_irqs(hba->dev, nr_irqs,
ufs_qcom_write_msi_msg);
if (ret) {
dev_err(hba->dev, "Failed to request Platform MSI %d\n", ret);
return ret;
@ -1742,7 +1742,7 @@ static int ufs_qcom_config_esi(struct ufs_hba *hba)
devm_free_irq(hba->dev, desc->irq, hba);
}
msi_unlock_descs(hba->dev);
platform_msi_domain_free_irqs(hba->dev);
platform_device_msi_free_irqs_all(hba->dev);
} else {
if (host->hw_ver.major == 6 && host->hw_ver.minor == 0 &&
host->hw_ver.step == 0)
@ -1818,7 +1818,7 @@ static void ufs_qcom_remove(struct platform_device *pdev)
pm_runtime_get_sync(&(pdev)->dev);
ufshcd_remove(hba);
platform_msi_domain_free_irqs(hba->dev);
platform_device_msi_free_irqs_all(hba->dev);
}
static const struct of_device_id ufs_qcom_of_match[] __maybe_unused = {

17
include/linux/irqdomain.h
View File

@ -619,6 +619,23 @@ static inline bool irq_domain_is_msi_device(struct irq_domain *domain)
#endif /* CONFIG_IRQ_DOMAIN_HIERARCHY */
#ifdef CONFIG_GENERIC_MSI_IRQ
int msi_device_domain_alloc_wired(struct irq_domain *domain, unsigned int hwirq,
unsigned int type);
void msi_device_domain_free_wired(struct irq_domain *domain, unsigned int virq);
#else
static inline int msi_device_domain_alloc_wired(struct irq_domain *domain, unsigned int hwirq,
unsigned int type)
{
WARN_ON_ONCE(1);
return -EINVAL;
}
static inline void msi_device_domain_free_wired(struct irq_domain *domain, unsigned int virq)
{
WARN_ON_ONCE(1);
}
#endif
#else /* CONFIG_IRQ_DOMAIN */
static inline void irq_dispose_mapping(unsigned int virq) { }
static inline struct irq_domain *irq_find_matching_fwnode(

2
include/linux/irqdomain_defs.h
View File

@ -26,6 +26,8 @@ enum irq_domain_bus_token {
DOMAIN_BUS_DMAR,
DOMAIN_BUS_AMDVI,
DOMAIN_BUS_PCI_DEVICE_IMS,
DOMAIN_BUS_DEVICE_MSI,
DOMAIN_BUS_WIRED_TO_MSI,
};
#endif /* _LINUX_IRQDOMAIN_DEFS_H */

24
include/linux/msi.h
View File

@ -412,6 +412,7 @@ bool arch_restore_msi_irqs(struct pci_dev *dev);
struct irq_domain;
struct irq_domain_ops;
struct irq_chip;
struct irq_fwspec;
struct device_node;
struct fwnode_handle;
struct msi_domain_info;
@ -431,6 +432,8 @@ struct msi_domain_info;
* function.
* @msi_post_free: Optional function which is invoked after freeing
* all interrupts.
* @msi_translate: Optional translate callback to support the odd wire to
* MSI bridges, e.g. MBIGEN
*
* @get_hwirq, @msi_init and @msi_free are callbacks used by the underlying
* irqdomain.
@ -468,6 +471,8 @@ struct msi_domain_ops {
struct device *dev);
void (*msi_post_free)(struct irq_domain *domain,
struct device *dev);
int (*msi_translate)(struct irq_domain *domain, struct irq_fwspec *fwspec,
irq_hw_number_t *hwirq, unsigned int *type);
};
/**
@ -547,6 +552,10 @@ enum {
MSI_FLAG_ALLOC_SIMPLE_MSI_DESCS = (1 << 5),
/* Free MSI descriptors */
MSI_FLAG_FREE_MSI_DESCS = (1 << 6),
/* Use dev->fwnode for MSI device domain creation */
MSI_FLAG_USE_DEV_FWNODE = (1 << 7),
/* Set parent->dev into domain->pm_dev on device domain creation */
MSI_FLAG_PARENT_PM_DEV = (1 << 8),
/* Mask for the generic functionality */
MSI_GENERIC_FLAGS_MASK = GENMASK(15, 0),
@ -572,6 +581,11 @@ enum {
* struct msi_parent_ops - MSI parent domain callbacks and configuration info
*
* @supported_flags: Required: The supported MSI flags of the parent domain
* @required_flags: Optional: The required MSI flags of the parent MSI domain
* @bus_select_token: Optional: The bus token of the real parent domain for
* irq_domain::select()
* @bus_select_mask: Optional: A mask of supported BUS_DOMAINs for
* irq_domain::select()
* @prefix: Optional: Prefix for the domain and chip name
* @init_dev_msi_info: Required: Callback for MSI parent domains to setup parent
* domain specific domain flags, domain ops and interrupt chip
@ -579,6 +593,9 @@ enum {
*/
struct msi_parent_ops {
u32 supported_flags;
u32 required_flags;
u32 bus_select_token;
u32 bus_select_mask;
const char *prefix;
bool (*init_dev_msi_info)(struct device *dev, struct irq_domain *domain,
struct irq_domain *msi_parent_domain,
@ -627,9 +644,6 @@ struct msi_domain_info *msi_get_domain_info(struct irq_domain *domain);
struct irq_domain *platform_msi_create_irq_domain(struct fwnode_handle *fwnode,
struct msi_domain_info *info,
struct irq_domain *parent);
int platform_msi_domain_alloc_irqs(struct device *dev, unsigned int nvec,
irq_write_msi_msg_t write_msi_msg);
void platform_msi_domain_free_irqs(struct device *dev);
/* When an MSI domain is used as an intermediate domain */
int msi_domain_prepare_irqs(struct irq_domain *domain, struct device *dev,
@ -656,6 +670,10 @@ int platform_msi_device_domain_alloc(struct irq_domain *domain, unsigned int vir
void platform_msi_device_domain_free(struct irq_domain *domain, unsigned int virq,
unsigned int nvec);
void *platform_msi_get_host_data(struct irq_domain *domain);
/* Per device platform MSI */
int platform_device_msi_init_and_alloc_irqs(struct device *dev, unsigned int nvec,
irq_write_msi_msg_t write_msi_msg);
void platform_device_msi_free_irqs_all(struct device *dev);
bool msi_device_has_isolated_msi(struct device *dev);
#else /* CONFIG_GENERIC_MSI_IRQ */

18
include/linux/soc/andes/irq.h Normal file
View File

@ -0,0 +1,18 @@
/* SPDX-License-Identifier: GPL-2.0-only */
/*
* Copyright (C) 2023 Andes Technology Corporation
*/
#ifndef __ANDES_IRQ_H
#define __ANDES_IRQ_H
/* Andes PMU irq number */
#define ANDES_RV_IRQ_PMOVI 18
#define ANDES_RV_IRQ_LAST ANDES_RV_IRQ_PMOVI
#define ANDES_SLI_CAUSE_BASE 256
/* Andes PMU related registers */
#define ANDES_CSR_SLIE 0x9c4
#define ANDES_CSR_SLIP 0x9c5
#define ANDES_CSR_SCOUNTEROF 0x9d4
#endif /* __ANDES_IRQ_H */

28
kernel/irq/irqdomain.c
View File

@ -29,6 +29,7 @@ static int irq_domain_alloc_irqs_locked(struct irq_domain *domain, int irq_base,
unsigned int nr_irqs, int node, void *arg,
bool realloc, const struct irq_affinity_desc *affinity);
static void irq_domain_check_hierarchy(struct irq_domain *domain);
static void irq_domain_free_one_irq(struct irq_domain *domain, unsigned int virq);
struct irqchip_fwid {
struct fwnode_handle fwnode;
@ -448,7 +449,7 @@ struct irq_domain *irq_find_matching_fwspec(struct irq_fwspec *fwspec,
*/
mutex_lock(&irq_domain_mutex);
list_for_each_entry(h, &irq_domain_list, link) {
if (h->ops->select && fwspec->param_count)
if (h->ops->select && bus_token != DOMAIN_BUS_ANY)
rc = h->ops->select(h, fwspec, bus_token);
else if (h->ops->match)
rc = h->ops->match(h, to_of_node(fwnode), bus_token);
@ -858,8 +859,13 @@ unsigned int irq_create_fwspec_mapping(struct irq_fwspec *fwspec)
}
if (irq_domain_is_hierarchy(domain)) {
virq = irq_domain_alloc_irqs_locked(domain, -1, 1, NUMA_NO_NODE,
fwspec, false, NULL);
if (irq_domain_is_msi_device(domain)) {
mutex_unlock(&domain->root->mutex);
virq = msi_device_domain_alloc_wired(domain, hwirq, type);
mutex_lock(&domain->root->mutex);
} else
virq = irq_domain_alloc_irqs_locked(domain, -1, 1, NUMA_NO_NODE,
fwspec, false, NULL);
if (virq <= 0) {
virq = 0;
goto out;
@ -914,7 +920,7 @@ void irq_dispose_mapping(unsigned int virq)
return;
if (irq_domain_is_hierarchy(domain)) {
irq_domain_free_irqs(virq, 1);
irq_domain_free_one_irq(domain, virq);
} else {
irq_domain_disassociate(domain, virq);
irq_free_desc(virq);
@ -1755,6 +1761,14 @@ void irq_domain_free_irqs(unsigned int virq, unsigned int nr_irqs)
irq_free_descs(virq, nr_irqs);
}
static void irq_domain_free_one_irq(struct irq_domain *domain, unsigned int virq)
{
if (irq_domain_is_msi_device(domain))
msi_device_domain_free_wired(domain, virq);
else
irq_domain_free_irqs(virq, 1);
}
/**
* irq_domain_alloc_irqs_parent - Allocate interrupts from parent domain
* @domain: Domain below which interrupts must be allocated
@ -1907,9 +1921,9 @@ static int irq_domain_alloc_irqs_locked(struct irq_domain *domain, int irq_base,
return -EINVAL;
}
static void irq_domain_check_hierarchy(struct irq_domain *domain)
{
}
static void irq_domain_check_hierarchy(struct irq_domain *domain) { }
static void irq_domain_free_one_irq(struct irq_domain *domain, unsigned int virq) { }
#endif /* CONFIG_IRQ_DOMAIN_HIERARCHY */
#ifdef CONFIG_GENERIC_IRQ_DEBUGFS

28
kernel/irq/matrix.c
View File

@ -8,8 +8,6 @@
#include <linux/cpu.h>
#include <linux/irq.h>
#define IRQ_MATRIX_SIZE (BITS_TO_LONGS(IRQ_MATRIX_BITS))
struct cpumap {
unsigned int available;
unsigned int allocated;
@ -17,8 +15,8 @@ struct cpumap {
unsigned int managed_allocated;
bool initialized;
bool online;
unsigned long alloc_map[IRQ_MATRIX_SIZE];
unsigned long managed_map[IRQ_MATRIX_SIZE];
unsigned long *managed_map;
unsigned long alloc_map[];
};
struct irq_matrix {
@ -32,8 +30,8 @@ struct irq_matrix {
unsigned int total_allocated;
unsigned int online_maps;
struct cpumap __percpu *maps;
unsigned long scratch_map[IRQ_MATRIX_SIZE];
unsigned long system_map[IRQ_MATRIX_SIZE];
unsigned long *system_map;
unsigned long scratch_map[];
};
#define CREATE_TRACE_POINTS
@ -50,24 +48,32 @@ __init struct irq_matrix *irq_alloc_matrix(unsigned int matrix_bits,
unsigned int alloc_start,
unsigned int alloc_end)
{
unsigned int cpu, matrix_size = BITS_TO_LONGS(matrix_bits);
struct irq_matrix *m;
if (matrix_bits > IRQ_MATRIX_BITS)
return NULL;
m = kzalloc(sizeof(*m), GFP_KERNEL);
m = kzalloc(struct_size(m, scratch_map, matrix_size * 2), GFP_KERNEL);
if (!m)
return NULL;
m->system_map = &m->scratch_map[matrix_size];
m->matrix_bits = matrix_bits;
m->alloc_start = alloc_start;
m->alloc_end = alloc_end;
m->alloc_size = alloc_end - alloc_start;
m->maps = alloc_percpu(*m->maps);
m->maps = __alloc_percpu(struct_size(m->maps, alloc_map, matrix_size * 2),
__alignof__(*m->maps));
if (!m->maps) {
kfree(m);
return NULL;
}
for_each_possible_cpu(cpu) {
struct cpumap *cm = per_cpu_ptr(m->maps, cpu);
cm->managed_map = &cm->alloc_map[matrix_size];
}
return m;
}

192
kernel/irq/msi.c
View File

@ -726,11 +726,26 @@ static void msi_domain_free(struct irq_domain *domain, unsigned int virq,
irq_domain_free_irqs_top(domain, virq, nr_irqs);
}
static int msi_domain_translate(struct irq_domain *domain, struct irq_fwspec *fwspec,
irq_hw_number_t *hwirq, unsigned int *type)
{
struct msi_domain_info *info = domain->host_data;
/*
* This will catch allocations through the regular irqdomain path except
* for MSI domains which really support this, e.g. MBIGEN.
*/
if (!info->ops->msi_translate)
return -ENOTSUPP;
return info->ops->msi_translate(domain, fwspec, hwirq, type);
}
static const struct irq_domain_ops msi_domain_ops = {
.alloc = msi_domain_alloc,
.free = msi_domain_free,
.activate = msi_domain_activate,
.deactivate = msi_domain_deactivate,
.translate = msi_domain_translate,
};
static irq_hw_number_t msi_domain_ops_get_hwirq(struct msi_domain_info *info,
@ -830,8 +845,11 @@ static struct irq_domain *__msi_create_irq_domain(struct fwnode_handle *fwnode,
domain = irq_domain_create_hierarchy(parent, flags | IRQ_DOMAIN_FLAG_MSI, 0,
fwnode, &msi_domain_ops, info);
if (domain)
if (domain) {
irq_domain_update_bus_token(domain, info->bus_token);
if (info->flags & MSI_FLAG_PARENT_PM_DEV)
domain->pm_dev = parent->pm_dev;
}
return domain;
}
@ -945,9 +963,9 @@ bool msi_create_device_irq_domain(struct device *dev, unsigned int domid,
void *chip_data)
{
struct irq_domain *domain, *parent = dev->msi.domain;
const struct msi_parent_ops *pops;
struct fwnode_handle *fwnode, *fwnalloced = NULL;
struct msi_domain_template *bundle;
struct fwnode_handle *fwnode;
const struct msi_parent_ops *pops;
if (!irq_domain_is_msi_parent(parent))
return false;
@ -970,7 +988,19 @@ bool msi_create_device_irq_domain(struct device *dev, unsigned int domid,
pops->prefix ? : "", bundle->chip.name, dev_name(dev));
bundle->chip.name = bundle->name;
fwnode = irq_domain_alloc_named_fwnode(bundle->name);
/*
* Using the device firmware node is required for wire to MSI
* device domains so that the existing firmware results in a domain
* match.
* All other device domains like PCI/MSI use the named firmware
* node as they are not guaranteed to have a fwnode. They are never
* looked up and always handled in the context of the device.
*/
if (bundle->info.flags & MSI_FLAG_USE_DEV_FWNODE)
fwnode = dev->fwnode;
else
fwnode = fwnalloced = irq_domain_alloc_named_fwnode(bundle->name);
if (!fwnode)
goto free_bundle;
@ -997,7 +1027,7 @@ bool msi_create_device_irq_domain(struct device *dev, unsigned int domid,
fail:
msi_unlock_descs(dev);
free_fwnode:
irq_domain_free_fwnode(fwnode);
irq_domain_free_fwnode(fwnalloced);
free_bundle:
kfree(bundle);
return false;
@ -1431,6 +1461,51 @@ int msi_domain_alloc_irqs_all_locked(struct device *dev, unsigned int domid, int
return msi_domain_alloc_locked(dev, &ctrl);
}
static struct msi_map __msi_domain_alloc_irq_at(struct device *dev, unsigned int domid,
unsigned int index,
const struct irq_affinity_desc *affdesc,
union msi_instance_cookie *icookie)
{
struct msi_ctrl ctrl = { .domid = domid, .nirqs = 1, };
struct irq_domain *domain;
struct msi_map map = { };
struct msi_desc *desc;
int ret;
domain = msi_get_device_domain(dev, domid);
if (!domain) {
map.index = -ENODEV;
return map;
}
desc = msi_alloc_desc(dev, 1, affdesc);
if (!desc) {
map.index = -ENOMEM;
return map;
}
if (icookie)
desc->data.icookie = *icookie;
ret = msi_insert_desc(dev, desc, domid, index);
if (ret) {
map.index = ret;
return map;
}
ctrl.first = ctrl.last = desc->msi_index;
ret = __msi_domain_alloc_irqs(dev, domain, &ctrl);
if (ret) {
map.index = ret;
msi_domain_free_locked(dev, &ctrl);
} else {
map.index = desc->msi_index;
map.virq = desc->irq;
}
return map;
}
/**
* msi_domain_alloc_irq_at - Allocate an interrupt from a MSI interrupt domain at
* a given index - or at the next free index
@ -1460,49 +1535,58 @@ struct msi_map msi_domain_alloc_irq_at(struct device *dev, unsigned int domid, u
const struct irq_affinity_desc *affdesc,
union msi_instance_cookie *icookie)
{
struct msi_ctrl ctrl = { .domid = domid, .nirqs = 1, };
struct irq_domain *domain;
struct msi_map map = { };
struct msi_desc *desc;
int ret;
struct msi_map map;
msi_lock_descs(dev);
domain = msi_get_device_domain(dev, domid);
if (!domain) {
map.index = -ENODEV;
goto unlock;
}
desc = msi_alloc_desc(dev, 1, affdesc);
if (!desc) {
map.index = -ENOMEM;
goto unlock;
}
if (icookie)
desc->data.icookie = *icookie;
ret = msi_insert_desc(dev, desc, domid, index);
if (ret) {
map.index = ret;
goto unlock;
}
ctrl.first = ctrl.last = desc->msi_index;
ret = __msi_domain_alloc_irqs(dev, domain, &ctrl);
if (ret) {
map.index = ret;
msi_domain_free_locked(dev, &ctrl);
} else {
map.index = desc->msi_index;
map.virq = desc->irq;
}
unlock:
map = __msi_domain_alloc_irq_at(dev, domid, index, affdesc, icookie);
msi_unlock_descs(dev);
return map;
}
/**
* msi_device_domain_alloc_wired - Allocate a "wired" interrupt on @domain
* @domain: The domain to allocate on
* @hwirq: The hardware interrupt number to allocate for
* @type: The interrupt type
*
* This weirdness supports wire to MSI controllers like MBIGEN.
*
* @hwirq is the hardware interrupt number which is handed in from
* irq_create_fwspec_mapping(). As the wire to MSI domain is sparse, but
* sized in firmware, the hardware interrupt number cannot be used as MSI
* index. For the underlying irq chip the MSI index is irrelevant and
* all it needs is the hardware interrupt number.
*
* To handle this the MSI index is allocated with MSI_ANY_INDEX and the
* hardware interrupt number is stored along with the type information in
* msi_desc::cookie so the underlying interrupt chip and domain code can
* retrieve it.
*
* Return: The Linux interrupt number (> 0) or an error code
*/
int msi_device_domain_alloc_wired(struct irq_domain *domain, unsigned int hwirq,
unsigned int type)
{
unsigned int domid = MSI_DEFAULT_DOMAIN;
union msi_instance_cookie icookie = { };
struct device *dev = domain->dev;
struct msi_map map = { };
if (WARN_ON_ONCE(!dev || domain->bus_token != DOMAIN_BUS_WIRED_TO_MSI))
return -EINVAL;
icookie.value = ((u64)type << 32) | hwirq;
msi_lock_descs(dev);
if (WARN_ON_ONCE(msi_get_device_domain(dev, domid) != domain))
map.index = -EINVAL;
else
map = __msi_domain_alloc_irq_at(dev, domid, MSI_ANY_INDEX, NULL, &icookie);
msi_unlock_descs(dev);
return map.index >= 0 ? map.virq : map.index;
}
static void __msi_domain_free_irqs(struct device *dev, struct irq_domain *domain,
struct msi_ctrl *ctrl)
{
@ -1628,6 +1712,30 @@ void msi_domain_free_irqs_all(struct device *dev, unsigned int domid)
msi_unlock_descs(dev);
}
/**
* msi_device_domain_free_wired - Free a wired interrupt in @domain
* @domain: The domain to free the interrupt on
* @virq: The Linux interrupt number to free
*
* This is the counterpart of msi_device_domain_alloc_wired() for the
* weird wired to MSI converting domains.
*/
void msi_device_domain_free_wired(struct irq_domain *domain, unsigned int virq)
{
struct msi_desc *desc = irq_get_msi_desc(virq);
struct device *dev = domain->dev;
if (WARN_ON_ONCE(!dev || !desc || domain->bus_token != DOMAIN_BUS_WIRED_TO_MSI))
return;
msi_lock_descs(dev);
if (!WARN_ON_ONCE(msi_get_device_domain(dev, MSI_DEFAULT_DOMAIN) != domain)) {
msi_domain_free_irqs_range_locked(dev, MSI_DEFAULT_DOMAIN, desc->msi_index,
desc->msi_index);
}
msi_unlock_descs(dev);
}
/**
* msi_get_domain_info - Get the MSI interrupt domain info for @domain
* @domain: The interrupt domain to retrieve data from