As of 07940c369a ("PCI: dwc: Fix MSI page leakage in suspend/resume"),
the PCIe designware host driver has been using the driver data allocation
for the msi_msg DMA mapping which can result in a DMA_MAPPING_ERROR due to
the DMA overflow check in dma_direct_map_page() when the address is greater
than 32 bits (reported in [1]). The commit was trying to address a memory
leak on suspend/resume by moving the MSI mapping to dw_pcie_host_init(),
but subsequently dropped the page allocation thinking it wasn't needed.
To fix the DMA mapping issue as well as make msi_msg DMA'able, switch back
to allocating a 32-bit page for the msi_msg. To avoid the suspend/resume
leak, allocate the page in dw_pcie_host_init() since that shouldn't be
called during suspend/resume.
[1] https://lore.kernel.org/all/Yo0soniFborDl7+C@google.com/
Signed-off-by: Will McVicker <willmcvicker@google.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Rob Herring <robh@kernel.org>
The DWC PCIe RC/EP/DM IP core configuration parameters determine the number
of inbound and outbound iATU windows, alignment requirements (which is also
the minimum window size), minimum and maximum sizes. If internal ATU is
enabled, the former settings are determined by CX_ATU_MIN_REGION_SIZE; the
latter are determined by CX_ATU_MAX_REGION_SIZE.
Determine the required alignment and maximum size supported by the
controller and log it to help verify whether the requested inbound or
outbound memory mappings can be fully created.
Note 1. The extended iATU regions have been supported since DWC PCIe
v4.60a. There is no need in testing the upper limit register availability
for the older cores.
Note 2. The regions alignment is determined with using the fls() method
since the lower four bits of the ATU Limit register can be occupied with
the Circular Buffer Increment setting, which can be initialized with zeros.
Link: https://lore.kernel.org/r/20220624143947.8991-13-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Previously __dw_pcie_prog_outbound_atu() duplicated a lot of code between
the iatu_unroll_enabled version and the PCIE_ATU_VIEWPORT version:
__dw_pcie_prog_outbound_atu
if (iatu_unroll_enabled)
dw_pcie_prog_outbound_atu_unroll
dw_pcie_writel_ob_unroll(PCIE_ATU_UNR_LOWER_BASE, ...)
dw_pcie_writel_ob_unroll(PCIE_ATU_UNR_UPPER_BASE, ...)
...
return
dw_pcie_writel_dbi(PCIE_ATU_VIEWPORT, ...)
dw_pcie_writel_dbi(PCIE_ATU_LOWER_BASE, ...)
dw_pcie_writel_dbi(PCIE_ATU_UPPER_BASE, ...)
...
Unify those by pushing the unroll address computation and viewport
selection down into dw_pcie_writel_atu() so we can use the same
dw_pcie_writel_atu_ob() accessor for both paths:
__dw_pcie_prog_outbound_atu
dw_pcie_writel_atu_ob(PCIE_ATU_LOWER_BASE, ...)
dw_pcie_writel_atu
dw_pcie_select_atu # new
if (iatu_unroll_enabled)
return pci->atu_base + PCIE_ATU_UNROLL_BASE(...)
dw_pcie_writel_dbi(PCIE_ATU_VIEWPORT, ...)
return pci->atu_base
dw_pcie_write(base + reg)
dw_pcie_writel_atu_ob(PCIE_ATU_UPPER_BASE, ...)
...
In the non-unroll case, this does involve more MMIO writes to
PCIE_ATU_VIEWPORT, but it's mainly in initialization paths and the code
simplification is significant.
[bhelgaas: commit log, simplify dw_pcie_select_atu()]
Link: https://lore.kernel.org/r/20220624143947.8991-12-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Previously callers of dw_pcie_disable_atu() supplied enum
dw_pcie_region_type (DW_PCIE_REGION_INBOUND, DW_PCIE_REGION_OUTBOUND),
which dw_pcie_disable_atu() converted to the PCIE_ATU_REGION_DIR_IB or
PCIE_ATU_REGION_DIR_OB values needed to program the ATU registers.
Simplify the code by dropping the dw_pcie_region_type enum and passing
PCIE_ATU_REGION_DIR_IB or PCIE_ATU_REGION_DIR_OB directly.
Reorder dw_pcie_disable_atu() arguments to (dir, index) since "index"
indicates an ATU window in the regions of the corresponding direction.
[bhelgaas: commit log]
Link: https://lore.kernel.org/r/20220624143947.8991-11-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Previously dw_pcie_ep_set_bar() converted the BAR PCI_BASE_ADDRESS_SPACE
bit to the internal dw_pcie_as_type enum (DW_PCIE_AS_MEM, DW_PCIE_AS_IO)
and passed it down to dw_pcie_prog_inbound_atu(), which converted the enum
to the PCIE_ATU_TYPE_MEM/PCIE_ATU_TYPE_IO values needed to program the ATU
registers.
Simplify the code by dropping the dw_pcie_as_type enum and passing
PCIE_ATU_TYPE_MEM or PCIE_ATU_TYPE_IO directly.
Reorder inbound ATU function arguments to match the outbound functions,
with address-related parameters at the end.
[bhelgaas: commit log]
Link: https://lore.kernel.org/r/20220624143947.8991-10-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Since DWC PCIe v4.70a, the controller version and version type can be read
from the PORT_LOGIC.PCIE_VERSION_OFF and PORT_LOGIC.PCIE_VERSION_TYPE_OFF
registers respectively.
Read the version from those registers and warn if if's different from the
version we got from the device tree.
We can only read the version after platform-specific drivers have done any
DBI-related initialization, such as reference clock activation.
[bhelgaas: commit log]
Link: https://lore.kernel.org/r/20220624143947.8991-5-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Factor out this pattern:
if (!pci->ops || !pci->ops->start_link)
return -EINVAL;
return pci->ops->start_link(pci);
into a new dw_pcie_start_link() wrapper and do the same for the stop_link()
method.
Note that dw_pcie_ep_start() previously returned -EINVAL if there was no
platform start_link() method, which didn't make much sense since that is
not an error. It will now return 0 in that case.
As a side-effect, drop the empty start_link() and dummy dw_pcie_ops
instances from the generic DW PCIe and Layerscape EP platform drivers.
[bhelgaas: commit log]
Link: https://lore.kernel.org/r/20220624143428.8334-14-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
There are several places in the common DW PCIe code with incoherent local
variable usage: a variable is defined and initialized with a structure
field, but the structure pointer is dereferenced to access that field
anyway; the local variable is defined and initialized but either used just
once or not used afterwards in the main part of the subsequent method. It
mainly concerns the pcie_port.dev field. Fix that in the relevant places.
Link: https://lore.kernel.org/r/20220624143428.8334-12-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
The unrolled version of the internal ATU has been available since the DWC
PCIe v4.80a IP core, but it may not be enabled. Per [1], if unrolled ATU
is enabled, the PCIE_ATU_VIEWPORT does not exist and reads as 0xffffffff;
while if unrolled ATU is disabled, PCIE_ATU_VIEWPORT will contain some
zeros.
Simplify dw_pcie_iatu_unroll_enabled() by checking the value of
PCIE_ATU_VIEWPORT.
[1] DesignWare Cores, PCI Express Controller, Register Desciptions,
v.4.90a, December 2016, p.855
[bhelgaas: commit log]
Link: https://lore.kernel.org/r/20220624143428.8334-10-Sergey.Semin@baikalelectronics.ru
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Reviewed-by: Rob Herring <robh@kernel.org>
We program the 64-bit ATU limit address (in PCIE_ATU_LIMIT/
PCIE_ATU_UPPER_LIMIT or PCIE_ATU_UNR_LOWER_LIMIT/PCIE_ATU_UNR_UPPER_LIMIT),
but in addition, the PCIE_ATU_INCREASE_REGION_SIZE bit must be set if the
upper 32 bits of the limit address differ from the upper 32 bits of the
base address (see [1,2]).
5b4cf0f653 ("PCI: dwc: Add upper limit address for outbound iATU") set
PCIE_ATU_INCREASE_REGION_SIZE, but only when the *size* was greater than
4GB. It did not set it when a smaller region crossed a 4GB boundary, e.g.,
[mem 0x0_f0000000-0x1_0fffffff].
Set PCIE_ATU_INCREASE_REGION_SIZE whenever PCIE_ATU_UPPER_LIMIT is
greater than PCIE_ATU_UPPER_BASE.
[1] DesignWare Cores PCI Express Controller Databook - DWC PCIe Root Port,
v5.40a, March 2019, fig.3-36, p.175
[2] DesignWare Cores PCI Express Controller Databook - DWC PCIe Root Port,
v5.40a, March 2019, fig.3-37, p.176
[bhelgaas: commit log]
Fixes: 5b4cf0f653 ("PCI: dwc: Add upper limit address for outbound iATU")
Link: https://lore.kernel.org/r/20220624143428.8334-5-Sergey.Semin@baikalelectronics.ru
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Reviewed-by: Rob Herring <robh@kernel.org>
dw_pcie_disable_atu() was introduced by f8aed6ec62 ("PCI: dwc:
designware: Add EP mode support") and supported only the viewport version
of the iATU CSRs.
DW PCIe IP cores v4.80a and newer also support unrolled iATU/eDMA space.
Callers of dw_pcie_disable_atu(), including pci_epc_ops.clear_bar(),
pci_epc_ops.unmap_addr(), and dw_pcie_setup_rc(), don't work correctly when
it is enabled.
Add dw_pcie_disable_atu() support for controllers with unrolled iATU CSRs
enabled.
[bhelgaas: commit log]
Fixes: f8aed6ec62 ("PCI: dwc: designware: Add EP mode support")
Link: https://lore.kernel.org/r/20220624143428.8334-3-Sergey.Semin@baikalelectronics.ru
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Reviewed-by: Rob Herring <robh@kernel.org>
It's logically correct to undo everything that was done when an error is
discovered or in the corresponding cleanup counterpart. Otherwise the host
controller will be left in an undetermined state. Since the link is set up
in the host_init method, deactivate it there in the cleanup-on-error block
and stop the link in the antagonistic routine - dw_pcie_host_deinit(). Link
deactivation is platform-specific and should be implemented in
dw_pcie_ops.stop_link().
Fixes: 886a9c1347 ("PCI: dwc: Move link handling into common code")
Link: https://lore.kernel.org/r/20220624143428.8334-2-Sergey.Semin@baikalelectronics.ru
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Reviewed-by: Rob Herring <robh@kernel.org>
Pull pci updates from Bjorn Helgaas:
"Resource management:
- Restrict E820 clipping to PCI host bridge windows (Bjorn Helgaas)
- Log E820 clipping better (Bjorn Helgaas)
- Add kernel cmdline options to enable/disable E820 clipping (Hans de
Goede)
- Disable E820 reserved region clipping for IdeaPads, Yoga, Yoga
Slip, Acer Spin 5, Clevo Barebone systems where clipping leaves no
usable address space for touchpads, Thunderbolt devices, etc (Hans
de Goede)
- Disable E820 clipping by default starting in 2023 (Hans de Goede)
PCI device hotplug:
- Include files to remove implicit dependencies (Christophe Leroy)
- Only put Root Ports in D3 if they can signal and wake from D3 so
AMD Yellow Carp doesn't miss hotplug events (Mario Limonciello)
Power management:
- Define pci_restore_standard_config() only for CONFIG_PM_SLEEP since
it's unused otherwise (Krzysztof Kozlowski)
- Power up devices completely, including anything platform firmware
needs to do, during runtime resume (Rafael J. Wysocki)
- Move pci_resume_bus() to PM callbacks so we observe the required
bridge power-up delays (Rafael J. Wysocki)
- Drop unneeded runtime_d3cold device flag (Rafael J. Wysocki)
- Split pci_raw_set_power_state() between pci_power_up() and a new
pci_set_low_power_state() (Rafael J. Wysocki)
- Set current_state to D3cold if config read returns ~0, indicating
the device is not accessible (Rafael J. Wysocki)
- Do not call pci_update_current_state() from pci_power_up() so BARs
and ASPM config are restored correctly (Rafael J. Wysocki)
- Write 0 to PMCSR in pci_power_up() in all cases (Rafael J. Wysocki)
- Split pci_power_up() to pci_set_full_power_state() to avoid some
redundant operations (Rafael J. Wysocki)
- Skip restoring BARs if device is not in D0 (Rafael J. Wysocki)
- Rearrange and clarify pci_set_power_state() (Rafael J. Wysocki)
- Remove redundant BAR restores from pci_pm_thaw_noirq() (Rafael J.
Wysocki)
Virtualization:
- Acquire device lock before config space access lock to avoid AB/BA
deadlock with sriov_numvfs_store() (Yicong Yang)
Error handling:
- Clear MULTI_ERR_COR/UNCOR_RCV bits, which a race could previously
leave permanently set (Kuppuswamy Sathyanarayanan)
Peer-to-peer DMA:
- Whitelist Intel Skylake-E Root Ports regardless of which devfn they
are (Shlomo Pongratz)
ASPM:
- Override L1 acceptable latency advertised by Intel DG2 so ASPM L1
can be enabled (Mika Westerberg)
Cadence PCIe controller driver:
- Set up device-specific register to allow PTM Responder to be
enabled by the normal architected bit (Christian Gmeiner)
- Override advertised FLR support since the controller doesn't
implement FLR correctly (Parshuram Thombare)
Cadence PCIe endpoint driver:
- Correct bitmap size for the ob_region_map of outbound window usage
(Dan Carpenter)
Freescale i.MX6 PCIe controller driver:
- Fix PERST# assertion/deassertion so we observe the required delays
before accessing device (Francesco Dolcini)
Freescale Layerscape PCIe controller driver:
- Add "big-endian" DT property (Hou Zhiqiang)
- Update SCFG DT property (Hou Zhiqiang)
- Add "aer", "pme", "intr" DT properties (Li Yang)
- Add DT compatible strings for ls1028a (Xiaowei Bao)
Intel VMD host bridge driver:
- Assign VMD IRQ domain before enumeration to avoid IOMMU interrupt
remapping errors when MSI-X remapping is disabled (Nirmal Patel)
- Revert VMD workaround that kept MSI-X remapping enabled when IOMMU
remapping was enabled (Nirmal Patel)
Marvell MVEBU PCIe controller driver:
- Add of_pci_get_slot_power_limit() to parse the
'slot-power-limit-milliwatt' DT property (Pali Rohár)
- Add mvebu support for sending Set_Slot_Power_Limit message (Pali
Rohár)
MediaTek PCIe controller driver:
- Fix refcount leak in mtk_pcie_subsys_powerup() (Miaoqian Lin)
MediaTek PCIe Gen3 controller driver:
- Reset PHY and MAC at probe time (AngeloGioacchino Del Regno)
Microchip PolarFlare PCIe controller driver:
- Add chained_irq_enter()/chained_irq_exit() calls to mc_handle_msi()
and mc_handle_intx() to avoid lost interrupts (Conor Dooley)
- Fix interrupt handling race (Daire McNamara)
NVIDIA Tegra194 PCIe controller driver:
- Drop tegra194 MSI register save/restore, which is unnecessary since
the DWC core does it (Jisheng Zhang)
Qualcomm PCIe controller driver:
- Add SM8150 SoC DT binding and support (Bhupesh Sharma)
- Fix pipe clock imbalance (Johan Hovold)
- Fix runtime PM imbalance on probe errors (Johan Hovold)
- Fix PHY init imbalance on probe errors (Johan Hovold)
- Convert DT binding to YAML (Dmitry Baryshkov)
- Update DT binding to show that resets aren't required for
MSM8996/APQ8096 platforms (Dmitry Baryshkov)
- Add explicit register names per chipset in DT binding (Dmitry
Baryshkov)
- Add sc7280-specific clock and reset definitions to DT binding
(Dmitry Baryshkov)
Rockchip PCIe controller driver:
- Fix bitmap size when searching for free outbound region (Dan
Carpenter)
Rockchip DesignWare PCIe controller driver:
- Remove "snps,dw-pcie" from rockchip-dwc DT "compatible" property
because it's not fully compatible with rockchip (Peter Geis)
- Reset rockchip-dwc controller at probe (Peter Geis)
- Add rockchip-dwc INTx support (Peter Geis)
Synopsys DesignWare PCIe controller driver:
- Return error instead of success if DMA mapping of MSI area fails
(Jiantao Zhang)
Miscellaneous:
- Change pci_set_dma_mask() documentation references to
dma_set_mask() (Alex Williamson)"
* tag 'pci-v5.19-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci: (64 commits)
dt-bindings: PCI: qcom: Add schema for sc7280 chipset
dt-bindings: PCI: qcom: Specify reg-names explicitly
dt-bindings: PCI: qcom: Do not require resets on msm8996 platforms
dt-bindings: PCI: qcom: Convert to YAML
PCI: qcom: Fix unbalanced PHY init on probe errors
PCI: qcom: Fix runtime PM imbalance on probe errors
PCI: qcom: Fix pipe clock imbalance
PCI: qcom: Add SM8150 SoC support
dt-bindings: pci: qcom: Document PCIe bindings for SM8150 SoC
x86/PCI: Disable E820 reserved region clipping starting in 2023
x86/PCI: Disable E820 reserved region clipping via quirks
x86/PCI: Add kernel cmdline options to use/ignore E820 reserved regions
PCI: microchip: Fix potential race in interrupt handling
PCI/AER: Clear MULTI_ERR_COR/UNCOR_RCV bits
PCI: cadence: Clear FLR in device capabilities register
PCI: cadence: Allow PTM Responder to be enabled
PCI: vmd: Revert 2565e5b69c ("PCI: vmd: Do not disable MSI-X remapping if interrupt remapping is enabled by IOMMU.")
PCI: vmd: Assign VMD IRQ domain before enumeration
PCI: Avoid pci_dev_lock() AB/BA deadlock with sriov_numvfs_store()
PCI: rockchip-dwc: Add legacy interrupt support
...
According to the PCIe standard the PERST# signal (reset-gpio in
fsl,imx* compatible dts) should be kept asserted for at least 100 usec
before the PCIe refclock is stable, should be kept asserted for at
least 100 msec after the power rails are stable and the host should wait
at least 100 msec after it is de-asserted before accessing the
configuration space of any attached device.
From PCIe CEM r2.0, sec 2.6.2
T-PVPERL: Power stable to PERST# inactive - 100 msec
T-PERST-CLK: REFCLK stable before PERST# inactive - 100 usec.
From PCIe r5.0, sec 6.6.1
With a Downstream Port that does not support Link speeds greater than
5.0 GT/s, software must wait a minimum of 100 ms before sending a
Configuration Request to the device immediately below that Port.
Failure to do so could prevent PCIe devices to be working correctly,
and this was experienced with real devices.
Move reset assert to imx6_pcie_assert_core_reset(), this way we ensure
that PERST# is asserted before enabling any clock, move de-assert to the
end of imx6_pcie_deassert_core_reset() after the clock is enabled and
deemed stable and add a new delay of 100 msec just afterward.
Link: https://lore.kernel.org/all/20220211152550.286821-1-francesco.dolcini@toradex.com
Link: https://lore.kernel.org/r/20220404081509.94356-1-francesco.dolcini@toradex.com
Fixes: bb38919ec5 ("PCI: imx6: Add support for i.MX6 PCIe controller")
Signed-off-by: Francesco Dolcini <francesco.dolcini@toradex.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Acked-by: Richard Zhu <hongxing.zhu@nxp.com>
