- Remove unused struct dw_edma_chip.irq (Frank Li)
- Move eDMA private data from struct dw_edma to struct dw_edma_chip (Frank
Li)
- Convert "struct dw_edma_region rg_region" to "void __iomem *reg_base"
since only the virtual address (not physical address or size) is used
(Frank Li)
- Rename "*_ch_cnt" to "ll_*_cnt" to reflect actual usage (Frank Li)
- Drop dma_slave_config.direction field usage (Serge Semin)
- Fix eDMA Rd/Wr-channels and DMA-direction semantics (Serge Semin)
- Add chip-specific DW_EDMA_CHIP_LOCAL flag to indicate that local eDMA
doesn't require generating MSIs to remote (Frank Li)
- Enable DMA tests for endpoints that support it (Frank Li)
* pci/ctrl/dwc-edma:
PCI: endpoint: Enable DMA tests for endpoints with DMA capabilities
dmaengine: dw-edma: Add support for chip-specific flags
dmaengine: dw-edma: Fix eDMA Rd/Wr-channels and DMA-direction semantics
dmaengine: dw-edma: Drop dma_slave_config.direction field usage
dmaengine: dw-edma: Rename wr(rd)_ch_cnt to ll_wr(rd)_cnt in struct dw_edma_chip
dmaengine: dw-edma: Change rg_region to reg_base in struct dw_edma_chip
dmaengine: dw-edma: Detach the private data and chip info structures
dmaengine: dw-edma: Remove unused irq field in struct dw_edma_chip
Some PCI Endpoint controllers integrate an eDMA (embedded DMA). eDMA can
bypass the outbound memory address translation unit to access all RC memory
space.
Add eDMA support for pci-epf-test.
Depending on HW availability, the EPF test can use either eDMA or general
system DMA controllers to perform DMA. The test tries to use eDMA first and
falls back to general system DMA controllers if there's no eDMA
Separate dma_chan to dma_chan_tx and dma_chan_rx. Search for an eDMA
channel first, then search for a memory-to-memory DMA channel. If general
memory to memory channels are used, dma_chan_rx = dma_chan_tx.
Add dma_addr_t dma_remote in pci_epf_test_data_transfer() because eDMA uses
remote RC physical address directly.
Add enum dma_transfer_direction dir in pci_epf_test_data_transfer() because
eDMA chooses the correct RX/TX channel by dir.
The overall steps are:
1. Execute dma_request_channel() and filter function to find correct eDMA
RX and TX Channel. If a channel does not exist, fallback to try to
allocate general memory to memory DMA channel.
2. Execute dmaengine_slave_config() to configure remote side physical
address.
3. Execute dmaengine_prep_slave_single() to create transfer descriptor.
4. Execute tx_submit().
5. Execute dma_async_issue_pending()
[bhelgaas: squash in fix from Dan Carpenter <dan.carpenter@oracle.com>:
https://lore.kernel.org/r/Ys2GSTnZhuLzzQG5@kili, also previously posted by
Peng Wu <wupeng58@huawei.com>:
https://lore.kernel.org/all/CANXvt5rK98-cEMgpzopY9POOK8a5=VDib8uKPLgJakOG=hRfwQ@mail.gmail.com/]
Link: https://lore.kernel.org/r/20220524152159.2370739-9-Frank.Li@nxp.com
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-by: Kishon Vijay Abraham I <kishon@ti.com>
Acked-By: Vinod Koul <vkoul@kernel.org>
Unbinding an endpoint function from the endpoint controller shouldn't stop
the controller. This is especially a problem for multi-function endpoints
where other endpoints may still be active.
Don't stop the controller when unbinding one of its endpoints. Normally
the controller is stopped via configfs.
Fixes: 349e7a85b2 ("PCI: endpoint: functions: Add an EP function to test PCI")
Link: https://lore.kernel.org/r/20220622040924.113279-1-mie@igel.co.jp
Signed-off-by: Shunsuke Mie <mie@igel.co.jp>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Kishon Vijay Abraham I <kishon@ti.com>
- Sort Intel Device IDs by value (Andy Shevchenko)
- Change Capability offsets to hex to match spec (Baruch Siach)
- Correct misspellings (Krzysztof Wilczyński)
- Terminate statement with semicolon in pci_endpoint_test.c (Ming Wang)
* pci/misc:
misc: pci_endpoint_test: Terminate statement with semicolon
PCI: Correct misspelled words
PCI: Change capability register offsets to hex
PCI: Sort Intel Device IDs by value
Fix a number of misspelled words, and while at it, correct two phrases used
to indicate a status of an operation where words used have been cleverly
truncated and thus always trigger a spellchecking error while performing a
static code analysis over the PCI tree.
[bhelgaas: reverse sense of quirk ternary]
Link: https://lore.kernel.org/r/20220107225942.121484-1-kw@linux.com
Signed-off-by: Krzysztof Wilczyński <kw@linux.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
- Use sysfs_emit() in "show" functions instead of sprintf() to avoid buffer
overruns (Kunihiko Hayashi)
* remotes/lorenzo/pci/endpoint:
PCI: endpoint: Use sysfs_emit() in "show" functions
- Tidy setup-irq.c comments (Pranay Sanghai)
- Fix misspellings (Krzysztof Wilczyński)
- Fix sprintf(), sscanf() format mismatches (Krzysztof Wilczyński)
- Tidy cpqphp code formatting (Krzysztof Wilczyński)
- Remove unused pci_pool wrappers, which have been replaced by dma_pool
(Cai Huoqing)
- Remove a redundant initialization in __pci_reset_function_locked() (Colin
Ian King)
- Use 'unsigned int' instead of 'unsigned' (Krzysztof Wilczyński)
- Update PCI subsystem information in MAINTAINERS (Krzysztof Wilczyński)
- Include generic <linux/> headers instead of <asm/> for cpqphp and vmd
(Krzysztof Wilczyński)
* pci/misc:
PCI: vmd: Drop redundant includes of <asm/device.h>, <asm/msi.h>
PCI: cpqphp: Use <linux/io.h> instead of <asm/io.h>
MAINTAINERS: Update PCI subsystem information
PCI: Prefer 'unsigned int' over bare 'unsigned'
PCI: Remove redundant 'rc' initialization
PCI: Remove unused pci_pool wrappers
PCI: cpqphp: Format if-statement code block correctly
PCI: Use unsigned to match sscanf("%x") in pci_dev_str_match_path()
PCI: hv: Remove unnecessary use of %hx
PCI: Correct misspelled and remove duplicated words
PCI: Tidy comments
Most of the "store" functions that handle userspace input via sysfs return
-EINVAL should the value fail validation and/or type conversion. This
error code is a clear message to userspace that the value is not a valid
input.
However, some of the "show" functions return input parsing error codes
as-is, which may be either -EINVAL or -ERANGE. The former would often be
from kstrtobool(), and the latter typically from other kstr*() functions
such as kstrtou8(), kstrtou32(), kstrtoint(), etc.
-EINVAL is commonly returned as the error code to indicate that the value
provided is invalid, but -ERANGE is not very useful in userspace.
Therefore, normalize the return error code to be -EINVAL for when the
validation and/or type conversion fails.
Link: https://lore.kernel.org/r/20210915230127.2495723-2-kw@linux.com
Signed-off-by: Krzysztof Wilczyński <kw@linux.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Add support to add virtual function in endpoint core. The virtual
function can only be associated with a physical function instead of a
endpoint controller. Provide APIs to associate a virtual function with
a physical function here.
[weiyongjun1@huawei.com: PCI: endpoint: Fix missing unlock on error in
pci_epf_add_vepf() - Reported-by: Hulk Robot <hulkci@huawei.com>]
Link: https://lore.kernel.org/r/20210819123343.1951-3-kishon@ti.com
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Wei Yongjun <weiyongjun1@huawei.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
The driver core ignores the return value of pci_epf_device_remove()
(because there is only little it can do when a device disappears) and
there are no pci_epf_drivers with a remove callback.
So make it impossible for future drivers to return an unused error code
by changing the remove prototype to return void.
The real motivation for this change is the quest to make struct
bus_type::remove return void, too.
Acked-by: Kishon Vijay Abraham I <kishon@ti.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
Link: https://lore.kernel.org/r/20210713193522.1770306-2-u.kleine-koenig@pengutronix.de
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
get_features ops of pci_epc_ops may return NULL, causing NULL pointer
dereference in pci_epf_test_alloc_space function. Let us add a check for
pci_epc_feature pointer in pci_epf_test_bind before we access it to avoid
any such NULL pointer dereference and return -ENOTSUPP in case
pci_epc_feature is not found.
When the patch is not applied and EPC features is not implemented in the
platform driver, we see the following dump due to kernel NULL pointer
dereference.
Call trace:
pci_epf_test_bind+0xf4/0x388
pci_epf_bind+0x3c/0x80
pci_epc_epf_link+0xa8/0xcc
configfs_symlink+0x1a4/0x48c
vfs_symlink+0x104/0x184
do_symlinkat+0x80/0xd4
__arm64_sys_symlinkat+0x1c/0x24
el0_svc_common.constprop.3+0xb8/0x170
el0_svc_handler+0x70/0x88
el0_svc+0x8/0x640
Code: d2800581 b9403ab9 f9404ebb 8b394f60 (f9400400)
---[ end trace a438e3c5a24f9df0 ]---
Link: https://lore.kernel.org/r/20210324101609.79278-1-shradha.t@samsung.com
Fixes: 2c04c5b8ee ("PCI: pci-epf-test: Use pci_epc_get_features() to get EPC features")
Signed-off-by: Sriram Dash <dash.sriram@gmail.com>
Signed-off-by: Shradha Todi <shradha.t@samsung.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: Pankaj Dubey <pankaj.dubey@samsung.com>
Reviewed-by: Kishon Vijay Abraham I <kishon@ti.com>
Documentation/PCI/endpoint/pci-endpoint-cfs.rst explains how a user has to
create a directory in-order to create a 'EPF Device' that can be
configured/probed by 'EPF Driver'.
Allow user to create a sub-directory of 'EPF Device' directory for any
function specific attributes that has to be exposed to the user.
Link: https://lore.kernel.org/r/20210201195809.7342-11-kishon@ti.com
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
In addition to the attributes that are generic across function drivers
documented in Documentation/PCI/endpoint/pci-endpoint-cfs.rst, there could
be function-specific attributes that has to be exposed by the function
driver to be configured by the user. Add ->add_cfs() in pci_epf_ops to be
populated by the function driver if it has to expose any function-specific
attributes and pci_epf_type_add_cfs() to be invoked by pci-ep-cfs.c when
sub-directory to main function directory is created.
Link: https://lore.kernel.org/r/20210201195809.7342-10-kishon@ti.com
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Add pci_epc_ops to map physical address to MSI address and return MSI data.
The physical address is an address in the outbound region. This is required
to implement doorbell functionality of NTB (non-transparent bridge) wherein
EPC on either side of the interface (primary and secondary) can directly
write to the physical address (in outbound region) of the other interface
to ring doorbell using MSI.
Link: https://lore.kernel.org/r/20210201195809.7342-9-kishon@ti.com
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Now that PCI endpoint core supports to add secondary endpoint controller
(EPC) with endpoint function (EPF), Add support in configfs to associate
two EPCs with EPF. This creates "primary" and "secondary" directory inside
the directory created by users for EPF device. Users have to add a symlink
of endpoint controller (pci_ep/controllers/) to "primary" or "secondary"
directory to bind EPF to primary and secondary EPF interfaces respectively.
Existing method of linking directory representing EPF device to directory
representing EPC device to associate a single EPC device with a EPF device
will continue to work.
Link: https://lore.kernel.org/r/20210201195809.7342-8-kishon@ti.com
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
In the case of standard endpoint functions, only one endpoint controller
(EPC) will be associated with an endpoint function (EPF). However for
providing NTB (non transparent bridge) functionality, two EPCs should be
associated with a single EPF. Add support to associate secondary EPC with
EPF. This is in preparation for adding NTB endpoint function driver.
Link: https://lore.kernel.org/r/20210201195809.7342-7-kishon@ti.com
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
pci_epc_get_first_free_bar() uses only "reserved_bar" member in
epc_features to get the first unreserved BAR. However if the reserved BAR
is also a 64-bit BAR, then the next BAR shouldn't be returned (since 64-bit
BAR uses two BARs).
Make pci_epc_get_first_free_bar() take into account 64 bit BAR while
returning the first free unreserved BAR.
Link: https://lore.kernel.org/r/20210201195809.7342-3-kishon@ti.com
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
R-Car PCIe controller has support to map multiple memory regions for
mapping the outbound memory in local system also the controller limits
single allocation for each region (that is, once a chunk is used from the
region it cannot be used to allocate a new one). This features inspires to
add support for handling multiple memory bases in endpoint framework.
With this patch pci_epc_mem_init() initializes address space for endpoint
controller which support single window and pci_epc_multi_mem_init()
initializes multiple windows supported by endpoint controller.
Link: https://lore.kernel.org/r/1588854799-13710-6-git-send-email-prabhakar.mahadev-lad.rj@bp.renesas.com
Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Acked-by: Kishon Vijay Abraham I <kishon@ti.com>
When unbinding pci_epf_test, pci_epf_test_clean_dma_chan() is called in
pci_epf_test_unbind() even though epf_test->dma_supported is false.
As a result, dma_release_channel() will trigger a NULL pointer
dereference because dma_chan is not set.
Avoid calling dma_release_channel() if epf_test->dma_supported
is false.
Link: https://lore.kernel.org/r/1587540287-10458-1-git-send-email-hayashi.kunihiko@socionext.com
Fixes: 5ebf3fc59b ("PCI: endpoint: functions/pci-epf-test: Add DMA support to transfer data")
Signed-off-by: Kunihiko Hayashi <hayashi.kunihiko@socionext.com>
[lorenzo.pieralisi@arm.com: commit log]
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Kishon Vijay Abraham I <kishon@ti.com>
commit beb4641a78 ("PCI: dwc: Add MSI-X callbacks handler"),
in order to raise MSI-X interrupt, obtained MSIX table address from
Base Address Register (BAR). However BAR only holds PCI address
programmed by the host whereas the MSI-X table should be in the local
memory.
Store the MSI-X table address (virtual address) as part of ->set_bar()
callback and use that to get the message address and message data
here.
Fixes: beb4641a78 ("PCI: dwc: Add MSI-X callbacks handler")
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
commit 8963106eab ("PCI: endpoint: Add MSI-X interfaces") while
adding support to raise MSI-X interrupts from endpoint didn't include
BAR Indicator register (BIR) configuration and MSI-X table offset as
arguments in pci_epc_set_msix(). This would result in endpoint
controller register using random BAR indicator register, the memory
for which might not be allocated by the endpoint function driver.
Add BAR indicator register and MSI-X table offset as arguments in
pci_epc_set_msix() and allocate space for MSI-X table and pending
bit array (PBA) in pci-epf-test endpoint function driver.
Fixes: 8963106eab ("PCI: endpoint: Add MSI-X interfaces")
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Print throughput information in KB/s after every completed transfer,
including information on whether DMA is used or not.
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Tested-by: Alan Mikhak <alan.mikhak@sifive.com>
After an endpoint is started through configfs, if 0 is written to the
configfs entry 'start', the controller stops but the epc_group->start
value remains 1.
A subsequent unlinking of the function from the controller would trigger
a spurious WARN_ON_ONCE() in pci_epc_epf_unlink() despite right
behavior.
Fix it by setting epc_group->start = 0 when a controller is stopped
using configfs.
Fixes: d746799116 ("PCI: endpoint: Introduce configfs entry for configuring EP functions")
Signed-off-by: Kunihiko Hayashi <hayashi.kunihiko@socionext.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Kishon Vijay Abraham I <kishon@ti.com>
Cc: Kishon Vijay Abraham I <kishon@ti.com>
Add support to defer core initialization for the endpoint mode of
operation.
This would enable support for implementations where the core
initialization needs to be deferred until the PCIe reference clock is
available from the host system.
Signed-off-by: Vidya Sagar <vidyas@nvidia.com>
[lorenzo.pieralisi@arm.com: commit log]
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Kishon Vijay Abraham I <kishon@ti.com>
Add support to send notifications to EPF from EPC once the core
registers initialization is complete.
Signed-off-by: Vidya Sagar <vidyas@nvidia.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Kishon Vijay Abraham I <kishon@ti.com>
The PCIe endpoint core relies on the drivers that invoke the
pci_epc_add_epf() API to allocate and assign a function number
to each physical function (PF). Since endpoint function device can
be created by multiple mechanisms (configfs, devicetree, etc..),
allowing each of these mechanisms to assign a function number
would result in mutliple endpoint function devices having the
same function number. In order to avoid this, let EPC core assign
a function number to the endpoint device.
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Protect concurrent access to pci_epf_ops with a mutex.
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
pci-epc-mem uses a bitmap to manage the Endpoint outbound (OB) address
region. This address region will be shared by multiple endpoint
functions (in the case of multi function endpoint) and it has to be
protected from concurrent access to avoid updating an inconsistent state.
Use a mutex to protect bitmap updates to prevent the memory
allocation API from returning incorrect addresses.
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: stable@vger.kernel.org # v4.14+
The pci_epc_ops is not intended to be invoked from interrupt context.
Hence replace spin_lock_irqsave and spin_unlock_irqrestore with
mutex_lock and mutex_unlock respectively.
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
